Click here to download the PDF version of the transcript.
Stan Skrabut: Are you ready for another episode of In the Classroom podcast? I certainly am. Welcome back. Once again thank you ever so much for joining me. I really appreciate it. I know all the other things that you can be doing and you’re hanging out with me. I can’t thank you enough. Today is the last episode in this series that we have dealing with the OSCQR Rubric. The OSCQR Rubric is an online course quality review rubric. It looks at the course through instructional technologists or instructional designers’ viewpoint to help make the best course that you can possibly make, at least from the design aspect of it.
This is the sixth section. We’ve already gone through five different sections and we started at episode number 19 if you want to get go and review all those episodes maybe you haven’t checked them out yet. Today we are going to focus on assessment and feedback and of the 50 standards, there are seven standards in this particular section. Let’s get started. Standard number 44, “Course grading policies, including the consequences of late submissions, are clearly stated in the course information area or syllabus.” Nothing drives a learner more crazy than obscure instructions requirements and so, by being as clear upfront as possible, you’re just going to make everybody’s life that much easier.”
Learners need to know how their work will be assessed in a clear and transparent manner. Grading policies can guide learner progress and promote fair and objective review and assessment of all graded work. The research is showing that grading policies directly impact motivation.” If students are not clear about it, they get frustrated, their motivation goes down and you can help this just by making things more clear. All assignments and graded activities should have clear goals and should tie back to your objectives. Do not just have an assignment for the sake of an assignment, but it should address one of those specific goals.
Different ways that you can do it, you can make your grading policies clearer in both the syllabus, but also the course information document so that other area out there you can lay those things out and reference to them from the syllabus. Ideas that you may want to incorporate dealing with this particular standard. If you do peer review grade work, provide a grading system, a rubric specifically, for the learners and ask for feedback how well that the system is working. For group projects, include a team reporting tool with a grading rubric for learners, provide feedback on how the other learners fulfilled their role in the team, establish criteria that ties back to program, course and module objectives.
Talked about that and how that rubric addresses things such as clarity and precision, grammar and spelling, creativity, other skills that you might want to measure. You may want to pull this together into a handbook on grading policies and rubrics that the learners could download and have on hand. It’s a quick access guide. Keep things simple. If an assignment or graded activity can be measured bypass/fail, use a simplified scale. I am a huge fan of competency–based activities where either it’s a go or a no go. In the military, that it either meets the standard or doesn’t meet the standard. The fact that it’s like, “Well, it’s 90% or 95%,” I don’t really care about that.
Things can always be improved. They can always be better. Things can always be worse. The bottom line is can the learner do it and meet a certain standard that is an acceptable standard? That has served me well for all these years and that’s the one that I like, and I stick to it. It’s very easy with that type of standard, you just kick it back and say, “No, it’s not good enough. Here’s some feedback. Let’s do it again.” That’s the grading that I like. I like this opportunity for students to– Learning has to be messy. It’s not just one-and-done, but let’s try to really work on getting it right. If you have regrading rules, stick to them.
Here’s the policy on whatever it is, and this will help to mitigate disputes later, but I have a regrading policy. I grade everything until the last day of class. So far, it’s worked out. Number 45, “Course includes frequent and appropriate methods to assess learners’ mastery”. We talked about in episodes eight, nine and 10 about universal design for learning and this, I think, really ties into this learning design or universal design for learning that one of the things that you want to do is not just have the same type of assessment over and over and over again because the only individuals you’re benefitting are the individuals that are good at those types of assessment.
You need to have choice in the assessments and I would also encourage to have a lot more assess, that you don’t have one assessment that’s worth half the grade and they get it wrong, they’ve already failed the class, that having multiple assessments, you can measure things more incrementally and it gives them an opportunity to be successful over the long period of time. As the rubric indicates, “Consistent, regular assessments help learners demonstrate their progress and deficiencies. As learners move through an online course, they should encounter regular assignments, activities, and interactions designed to assess how well they have mastered the learning content and how close they are to meeting the program, course or module learning objectives.”
More opportunities to measure this, I think, for me this is important. Otherwise, you can seriously lose a student if they are not engaged in the course on a regular basis, especially for an online course. If you don’t have interaction for two, three, four weeks, you’re pretty much going to lose that student, so you need to keep them on task. Ideas that you may want to weave in. It could be quizzes. It could be your textbook publisher talks about quizzes. In episode number 18, I talked about test pools, being able to build test pools that you can regularly test your students’ knowledge and how you can weave that in, so being able to use the grade book and the grading and the rubric’s tools that are part of your learning management system to help develop your assessment that you should give.
Those are great tools to build your assessment. It provides some immediate feedback depending on how you use them. Everything should be mapped back to your learning objectives and that way you can assess how well that you met those particular objectives. Those are different ways that you want to consider how to building those assessments for mastery. Now, naturally, I have some of my personal thoughts weaved in there. Number 46, “Criteria for the assessment of a graded assignment are clearly articulated”. This goes back to a lot of the other things that we have said, dealing with consistency, but clarity.
Rubrics are a great way to help define clarity, that for these different assignments they’re easy to measure a student. They can be given to the students ahead of time so they can sell measure their work to make sure that they’re on track. It makes the learning targets clearer. It helps guide the design and delivery of the instruction and normalizes the assessment process, that you’re assessing each one pretty much the same way. Other things that you want to do here is if you have some exemplary work, other students who have mastered this, get permission to put those upfront and use them as models that other students could use.
Things that you want to try out in this area, learn how to use some rubrics. Instructional technologists would be happy to give you a tour on how to use a grading rubric and how to build one for your particular course. Keep examples so that exemplary work. Actually, you have to have permission to use it, but that would be great. I think students really appreciate it when their information is known. I’m going to put a link into the show notes of some course rubrics that are out there. There’s some good examples that you can benefit from, that you can use to build yours and then, there’s also some rubric tools.
I’ll put those all into the show notes. Number 47, “Learners have opportunities to review their performance and assess their own learning throughout the course (pre-test, automated self-tests, reflective assignments, etcetera)”. Great ways of doing it. As I mentioned in that episode 18, I talked about test pools. Test pools, you can build these really massive test banks that allow students to go in and self-test on different aspects of the course, really powerful tool, something that you should consider. Self-assessment fosters learners’ ability, construct meaning and promotes self metacognition by asking them to check their own skill mastery before you do it will help them examine their own reasoning processes and decision-making processes.
That’s some of the reasons that you want to do it. Suggestions that the rubric offers is have learners develop a personal goal statement for the course. Check-in on this at the midterm and the end of the course to see where they’re at with this particular goal statement. Include non-graded test your knowledge quizzes at the end of each module, with that you can use extra credit points for example. Maybe you have to get to 100%, then they get the extra credit points.
You can add multiple tests in order to do this, different ways that you can approach it. When you have a reflective exercise, provide clear guidance on what learners should provide . You may demonstrate writing samples, very often in a class is like okay, we want you to do this but there was having never done it before, very often didn’t know what I needed to do in order to meet the expectations of the instructor. Having some examples are really useful for learners. The nice thing about online courses is you can provide those types of examples. Another thing that you can do is have the learners rate their own participation in discussion forums or other assignments.
Have them have that reflective piece to let them self-analyze how well that they performed in that particular activity. E-portfolios, another powerful tool that you can use to collect what students have put together and show progress over the course of a term. Number 48, learners are informed when a time response is required. Proper lead time is provided to ensure there’s an opportunity to prepare an accommodation. Another important one, there has to be some latitude especially for online learners because I’ve watched some instructors saying that you will do this at 3 o’clock and 4 o’clock the test close.
Well, with online learners, you have no idea what they’re doing at three o’clock and they can’t just simply drop everything to access your test. We’re not thinking about if we do that, we’re not thinking about what it is to be an online student. Having that latitude to make it available. If there’s going to be something that has to be a specific deadline so learners need to know those well upfront so they can work them into their schedule and they need to know everything about that assignment so they can clearly meet that deadline for time responses. Those usually come in as far as quizzes and things like that.
Instructors benefit from promoting effective time management strategies for students in addition to providing information when time responses are required. Instructors can explain the importance of respecting set deadlines and the courtesy of timeliness. This goes back to that idea why and I talked about that in a previous episode but why are we doing things and why are we doing the things the way that we’re doing. This idea of universal design for learning talks about modeling expert performance. Instructors are experts and they have built all this knowledge and it becomes second nature. For novice learners who are new to a discipline, this is all new.
It’s really imperative that instructors go out and explain why things are done certain ways. In order to help them learn to think as an expert performer, this is part of that discussion. If you go back to the Universal Design for Learning episodes which were eight through 10, I talked a little bit more about that. If learners are expected to post discussion forums by a specific day and time, that information needs to be clearly detailed in all discussion form, descriptions and overviews. Not hidden away in one area but every single discussion forum needs to have those clear expectations.
You may say it’s overkill that they should be able to know this knowledge but if you want to improve the results of your course, taking time to do these things will certainly help. Other mechanisms that you can provide to help are a downloadable, printable weekly checklist with details on when everything is due each day of the week throughout the course module. That module checklist, hey these things are due. Different ways you can do it. I know in blackboard there’s a tool that you could turn on where students check off those particular items.
Having that as part of the opening to the module, here’s what you’re expected to do when everything’s due, that’ll do wonders for your course. Other things that you can do is set up an automated email that reminds students or learners at least once a day in advance that they need to respond or take action in the course by a specific date and time. There’s tools that you can use, I use a tool called MailChimp and I can load my students in there and for the course, I can set up the emails all the way through the course so it runs automatic and just advises them and reminds them when to do that.
Let’s see, highlight specific dates and times in module overviews or weekly announcements so we talked a little bit about that. Number 49, learners have easy access to a well-designed and updated grade book. Grade books are the number one tool that students check out in a learning management system that they’re constantly going to the grade book to see what to do, where they stand, how well they’re doing, they need this feedback and guidance to stay on track especially in online courses because they don’t run into the professor every day so having that grade book keeps them on track on what they still need to do and how to get there.
Online grade books also provide instructors with the opportunity to automate, customize and share grades and feedback with students, that this is a communication tool as well as just a feedback tool for students but it’s a way that you can communicate to students if they’re on track or not. There should be a link off your navigation structure for that grade book. You should provide it also a video overview on how the grade book works. How students can see the feedback for your assignments and that has been frustrating for a lot of instructors that they have said, hey I put feedback in your assignments, but students don’t know how to access it and it’s been frustrating for students because they want to get feedback.
Those are different ways to do it. When you are labeling your grade book or putting together your assignments, having short titles helps to keep the grade book clean. Those are some strategies that you can have. If you need a grade book tutorial, talk to your instructional designers, they’ll be happy to provide one for you. Encourage your students to check the grade book after every assignment just to make sure that nothing is missing. Don’t let them go weeks and months without checking these things. They need to be able to stay on track. All right, we have come down to number 50, the last one.
Number 50, learners have multiple opportunities to provide descriptive feedback on course design, course content, course experience and ease of online technology. Basically, what this is saying is you need to provide a feedback mechanism that where students can talk to you, bow areas of your course that you can improve. If they find navigation difficult or content is not great, there should be a place where they can tell you about that and not have to wait to the end of the course that there should be maybe surveys along the way or an open discussion forum where you can gather that input.
When I was early on, I would create surveys and do them every month asking if we were on track, how everybody was doing. This is certainly a way that you can do that. Putting a suggestion box for them to collect informal feedback, having the informal surveys, distributing surveys, different parts of the course. You want to try to make this a learning opportunity that your course, in the end, the goal is for students to learn. If something is impeding their learning, it would be great to know about it as soon as possible so you can make the adjustments.
I had a course where students didn’t like the times when I opened up a discussion and so we did a quick survey and they told me when they preferred to open up the discussions and I made the adjustment. It wasn’t big to me but it helped the course move along. Other things that you may want to do is invite students to do a full course review. Have them look things over module by module and offer opportunities for improvement. Other ways are like the military’s after-action review where you ask for three things that they liked about the course, three things that could stand improvement and by doing this after-action review, you get all great feedback to help improve your course.
As I noted before, turn on comments for Google Documents so students can comment on different parts of the instructions that you provide and I happen to use Google Documents where I could improve on those particular instructions to make them clearer or to provide more explanation, what have you. Different ways of doing that. We hit number 50 but this was the last part, focused on assessment and feedback. Here’s a recap of the different standards that we just talked about. Number 44, course grading policies including the consequences of late submissions are clearly stated in the Course Information area or Syllabus. 45, course includes frequent or appropriate methods to assess learners’ mastery of content. 46, criteria for the assessment of a graded assignment are clearly articulated in rubrics and exemplary work. 47, learners have opportunities to review their performance and assess their own learning throughout the course through pre-test, automated self-tests, reflective assignments, et cetera. 48, learners are informed when a timed response is required. Properly, lead time is provided to ensure there’s the opportunity to prepare and accommodation.
49, learners have easy access to a well-designed and up to date, grade book. Finally, number 50, learners have multiple opportunities to provide descriptive feedback and course design, course content learning, or course experience and ease of online technology. Well, there you have it. So episodes 19 through this one which happens to be 24, we talked about the OSCQR online course quality review rubric. These are 50 standards that will help you just improve your course. If you put a number of these into play, your course will definitely improve. I can guarantee that. [sound cut] In the meantime, here’s a quick plug for my book, Read to Succeed.
