Click here to download the PDF version of the transcript.
Stan Skrabut: Thanks, everyone, so much for taking time to listen to this podcast. It certainly means a lot. I know you could be doing other things, perhaps you are, but you’re still hanging out with me, and I really appreciate it. What are you doing this summer? When I’m recording this, this is definitely during the summer. If you’re normal faculty, then you’re probably out, not in the classroom, but doing other things, perhaps pursuing some research or just maybe just getting back some energy after this whole COVID experience. I don’t blame you. Have you been out because the COVID restrictions are being rolled back?
What are you doing? Let me know. This week, we’re going to take a look at after-action reviews. Have you ever put together a class activity and expected one thing to happen, but something entirely different occurred, or are you looking for an easy but effective strategy for identifying what’s working in your classroom and what needs to improve? After-action reviews are tools that you’re looking for.
I have used after-action reviews for the last 40 some years. I started using them while I was in the military. In the air force, we did debriefings all the time, but come to find out, we were following a lot of the strategies that the army put in place, and they owned the territory for after-action reviews, although a lot of businesses have started using those. I’ve introduced them into my classroom. We are going to take a look at after-action reviews and how they can improve your courses, and also improve student learning. Let’s get started. First of all, what is an after-action review? According to the army, an after-action review is a professional discussion of an event focused on performance standards that enable soldiers to discover for themselves what happened, why it happened, and how to sustain strengths and improve on weaknesses. If you switch out the word “soldiers” with “students,” we have a great tool that can be used effectively in the classroom.
During my graduate-level program planning evaluation course, I conduct two drafts and peer review sessions. They have to write a program plan. They have to do two drafts of this and turn in a final product. After the drafts, there’s peer reviews and ideas, they take all that information, make a better program plan. In addition, I have discussions every week. After the drafts are done, after the peer reviews are done, I conduct an after-action reviews. I use one of those discussions, 1 of the 15, actually I use 2 of them, and I do after-action reviews.
This has helped me continue to improve the course. I always learn things that I can use to make the course better, but the students also have benefited from the sessions in the fact that they get to reflect on the learning process. In every course, every activity, there’s always some weaknesses that need to be improved. There’s always something really great about the course that we want to sustain and keep making better. After-action reviews are not just a meeting, but there’s a practice. What I mean is that you should be doing them regularly after every learning event to reflect on what’s happening.
They’re focused on continuous improvement. We can look at after-action reviews as an emergent learning practice. The guidance for developing an emergent learning practice, this came from Charles Parry and Marilyn Darling’s article, Emergent Learning in Action: The After Action Review. In this, they talk about four patterns that characterize emergent learning and are consistently found in successful after-action review applications. These four patterns are localness, forward-focus, punctuation, and iteration.
Localness refers to the task proximity, the group that’s directly responsible for the task and the results. This could be defined as groups in your classroom or for the entire class, depending on the scope of the activity. If you have an activity broken down into groups, then you would probably have each of those groups do an action review, or in the case where I’m doing, everybody’s involved in a peer review, the entire class, then I would have them, the entire class, participate in that peer review.
One of the nice things about this localness is that members get to see how their actions affect others and what else is transpiring in the class and the activities. Forward-focus means that you’re looking towards the future first. As you’re building your lessons, these lessons learned through these after-action reviews, what you’re doing is, you’re looking how you can apply them to the next opportunity. You are learning from what you’ve done in order to do what you’re going to do better.
Punctuation is focused. If you’re looking at the entire term, which, of course, evaluation does, it looks at the entire term. Too much time has transpired to really capture those opportunities to make your course better. There are certainly things that you can learn from a course evaluation, but if you do it right after the activity or the assignment and do it, you have a greater chance of capturing what can be useful to continuously improve that activity.
You want to take your term and break it down into those unique activities and do an after-action review after each one of those. In terms of iteration, the army thinks learning has not occurred until insight shapes actual behavior and is validated by results, which means you need to do it again. Learning occurs when there’s more than one opportunity to put actions into play. It’s not one and done. Very often, in higher ed, we do one-and-done things. That’s why I switched when I did my program plan to have multiple opportunities to write and continuously improve the plan instead of “just turn a plan in at the end of the term and I’ll grade it.” I want them to actually keep improving.
Here are some key elements to consider when doing an after-actions review. First of all, conduct this during or immediately after the event. Once you have an activity, make sure that you’ve also scheduled time to immediately do an after-actions review. Focus on intended learning objectives. What was the overall intent of that activity? Did that activity achieve that intent? Making sure that everyone understands what that activity, what the purpose of that activity is. Focus on student or team performance. This is where you’re not really trying to isolate individual students, but you’re trying to find those trends that are happening across all the students.
You’re looking for opportunities where they can improve, and acknowledging those activities during that after-action review session, whether that was an adequate performance, something that they should continue, or something that they should change, so definitely, it should be a learning opportunity as you’re doing this after-actions review. Also importantly, is that everyone participates in the discussion. One way to do this is just go around the room and ask each person. What I like to do is, I like to go around the room and ask them for what went right first. We’ll talk about this in a second. I’m looking for unique responses, so I’m not looking for duplication.
I’m going around the room asking for something that went right during this activity, and we’ll certainly capture that. When doing that, you want to also know why it went right. [unintelligible 00:07:43] all possible use open-ended questions, but you’re looking for brief responses. You are not looking for war and peace, right? You’re looking for something, this occurred, this is why it’s believed to be a good practice that we should continue and move on.
Try to make it a quick and engaging activity, but let’s not dwell on the details. You can later go back and look for additional details, if necessary. As I had mentioned earlier, you are looking for specific standards, whether or not they were met. This is not tied to a person. You’re not looking for a blame game. You’re not trying to assess blame or give praise unnecessarily, right?
You’re trying to keep names out of this as much as possible. What you’re looking for are what the strengths are, what the weaknesses are, and you want to collect these responses so you can later do something with them, to have a list that you work through. In that way, you can go back to your course, go back to that activity, and work to improve it. In order to get everybody’s participation, so when you’re conducting after-actions review, you’re trying to solicit honest feedback from everyone involved.
You need to create the right atmosphere. To do that, you may need to set some ground rules. These particular ground rules, I think it’s a great list, come from the guide to after-action review, and where I get my information, I will include it in the show notes. Please, go check out those show notes, but here are some ground rules that you may want to consider before you dive into this after-action review.
First of all, active participation, it’s important for everyone to participate. However, I know through sessions that I’ve conducted, when you start whittling down areas for improvements or strengths that suddenly folks have no new ideas, and so, having a pass is allowed. It’s also important that everyone’s views have equal value. Now, in the military, there was a rank structure. What was important is that everybody in that room had equal value in their views and presenting those views equally. Someone who is an airman and a general, they got the same say in that opportunity because, in those cases, you were looking to improve operations. Who better to understand the operations than the individuals actually doing it? With that, you are not assigning blame. You’re trying not to assign blame. You are trying to fix processes. You’re trying to improve learning. You’re not assessing blame. You’re just trying to correct maybe misconceptions, or faulty processes, or somebody didn’t understand the activity. Then you need to write the activity better.
Actually, next week, we’re going to talk about transparent assignment design. Come back for that. Also, there are no right or wrong answers. This is somebody’s viewpoint, so it’s not right or wrong. Also, be open to new ideas when you’re looking to improve things. I’ve received feedback on some of the assignments that I have put into place before. I’ve had students come back and say, “Why don’t you try it this way?” It’s like, “I never thought of that. That’s a wonderful idea.”
Think in the manner of Pixar also, that you are looking for a “yes” and rather than “no, but,” or “either/or” type thinking. “Yes, and that’s great” and “We can do it a different way.” Trying to gather consensus with possible. If you can’t get consensus, ask for clarification. When you’re doing this and you come to those areas where you’re talking about weaknesses, something that’s not going right, what you’re also looking for is ways to improve.
That individual should also come up with recommendations for improved approaches. When you are capturing a record of the discussion, you need to make sure that everybody’s onboard before you distribute this outside of that group, and also that when you capture quotes, they are not tied to any specific individual. That is some ground rules. By presenting those to everybody upfront, you can lay out those ground rules in order to build trust, in order to get everybody to participate.
As far as tips for success, try to do the after-actions review as soon as possible. Start with what went right. This gets everybody contributing. You may also want to give participants a couple minutes to think about and perhaps write down their ideas before anyone speaks. You’re really going through two questions. There’s a third component, but when it comes time for the questions, you’re looking at what went right and why, and what can be improved and how.
Having those questions out there and giving folks time to think about it, maybe jot some notes, that’ll help with participation. Go around the room to give everyone a chance to speak and ensure that you are giving everyone a chance to speak, even the quieter ones who tend not to. You want to make sure that you’re getting maximum participation. Also, when you’re trying to get to the root cause of a problem that you identify a problem, the five whys. Why is this happening? Why did this occur? Why does it continue to occur? Why, why, why? You want to really tap into that.
Then, when you’re also looking at weaknesses, you ask the question, “What would you do differently next time?” If they had an opportunity– Looking at the classroom situation that you built an activity, and it did not go necessarily the way that you would hope, you can ask a question, “What would you do in order to ensure that this was a successful activity?” Get responses that way.
When you’re capturing responses, try to do it in a visible manner. You can have a flip chart on an easel and writing things down and displaying those all over the room, or use an electronic capture. You can have it displayed maybe on the screen, or we’ve been working in a remote environment, so on Zoom. Bring up a document and start capturing the responses that way.
Probably the most important tip for success is the fact that you have to do something with the insights that you gain. You can’t just let them die on the vine that you just did this activity, collected the input. You need to do something with that information. Now, you can probably see that this can be something more than just an activity. It can be also brought out of the classroom for maybe a department, try to implement something, do an after-action review after that, or for the entire school. We have orientation, for example, for faculty. I’m new to the process. One of the things I asked everybody to do after the [unintelligible 00:14:41] was an after-actions review because it did not go as smooth as I had hoped, even being the new guy. I wanted to see what we could do to improve the process, because the more effective that faculty are brought on board, the smoother that the term is going to roll out.
After-action reviews can be very formal, where you have potentially hundreds of people involved, and they’re being facilitated and take hours to do, or they can be very informal, where you, as the instructor– Take 15 minutes. It does not take long in a classroom to go through these couple of questions and get the responses. It does not take very much time at all. Now, what does take time is if you want to elaborate and have discussion around some of those items, and that certainly can take a lot more time.
Here’s the method that I typically do for after-actions review. When I talk about this, this is normally for a face-to-face situation, that some type of activity has occurred. I change it slightly for an online environment when I do it in discussion. For the face-to-face, first thing I do is, I brief, or I explain what should have occurred during the activity, what I was expecting to see during that particular activity.
Then I will ask the students to supply three inputs based on what they believe went well and why. In doing this, I go around the classroom only getting one input at a time. I’m not asking students for all three of their inputs at one time, I do it one at a time and go around the classroom because it’s very possible that you could run out of input right away, depending on who you choose in your classroom.
You want to give everybody an opportunity to just listen to one thing at a time from one person at a time, and they can also then formulate new thoughts, things that they didn’t consider, and add them to their list and provide them as they come around. I’ve found that to work extremely well. Then, I ask the students for three areas of improvement and how. I repeat the same process that I did.
Once again, I go around the classroom multiple times, three times, trying to get individual input from each student each time around. I capture all this input on a document that I share with the students. It’s also, in some ways, almost a to-do list for me as I go into make improvements to the course. That’s how I do it for a face-to-face class. Now, for an online class, it’s slightly different.
For an online class, I’m doing it through a threaded discussion, so they have a week to look at this. I will ask them for their three areas where they thought things went well and why, but also three areas that they believe could be improved and how it can be improved. Also, part of that discussion is, I’m asking them to try not to repeat what other students have said, but maybe add on and keep the discussion going. That’s basically an after-actions review. Now, after action reviews can certainly be used for any classroom activity, but they can also be used by students to reflect upon their individual performance on an assignment, their quiz preparation, or any other activity.
It’s an really nice reflective tool that gets everybody looking at the learning process and what was expecting to happen and what actually happened and how we can improve that for the next time that it goes around. I encourage you to weave in these after-action reviews into your classroom. One of the other things that I want to encourage you to do is check out my book. Here’s a quick plug for that book, Read to Succeed.