Layering Game Mechanics on Top of a Civil Air Patrol Summer Encampment, Part 3

In this post, I will let you know the results of the gamification experiment I conducted at a Civil Air Patrol summer encampment. You can read about this in this post, Layering game mechanics on top of a Civil Air Patrol summer encampment, Part 1. 

Months before the encampment, I began preparing the materials and methodology that I was going to use. During this process, I explained my concept to key members of the encampment staff. Each conversation helped to shape the encampment. It was important to me that we conducted an encampment different than previous encampments. I believed it important to change the culture of encampments from a harsh hazing atmosphere to one of teaching and encouragement. The key staff bought into this idea.

Two days prior to the in-flight cadets arriving, I had an opportunity to brief the entire staff and explain my idea. It took time to shape the new roles that they were in from roles they had fixed in their minds. In addition to group discussions, I had to occasionally pull cadre aside and provide more specific direction. However, when the in-flight cadets arrived, I believe most cadets knew how to perform as cadre. On the first day of the encampment, I explained to the in-flight cadets how the game would work. I also explained that they did not have to participate if they chose not to, but that it would not help their flight’s success.

Game Element Results

Here are the results for key elements:

Knowledge Book Memorization

I was perhaps most anxious about this element. Would cadets participate or not? I pleased to discover that all 53 in flight cadets participated at one level or another. Out of a possible 2,180 achievable points, four cadets were able to achieve maximum points. Two cadets achieved maximum points thus recited the entire booklet within the first three days. The mean score was 617 with a mode of 300. The minimum score was 50 point.

Flights were able to earn points on nine unique items. All flights earned points on only one same item, quotation #7. The difference between the top flight and the last flight was 360 points; this is essentially learning two quotes. Here is a picture of the results:

Quote results

Room Inspection

Based on previous encampments, I expected that rooms would gradually improve over the week (we are talking about teenagers making their beds). I expected to see a lot of reds during the beginning of the week, and progressively more yellows and greens as the week progressed. I was not disappointed. If a cadet received an excellent rating on all items throughout the week, they could earn 840 points. If they earned a satisfactory, they would have earned 420 points. The final results show a minimum of 70 and a maximum of 570, mean of 194 and a mode of 170. If I had another week, I would have seen significant improvement. A picture helps to tell the whole story.

Flights managed to earn points on only four unique inspection items out of 42 possible items. I personally was hoping for more tactical planning on the part of flight commanders. It was towards the end of the week when the flight commanders learned how to “game” the system and encouraged their flight to focus more energy to selected parts of the inspection in an effort to put points on the board.

Room Inspection Results

Open Ranks Inspections

There was no mystery regarding the open ranks inspections. I expected results would improve throughout the week and they did. The difference between first place and third was only 132 points. A flight could earn a possible 400 points per day, and there were four inspections for a total of 1,600 points. The lead flight earned 1,344 points.

Uniform Inspections

Three of the uniform inspection were on battle dress uniforms (BDUs) and one was on the blues uniform. The BDUs are hard to fix because of all the items that must be sewn on. On the other hand, the blues has more attention to detail. Across the four inspections, cadets could earn a maximum of 40 points. On average, cadets earned 20 points with a mode of 20 and a minimum of 7 and maximum of 29 points. No cadets had a perfect uniform on any given day.

The top flight and last flight were separated by only 57 points.

Cadet Physical Fitness Test

During the cadet physical fitness test, cadets could earn a maximum of 40 points. Out of 53 cadets, 45% (N=24) cadets passed all four parts of the test, and 4 cadets did not participate or failed all portions of the test.

The difference between the top flight from the last place flight was 60 points.

Sports

There were four opportunities to earn points through sports competition. One flight won three of the four engagements, and one flight did not win a single event.

Aerospace Education Quizzes

Across seven quiz bowls, there were a possible 4,510 points available. The difference between first and last was 670 points.

Overall Results

For individual cadet performance, the honor cadet earned 2,560 points out of a possible 3,100 points. The mean was 872 and mode was 440 points with a minimum of 216 and maximum of 2,560 points.  This cadet only earned maximum points on one item.

The top-flight achieved 6,632 points and lead the last place flight by 1,539 points. They managed to place first six out of nine events.

Lesson Learned

Overall, I am pleased with how the encampment ran and the results. Many members of the staff also reported pleasure with how the encampment ran. Naturally, we learned some lessons.

First of all, the spreadsheet was not perfect when we began, and we had to make tweaks along the way. Fortunately, we have significantly improved the spreadsheet and it is ready for the next encampment.

The cadet leadership needs more direction, and they need to be cautioned that it is only a game and not worth anything other than bragging rights in the end. In one case, one of the flight commanders wanted one of his cadets sent home because he did not believe she was pulling her weight. After reviewing the metrics, it was determined she had twelve other cadets behind her in terms of points. She was retained.

I also need to continue to look at the game elements to see if more balancing can be done.

In the end, I believe the change to gamify the Civil Air Patrol encampment resulted in a better product. I will be repeating the process for the next encampment.

Additional Reading

• Civil Air Patrol a Key Player in Operation Moonwatch
• Layering Game Mechanics on Top of a Civil Air Patrol Summer Encampment, Part 1
• Layering Game Mechanics on Top of a Civil Air Patrol Summer Encampment, Part 2
• How to be a Successful #CivilAirPatrol Commander
• How Technology Helped Us Find a Missing Aircraft