Layering Game Mechanics on Top of a Civil Air Patrol Summer Encampment, Part 3

In this post, I will let you know the results of the gamification experiment I conducted at a Civil Air Patrol summer encampment. You can read about this in this post, Layering game mechanics on top of a Civil Air Patrol summer encampment, Part 1. 

Months before the encampment, I began preparing the materials and methodology that I was going to use. During this process, I explained my concept to key members of the encampment staff. Each conversation helped to shape the encampment. It was important to me that we conducted an encampment different than previous encampments. I believed it important to change the culture of encampments from a harsh hazing atmosphere to one of teaching and encouragement. The key staff bought into this idea.

Two days prior to the in-flight cadets arriving, I had an opportunity to brief the entire staff and explain my idea. It took time to shape the new roles that they were in from roles they had fixed in their minds. Besides group discussions, I occasionally had to pull cadre aside and provide more specific direction. However, when the in-flight cadets arrived, I believe most cadets knew how to perform as cadre. On the first day of the encampment, I explained to the in-flight cadets how the game would work. I also explained that they did not have to participate if they chose not to, but that it would not help their flight’s success.

Game Element Results

Here are the results for key elements:

Knowledge Book Memorization

I was perhaps most anxious about this element. Would cadets participate or not? I pleased to discover that all 53 in-flight cadets participated. Out of a possible 2,180 achievable points, four cadets achieved maximum points. Two cadets achieved maximum points, thus recited the entire booklet within the first three days. The mean score was 617, with a mode of 300. The minimum score was 50 points.

Flights could earn points on nine unique items. All flights earned points on the same item, quotation #7. The difference between the top flight and the last flight was 360 points. This is essentially learning two quotes. Here is a picture of the results:

Quote results

Room Inspection

Based on previous encampments, I expected rooms would gradually improve over the week (we are talking about teenagers making their beds). I expected to see a lot of reds during the beginning of the week, and progressively more yellows and greens as the week progressed. I was not disappointed. If a cadet received an excellent rating on all items throughout the week, they could earn 840 points. If they earned a satisfactory, they would have earned 420 points. The ultimate results show a minimum of 70 and a maximum of 570, mean of 194 and a mode of 170. If I had another week, I would have seen significant improvement. A picture helps to tell the complete story.

Flights earned points on only four unique inspection items out of 42 items. I was hoping for more tactical planning on the part of flight commanders. It was towards the end of the week when the flight commanders learned how to “game” the system and encouraged their flight to focus more energy on selected parts of the inspection to put points on the board.

Room Inspection Results

Open Ranks Inspections

There was no mystery regarding the open ranks inspections. I expected results would improve throughout the week, and they did. The difference between first place and third was only 132 points. A flight could earn a possible 400 points per day, and there were four inspections for 1,600 points. The lead flight earned 1,344 points.

Uniform Inspections

Three of the uniform inspections were on battle dress uniforms (BDUs) and one was on the blues uniform. The numerous items that need to be sewn on make repairing the BDUs difficult. On the other hand, the blues has more attention to detail. Across the four inspections, cadets could earn a maximum of 40 points. On average, cadets earned 20 points with a mode of 20 and a minimum of 7 and a maximum of 29 points. No cadet had a perfect uniform on any given day.

Only 57 points separated the top flight and last flight.

Cadet Physical Fitness Test

During the cadet physical fitness test, cadets could earn a maximum of 40 points. Out of 53 cadets, 45% (N=24) cadets passed all four parts of the test, and 4 cadets did not participate or failed all portions of the test.

The difference between the top flight and the last place flight was 60 points.

Sports

There were four opportunities to earn points through sports competition. One flight won three of the four engagements, and one flight did not win a single event.

Aerospace Education Quizzes

Across seven quiz bowls, there were a possible 4,510 points available. The difference between first and last was 670 points.

Overall Results

For individual cadet performance, the honor cadet earned 2,560 points out of a possible 3,100 points. The mean was 872, and the mode was 440 points with a minimum of 216 and maximum of 2,560 points. This cadet only earned maximum points on one item.

The top-flight achieved 6,632 points and led the last place flight by 1,539 points. They placed first in six out of nine events.

Lesson Learned

Overall, I am pleased with how the encampment went and the results. Many members of the staff also reported satisfaction with how the encampment ran. Naturally, we learned some lessons.

First, the spreadsheet was not perfect when we began, and we had to make tweaks along the way. Fortunately, we have significantly improved the spreadsheet, and it is ready for the next encampment.

The cadet leadership needs more direction, and they need to be cautioned that it is only a game and not worth anything other than bragging rights in the end. In one situation, a flight commander wanted to send one of his cadets home because he thought she wasn’t doing her fair share of work. After reviewing the metrics, it was determined she had twelve other cadets behind her in terms of points. The command chose to keep her.

I also need to continue to look at the game elements to see if more balancing can be done.

In the end, I believe the change to gamify the Civil Air Patrol encampment resulted in a better product. I will repeat the process for the next encampment.

Additional Reading

• Civil Air Patrol a Key Player in Operation Moonwatch
• Layering Game Mechanics on Top of a Civil Air Patrol Summer Encampment, Part 1
• Layering Game Mechanics on Top of a Civil Air Patrol Summer Encampment, Part 2
• How to be a Successful #CivilAirPatrol Commander
• How Technology Helped Us Find a Missing Aircraft