Authentic assessment asks the learner to perform real-world tasks and demonstrate a meaningful application of skills and competencies. Authentic assessment lies at the heart of training today’s aviation learner to use critical thinking skills. Rather than selecting from predetermined responses, learners must generate responses from skills and concepts they have learned. By using open-ended questions and established performance criteria, authentic assessment focuses on the learning process, enhances the development of real-world skills, encourages higher order thinking skills, and teaches learners to assess their own work and performance.

Learner-Centered Assessment

There are several aspects of effective authentic assessment. The first is the use of open-ended questions in what might be called a “collaborative critique,” which is a form of learner-centered grading. As described in the scenario that introduced this site, the instructor begins by using a four-step series of open-ended questions to guide the learner through a complete self-assessment.

Replay—the instructor asks the learner to verbally replay the flight or procedure. While the learner speaks, the instructor listens for areas where the account does not seem accurate. At the right moment, the instructor discusses any discrepancy with the learner. This approach gives the learner a chance to validate his or her own perceptions, and it gives the instructor critical insight into the learner’s judgment abilities.

Reconstruct—the reconstruction stage encourages learning by identifying the key things that the learner would have, could have, or should have done differently during the flight or procedure.

Reflect—insights come from investing perceptions and experiences with meaning, requiring reflection on the events. For example:

  1. What was the most important thing you learned today?
  2. What part of the session was easiest for you? What part was hardest?
  3. Did anything make you uncomfortable? If so, when did it occur?
  4. How would you assess your performance and your decisions?
  5. How did your performance compare to the standards in the ACS?

Redirect—the final step is to help the learner relate lessons learned in this session to other experiences and consider how they might help in future sessions. Questions might include:

  • How does this experience relate to previous lessons?
  • What might be done to mitigate a similar risk in a future situation?
  • Which aspects of this experience might apply to future situations, and how?
  • What personal minimums should be established, and what additional proficiency flying and/or training might be useful?

Any self-assessment stimulates growth in the learner’s thought processes and, in turn, behaviors. An in-depth discussion between the instructor and the learner may follow, which compares the instructor’s assessment to the learner’s self-assessment. Through this discussion, the instructor and the learner jointly determine the learner’s progress. The progress may be recorded on a rubric as part of a training program. As explained earlier, a rubric is a guide for scoring performance assessments in a reliable, fair, and valid manner. It is generally composed of dimensions for judging learner performance, a scale for rating performances on each dimension, and standards of excellence for specified performance levels.

The collaborative assessment process in learner-centered grading uses two broad rubrics: one that assesses the learner’s level of proficiency on skill-focused maneuvers or procedures, and one that assesses the learner’s level of proficiency on single-pilot resource management (SRM), which is the cognitive or decision-making aspect of flight training.

The performance assessment dimensions for each type of rubric are as follows:

Maneuver or Procedure “Grades”

  • Describe—at the completion of the scenario, the learner is able to describe the physical characteristics and cognitive elements of the scenario activities but needs assistance to execute the maneuver or procedure successfully.
  • Explain—at the completion of the scenario, the learner is able to describe the scenario activity and understand the underlying concepts, principles, and procedures that comprise the activity, but needs assistance to execute the maneuver or procedure successfully.
  • Practice—at the completion of the scenario, the learner is able to plan and execute the scenario. Coaching, instruction, and/or assistance will correct deviations and errors identified by the instructor.
  • Perform—at the completion of the scenario, the learner is able to perform the activity without instructor assistance. The learner will identify and correct errors and deviations in an expeditious manner. At no time will the successful completion of the activity be in doubt. (“Perform” is used to signify that the learner is satisfactorily demonstrating proficiency in traditional piloting and systems operation skills).
  • Not observed—any event not accomplished or required.

For example, a learner can describe a landing and can tell the flight instructor about the physical characteristics and appearance of the landing. On a good day, with the wind straight down the runway, the learner may be able to practice landings with some success while still functioning at the rote level of learning. However, on a gusty crosswind day the learner needs a deeper level of understanding to adapt to the different conditions. If a learner can explain all the basic physics associated with lift/drag and crosswind correction, he or she is more likely to practice successfully and eventually perform a landing under a wide variety of conditions.

Assessing Risk Management Skills

  • Explain—the learner can verbally identify, describe, and understand the risks inherent in the flight scenario, but needs to be prompted to identify risks and make decisions.
  • Practice—the learner is able to identify, understand, and apply SRM principles to the actual flight situation. Coaching, instruction, and/or assistance quickly corrects minor deviations and errors identified by the instructor. The learner is an active decision maker.
  • Manage-Decide—the learner can correctly gather the most important data available both inside and outside the flight deck, identify possible courses of action, evaluate the risk inherent in each course of action, and make the appropriate decision. Instructor intervention is not required for the safe completion of the flight.

In SRM, the learner may be able to describe basic SRM principles during the first flight. Later, he or she is able to explain how SRM applies to different scenarios that are presented on the ground and in the air. When the learner actually begins to make quality decisions based on good SRM techniques, he or she earns a grade of manage-decide. The advantage of this type of grading is that both flight instructor and learner know exactly where the learning has progressed.

Let’s look at how the rubric in Figure might be used in a flight training scenario. During the postflight debriefing, flight instructor Linda asks her learner, Brian, to assess his performance for the day using the Replay, Reconstruct, Reflect, and Redirect guided discussion questions described in the Learner-Centered Assessment section presented earlier. Based on this assessment, she and Brian discuss where Brian’s performance falls in the rubrics for maneuvers/procedures and SRM. This part of the assessment may be verbally discussed or, alternatively, Brian and Linda separately create an assessment sheet for each element of the flight.

Authentic Assessment, Aviation instructor
Rubric for assessing flight training maneuvers

When Brian studies the performance levels, he decides he was at the “Perform” level since he had not made any mistakes. Where he had rated the item as “Perform,” Linda had rated it as “Practice.” During the ensuing discussion, Brian understands where he needs more practice before his performance is at the “Perform” level.

This approach to assessment has several key advantages. One is that it actively involves the learner in the assessment process and establishes the habit of healthy reflection and self-assessment that is critical to being a safe pilot. Another is that these grades are not self-esteem related, since they do not describe a recognized level of prestige (such as A+ or “Outstanding”), but rather a level of performance. The learner cannot flunk a lesson. Instead, he or she demonstrates a particular level of flight and SRM skills.

Both instructors and learners may initially be reluctant to use this method of assessment. Instructors may think it requires more time, when in fact it is merely a more structured, effective, and collaborative version of a traditional postflight critique. Also, instructors who learned in the more traditional assessment structure must be careful not to equate or force the dimensions of the rubric into the traditional grading mold of A through F. One way to avoid this temptation is to remember that evaluation should be progressive: the learner may achieve a new level of learning during each lesson. For example, in flight one, a task might be a “describe” item. By flight three, it is a “practice” item, and by flight five, it is a “manage-decide” item.The learner may be reluctant to self-assess if he or she has not had the chance to participate in such a process before. Therefore, the instructor may need to teach the learner how to become an active participant in the collaborative assessment.