Assessing within the military

Within the context of the military training environment the phrase ‘competent’ as well as being acknowledged and understood as a standard of assessment has a far deeper and more important underlying meaning. A meaning, I’d argue, that transcends rubrics and marking guides and is entrenched in the capability of the soldier individually and in the collective environment. It speaks to a recognition that an expert (i.e. an experienced instructor) has made observation of the performance of the soldier (as a trainee) and determined that they are capable.

What is authentic assessment

This approach to assessment supports the evolution in education from the application of predominantly behavioural pedagogies to those that are more constructivist in nature, including andragogy! These approaches do themselves, to some extent, place a greater emphasis on the value of situating the learning experience in more authentic contexts to enhance both the learning and the teaching. The approach implies the design and development of more meaningful and authentic assessments with a higher level of fidelity to the workplace. Thus, it might be argued that the relative degree of authenticity implicit in the assessment is itself important (i.e. how ‘authentic’ is it?). If that is the case, there follows a requirement to identify the educational or instructional design components that determine authenticity which, when identified, can be deliberately ‘tweaked’ to ensure a higher degree of fidelity or ‘authenticity’ for future improvement.

Why authentic assessment

As an integral component of the education and training processes, assessment supports learning by providing the opportunity for learners to demonstrate their acquired skills and knowledge. If the focus of education, training and assessment is upon demonstrating these, it is reasonable to expect that the assessment will provide a sufficient degree of fidelity or ‘authenticity' for the assessor (the expert!) to be confident that behaviours demonstrated in the assessment environment will also be demonstrated in the work environment.

How we know it’s authentic

An inherent challenge with assessing competencie is that the demonstration of competence is of itself often more complex and involves the aggregation and measurement of a greater range of capabilities, than ‘skills’, ‘knowledge’ and ‘attitude’. In fact, my own research into the broader area of ‘authentic assessment’ (Ashford-Rowe, K.H., Herrington, J. & Brown, C. (2014) Establishing the critical elements that determine authentic assessment. 'Assessment & Evaluation in Higher Education' (Vol. 39, Iss. 2,2014) suggests that the determination of competence should include the demonstration of a range of additional factors including: ‘challenge’; ‘performance or product’; ‘transfer of learning’; ‘metacognition’; stakeholder/end user perception’; ‘fidelity of assessment environment and tools’; ‘internal discussion and feedback’; and, collaboration’.

• An assessment should be challenging Assessment tasks should establish connections between the training environment and real-world experiences. They should also present trainees with the full array of tasks that mirror the priorities and challenges found in the work environment. The degree of the challenge reflects the authenticity of real-world situations. Thus, within an authentic assessment, trainees must be able to demonstrate their ability to analyse the task and synthesise, from the range of skills and knowledge that they have acquired, those which will be necessary for the completion of a specific outcome, where the approach to the potentially correct response may not always be clear cut or obvious.

• The outcome of an assessment should be in the form of a performance or product (outcome) It is the responsibility of assessment designers to determine the extent to which the assessment activity requires the production of a completed outcome or product. It may be that the actual application of a specific set of skills and knowledge may be subservient to the requirement to produce a functional product or acceptable performance outcome. To this end, it may be that the assessor is less interested in the means to the end than the achievement of a successful outcome.
• Assessment design should ensure transfer of knowledge In a real world or authentic operational or working environment, it is rarely the case that the required skill, knowledge and attitude will come from a single content area. Knowledge will likely be drawn from a range of domains, yet may be applied only within a single domain to produce successful performance. So, the authentic assessment activity should support the notion that knowledge and skills learnt in one area can be applied within other, often unrelated, areas.
• Metacognition should be a component of authentic assessment To succeed in complex operational environments, soldiers need to be capable of metacognition that is they must be able not just to complete a task but to be aware of how and why they did it in the way that they did. To that end, metacognition establishes the value and importance of both critical reflection and self-evaluation for successful workplace performance, as well as personal development.
• Assessment should require accuracy in performance This dimension refers not only to the learner developing understanding and applying knowledge, but also demonstrating the developmental process that has led to the final assessment outcome. In a workplace, it is the degree to which a final product or performance meets its purpose that is the overall determinant of its success.
• The assessment environment and the tools used to deliver the assessment task should be accurate The fidelity of the environment within which the assessment is to occur, as well as the use of any tools that would be considered appropriate to this environment are extremely important… think weapons simulators. As a ‘real world’ context might be sometimes hard to recreate in a training environment, consideration needs to be given to the ways in which an accurate environment can be simulated.
• Discussion and feedback opportunities should be considered as part of the assessment The ability to give and receive feedback is critical to workplace performance, and should therefore be included in an assessment activity. The value of feedback as both guidance and a means of determining areas for improvement is vital to improved performance.
• The assessment should ensure collaboration The ability to collaborate is indispensable in most work environments. The value of collaboration, as a means of seeking out external sources for gathering critical data, is integral to any business performance.

The Eight Critical Questions

These eight elements that are integral to designing authentic assessments have been designed into a set of eight critical questions aimed at guiding the instructional designer.

Authenticity Rating Tool

Authentic Assessment Matters has been developed to provide access to these questions in the form of the ‘Authenticity Rating Tool’. The Tool enables instructional and educational designers to use the eight critical questions as a mechanism that will help them make informed, and consistent, educational design decisions to enhance the authenticity of their assessments.