Cognitive Skill Assessment in a Virtual Environment
(Room 320E)
04 Dec 19
9:00 AM
-
9:30 AM
Tracks:
Full Schedule, Wednesday Schedule
Adaptability is a critical skill necessary for current and future operating forces. In complex arenas where problems are shifting continually, adaptability is essential for achieving mission success . However, efforts to operationalize and assess the construct of adaptability have fallen short. Existing adaptability assessments do not meet the military’s need for an objective, tactically realistic, and diagnostic capability. In addition, many are resource intensive to maintain and administer, and therefore are not sustainable. The Assessment Toolkit for Leader Adaptability Skills (ATLAS) addresses these challenges and extends the research and development in the area of complex skill assessment.
The ATLAS tool measures adaptability in tactical vignettes representing mission sets faced by maneuver small unit leaders. To eliminate the need for a resource-intensive human rater scoring process, five standard and objective measures of adaptability are captured from user actions and scored using automated algorithms. The measures of adaptability align with steps of a framework hypothesizing the flow of cognitive processes that must occur for a favorable adaptive outcome and are the basis for the diagnostic feedback provided to users.
To date, the measures and prototype vignettes have been instantiated in a Unity simulation environment for hypothesis testing. Pilot testing resulted in an expert performance model against which user responses are assessed. In addition, a detailed, diagnostic scoring approach has been developed. Future phases of research will validate the adaptability measures in a large-scale field test and examine the validity of the measures when a rapid content generation function is applied. This function will automatically modify the repository of vignettes for increased sustainability of the tool. The outcomes of this effort contribute to the research community by providing a framework, method, and automated measures for assessing individual developmental needs for adaptability in the context of engaging and realistic simulations.