Psychomotor Skills Assessment via Human Experts, Simulators, and Artificial Intelligence
(Room 320E)
05 Dec 19
10:30 AM
-
11:00 AM
Tracks:
Full Schedule, Thursday Schedule
Surgical education programs present significant challenges for the creation of accurate and executable assessment methods and metrics for psychomotor skills. This assessment has historically been accomplished by (1) observation and scoring by a qualified expert, (2) examination and scoring of a finished product, or (3) simulator data collection and mathematic scoring of both performance and final product. Each of these offer unique advantages and limitations during real execution, as described in detail in the paper. Surgical courses at the Nicholson Center combine all three of these assessment methods, while also investigating new technologies that offer improvements. Given the advancements that are being reported for artificial intelligence and deep learning we have studied the applicability of these techniques to the assessment of surgical skills. This paper describes the evaluation of techniques for processing video of surgical skills and for classifying the scores assigned to a performance. The techniques investigated include convolutional, recursive, and long short-term memory neural networks for the former and k-nearest neighbor, naive bayes, random forests, and support vector machines for the latter. These techniques deliver impressive results in recognizing static objects in individual pictures and are beginning to address recognition of dynamic activities in video. For surgical skills performance, these techniques must (1) recognize stationary objects in the scene, (2) identify the dynamic activities that are demonstrated in the video, and (3) classify the quality of performance of the activity. The techniques are considered for processing both video of the live activity and simulator data streams. We found that automated assessment with deep learning presents several challenges for the humans who must specify the target answers and for the algorithms which must process the video and data. These hurdles are equally present for the assessment of psychomotor surgical, military, or industrial skills.