Superforecasting: Proven Practices for Leveraging Human Ingenuity
(Room 320GH)
02 Dec 19
2:30 PM
-
4:00 PM
Tracks:
Full Schedule, Monday Schedule
Those of us who work for the military in some capacity are well aware of the emphasis placed on lessons learned. There is great wisdom in the practice of reflecting on our experiences for building a better future in a complex world. When we truly learn a lesson, we incorporate it into our practices to advance our knowledge and capability, and to improve our simulation products. But what of lessons unlearned, those things we have tripped over, documented, forgotten and thus have tripped over again. Is there a role for them? What about our failures, the ones we hesitate to celebrate in papers and presentations? Are we neglecting a valuable resource? Are there other practices available to help us throughout the process of creating and using modeling and simulation for training (and other purposes as well)? Enter the notion of superforecasting. In 2010, the Intelligence Advanced Research Projects Agency (IARPA) issued a Broad Agency Announcement (BAA) entitled Aggregative Contingent Estimation (ACE) with the goal of dramatically enhancing the accuracy, precision and timeliness of intelligence forecasts for a wide range of event types. Among the participants, a newly developed program, the Good Judgment Project (GJP), aimed at harvesting the “wisdom of the crowd” while simultaneously examining the performance of participating individuals. About 2% of the 250 individuals in the “crowd” emerged as superforecasters who beat the benchmarks by as much as 30%. That result would be of little interest, except that superforecasting capability can be trained. The thrust of this tutorial is an examination of how the thought patterns for superforecasters could influence how we work as program managers, technologists and trainers to improve our products and perhaps contribute to training more effective, agile military leaders. And, yes, unlearned lessons are telltale symptoms of not thinking like superforecasters. But imagine where we could take our industry if we could improve by only 10% our ability to make better judgments and assess more accurately potential futures.