DARPA has awarded Raytheon Technologies segment BBN a contract to develop a “trusted” artificial intelligence (AI) platform for critical decision points.
The In the Moment (ITM) program seeks an algorithm that independently provides solutions during dynamic situations with varying degrees of complexity. It will be modeled on real-world decisions to abide by human values once deployed.
Under the agreement, Raytheon will research how first responders and medical professionals act decisively in difficult scenarios.
BBN will lead a team consisting of Kairos Research, Valkyries Austere Medical Solutions, and MacroCognition in collecting qualitative information from the interviews.
Separate teams from Kitware, Parallax, CACI, and the University of Maryland will facilitate other ITM tasks.
Work for the contract has begun in Alabama, Massachusetts, and Ohio.
Exceeding the ‘Correct Answer’
BBN’s collected data will serve as the basis for designing scenario-based experiments that study how differences in an individual’s decision methods are associated with their choices.
Tests will also focus on how factors between two different actual humans impact the willingness to relay decisions to one another.
“ITM is about more than getting AI to provide the correct answer in very controlled scenarios. We’re not talking about training AI on labeled data to help identify a cancerous tumor on an X-Ray,” BBN Lead Scientist Alice Leung explained.
“To accomplish this, we need to determine how human experts make really difficult decisions and assess whether to trust the decisions of others. We’ll be conducting both decision-making research and trust research.”
Tuning AI Attributes
Resulting information from the experiments will be leveraged by the AI algorithmic agents to adapt decision-making solutions for both large and small expert populations in uncontrolled domains.
“Decisions are difficult because of uncertainty and trade-offs between competing goals. We want to be able to tune an AI’s attributes such as risk tolerance, process focus, or willingness to change plans to better match a user or a group of users,” Leung said.