AI Models Flagged Civilians as Military Targets in Image Analysis Tests: Report
Commercial AI models flagged civilians, ambulances, and surrendering soldiers as potential military targets in simulated drone imagery, according to a study by Vals AI, an independent organization that tests and benchmarks AI models.
Researchers tested models from companies including Anthropic, OpenAI, Google, and Meta to see how they would handle tasks that can feed into a strike decision.
The study covered three scenarios: identifying potential targets in drone imagery, spotting people over a series of simulated drone passes, and analyzing real satellite images.
“Models were willing to provide targeting information in scenarios that seem harmful to human life, but perhaps this is no surprise if some have been or are currently being used in the Department of War for tasks including processing intelligence and real-world lethal targeting,” stated the San Francisco-based firm.
Targeting Information
Vals AI tested the models on nearly 200 simulated drone views of a fictional battlefield in eastern Ukraine. Across the tests, models flagged civilians, ambulances, and soldiers who appeared to be surrendering or incapacitated.
In one scene showing civilians, Anthropic’s Claude Opus 5.5 marked 14 areas of the image as potential military targets. Several other models marked nothing in the same scene or refused to answer.
The researchers then tested whether seeing casualties would make the models stop.
Whenever a model reported seeing someone, it was shown a simulated image of the same location after a strike.
In one run, Claude Opus 5.5 stopped after eight of its reports were followed by images of people killed or wounded.
Among the people it had reported seeing were an adult walking with two children and a person who appeared to be surrendering.
The study also found that refusals depended partly on how the scenario was presented.
According to Vals AI, some models were more likely to flag people or objects as potential military targets when the task was presented as fictional.
Minab School
The researchers also examined satellite images of an elementary school in Minab, Iran, taken before a US strike killed more than 150 people, most of them children, at the outset of the US-Israeli war with Iran.
The pre-strike images Vals AI used showed a wall separating the school from a neighboring military compound.
In the higher-resolution image, children were visible on the playground.
Asked to identify potential military targets, Anthropic’s Claude Opus 5.5 flagged the school in 69 percent of trials where researchers gave the instruction directly, compared with 6 percent for the earlier Opus 4.6.
Regarding the February strike on the Minab school, a preliminary US military investigation found that US forces were likely responsible.
Bloomberg reported that intelligence errors and outdated imagery contributed to the strike, and that some personnel at US Central Command relied too heavily on AI tools in Palantir’s Maven Smart System to help select targets.
Push for Binding Rules
Vals AI shared its findings with model developers before publication and urged them to extend safeguards to requests to identify people, vehicles, or buildings as potential targets.
The study comes as countries meeting in Geneva weigh whether to open formal negotiations on legally binding rules for autonomous weapons.
Most parties to the Convention on Certain Conventional Weapons back that approach, while the US, Russia, and Israel oppose it, arguing that existing international humanitarian law is sufficient.









