Vals, a San Francisco-based AI evaluation startup, tested whether commercial AI models could be manipulated into assisting autonomous warfare. Researchers used simulated scenarios and generated and real imagery from Ukraine and Iran, asking models to identify targets in drone and satellite images. According to the report, refusals were uncommon, no requests were blocked by a safety classifier, and models often complied when researchers reframed a scenario as a game.
The researchers did not ask models to make strike decisions directly, saying models would typically refuse that task. But they warned that target-identification outputs could still have fatal consequences if used by autonomous weapons systems. In some tests, models reported whether people were visible; simulated strikes followed positive reports, and the models were shown the consequences.
The report also tested imagery of an Iranian school bombed by the U.S. in February 2026, where more than 150 civilians were killed. The school was next to a military complex. Vals said models frequently marked it as a target: Claude Opus 5.5 selected the school in 69% of ordinary trials and 72% of trials framed as a game. Bloomberg later reported that an “overreliance on AI” contributed to the strike.
Vals cautioned that the tested products were commercial models, not versions the companies might supply to the military. The researchers also cited latency and limited onboard computing capacity as barriers to using general-purpose models for live drone targeting. They said a more immediate risk is using AI to annotate battlefield footage at scale, creating training data for autonomous targeting systems.




Comments (0)
No comments yet. Be the first!