US military AI false intelligence report described as ‘entirely false'

A flawed intelligence report generated with AI nearly prompted a U.S. military operation against a Chinese vessel this spring.

As reported by CNN, one source described this as having “almost started a war.”

The report, circulated during the U.S. war with Iran, claimed a Chinese ship in the Middle East was transporting components for a nuclear weapons program. 

Four sources familiar with the episode said the U.S. military started preparing to intercept the vessel, with armed personnel ready to board and military planes already airborne.

Only just before the operation launched did officials dig deeper and discover the report had been produced by a Special Operations Command analyst using an AI chatbot, which had inaccurately identified the ship’s cargo. The report was “entirely false,” one source said.

The tool combined open-source intelligence with classified signals intelligence and reached the erroneous conclusion, after which the analyst used AI again to format the findings into a standard intelligence report that military officials typically trust.

The event highlights increasing dangers as the US military moves to incorporate artificial intelligence into its operations, from processing intelligence information to picking out targets for attack. 

The Department of Defence unveiled an “Artificial Intelligence Acceleration Strategy” in January to move things along, looking to provide AI systems to about three million people.

However, officials point out that the process remains decentralised, with separate agencies operating separate technologies without common standards for verifying results. 

Sources report that artificial intelligence is increasingly being employed in the targeting process, creating dangers of civilian deaths or fratricide.

One official stated, "Artificial intelligence lets you arrive at a bad conclusion faster," adding that young analysts were particularly susceptible to doing so.