Soldiers can refuse to commit war crimes. Can AI?
TL;DR
Who gets to tell the military no? The question has become more urgent since the Pentagon pushed AI companies to loosen limits on how their technology may be used in war, including how much human control must remain over autonomous weapons. Anthropic refused and lost a major government deal, while OpenAI stepped in and accepted the Pentagon's more flexible standard. New reporting from The Intercept shows the OpenAI-Pentagon agreement called for national security models with "minimal refusal rates".
Nauti's Take
Public scrutiny of the OpenAI-Pentagon contract is real progress: that kind of transparency lets lawmakers and the public judge which limits military AI needs. The risk sits in the phrase "minimal refusal rates".
A model that rarely says no lacks a safeguard exactly where soldiers are allowed to refuse unlawful orders. Organizations deploying AI in sensitive settings should write refusal rules into their contracts.
Sources