AI Models Overthink Problems—and It’s a Security Risk
TL;DR
Researchers from Zhejiang University and Alibaba presented an ICML 2026 study showing that logically inconsistent prompts can push reasoning models into excessively long chains of thought. The attack uses an evolutionary algorithm to corrupt math problems by swapping, deleting, or importing premises, creating tasks that look solvable but are logically broken.
Nauti's Take
This is a useful reality check: more thinking is not automatically better AI. Reasoning models need firm boundaries, or their main strength becomes an expensive weakness.
The study does not read like a ready-made mass attack, because pricing, rate limits, context windows, and defenses matter. But it exposes a real product gap: models need to get better at saying that the question itself is broken.
Briefingshow
Reasoning is the core selling point of modern AI systems, but this work turns it into an attack surface. If a model cannot reliably detect that a task is impossible or contradictory, it burns tokens, latency, and server capacity. Providers need to treat stopping behavior as a security feature, not just a cost-control setting.
Sources