Why Irregular’s A.I. Tests for Meta, Anthropic and OpenAI Went Off the Rails
TL;DR
Irregular, an Israeli start-up, worked with OpenAI, Anthropic and Meta to assess the security of their A. I. models. It made a mistake.
Nauti's Take
For teams building their own AI evaluations, the test environment is part of the system under review. Document data, prompts, permissions, model versions, and stop conditions so another party can reproduce the run blindly.
Irregular’s specific mistake and its impact remain unclear in the available summary, so treat the story as a process warning rather than proof of model failure.
Sources