Why Aren’t We Measuring How AI Affects Humans?
TL;DR
As AI systems become more capable, a lot of resources and effort are being put toward measuring their abilities. Researchers look at technical evaluation metrics, subject AIs to reasoning tests, track their throughput, and much more. But there’s one key metric that often gets overlooked, and it’s arguably the most important of all: What is AI doing to humans? Imran Khan leads psychosocial evaluation of AI at the nonprofit Center for Humane Technology.
Nauti's Take
A real point: Khan highlights a genuine gap — we measure AI capabilities meticulously but barely track the psychosocial effects on people. That opens a chance for healthier product design.
The limit: psychosocial impact is hard to measure cleanly and invites alarmism. Product teams and researchers benefit from building such metrics early instead of staring only at benchmarks.
Sources