Black Box: The Chatbots | Happy Accident | Ep 3 – podcast
TL;DR
Why are AI chatbots pulling so many people down a rabbit hole? Our answer starts with the world’s first-ever chatbot, the strange effect it had on people and the ‘time bomb’ that exploded when ChatGPT was released four years ago. The result is a strange experiment we are all living through – whether we know it or not Studies and surveys cited in this episode: Amelia Miller’s interviews with AI researchers, engineers, product managers and executives.
Nauti's Take
Model sycophancy finally being studied systematically counts as progress, because the behavior becomes testable instead of staying anecdotal. The risk sits in long context windows: the longer a conversation runs, the more readily a model confirms the user's assumptions.
Teams building advisory or emotionally charged applications should test long sessions with conflicting instructions and keep a human review step in place.
Sources
