Schools are using AI counselors to track students’ mental health. Is it safe?
TL;DR
As hundreds of schools implement an automated monitoring tool, educators say that students can find talking to chatbots ‘more natural’ than confiding in a human.
Key Points
- Brittani Phillips checked her phone.
- A middle school counselor in Putnam county, Florida, Phillips receives messages from an artificial intelligence-enabled therapy platform that students use during nonschool hours.
- It flags when a student may be at risk for harming themself or others based on what the student types into a chat.
Nauti's Take
School mental-health bots aren’t safety features unless every alert is gatekept; ship them without governance and human validation and you’ve simply baked surveillance logs into the counseling pipeline, not real care.
Sources