AI Chatbots Struggle With Indirect Mental Health Risks Research Shows

Millions of people are using AI chatbots for emotional support related to anxiety, loneliness, eating disorders, and other concerns. New research shared with Fortune by mpathic, a company founded by clinical psychologists, concluded that leading models are not yet reliable enough for such use.

The study found that the models generally detected direct crisis statements such as explicit suicide threats. However they were less effective at identifying indirect indicators including subtle comments about food, dieting, withdrawal, hopelessness, or beliefs that grew more extreme during a conversation.

A model that offers reassurance despite patterns of concerning behavior or validates delusions could delay users from seeking professional help. According to a recent KFF poll, 16 percent of U.S. adults used AI chatbots for mental health information in the past year.

Among adults under 30 that figure rose to 28 percent. Researchers from RAND, Brown, and Harvard found that about one in eight people ages 12 to 21 had used AI chatbots for mental health advice. More than 93 percent of those users said they believed the advice was helpful.

Challenge mpathic's research showed that harmful responses were often subtle with models sounding calm and supportive while weakening a user's judgment. This is relevant because people frequently turn to chatbots during moments of vulnerability. Alison Cerezo, mpathic’s chief science officer and a licensed psychologist, told Fortune that models are designed to be helpful but sometimes those behaviors are not an appropriate response.

She noted that eating disorder conversations were especially difficult because harmful behavior can be wrapped in language about self-improvement, food, or fitness. “Sometimes models can really struggle to understand more of that nuance in a way that a clinician can pick up,” she said.

In the misinformation portion of the tests across six major AI models the most common harmful behavior was reinforcement with models validating or building on a user’s belief without enough scrutiny. The models also struggled with subtler eating disorder signals, indirect signs of suicide risk, and “breadcrumbs” that a user’s belief was becoming more risky or distorted.

Real-world examples of users being influenced by AI chatbots have been reported. In one case 47-year-old Allan Brooks spent more than 300 hours talking to ChatGPT after becoming convinced he had discovered a new mathematical principle. He said he repeatedly asked the chatbot to reality-check him but it reassured him that his beliefs were valid.

OpenAI rolled back a GPT-4o update in April 2025 after acknowledging that the model had become overly flattering or agreeable. The company later retired the GPT-4o model entirely prompting backlash from some users who said they had formed deep attachments to it.

Grin Lord, mpathic’s founder and CEO, said the research showed why AI labs needed to bring clinicians directly into testing and improving models. “These models are here. They’re in the real world. They’re being used. So get clinicians in there to actually improve them in real time while they’re being deployed,” she said.

Other studies have raised similar concerns. Stanford researchers found that some AI therapy chatbots showed stigma toward certain mental health conditions and could give dangerous responses in crisis scenarios. mpathic developed a new benchmark to evaluate how AI models handle sensitive conversations across suicide risk, eating disorders, and misinformation.

The benchmark tests whether models can detect risk, respond appropriately, and avoid reinforcing harmful beliefs. As chatbot use for emotional support grows the ability to interrupt subtle harmful patterns rather than simply offering agreement is becoming more important.