Study Finds AI Chatbots Often Fail to Generate Accurate Differential Diagnoses for Symptoms

A new study published in JAMA Network Open on Monday assessed the reliability of AI chatbots for medical advice. Researchers from Mass General Brigham tested 21 general-purpose large language models, including versions of ChatGPT, DeepSeek, Claude, Gemini, and Grok.

The evaluation used information from 29 published medical cases involving common conditions such as heart failure and ectopic pregnancies.

The chatbots received data incrementally, starting with basic details like age, gender, and symptoms. They failed to produce the correct differential diagnosis— a list of possible causes— more than 80 percent of the time at this stage. Differential diagnosis is a standard process in medicine to identify potential conditions based on initial information.

Performance improved significantly when additional data was provided. After receiving results from physical examinations and laboratory tests, the chatbots identified the correct diagnosis more than 90 percent of the time. This suggests that AI tools can assist effectively once comprehensive information is available.

Limitations in Early-Stage Analysis The study revealed challenges in the initial phases of diagnosis.

Chatbots struggled with open-ended scenarios where limited information was available, which is common when patients first describe symptoms. This phase is critical, as it guides further testing and treatment decisions. Dr.

Marc Succi, executive director of the MESH Incubator at Mass General Brigham and associate professor of radiology at Harvard Medical School, stated in an interview that users should not fully trust chatbot outputs without verification. He emphasized the importance of human involvement, including patient interviews, medical history reviews, diagnostic tests, and physical exams to narrow down possibilities.

Succi noted that acting on incomplete AI advice could lead to unnecessary procedures, such as biopsies for non-cancerous conditions, or delays in urgent care, like treatment for headaches indicating a stroke.

“You can’t just trust what the chatbot says. " — Dr. Marc Succi (The Boston Globe) Arya Rao, lead author of the study, a MESH researcher, and MD-PhD student at Harvard Medical School, said the models excel at final diagnoses with complete data but falter at the outset.”

Rao's comment underscores the gap between AI capabilities and real-world diagnostic workflows, where information is often gathered progressively.

Context of Primary Care Shortages The research occurs amid a shortage of primary care physicians, affecting access to appointments.

Many patients face long waits when seeking care for symptoms, leading some to use AI chatbots for preliminary assessments. Mass General Brigham, the state's largest health care system, reported that thousands of its patients lack assigned primary care providers. In response to these challenges, Mass General Brigham launched an AI app called Care Connect in September of the previous year.

The app operates 24/7, handles patient inquiries, reviews medical records, and schedules telehealth appointments with physicians within as little as half an hour. It aims to streamline intake processes amid the provider shortage. Dr.

Rajesh Patel, vice president of digital patient experience at Mass General Brigham, stated that the Care Connect chatbot differs from general-purpose tools. It focuses on medical intake to expedite clinician appointments and does not provide diagnoses.

Patel added that patients always consult a real clinician for diagnosis, treatment, and follow-up, aligning with the study's emphasis on physician involvement.

Implications for AI in Healthcare The findings suggest that while AI chatbots show promise in later diagnostic stages, they require supervision to avoid errors in early assessments.

Healthcare providers can use these tools to support, but not replace, human judgment in complex cases. Future developments may involve integrating AI more closely with clinical workflows to enhance accuracy. This study provides evidence-based insights into AI limitations, informing how health systems deploy such technologies.

As adoption grows, balancing innovation with safety remains key, particularly for vulnerable patients relying on timely care.