Study Finds AI Chatbots Often Fail to Generate Accurate Differential Diagnoses for Symptoms

Succi noted that acting on incomplete AI advice could lead to unnecessary procedures, such as biopsies for non-cancerous conditions, or delays in urgent care, like treatment for headaches indicating a stroke.

“You can’t just trust what the chatbot says. " — Dr. Marc Succi (The Boston Globe) Arya Rao, lead author of the study, a MESH researcher, and MD-PhD student at Harvard Medical School, said the models excel at final diagnoses with complete data but falter at the outset.”

Rao's comment underscores the gap between AI capabilities and real-world diagnostic workflows, where information is often gathered progressively.

The research occurs amid a shortage of primary care physicians, affecting access to appointments.

Many patients face long waits when seeking care for symptoms, leading some to use AI chatbots for preliminary assessments. Mass General Brigham, the state's largest health care system, reported that thousands of its patients lack assigned primary care providers. In response to these challenges, Mass General Brigham launched an AI app called Care Connect in September of the previous year.

The app operates 24/7, handles patient inquiries, reviews medical records, and schedules telehealth appointments with physicians within as little as half an hour. It aims to streamline intake processes amid the provider shortage. Dr.

Rajesh Patel, vice president of digital patient experience at Mass General Brigham, stated that the Care Connect chatbot differs from general-purpose tools. It focuses on medical intake to expedite clinician appointments and does not provide diagnoses.

Patel added that patients always consult a real clinician for diagnosis, treatment, and follow-up, aligning with the study's emphasis on physician involvement.

The findings suggest that while AI chatbots show promise in later diagnostic stages, they require supervision to avoid errors in early assessments.

Healthcare providers can use these tools to support, but not replace, human judgment in complex cases. Future developments may involve integrating AI more closely with clinical workflows to enhance accuracy. This study provides evidence-based insights into AI limitations, informing how health systems deploy such technologies.

As adoption grows, balancing innovation with safety remains key, particularly for vulnerable patients relying on timely care.