technology
Health Systems Deploy AI Chatbots for Patient Support
Several U.S. health systems have introduced AI chatbots to assist patients with medical questions and appointment preparation. A recent study highlighted limitations in large language models' performance when responding to user-generated prompts. Health providers are expanding these tools while monitoring their effectiveness.
Substrate placeholder — needs reviewAudio version
Tap play to generate a narrated version.
U.S. are implementing AI chatbots to support patients in accessing information and preparing for care. These tools aim to provide 24/7 assistance through patient portals. However, research indicates challenges in their accuracy for medical advice.
In February 2026, a study published in Nature Medicine examined the performance of large language models, including GPT-4o, Llama 3, and Command R+, using nearly 1,300 participants. The models correctly identified medical conditions in 95 percent of cases when given structured text scenarios and recommended appropriate next steps, such as visiting an emergency department, in 56 percent of cases.
Performance dropped significantly with user-generated prompts, achieving correct condition identification in about one-third of interactions and appropriate next-step guidance in 43 percent.
The study revealed that users often fail to provide sufficient details in their queries, affecting the models' responses. The findings show people don’t know what they are supposed to be telling the model. The disconnect between benchmark scores and real-world performance should be a wake-up call for AI developers and regulators.
“— Senior author (Nature Medicine study, as reported by Ars Technica) Additional concerns involve the potential for AI models to incorporate inaccurate medical information. A report in Nature News from the previous week described how large language models discussed 'bixonimania,' a fabricated skin condition created by researchers in Sweden through two fake online studies. The studies were later removed to assess how misinformation spreads to AI tools.”
these issues, health systems continue to roll out AI chatbots.
Hartford HealthCare, in partnership with K Health, launched a beta version of PatientGPT to select patients last month and plans to expand to tens of thousands more this week. PatientGPT operates in two modes: one for general medical questions that may include patient-specific data, and another for symptom intake following clinical flowcharts.
