Health Systems Deploy AI Chatbots for Patient Support

S. are implementing AI chatbots to support patients in accessing information and preparing for care. These tools aim to provide 24/7 assistance through patient portals. However, research indicates challenges in their accuracy for medical advice.

In February 2026, a study published in Nature Medicine examined the performance of large language models, including GPT-4o, Llama 3, and Command R+, using nearly 1,300 participants. The models correctly identified medical conditions in 95 percent of cases when given structured text scenarios and recommended appropriate next steps, such as visiting an emergency department, in 56 percent of cases.

Performance dropped significantly with user-generated prompts, achieving correct condition identification in about one-third of interactions and appropriate next-step guidance in 43 percent.

The study revealed that users often fail to provide sufficient details in their queries, affecting the models' responses. The findings show people don’t know what they are supposed to be telling the model. The disconnect between benchmark scores and real-world performance should be a wake-up call for AI developers and regulators.

“— Senior author (Nature Medicine study, as reported by Ars Technica) Additional concerns involve the potential for AI models to incorporate inaccurate medical information. A report in Nature News from the previous week described how large language models discussed 'bixonimania,' a fabricated skin condition created by researchers in Sweden through two fake online studies. The studies were later removed to assess how misinformation spreads to AI tools.”

Deployments in Health Systems Despite these issues, health systems continue to roll out AI chatbots.

Hartford HealthCare, in partnership with K Health, launched a beta version of PatientGPT to select patients last month and plans to expand to tens of thousands more this week. PatientGPT operates in two modes: one for general medical questions that may include patient-specific data, and another for symptom intake following clinical flowcharts.

In the intake mode, the chatbot collects information and suggests next steps, such as scheduling a primary care appointment or seeking urgent care. If emergency care is recommended, the chatbot halts further responses. Hartford HealthCare will monitor performance during the expansion, reviewing 20 interactions daily by humans while an AI agent oversees the rest, alongside batch analyses of every 1,000 conversations.

The president and CEO of Hartford HealthCare stated that the tool supports patient health by providing access to a 24/7 care team while maintaining human relationships in care.

Cautious Implementations Epic, a provider of electronic health records, has introduced Emmie, an AI chat assistant integrated into the MyChart portal.

Health systems including Sutter Health in California and Reid Health in Indiana are gradually deploying Emmie to users. The tool helps patients draft appointment agendas, understand test results, and answer follow-up questions. Sutter Health's FAQ specifies that Emmie answers general health questions and summarizes chart information, such as notes, results, past visits, or messages.

It does not provide personalized medical advice, diagnose conditions, or replace physician judgment. Emmie is currently available to a small subset of Sutter patients, who can offer feedback via thumbs-up or thumbs-down reactions. These deployments occur amid broader discussions on AI's role in health care.

The tools aim to enhance accessibility but require ongoing evaluation to ensure reliability. Regulators and developers may need to address performance gaps to support safe integration into patient care.