Ai falls short in the emotional skills that doctors use in their practice. (© BiancoBlue | Dreamstime.com)
In a nutshell
- AI-generated medical responses were more accurate and professionally written than those from human clinicians, according to a study analyzing over 7,000 medical queries across the U.S. and Australia.
- Despite their technical strengths, AI responses lacked emotional nuance and empathy, which human doctors expressed more effectively through varied tone and personalized language.
- Researchers emphasize that AI should support, not replace, healthcare professionals, especially as emotional connection remains a crucial component of effective patient care.
ORONO, Maine — When you’re sick and need medical advice, would you rather talk to a doctor or a computer? A new study from the University of Maine suggests that artificial intelligence might actually give you more accurate answers than human physicians, but it is still lacking in an area that is really important to patients.
Researchers compared AI-generated responses to complex medical questions against answers from human doctors across healthcare systems in the United States and Australia. AI consistently delivered more accurate and professional responses than human clinicians. But while the computers excelled at getting the facts right, they fell short in one critical area that patients desperately need: emotional connection and empathy.
This research, published in the Journal of Health Organization and Management, analyzed over 7,000 medical queries to understand whether AI could match human expertise in handling sensitive, nuanced medical cases. What they found reveals both the incredible promise and concerning limitations of our digital future in healthcare.
Researchers used a dataset called MEDIQA-QA, which contains medical questions and expert responses from both AI systems and human doctors. They evaluated responses on multiple criteria, including accuracy, professionalism, completeness of information, and clarity of language. Each AI response was scored on a scale of 1 to 10 across these different measures.
Scientists examined responses from healthcare systems in both the United States and Australia to understand how AI performance might vary across different medical environments and cultural contexts. They used sophisticated analysis techniques to compare everything from response length and sentiment to the actual medical terminology used by both AI and human respondents.
AI Scores High, But Lacks Heart
Results showed AI responses averaging around 8 out of 10 in overall quality scores, with most falling between 7 and 9. This suggests that AI systems are generally aligning well with expert medical standards when it comes to providing factually accurate information.
However, the differences between AI and human responses revealed something important about what patients actually need from their healthcare providers. While AI responses maintained a consistently neutral, professional tone, human responses showed a much broader range of emotional expression that likely resonated better with patients seeking care.
This emotional range is essential for effective patient care. When someone is worried about mysterious symptoms or dealing with a chronic condition, they need more than just clinical facts. They need reassurance, understanding, and sometimes just the feeling that someone cares about their well-being.
“Healthcare professionals offer healing that is grounded in human connection, through sight, touch, presence and communication — experiences that AI cannot replicate,” says Kelley Strout from the University of Maine, who was not involved with the study.
Communication Style Differences
The vocabulary analysis revealed that AI systems frequently used appropriate clinical terms like “treatment” and “management,” reinforcing their medical relevance. However, human responses more commonly included words like “people,” “children,” and “health,” suggesting a broader, more person-centered approach to medical communication.
AI responses were also notably more consistent in length compared to human responses, which varied much more widely. While this consistency might suggest AI follows a structured approach to answering questions, human doctors often adjust their response length and depth based on the complexity of the question and what they think the patient needs to know.
The study compared prior research on healthcare metrics between the U.S. and Australian systems, providing context for how AI might perform differently across various healthcare environments. Australian patients reported higher satisfaction rates (85% versus 78% in the US) and better access to specialists (85% versus 75%).
Healthcare costs differed dramatically between the two countries, with U.S. visits averaging $200 compared to Australia’s $50. Wait times were also shorter in Australia (15 minutes versus 30 minutes), though both countries showed similar treatment effectiveness scores around 90%.
These differences matter because they suggest that AI systems might need to be tailored to different healthcare environments and patient expectations.
Trust and Bias Concerns
Previous research has shown that how medical experts communicate can have a big impact on whether patients trust and follow their advice. This could be a challenge for AI systems. While they excel at providing consistent, accurate information, they may struggle to adapt their communication style to individual patients’ needs, education levels, or emotional states.
“Technology is only one part of the solution,” says study author C. Matt Graham, Ph.D., from Maine University, in a statement. “We need regulatory standards, human oversight, and inclusive datasets. Right now, most AI tools are trained on limited populations. If we’re not careful, we risk building systems that reflect and even magnify existing inequalities.”
Researchers found that most AI and human response pairs showed low similarity in content, suggesting that the two approaches often differ in how they address the same medical questions. This isn’t necessarily bad; it might mean that AI and human doctors are complementing each other rather than competing.
A Partnership, Not Replacement
Healthcare systems worldwide face increasing pressure from aging populations, staff shortages, and rising costs. AI could potentially be a tool for improving efficiency and reducing physician burnout.
“This isn’t about replacing doctors and nurses,” says Graham. “It’s about augmenting their abilities. AI can be a second set of eyes; it can help clinicians sift through mountains of data, recognize patterns, and offer evidence-based recommendations in real time.”
AI systems could handle routine medical inquiries, freeing up human doctors to focus on complex cases that require emotional intelligence and nuanced decision-making. They could also provide consistent, high-quality information to patients in underserved areas where access to specialists is limited.
However, the emotional and relational aspects of healthcare remain uniquely human. Patients dealing with serious diagnoses, chronic pain, or mental health issues need providers who can offer not just medical expertise but also compassion, understanding, and hope.
“Technology should enhance the humanity of medicine, not diminish it,” says Graham. “That means designing systems that support clinicians in delivering care, not replacing them altogether.”
Healthcare systems will need to carefully integrate AI tools while preserving the human elements that make medicine an art of caring for people during their most vulnerable moments.
Paper Summary
Methodology
The researchers used the MEDIQA-QA dataset containing over 7,000 medical questions and expert responses from both AI systems and human doctors. They analyzed responses from healthcare systems in the United States and Australia, evaluating each response on a scale of 1-10 across multiple criteria including relevance, factual accuracy, completeness, clarity, coherence, professional tone, and ethical content. The study used Python programming with various analytical tools to compare response quality, length, sentiment, and semantic similarity between AI and human responses. The researchers also compared healthcare metrics like patient satisfaction, specialist access, costs, wait times, and treatment effectiveness between the two countries.
Results
AI responses averaged around 8/10 in quality scores, with most falling between 7-9, suggesting strong alignment with expert medical standards. AI responses were more consistent in length compared to highly variable human responses. While AI maintained neutral, professional tones, human responses showed broader emotional range that appeared more empathetic. Vocabulary analysis revealed AI frequently used clinical terms like “treatment” and “management,” while humans more often used person-centered language including “people,” “children,” and “health.” Most AI and human response pairs showed low content similarity, indicating significantly different approaches to addressing the same medical questions.
Limitations
The study only examined healthcare systems in the USA and Australia, potentially limiting global applicability. The focus on specific metrics like accuracy and professionalism may have overlooked other important aspects such as long-term patient outcomes or system-wide impacts. The research relied on current AI technologies, meaning rapid technological advancement could alter these findings. The study was exploratory in nature and did not conduct inferential statistical tests, focusing instead on descriptive comparisons.
Funding and Disclosures
The paper does not explicitly mention funding sources or author disclosures in the provided content.
Publication Information
The paper “Artificial intelligence vs human clinicians: a comparative analysis of complex medical query handling across the USA and Australia” is authored by C. Matt Graham, Ph.D. It was published in the Journal of Health Organization and Management in 2025.
link