Large language models pass medical licensing exams, and the natural inference is that describing your symptoms to one should get you a doctor-quality answer. A randomised trial has now tested that inference directly, and it failed in an instructive way. The models knew the answers. The people using them did not get them. This article explains what happened, why it matters more than the exam scores, and how it compares with the symptom checkers that came before.
The exam scores are real
Tested on their own, current models are accurate on medical vignettes. In the trial described below, the researchers first fed ten scenarios to the models directly, and the models were correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average, where disposition means the right course of action, from self-care to calling an ambulance.
That is the number that gets quoted. It measures the model reading a complete, well-written case. It does not measure what happens when a worried person types.
The trial: models with humans attached
Researchers at Oxford ran a preregistered, controlled study with 1,298 participants in the UK. Each person was given one of ten medical scenarios written by doctors and asked to work out what might be wrong and what to do. Participants were randomly assigned to use one of three chatbots (GPT-4o, Llama 3 or Command R+) or whatever they normally would, which for most meant a search engine or their own judgement.
With a chatbot in hand, participants using the same LLMs identified relevant conditions in less than 34.5% of cases, and chose the right disposition in less than 44.2%. Neither was better than the control group. The same models that scored 94.9% when reading the scenario themselves delivered roughly a third of that accuracy once a human was doing the asking.
Why the gap exists
The authors attribute the failure to the interaction rather than the model. People left out details the model needed, described symptoms in ways that steered it wrongly, and sometimes received a correct answer buried among others and did not act on it. The model’s knowledge was intact; the conversation did not transmit it.
That matches what has been found in the human-factors literature on decision aids, where the quality of the aid and the quality of the human-plus-aid system are different things. A tool that is accurate when given perfect input can be useless in practice if the input is never perfect.
How this compares with symptom checkers
Chatbots are the successors of rule-based online symptom checkers, and those have been evaluated for years. A 2022 systematic review found that of the 177 studies retrieved, 10 studies met the inclusion criteria, covering a range of conditions. The diagnostic accuracy of the primary diagnosis was low (range: 19% to 36%) across the checkers tested, with triage accuracy typically higher.
Put side by side: older symptom checkers put the right diagnosis first 19% to 36% of the time; humans using modern chatbots identified relevant conditions under 34.5% of the time. The models are far more knowledgeable than the old tools. The outcome for the person at the keyboard has not moved as much as the technology has.
What the trial does not show
The scenarios were hypothetical, so participants had no real symptoms and no stake in the answer. That could cut either way: real patients may describe their own bodies better, or worse, than a vignette. The trial tested three specific models in 2024 and 2025 conditions; models change. It measured a single session, not repeated use by someone who learns to prompt well. And it did not test a chatbot built and validated for medical triage, only general assistants.
What it does show is that benchmark accuracy is not a safe proxy for real-world usefulness, and that the authors’ recommendation, systematic human user testing before deployment, is not yet standard.
What this means in practice
Use a chatbot to understand terms, prepare questions for an appointment, or learn what a condition generally involves. Do not use it to decide whether something is serious. In the one controlled test, people with a chatbot were no better at that decision than people without one, and the disposition accuracy, under 44.2%, is the number that matters most when the question is “do I need to go in”.
This is not individual medical advice. If you have symptoms that worry you, the evidence supports talking to a clinician, not a chatbot, to decide what to do.
Common questions
Can ChatGPT diagnose symptoms accurately?
Alone, models identified the correct condition in 94.9% of trial scenarios. When 1,298 real participants used the same models to assess scenarios, they identified relevant conditions in under 34.5% of cases, no better than people using a search engine.
Is an AI chatbot better than a symptom checker?
The models are much more capable, but the outcome for users has not improved as much. Older symptom checkers ranked the correct diagnosis first 19% to 36% of the time; chatbot users in the trial identified relevant conditions under 34.5% of the time.
Why do AI chatbots give worse medical advice to real people?
The trial’s authors point to the interaction: people omit details, phrase things in ways that mislead the model, and do not act on correct answers when given. The model’s knowledge is not the bottleneck.
Should I use AI to decide whether to see a doctor?
The trial found chatbot users chose the right course of action in under 44.2% of scenarios, no better than the control group. For that decision, the evidence favours a clinician or an established triage service.
The bottom line
The chatbot knows more medicine than you do, and that is not the same as you getting a better answer. In the only randomised test so far, people with an AI assistant were no better at identifying what was wrong or what to do than people without one. Use it to learn; do not use it to triage.
Did this answer your question?
Thanks. That helps us decide what to fix.
Leave a comment