Artificial Intelligence in medicine has become remarkably good at answering one question: can an algorithm perform a clinical task accurately?
AI systems can now detect abnormalities in medical images, predict disease risk, interpret electrocardiograms, support clinical decision-making, and assist healthcare professionals through increasingly sophisticated large language models. In the United States alone, the Food and Drug Administration reports that more than 1,600 AI-enabled medical devices had been authorised for marketing by September 2026 [1].
Yet, as these technologies move from research laboratories into hospitals and clinics, a more pressing question is coming into focus: does using AI improve patient health? For many years, much of medical AI research has focused on measures such as accuracy, sensitivity, specificity and area under the receiver operating characteristic curve. These metrics matter because they show how well an algorithm can classify, detect, or predict a clinical event. However, they do not necessarily tell us what happens once that prediction enters real-world care.
An algorithm may identify a high-risk patient with impressive accuracy, but does that information change the physician’s decision? Does the patient receive treatment sooner? Are unnecessary investigations avoided? Are complications reduced? Is hospitalisation less likely? And, ultimately, does the patient experience a better health outcome?
This distinction is becoming increasingly important. An editorial in Nature Medicine in April 2026 [2] emphasised that claims about the value of medical AI should be supported by adequate clinical evidence, noting that evidence of direct benefit for patients, healthcare professionals and healthcare systems remains more limited than rapid technological progress might suggest.
The imbalance between technological development and clinical evaluation is also evident in the scientific literature. A large analysis published in 2026 examined 4,667 primary medical AI studies and found that 88.2% were preclinical, while only 2.4% were randomised controlled trials. Most of these trials were conducted at a single centre, highlighting how uncommon large-scale, multicentre evaluations remain [3].
Recent clinical studies show why this type of evidence matters. In June 2026, a pragmatic cluster-randomised trial published in Nature Medicine evaluated a generative AI-enabled clinical decision support system across 16 primary care facilities in Kenya, involving 9,691 patients and 103 clinical officers. The system was designed to support clinicians during routine care. However, treatment failure within 14 days occurred in 2.2% of patients in the AI-supported group and 2.0% in the control group, with no statistically significant difference. Importantly, the study found no relevant safety signal, but it also showed no significant improvement in the primary clinical outcome [4].
This kind of result should not necessarily be interpreted as a failure of AI. Instead, it illustrates one of the most important lessons in digital health: even a technically robust algorithm may have limited impact if clinicians do not act on its recommendations, if the timing of the intervention is suboptimal, if usual care is already highly effective, or if the tool does not integrate naturally into the clinical workflow.
In other contexts, however, AI-assisted care has been linked to measurable patient benefits. A pragmatic, randomised trial of an AI-enabled electrocardiography alert system involving 15,965 hospitalised patients found lower 90-day all-cause mortality among patients whose physicians received AI-generated risk alerts than among those receiving usual care [5]. Large multicentre trials are also beginning to assess AI within broader clinical pathways. One recent cluster-randomised study involving more than 21,000 patients with acute ischaemic stroke across 77 hospitals evaluated an AI-supported decision system that combined imaging analysis, stroke classification, and evidence-based treatment recommendations [6].
These studies point to another important shift in how medical AI should be evaluated. In real-world clinical practice, the relevant comparison is rarely “AI versus doctor”. More often, it is “doctor using AI versus doctor providing usual care”.
That distinction matters because healthcare is not delivered by algorithms in isolation. AI systems interact with physicians, nurses, patients, electronic health records, organisational workflows and clinical protocols. Their value therefore depends not only on model performance but also on how people interpret and use their outputs.
A 2026 systematic review and meta-analysis of collaboration between healthcare professionals and large language models found promising improvements in certain aspects of diagnostic and management performance but also reported substantial heterogeneity and uncertainty across studies [7]. Improvements in areas such as documentation quality could coexist with factual errors and inconsistent effects on clinical decision-making.
For this reason, the next phase of medical AI research will need to move progressively beyond retrospective datasets and algorithm benchmarks. Prospective studies, randomised trials, multicentre validation, diverse patient populations and real-world monitoring will become increasingly important. Researchers will also need to consider outcomes that matter directly to patients and healthcare systems, including complications, mortality, quality of life, hospital admissions, waiting times, resource use and cost-effectiveness.
Safety will be equally important. An AI tool that performs well on average may still have unintended consequences, including automation bias, inappropriate reliance on recommendations, alert fatigue, or unequal performance across patient groups. Clinical evaluation should therefore assess not only whether an algorithm is correct, but also how it changes professional behaviour and whether those changes ultimately improve care.
Not every AI application will require the same level of evidence. A tool designed to summarise clinical documentation does not pose the same level of risk as an algorithm that influences cancer treatment or identifies patients at risk of imminent deterioration. The level of clinical validation should therefore be proportional to the intended use, potential risk and consequences of error.
Medical AI has already demonstrated extraordinary technical capabilities. The challenge for the coming years is different. Healthcare does not ultimately need algorithms with the highest benchmark scores. It needs technologies that help healthcare professionals make better decisions, reduce preventable errors, improve access to care, and deliver meaningful benefits for patients.
The most important question may therefore no longer be simply “How accurate is the algorithm?” but rather “What happens to the patient when we use it?”
References
- Artificial Intelligence-Enabled Medical Devices: available at https://www.fda.gov/
- Lång, K. From algorithms to patient outcomes – lessons from one of the first randomized trials of AI in medicine. Nat Med (2026). https://doi.org/10.1038/s41591-026-04633-x.
- Wang, Q., Yang, Y., Qiu, L. et al. A quantitative analysis of global AI medical studies: gaps in randomized controlled trials. npj Digit. Med. 9, 403 (2026). https://doi.org/10.1038/s41746-026-02698-z.
- Agweyu, A., Mwaniki, P., Menon, V. et al. Generative AI-enabled clinical decision support system in primary care: a pragmatic, cluster-randomized trial. Nat Med 32, 3032–3039 (2026). https://doi.org/10.1038/s41591-026-04503-6.
- Lin, CS., Liu, WT., Tsai, DJ. et al. AI-enabled electrocardiography alert intervention and all-cause mortality: a pragmatic randomized clinical trial. Nat Med 30, 1461–1470 (2024). https://doi.org/10.1038/s41591-024-02961-4.
- Zhang X, Ding L, Jing J, Wang C, Gu H, Jiang Y et al. Effect of a clinical decision support system on stroke care quality and outcomes in patients with acute ischaemic stroke (GOLDEN BRIDGE II): cluster randomised clinical trial BMJ 2026; 392 :e085810 doi:10.1136/bmj-2025-085810.
- Wang G, Zhang K, Jiang J, Wang C, Bi H, Liang H, Qi Z, Huang Y, Li Y, Yang X. Human–large language model collaboration in clinical medicine: a systematic review and meta-analysis. npj Digital Medicine. 2026;9:195. doi:10.1038/s41746-026-02382-2.
