Microsoft’s AI beats doctors on hard cases

Microsoft says its AI can diagnose complex medical cases far more accurately than doctors, and the claim now comes with a peer-reviewed study behind it. On October 22, 2026, the journal Science published results showing that Microsoft’s diagnostic system got about 83% of difficult cases right. Generalist physicians working on the same cases scored 36% with reference materials and 22% without them.

The system is called MAI-DxO, short for Microsoft AI Diagnostic Orchestrator. Microsoft first described it in a preprint in June 2025, and the new paper is the peer-reviewed version. “It is finally here,” Mustafa Suleyman, CEO of Microsoft AI, told The Verge, speaking about AI’s long-promised arrival in healthcare.

How it works

MAI-DxO coordinates several AI agents that act like a panel of doctors. They go through the patient’s symptoms and history, decide which tests to order, argue over possible explanations and then settle on a diagnosis.

The test cases come from the New England Journal of Medicine (NEJM), which publishes difficult real-life cases that doctors have used as training puzzles for decades. The AI doesn’t get the full case file at the start. Like a doctor during a real workup, it has to ask questions and order tests to get information, and each test has an estimated cost. Microsoft gathered the cases into its own benchmark, SDBench, which it plans to release as open source.

Tested AI models

Microsoft tested MAI-DxO with models from OpenAI, Google, Anthropic, Meta and DeepSeek. The best results came with OpenAI’s GPT-5 and GPT-5.5. According to TechCrunch’s reading of the study, much of the improvement comes from the underlying models getting better, with Microsoft’s orchestration adding gains on top.

GPT-5 on its own, with a short basic prompt, scored 78.9% on complex cases at an estimated cost of about $8,400 per case. With MAI-DxO, the cost dropped by about $3,000 and accuracy rose by about four points. GPT-5.5, released in April 2026, went from 72% to nearly 87% on complex cases with MAI-DxO, at a slightly lower cost. Google’s smaller Gemini 2.5 Flash model jumped from 46.4% to nearly 69%.

On emergency department cases, the orchestrator made little difference. GPT-5.5 scored about 85% with or without it, at a similar cost.

So MAI-DxO helps most with the hardest cases and with weaker models. Even then, the gap between weak and strong models was bigger than the boost the orchestrator added. Microsoft’s researchers said gains naturally shrink as models get closer to the maximum possible score. They also said later testing shows that “improvements in model capability do not necessarily diminish the benefits” of having several agents work together.

The results

The main test used hundreds of NEJM cases, and the physicians taking part were primary care doctors and general internists. Their 22% score without references is the basis for Suleyman’s claim that the system is “up to four times” more accurate than doctors. Against doctors who had references, it was a bit more than twice as accurate.

The study also tested emergency department cases built from records of past patient admissions at Beth Israel Deaconess Medical Center in Boston. These are closer to everyday hospital work. MAI-DxO scored 80%, compared with an average of 47% for physicians. In both tests, the AI usually reached its diagnosis at a lower estimated cost than the doctors.

There is one gap in the comparison. Cases this complex would normally go to specialists or a team of doctors, and neither group was tested.

Next step

So far, MAI-DxO has only worked on past cases. “It needs clinical validation,” Suleyman said. “That is a leap, but this is a major milestone along that journey.”

That testing will happen at Beth Israel Deaconess. Patients will get standard care from their doctors. Those who agree to take part will also interact with MAI-DxO, and researchers will compare its recommendations with the doctors’ decisions. Dominic King, Microsoft AI’s health lead and a co-author of the study, said it will likely “get underway in the next couple of months,” but didn’t give a firm date.

Google is doing something similar. In October it published a study in The Lancet, also with Beth Israel, testing its AMIE diagnostic AI during real urgent care visits while physicians watched in real time.

Some real-world data on AI-assisted diagnosis already exists. In Kenya, the clinic network Penda Health used an AI assistant built on OpenAI models in 15 primary care clinics. In a 2025 study of 39,849 patient visits, clinicians using it made 16% fewer diagnostic errors and 13% fewer treatment errors.

Microsoft’s plan

The study is part of Microsoft’s push toward what it calls “medical superintelligence,” meaning AI that can outperform the best human-led health systems. In a blog post and an article in the journal BMJ Digital Health & AI, both published the same day as the study, Microsoft named five more areas AI will need to master beyond diagnosis. These include coordinating care, making treatment decisions and handling the human side of medicine.

Microsoft is also working with the Mayo Clinic on a healthcare foundation model planned for 2027.

Microsoft hasn’t said when or how MAI-DxO might become a product. The company already sells dictation and radiology tools to clinicians, and health is one of the most popular uses of its Copilot chatbot. King said strict regulatory and compliance rules in medicine mean any rollout has to be done responsibly.

Suleyman expects “extremely impressive deployments of intelligent systems in production” as early as next year, and “something that is indistinguishable from human performance on medical superintelligence in the next couple of years.” Even so, he says human clinicians will always have a central role, and he hopes AI will take some of “the drudgery” out of medicine so doctors can spend more time with patients.

For now, MAI-DxO is a strong result on historical cases. The upcoming study in Boston will show how it handles real patients in a busy hospital.

Share the Post:

Related Posts