Microsoft’s MAI-DxO scored 83.2% on a set of hard clinical cases drawn from the New England Journal of Medicine’s clinicopathological conferences. Generalist physicians scored 36.1% on the same cases. On emergency department admissions from MIMIC-IV-ED, the system scored 80.3% against 46.8% for the physicians.

The figures come from the abstract of “Sequential diagnosis with language models,” published in Science on 22 October 2026 (doi 10.1126/science.aeh7930). The paper’s body is paywalled. Both the DOI landing page and the PDF link returned HTTP 403 when the research behind this article was assembled on 24 October, so the numbers here come from the publisher’s deposited abstract, the benchmark and datasets the authors released themselves, Microsoft’s launch post and TechCrunch’s reporting.

The comparison arm is the part to read twice. It is primary care doctors and general internists working individually with reference materials, as TechCrunch describes the arm, and the cases are the kind hospitals normally route to specialists or to a team. TechCrunch’s report on the study notes that those specialists and physician teams were not part of the comparison.

What the study compared

Case set MAI-DxO Generalist physicians
NEJM clinicopathological conference cases 83.2% 36.1%
MIMIC-IV-ED emergency department admissions 80.3% 46.8%

Microsoft’s post puts the emergency department result as “1.7 times more accurate” than physicians. Dividing 80.3 by 46.8 gives 1.72, so that claim holds against the abstract.

The abstract describes “an AI system that simulates a panel of physicians.” MAI-DxO is Microsoft’s name for it, and the authors’ released benchmark uses the same Diagnostic Orchestrator label.

Three terms, one line each

Case-vignette simulation: the system works through a written case summary instead of examining a patient.

Agent panel: MAI-DxO spreads the task across role-playing agents that propose diagnoses and order tests, with a final decision on when to commit.

Differential-diagnosis accuracy: the benchmark scores the committed diagnosis on a five-point scale and counts 4 or 5 as correct. It does not measure whether treatment helped.

What the text simulation cannot test

The authors state their own limits in the dataset release on PhysioNet. The benchmark is text-only. It cannot test patient communication, how a condition changes over time, response to treatment, or the reading of images, waveforms and pathology slides. Discharge summaries used as source material may omit findings a real chart would have shown.

One design detail matters for anyone reading the accuracy numbers. When an agent requests a test whose result is not in the case text, a gatekeeper model returns a synthetic result, and a separate judge model scores the final diagnosis against the known answer. The score measures how well the agent navigates a text case with a simulated test result.

The authors limit their own conclusion in the same place: the benchmark assesses diagnostic correctness, not treatment safety, longitudinal management or patient outcomes. Costs use standardized U.S. prices from the 2023 Kaiser Permanente Washington Central Hospital charge master, rather than what a real admission would bill.

DxO adds about 3.7 points over the base model on the MIMIC cases

The author-released MIMIC data gives the cleaner comparison because it includes the baseline. On 325 validation and test cases, a base GPT-5 agent scored 76.6% and the DxO version 80.3%, a gain of roughly 3.7 points. Workup cost was about US$4,240 per case without the orchestration and about US$4,980 with it. The 80.3% configuration ran without budget constraints.

That pattern matters when the scaffolding gets the credit. On the emergency department cases the orchestration adds little over the model beneath it, while the baseline agent already clears the physician baseline. The released configuration pins the gatekeeper and judge models, so the results belong to a specific model stack rather than to MAI-DxO as a fixed design. The released material does not settle which model produced the MIMIC numbers: PhysioNet credits them to GPT-5, while the SDBench README says the Science runs reported the validation and test splits for openai/gpt-5.5.

On the harder NEJM cases the orchestration appears to add more. Microsoft’s figures, relayed by TechCrunch, describe a rise from 72% to nearly 87% for GPT-5.5 with DxO and about four points for GPT-5. Neither figure appeared in the abstract or could be read in the paywalled body.

Microsoft’s claims about its own system

Microsoft published its launch post the same day under the heading “Building medical superintelligence.” The post calls the system “more than twice” as accurate on complex cases, which the abstract supports, and “nearly four times” as accurate when physicians diagnose without those resources, which the abstract does not. The second claim implies a physician arm near 21% that appears nowhere in the abstract and could not be read in the body, so it is not used here.

The post says the paper was “reviewed by independent experts before publication.” That describes Science’s standard peer review, not a clinical evaluation by outside clinicians.

Microsoft also describes an order-of-magnitude improvement within 24 to 36 months and a push toward what it calls medical superintelligence. Those are statements about the company’s plans.

Figures that come from Microsoft’s chart, not the paper

One number travels with the launch without surviving a check. GPT-5 alone on complex cases is reported at 78.9% and about US$8,400 per case. That appears in TechCrunch’s account of Microsoft’s data and not in anything read at source here, and the author-released MIMIC figures are roughly half that cost in a different configuration.

The same caution applies to two other pairs from Microsoft’s chart: GPT-5.5 at 72% rising to about 87%, and Gemini 2.5 Flash at 46.4% rising to about 69%. Both come from Microsoft’s own data as relayed by TechCrunch.

Who ran the study

All 15 authors are Microsoft employees, and the conflict statement declares no other interests. Harsha Nori is first author and Eric Horvitz is last, with Mustafa Suleyman and Dominic King among the rest. No independent expert assessment of the result was found. The case sets carry access conditions of their own: the NEJM conference cases must be licensed from the journal, and MIMIC requires PhysioNet credentialing.

An earlier version of this work appeared as an arXiv preprint in mid-2025 with different arms and numbers: 80% for the system against a 20% generalist-physician average on 304 NEJM cases, using a different underlying model. The two sets of figures should stay apart.

The clinical evaluations are planned, not finished

Microsoft says Mayo Clinic will build a foundation model next year, and Beth Israel Deaconess Medical Center will run a real-world evaluation in which patients receive standard care while consenting patients also work through DxO in parallel. Neither project has published results. Google’s AMIE study with Beth Israel, published in the Lancet in October 2026, ran real-world urgent care visits with physicians monitoring in real time, as TechCrunch reports.