83% Accuracy: Microsoft AI Beats Doctors in Complex Diagnosis Trial

Microsoft’s new diagnostic AI system correctly identified complex medical conditions in a simulation more than twice as often as experienced generalist physicians.

The study, published this Thursday in the journal Science, reveals that the system, known as MAI-DxO or simply DxO, achieved an 83% accuracy rate on complex cases. By comparison, primary care doctors and general internists, who had access to reference materials, were correct only 36% of the time. The system works by coordinating several AI agents that function like a panel of specialists, questioning virtual patients, ordering tests, and debating potential diagnoses.

Mustafa Suleyman, the CEO of Microsoft AI, described the results as a significant step toward what he calls medical superintelligence. He noted that the company is currently working with the Mayo Clinic to develop a foundation model for healthcare, with a release planned for next year. Suleyman believes this infrastructure will allow for substantial leaps in clinical reasoning and prediction capabilities.

The findings place Microsoft in a competitive group with Google and OpenAI, both of which have recently claimed that their AI tools can match or exceed human doctor performance. However, the data suggests that much of the improvement in DxO stems from using powerful underlying models. Microsoft tested the orchestration system with models from OpenAI, Google, Anthropic, Meta, and DeepSeek, but found the best results when paired with OpenAI’s GPT-5 and the newer GPT-5.5 model.

Cost and efficiency were also key metrics in the evaluation. When using GPT-5 with a basic prompt, the cost was approximately $8,400 per case with 78.9% accuracy. The DxO system reduced that cost by about $3,000 while improving accuracy by only four points. On emergency department cases, the gains were minimal for both accuracy and cost when using GPT-5. With the newer GPT-5.5 model, accuracy on complex cases rose from 72% to nearly 87% at a slightly lower cost, though emergency cases saw little change.

The orchestration layer proved most valuable when paired with less capable models. For instance, Gemini 2.5 Flash scored 46.4% on its own but jumped to nearly 69% with the assistance of DxO. Researchers noted that accuracy gains naturally diminish as models approach theoretical maximums, but the system helps optimize both accuracy and cost. A spokesperson for the company added that the orchestration addresses the broader challenge of balancing these factors.

To validate these results in the real world, Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston. The evaluation will compare DxO’s recommendations against human doctors in various hospital settings, including emergency and urgent care. Patients will receive standard treatment while consenting participants also engage with the AI system. Dominic King, Microsoft’s vice president of Health, stated that this will be the largest study of its kind, aiming to provide compelling evidence for broader deployment.

This research coincides with a manifesto outlining the path to medical superintelligence, defined as surpassing the strongest human-led health systems in reasoning and coordination. Microsoft aims to move beyond simple benchmarks like answering exam questions to integrating directly with clinicians and healthcare systems. Suleyman expressed confidence that healthcare models will see an order of magnitude improvement in capability within the next 24 to 36 months, driven by real-world usage and iterative improvements.