Microsoft says its diagnostic AI beats doctors on complex cases
The results come from a simulation, and testing in real hospitals is planned next.
Research#03
83%
Accuracy on hard cases, doctors scored 36%
Image: TechCrunch
83%Accuracy on hard cases, doctors scored 36%
The short version
- Microsoft published a study in Science on Thursday showing its diagnostic system, MAI-DxO, correctly diagnosed 83% of complex cases from the New England Journal of Medicine in a simulation.
- Generalist physicians working with reference materials were correct 36% of the time on the same cases.
- The system uses several AI agents that act like a panel of doctors, questioning a patient, ordering tests, and debating a diagnosis.
- Microsoft plans a study with Beth Israel Deaconess Medical Center to test the system in real clinical settings.
Why it matters
The study adds to claims by Google and OpenAI that their AI can match or outperform doctors. The data suggests much of the gain comes from stronger underlying AI models, with Microsoft's orchestration adding gains on top. For developers, the cost figures matter, since the system lowered cost on some runs but added little accuracy in others.
“It needs clinical validation”