Healthcare Is Quietly Moving Past RAG

For two years the story in clinical AI was simple: general models hallucinate, so you build a specialized one that retrieves over peer-reviewed literature and cites its sources. OpenEvidence became the poster child — an ensemble trained on the medical literature, roughly 15 million consultations a month, embedded in Epic at systems like Mount Sinai and Sutter. Retrieval-augmented generation over a trusted corpus. On paper, exactly what medicine should want.

The 2026 benchmarks are complicating that story.

The gap is closing, then reversing

Across a run of recent evaluations, general frontier models have caught up to the specialized tools on medical knowledge and, on the hardest questions, passed them. A Becker's-reported study found GPT-5.2, Gemini 3.1 Pro, and Claude Opus 4.6 outscoring OpenEvidence and UpToDate's expert AI across standard medical benchmarks — Gemini reportedly near 97% on MedQA against roughly 90% for OpenEvidence. On HealthBench Hard, a frontier general model landed well above OpenEvidence and other retrieval-first assistants on the same rubric.

None of this means the specialized tools are bad. It means their advantage was temporary. When the general model didn't know medicine, retrieval over the literature was a real moat. Now that the general model does know medicine, "we retrieve over PubMed" is a feature, not a defense.

Don't overcorrect

Here's the part the hype cycle will skip: on the dimension that matters most in a clinic — not being confidently wrong about an emergency — the specialized tools still hold up well. A triage study on medRxiv found OpenEvidence erring on the safe side, undertriaging a small fraction of emergencies where a general health assistant undertriaged far more. Retrieval-grounded, narrow-scope systems hallucinate less, and in medicine a lower false-confidence rate can matter more than a higher average score.

So this isn't "RAG is dead." It's narrower and more useful than that: retrieval quality alone stopped being the differentiator. Two systems can retrieve the same paper and still fail differently — one cites a guideline that was superseded eighteen months ago, one can't tell you which version informed its answer, one has no record of what it read.

The moat moves to governance

That is the real shift, and it is not unique to healthcare. When every model can retrieve, the questions that remain are governance questions. Was this guideline the current one when the agent used it? Can I reconstruct which version of which document produced this recommendation? Was that source even approved for clinical use? Retrieval doesn't answer any of those. It was never designed to.

I've argued for a while that context engineering is a strategic imperative and that governing the interaction matters more than the model on top. Medicine is where that argument stops being abstract. A clinical answer you can't attribute to a specific, current, approved source is not a safer answer because it came from a specialized tool. It's an unaudited one.

The next generation of clinical AI won't win on whose retrieval is smartest. The frontier models have made that a commodity. It'll win on whose context is governed — provenance, currency, integrity, and an audit trail of exactly what the system read before it spoke.