AI in the Radiology Reading Room: Where It Earns Its Keep and Where It Is Theater

Every radiology conference in 2026 will have a panel on AI. Most of them will be useless, because the question being debated has not changed since 2018: is AI going to replace radiologists. That is not a serious question and it has never been a serious question. The serious question — the one operators actually have to answer — is which AI tools earn their keep inside a reading workflow today, which ones are theater, and what to negotiate into the contract before you sign one.

This piece is the answer Yellowcross gives when a group asks us privately. It is not vendor-specific because the vendors keep moving and the lessons do not. The frame is what works, not who sells it.

The Three Slots Where AI Is Paying Back Right Now

Radiology AI in production is doing useful work in a small number of specific workflow slots. The slots have something in common: each one is a narrow, high-signal pattern-detection problem where the cost of a miss is high enough to justify the false-positive tax. None of them is “interpret the study and write the report.” All of them are upstream of the radiologist’s actual interpretation.

The first slot is worklist triage. The clearest production wins are intracranial hemorrhage on non-contrast head CT and pulmonary embolism on CTPA. A flagged study moves to the top of the worklist, the on-call radiologist reads it first, and the door-to-decision time for stroke and PE workups shortens in a way that the emergency department can feel. The radiologist still interprets the study. The AI did not interpret anything. It rearranged the queue based on a confident prior that this study is the one to read next. That is a real workflow improvement and the hospitals that have it deployed do not want to go back.

The second slot is measurement automation. Lesion size tracking across priors, cardiac measurements on chest CT, vertebral height loss, aortic diameter — anything where the radiologist would otherwise be calipering pixels by hand. Auto-measurement done well does not change the interpretation. It changes how long the interpretation takes. Five seconds saved per measurement compounds into a real RVU number over a year of reads, and the radiologist’s eyes stay on what actually requires judgment.

The third slot is targeted second-read on specific modalities, most credibly screening mammography and a growing set of chest CT applications. The model flags potential findings the radiologist may have missed. The radiologist reviews and adjudicates. In mammography this has the longest production track record, the cleanest peer-reviewed evidence, and the most settled reimbursement story, which is why it is the slot most groups should evaluate first. The other modalities are catching up but they are not there yet at the same level of evidence.

That is the list. There are interesting niches around bone age, fracture detection, lung nodule sizing, and a handful of others, and some of them will mature into a fourth slot before the decade is out. But if a vendor pitches you something that does not fit into triage, measurement, or targeted second-read, you should hear the pitch with both eyebrows up.

Where AI Is Theater

The honest counter-list is shorter and more controversial.

General-purpose interpretation is theater. There is no production model in 2026 that reads a full study, writes a report a radiologist would sign without rewriting, and does it at a quality bar that any group should put into a contractual workflow. The demos look impressive in conference halls. They do not look impressive at 2 a.m. on a complex abdominal pain workup with three priors and a stack of relevant labs the model cannot see.

“Replace the radiologist” tools are theater. They survive at the margin in narrow screening contexts where a strict yes/no output is acceptable and where regulatory and liability frameworks are willing to absorb the risk. Those contexts are real but they are not your group’s main read volume. The vendors that pitch radiologist replacement to a hospital procurement team are not selling a product. They are selling a hope, and the hope dies during the first quarterly performance review.

Generic “AI-powered reporting” is theater unless the vendor can show you a structured-reporting integration with your specific PACS and dictation stack and a real production deployment at a comparable group. The category exists. Most of the implementations do not. Ask for the reference call before you commit to anything.

Anything that produces an output the radiologist cannot easily audit is theater regardless of what it does. If a model flags a finding without showing the radiologist where on the image the finding is and what the model’s confidence is, the radiologist cannot defend the read in court, cannot teach the resident from it, and will not trust the tool over time. The auditability of the output matters more than the accuracy of it in production use.

The False-Positive Tax Nobody Quantifies in the Pitch

The variable that vendors do not voluntarily discuss is the false-positive rate at the operating point you would actually deploy, and the time cost that rate adds to the read.

A triage model with 95 percent sensitivity for intracranial hemorrhage at a 12 percent false-positive rate flags roughly one in eight non-bleed head CTs. Every flagged study moves to the top of the worklist and the radiologist has to read it as if it is a likely bleed, which means a more careful look than the same study would have gotten in normal queue order. Multiply that across an emergency department’s nighttime head CT volume and you can quantify the radiologist time spent on AI-induced over-attention. Sometimes the math works out. Sometimes it does not. It is the math that should be in the contract, not the sensitivity number on the slide.

The same calculation applies to second-read tools. The vendor will quote a cancer-detection improvement. The number that determines whether the tool earns its keep is not the detection improvement. It is the detection improvement net of the time cost of adjudicating the false positives, multiplied by the radiologist hourly cost, divided by the reimbursement uplift if any. Make the vendor show you that math at your read volume and at your case mix. If they cannot, the deployment will not behave the way the pitch suggested.

What to Negotiate Into the Contract Before You Sign

Five contract terms that should be non-negotiable for any radiology AI deployment in 2026.

A defined performance audit cadence and a defined remediation path if the model underperforms its contractual sensitivity, specificity, or latency on your data. You should not be discovering model drift through your own QA. The vendor should be reporting it.

A clear data and IP posture on whether your reads are used to retrain the model, who owns the resulting weights, and what happens to your data if the contract ends. The default vendor language on this is almost always too generous to the vendor.

A latency SLA. A triage model that flags hemorrhage 90 seconds after the study arrives at PACS is meaningfully different from one that takes seven minutes. Both can be marketed as “real-time.” Only one is useful for stroke workflow.

A termination clause that does not require you to keep paying for a model the FDA has reclassified, the payer has decided to stop reimbursing for, or the radiologist group has voted to stop using. These conditions are not hypothetical and the standard vendor contract assumes none of them happen.

An integration commitment specific to your PACS, dictation system, and worklist orchestration. Do not let the vendor sell you a model that is not actually plumbed into your environment. The plumbing is the deployment. The model is the easy part.

How to Choose the First Tool

If your group has not deployed any radiology AI yet, the right place to start is not the most impressive demo. It is the workflow slot where you have the clearest pain and the cleanest measurement. For most groups in 2026 that is one of three things: stroke triage for emergency department CTs, screening mammography second-read, or lesion-tracking auto-measurement for oncologic follow-up. Pick one. Deploy it. Measure it for six months. Decide whether to deploy the next one based on what you actually saw, not on what the next vendor pitches at the next conference.

Yellowcross helps radiology groups think through this evaluation when the procurement decision is approaching and the pitch decks are stacking up. If you want a second set of eyes on a specific AI contract or a specific deployment plan, that is exactly the kind of operational conversation we are built to have. Start at yellowcross.com or read the foundational piece on how to build the infrastructure that good AI deployments sit on top of: Building Your Practice: Successful Radiology Infrastructure.