Most clinical AI monitoring plans wait for ground truth. That's the wrong thing to wait for. Labels arrive months late or never. EHR upgrades and order-set changes shift your inputs overnight. Hosted LLMs change behavior under the same name. You can detect drift without labels by watching what you can see today: what clinicians do with your output, what your inputs look like, and how a frozen eval set scores on replay.
Build the response before the signal. For every alert, write down three things: the threshold, the action (roll back, suppress the card, or send output to human review), and who owns that call. A drift alert without an agreed response is a chart someone glances at on Fridays. I run production support on the same rule. A ping has to mean something, or people learn to ignore it. In a clinical system, an alert everyone ignores is worse than no alert, because it looks like coverage. If a signal doesn't map to one of those three actions, it doesn't get wired.
Detecting drift without labels
Classic drift monitoring compares predictions to outcomes. In clinical work, your model changes the outcome. If a card prompts an order and the order prevents the bad event, the "label" now says the risk was never there. Even when outcomes do arrive, they can't cleanly tell you the model was right.
Your best proxy is the clinician. Accepts, overrides and edits arrive within hours.
CDS Hooks feedback is your early label
CDS Hooks has a Feedback API built for this. The spec defines it as the way a service learns what happened to its recommendations. The EHR POSTs to {baseUrl}/cds-services/{service.id}/feedback. Each item carries the card's uuid, an outcome of accepted or overridden, the accepted suggestions, an optional coded override reason with a free-text comment, and a timestamp.
Three spec details decide whether you get any signal.
No uuid, no feedback. If a card or suggestion has no uuid, the EHR sends nothing. Generate a uuid for every card. At serve time, log it with the card's inputs, model version, and site. That log is the join key for everything below.
Ignored cards are silent. If the clinician never touches a card, the EHR doesn't report it. One reason is that an ignored card can still be acted on later. You have to infer ignore rate as cards served minus cards with feedback. Watch it as closely as override rate. A card nobody reads anymore has drifted out of relevance, even if nobody overrides it.
Feedback is optional on both sides. The service MAY support the endpoint, and the EHR SHOULD send feedback. Neither is required. Some sites will send you nothing. Know which ones before you trust a global number.
Ship a coded overrideReasons list with your cards. Each reason needs a display value, and the EHR SHOULD show the list on dismissal. "Not clinically relevant," "already addressed," and "data is wrong" tell you very different things. A spike in "data is wrong" points to input drift before model drift.
One card can produce several feedback rows: suggestions get accepted one at a time, and a card can be acted on late. A naive join double-counts those cards. Dedupe to the latest outcome per card first:
with latest as (
select distinct on (card_uuid) card_uuid, outcome
from card_feedback
order by card_uuid, outcome_timestamp desc
)
select s.site, s.card_type, s.model_id,
count(*) as served,
avg((l.card_uuid is null)::int) as ignore_rate,
count(*) filter (where l.outcome = 'overridden')::float
/ nullif(count(l.card_uuid override_rate
What counts as a normal override rate depends on the card, the site, and the EHR, so don't alert on a global number. Here's a starting heuristic, not a figure from my production data: keep a 4-week rolling baseline per site and card type. Alert when the weekly rate moves more than 2 standard deviations from it. Skip any slice with fewer than 50 cards served that week. Tune from there once you've seen a few months of your own noise.
Edit rates on drafted notes
Drafted notes have no feedback API. The signal is the diff between what you drafted and what the clinician signed. Store both.
Aggregate edit distance is a weak signal. A clinician who rewrites the assessment in their own style looks the same as one fixing a hallucinated medication. Break it down by section, track sections deleted and added, and flag changes to the fields that carry risk:
import difflib
RISK_FIELDS = ("med_name", "dose", "route", "laterality")
def note_edits(draft: dict, signed: dict, draft_fields: dict, signed_fields: dict):
sections = {}
for name in draft.keys() | signed.keys():
text, final = draft.get(name), signed.get(name)
if final is None:
sections[name] = {
A section the clinician wrote that you never drafted is its own signal. Either the draft stopped producing something it should, or the inputs that section depends on stopped arriving.
Roll these up per site the same way you roll up card feedback. A jump in medication or dose changes at one site after an upgrade is something to act on. A slow rise in total characters edited usually isn't.
Override rates tell you something broke. Input monitoring often tells you why, and sometimes it tells you first.
In an integrated clinical system, assume the inputs changed before the model did. A lab gets remapped to a new code. An order set gets rebuilt. A field that was always populated goes null after an upgrade. For each FHIR-derived feature, track these per site: missingness, unit mix, value histograms, and the share of the top codes in each coded field.
Hook behavior varies by EHR too. athenaOne's CDS Hooks integration guide notes that its order hook doesn't fire when orders are updated or deleted. If your service assumed it would, your input distribution at that site was never what you thought it was. (More on CDS Hooks integration.)
For value sets, fingerprint the code lists your features depend on, and alert when a fingerprint changes. Also alert when codes your model has never seen show up in a coded field:
import hashlib
def fingerprint(codes: set[str]) -> str:
return hashlib.sha256("\n".join(sorted(codes)).encode()).hexdigest()
def check_valueset(name, current_codes, seen_codes, store):
alerts, fp = [], fingerprint(current_codes)
baseline = store.get(name)
if baseline is None:
store.set(name fp
Run it daily, and on every announced EHR change. The fingerprint catches the order-set rebuild nobody told you about. The unseen-code check catches the new code your feature pipeline is quietly dropping into "other." The old baseline stays until someone signs off, so the alert stays open until a human closes it. It doesn't fire once and then go quiet.
Pinning a model ID isn't enough
Chen, Zaharia and Zou compared the March and June 2023 versions of GPT-3.5 and GPT-4 on seven tasks, including US medical licensing exam questions. Accuracy shifted, willingness to answer shifted, and formatting shifted. GPT-4's prime identification went from 84% to 51%. Some tasks got better and some got worse, and the two models often moved in opposite directions. Your outputs changed and you didn't deploy anything. That's what you need to catch.
So pin, and never call an alias in production. Anthropic's docs say each model ID is a pinned version. An alias like claude-sonnet-4-5 is different: it points to the latest snapshot and can move. Log the exact model ID on every output, next to the card uuid.
Pinning isn't the whole fix. The same page says the serving infrastructure (request router, safety classifiers, sampling logic) can change. Those updates occasionally shift observable behavior even when the ID and weights stay the same. Every pinned ID also has its own retirement schedule, so eventually you'll be forced to migrate. Treat that migration like any other model change.
Replay a frozen eval set
This is where pre-ship evals turn into production monitoring. Freeze a set of real, de-identified cases with expected outputs or graded rubrics, and don't touch it. Put new cases in a separate set so the frozen one stays comparable over time. If you don't have one yet, the three-step version is the fastest way to get one, and the longer guide covers binary scoring. If an LLM judge grades your rubrics, calibrate it against clinician grades first. An uncalibrated judge fails hardest on the cases that matter.
Replay it on these triggers:
- Any model ID change, including forced migrations off a retiring model.
- Any EHR upgrade at any site.
- Any value-set fingerprint change.
- Any prompt or feature-pipeline change.
- On a schedule (weekly is a sane start), because of the infrastructure changes above.
Diff each run against the last passing run case by case, not just on the aggregate score. A clinical model can hold its average while flipping the five cases that matter.
Make the pass/fail rule explicit. Tag a high-risk subset in the frozen set: medications, doses, and anything that would change an order. Any high-risk case that flips from pass to fail blocks the change. Outside that subset, every flip gets a human look before you ship.
Slice by site and population
An aggregate override rate is close to useless. Override behavior depends on the EHR UI as much as on the model. The spec lets the EHR decide whether to show override reasons. athenaOne's integration guide (v0.3.0, April 2025) says that when you supply reasons, the user must pick one before dismissing. Two sites on two vendors are measuring two different things.
Slice every signal by site, EHR vendor and version, card type, and model ID. Then slice by population, and pull each slice from a defined FHIR field so it means the same thing every week: age band from Patient.birthDate, sex from Patient.gender, language from Patient.communication, payer from Coverage, care setting from Encounter.class. Log those values at serve time with the card uuid. Recompute them later from a chart that has since changed and you're measuring a different population.
Drift rarely hits everyone at once. A remapped lab at one hospital disappears inside the global average. So does a model change that handles non-English notes worse. Alert per slice, and keep the volume floor so a ten-patient slice doesn't page anyone.
Decide what each model drift alert does
The answer should already be written down before the dashboard exists. Put it in config, not a wiki, and get clinical sign-off:
| Signal | Action | Owner |
|---|
| Override or ignore rate jump (per site + card type, vs. baseline) | Suppress the card at that site | Clinical lead |
| Value-set fingerprint change | Route outputs to human review until the replay passes | Integration engineer |
| Frozen eval replay regression | Roll back to the last passing pinned model ID | ML engineer |
| Medication or dose edit-rate jump on drafted notes | Human review | Clinical lead |
Each action has to be a switch you've already built and tested.
Suppression is already a pattern EHRs expect. athenaOne strongly recommends that services send a card only once, or use feedback to stop re-sending dismissed cards. A per-site, per-card kill switch is a small step past that.
Rollback means the previous pinned model ID is still callable and still passes the frozen set.
Human review means a queue exists and someone is staffed to work it. That's the same line I draw elsewhere. A clinician has to be able to independently review what the model produced, which is the bar in FDA's 2026 CDS guidance. It's also why I keep the LLM out of order-sign entirely.
Silent failures are the worst kind in production LLM systems. An alert with no action behind it is one of them. If you can't name the action, don't ship the alert.
