My desktop app was telling the model that stopped drugs were active. The model wasn't making that up. My code was.
Charts too big for one call get split, and each chunk carries a pinned summary header. That header lists every medication order under "Active medications" without checking status. Stopped, completed and entered-in-error orders reach the model as current drugs, under a label that says "active."
There is no single best FHIR format for an LLM. Pick a format per task, pin the projection in version control, and benchmark it on the edge cases that hurt patients. In my app, the wrong med lists traced to renderer bugs.
Why the same FHIR chart gives an LLM different medication lists
The desktop app I work on renders a chart three ways. The header bug above is one of them. The others are just as quiet.
Missing meds. Sometimes a drug is referenced rather than written inline. In that case the JSON compactor strips the referenced record, and the chat summary shows "Unknown medication." The app's own screens already resolve that reference. The prompt path doesn't.
Generic and brand counted twice. The header dedupes on display text. "metformin 500 MG" and "Glucophage 500 MG" are different strings, so both survive.
No dispenses at all. The chat summary leaves out dispense records entirely. A dispense with no matching order can never come up, because the model never sees one.
The prompt path is a second UI for the chart, and it needs the same tests the screens get. The screens already handle cases the prompt path doesn't, like referenced medications. Nobody had pointed those tests at the prompt.
Raw FHIR Bundle JSON vs. markdown summary: what I send an LLM
I use two formats for two jobs.
Skills get pruned, minified JSON. The trigger was a hard failure: prompt is too long: 1,302,962 tokens > 1,000,000 maximum. Pruning cuts the sample chart by 60%, and very large charts are split across several calls. That split is where the pinned-header bug lives.
Chat gets a flat markdown summary. On the same chart it's 94% smaller than the raw bundle.
I picked both formats on token limits and cost. I have not run an accuracy comparison between them. I'd bet most teams sending FHIR to a model are in the same spot. For trimming Bundles inside a tool layer and keeping PHI to the minimum, see The Best FHIR MCP Server Gives the Model No FHIR Tools.
What the research says about FHIR serialization for LLMs
Two recent papers measure accuracy, and they don't agree on a winner.
arXiv 2604.21076 tested four formats on medication reconciliation: Raw JSON, a Markdown table, a clinical narrative and a chronological timeline. It used five open-weight models and 200 Synthea patients. Narrative beat Raw JSON for small models. Mistral-7B went from 0.7247 to 0.9149 F1. The gain shrinks fast. Llama-3.1-8B moved from 0.918 to 0.947, and Phi-3.5-mini's gain wasn't significant. At 70B the order flipped: Raw JSON scored 0.9956 and narrative 0.9850, a gap that wasn't significant either. The timeline format was bad even at 70B, with 25 of 200 patients scoring zero (PDF).
- Their "Raw JSON" wasn't raw. It was capped at 100 resources, with ids and admin fields stripped. A real Bundle carries far more structural overhead.
- Omission was the main failure. Models missed active meds more often than they invented them. Small models plateaued around 7-10 concurrent active meds, so polypharmacy patients get the worst answers.
- The prompt told the model the status rule. Ground truth was only
MedicationRequest with status active. The paper had no MedicationStatement, no dispenses, no entered-in-error records and no contradictions.
FHIRBench tested six formats on four models. A condensed format beat Raw JSON on token F1 for three of the four. An LLM judge preferred Raw JSON for three of the four. Narrative kept 95% of Raw JSON's judged quality with 83% fewer tokens (337 vs 1,991 mean input tokens). The winner also depended on the model. GPT-5.4 favored Raw JSON, while Claude and the open-weight models favored compressed formats.
Neither paper tests my failure modes. Neither covers MedicationDispense, entered-in-error records, conflicts between statements and requests, or citing resource IDs. In 2604.21076 the model never sees an id, so nobody measured whether an answer traces back to a record.
Ollama's context window (num_ctx) silently truncates FHIR prompts
I also run clinical tasks on small local models. The first thing to break was the context window, silently.
Ollama defaults to 4,096 tokens, and we never set a larger one. Whole-chart prompts got cut down to their last 4,096 tokens. The model wrote confident notes with no problem list and no allergies. There was no error, just a shorter chart.
resp = ollama.chat(
model=model,
messages=messages,
options={"num_ctx": 8192, "num_predict": max_out},
)
We now request 8,192 tokens, cap output length, and prune and split the payload before sending. One problem is still open: the chunk sizes were tuned for cloud models, not an 8k window. Run a format benchmark on the small model you'll deploy, not just the hosted one. If the local model is silently truncating, every format comparison you run on it measures the truncation. I cover choosing small models in You Don't Need GPT-4 for That, and what long inputs do to output in Context Rot.
Here's the harness I'd point at my own renderers. You need a HAPI FHIR server, Synthea patients, and a handful of hand-made patients built to hit specific errors. Synthea alone isn't enough. It writes one MedicationRequest per refill, so you get long, clean histories. It won't produce the messy records that broke my app.
Hand-make these:
- A
MedicationStatement that says a drug is active while the MedicationRequest says stopped.
- A
MedicationDispense with no order behind it.
- Entered-in-error records.
- A med stopped with no end date.
- The same drug twice, once as an RxNorm generic and once as a brand.
- A medication that is referenced, not written inline.
Render every patient four ways:
- The raw Bundle.
- A pruned JSON projection.
- A clinical narrative with resource IDs kept inline (
metformin 500 MG [MedicationRequest/mr-07a]).
- SQL-on-FHIR view tables as CSV or TOON.
Run each one on a hosted model and on a small local model.
Write the expected answer for each patient as a fixture:
patient: Patient/edge-07
expect:
active:
- rxnorm: "<metformin 500 MG code>"
cite: [MedicationRequest/mr-07a]
flags:
- type: dispense_without_order
cite: [MedicationDispense/md-07b]
- type: duplicate_brand_generic
cite: [MedicationRequest/mr-07a, MedicationStatement/ms-07d]
never_active: [MedicationRequest/mr-07c]
Score three things for each format and model.
Accuracy per error type. A single F1 number hides the case that hurts. Report entered-in-error, dispense-without-order, duplicates and contradictions separately.
Citation. Every claim should cite a resource ID that exists and says what the model claims. A correct med with no citation fails.
Tokens, latency and cost. Report them per correct, cited med, not per call. A format that's cheap per call but misses half the meds isn't cheap.
Two rules for the scoring itself. Match on RxNorm codes, not exact strings. 2604.21076 used exact-string matching and called its scores a conservative lower bound for that reason. And don't make an LLM judge your only metric. In FHIRBench, Claude Sonnet 4.5 was last on F1 and first on judged quality. I've written about why your LLM evaluator lies. A fixture with expected IDs can't lie that way.
Using SQL-on-FHIR ViewDefinitions as LLM context
The format question gets easier once the projection is a file you can diff. A SQL-on-FHIR ViewDefinition is a tabular projection of one resource type, with columns written in FHIRPath. Here's the R4 shape for orders:
{
"resourceType": "ViewDefinition",
"name": "med_request",
"resource": "MedicationRequest",
"status": "active",
"select": [{
"column": [
{"name": "row_id", "path": "getResourceKey()"},
{"name": "patient", "path": "subject.getReferenceKey(Patient)"},
{"name": "status", "path": "status"},
{"name": "authored"
Two things fall out of this. First, the med_ref column makes the referenced-medication case explicit. My compactor dropped it silently. A view has a column that's either filled or empty, and a test can check it. Second, getResourceKey() gives every row a key the model can cite.
The spec also forces you to be honest about joins. A single ViewDefinition won't join resources. Reconciling statements, requests and dispenses means three views, joined downstream on getResourceKey() and getReferenceKey(). That's where status filtering and RxNorm dedup belong: in SQL you can test, not in a header nobody reads. The $run operation returns CSV, NDJSON, Parquet and more, so the same views feed the CSV arm of the harness.
The token savings are real. The accuracy effect is unmeasured. Flexpa used ViewDefinitions to cut 14 claims resources from about 240k tokens to about 19k, a 92% drop. That was billing data, and they measured tokens, not answers.
Keep the ViewDefinitions in version control next to the prompts. They're the contract between the EHR data and the model.
This is the rule I'm starting from. Treat it as a set of defaults until the harness says otherwise on your model.
Lookup ("Is she on lisinopril?"). Don't send a chart. Query the view and send the matching rows with their keys. A lookup is a filter, and filters belong in code.
Reconciliation. Do the deterministic work before the model sees anything: status filtering, entered-in-error removal, RxNorm dedup and joins. Then send the projection with IDs inline and require a citation for every claim. On 8B models and smaller, use a narrative with inline IDs; 2604.21076 found narrative ahead there. On 70B-class models, pruned JSON scored highest, though not significantly ahead of narrative. On small models, check patients past about seven active meds by hand.
Summarization. Use flat markdown or narrative, which is what my chat already does. Across FHIRBench's tasks, narrative kept 95% of judged quality at 83% fewer tokens. If you need the last 5% and can pay for it, Raw JSON got the best judge scores for most models. Skip chronological timelines.
Rerun the benchmark on every model or EHR change
Swap the model and the winning format can flip. Swap the EHR, or let a vendor change how it writes MedicationStatement, and your edge cases change under you. Treat the harness as a regression test, not a one-time study. Run it when the model changes, when the EHR feed changes, and when a ViewDefinition diff lands in a PR.
