LLM-powered documentation and retrieval for a regional health network
We partnered with a regional health network to modernize how clinicians interact with narrative documentation—without sacrificing privacy or control. The centerpiece was a retrieval-augmented LLM stack that pulls from approved corpora: internal protocols, formulary notes, and curated FAQs—not arbitrary internet sources.
Architecture choices
- Embeddings + vector store scoped per facility group, with ACLs enforced at query time
- Prompt contracts that require citations to retrieved chunks for clinical-adjacent answers
- Output filters for PII patterns and escalation when the model abstains
Evaluation that mattered
Beyond perplexity scores, we built reviewer workflows: clinicians and informaticists scored usefulness, safety, and alignment with policy on a fixed rubric. That feedback loop drove prompt and retrieval updates—not guesswork.
Operational reality
LLMs in healthcare are as much about release management as about models. We aligned with their change board, defined rollback, and monitored latency and error budgets so the assist layer stayed invisible when it needed to be.
Descriptions are generalized from delivery experience. Not medical advice; not an endorsement of any specific vendor model.