Configurable Clinical Information Extraction with Agentic RAG: What Works, What Breaks, and Why

We deploy ACIE (Agentic Clinical Information Extraction) at University Medicine Essen, an on-premise agentic RAG system that extracts structured data from complete patient contexts spanning hundreds of documents and thousands of FHIR resources. Clinicians configure what to extract through typed schemas, with no developer involvement.

Key findings from our deployment on one of the largest clinical FHIR repositories in Europe (~2B resources, 1.7M patients):

• Across 7,326 clinician judgments in a retrospective lymphoma registry study, physicians accepted 96.5% of extractions, with per-type acceptance ranging from 80% to 99%
• We quantify the metadata gap in real clinical data: 56.5% of documents carry timestamps outside their encounter period, standard document-type codes cover only 2.4% of documents, and a third of documents are duplicates
• We trace how these data quality failures shaped architectural decisions, from bypassing encounter-based scoping to using agentic search over fixed retrieval pipelines

The system runs entirely on-premise (Qwen 3.6 35B on 4xH100), with every extracted value grounded in cited source passages for clinician verification.

We'd love to hear from others working on clinical IE in production, especially around data quality challenges in real hospital systems.