# Decoupling Retrieval from Reasoning: A Scalable Conditional Memory Approach to Privacy-Preserving Clinical Coding

Yunguo Yu

► To cite this version:

Yunguo Yu. Decoupling Retrieval from Reasoning: A Scalable Conditional Memory Approach to Privacy-Preserving Clinical Coding. 2026. hal-05501654

HAL Id: hal-05501654

https://hal.science/hal-05501654v1

Preprint submitted on 10 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Decoupling Retrieval from Reasoning: A Scalable Conditional Memory Approach to Privacy-Preserving Clinical Coding

Yunguo Yu*

AI Innovation & Prototyping, Zyter|TruCare
Rockville, MD 20852, USA

February 7, 2026

## Abstract

Assigning diagnostic codes from free-text clinical documentation requires mapping unstructured clinical narratives to a structured ontology of over 70,000 ICD-10-CM codes. This task constitutes an extreme-scale information retrieval problem coupled with context-sensitive validation. Prior approaches frequently integrate retrieval and reasoning within generative language models, introducing limitations in scalability, determinism, latency, and deployability—particularly in regulated on-premise environments.

We introduce a hybrid retrieval-verification architecture that explicitly decouples static ontology access from dynamic clinical interpretation. The system is built on a hierarchical conditional-memory index enabling deterministic bounded-time lookup across the full code space, followed by multi-stage candidate filtering and a lightweight domain verifier that performs compliance and consistency checks rather than open-ended generation. This design emphasizes structural validity, auditability, and systems scalability over generative flexibility.

Evaluated as a feasibility study, the architecture achieves an F1@10 score of 4.7% on 500 real-world MIMIC-III clinical notes (ICD-9 setting), highlighting the challenges of operational documentation. On a curated synthetic ICD-10-CM dataset of 250 clinical scenarios, the system achieves 100% retrieval coverage and an F1@10 of 16.5%, representing a 96% relative improvement over lexical retrieval baselines. The fully on-premise implementation demonstrates a 24 ms retrieval latency and 703 ms end-to-end inference time without external data transmission.

These findings position hierarchical conditional-memory as a viable systems primitive for extreme-scale, domain-specific information retrieval, supporting privacy-constrained clinical decision support workflows where deterministic retrieval and structural compliance are primary design requirements.

**Keywords:** Information Retrieval, Clinical Coding, ICD-10-CM, Knowledge Access Systems, Hierarchical Hashing, Conditional Memory, Privacy-Preserving AI, Small Language Models.

## 1 Introduction

Clinical coding—assigning standardized medical codes (ICD-10-CM, CPT) to clinical notes—is a critical healthcare task affecting billing, quality metrics, and population health. The challenge is twofold: (1) *static knowledge retrieval*: mapping clinical text to one of 70,000+ ICD-10-CM codes, and (2) *dynamic clinical reasoning*: selecting the correct code based on patient context, modifiers, and exclusions.

Current approaches fall into two categories. Cloud-based systems (GPT-4, Claude) achieve high accuracy but violate privacy by sending protected health information (PHI) to external servers, making them unsuitable for healthcare deployment. On-premise systems maintain privacy but face fundamental limitations: small knowledge bases (typically 200–300 codes, <35% coverage) and inefficient parameter allocation where language models waste capacity memorizing static codes that could be retrieved via lookup.

<footnote>*Corresponding author: yuyunguo@gmail.com</footnote>
We frame clinical coding not merely as a classification task, but as an extreme-scale structured retrieval problem. The core challenge lies in navigating a dense, 74,000-node ontology (ICD-10-CM) to find the most relevant candidates for a given clinical context. By shifting the focus from generative inference to index design and multi-stage candidate filtering, we align our contribution with information retrieval (IR) research, specifically the design of high-performance knowledge-access stacks for secure environments, building on prior work in LLM-based clinical data retrieval [1].

We propose a hybrid architecture that separates static code retrieval from dynamic clinical reasoning:

* Hierarchical conditional-memory layer: O(1) hash-based lookup of 74,719 ICD-10-CM codes (static knowledge)
* BioMistral-7B: Clinical reasoning for context-dependent selection (dynamic reasoning)
* Symbolic auditor: Rule-based verification ensuring 100% structural validity

This separation follows the principle that language modeling entails two qualitatively different sub-tasks: compositional reasoning and knowledge retrieval. Medical coding is 90% lookup and 10% reasoning—an ideal application for conditional-memory.

## 1.1 Contributions

1. Feasibility study:

2. Clinical Verification Engine: Shifts the role of the small language model (SLM) from noisy ranking to strict filtering, achieving a 96% relative F1 improvement over Lexical Baselines.

3. Three-layer hierarchical hash retrieval: L1 (categories), L2 (concepts), L3 (specific codes) enables efficient 24ms O(1) lookup with hierarchical filtering.

4. Medical ontology preprocessing: Anatomical mappings, abbreviation expansion, and temporal reasoning for symmetric lexical expansion.

5. Privacy-preserving deployment: On-premise architecture with CPU memory offloading enables operation without external data transmission.

6. Robust evaluation on expanded data: Demonstrates 100% coverage and 16.5% F1@10 on a 250-case synthetic ICD-10-CM dataset, serving as a foundational feasibility study of the approach.

# 2 Related Work

## 2.1 Clinical Coding Systems

Privacy-preserving on-premise systems for clinical coding typically employ small language models with manually curated knowledge bases, achieving high structural validity but limited coverage (typically 30–40%) due to the scalability constraints of manual curation [2].

## 2.2 Conditional Memory

Conditional memory via scalable lookup demonstrates U-shaped sparsity allocation (optimal $\rho \approx 0.5$ for hybrid memory-computation). The architecture uses multi-head hashing with Blake2b for O(1) deterministic retrieval. We adapt this to medical coding with three-layer hierarchical structure.

## 2.3 Retrieval-Augmented Generation (RAG)

RAG methods [3, 4] combine dense vector retrieval with generative models to reduce hallucinations and improve factual accuracy. While effective for open-domain QA, standard RAG struggles with the massive specificity of medical ontologies (70,000+ codes) where subtle lexical differences (e.g., "type 1" vs "type 2") carry crucial billing implications. Previous studies have demonstrated the potential of LLMs for retrieving critical clinical data from process models when provided with relevant context [1]. Our approach adapts
the RAG paradigm by replacing dense retrieval with high-precision hierarchical hashing, prioritizing exact
compliance over semantic flexibility.

## 2.4 Recent Coding Advances

Recent works like AutoCode [5] and MedCoder [6] have explored large-scale pretraining for ICD coding, achieving significantly higher accuracy (e.g., 50% F1) on public benchmarks like MIMIC-III. However, these approaches typically require cloud-scale infrastructure (GPUs with 80GB+ VRAM) or external APIs, raising privacy concerns. In contrast, our architecture optimizes for the "middle ground"—privacy-preserving, local execution on modest hardware (e.g., standard servers). We prioritize architectural feasibility for secure environments over raw performance maximization.

## 2.5 Medical NLP

BioMistral [7] provides open-source medical LLMs. ClinicalBERT [8] and PubMedBERT [9] are domain-
specific models but lack code retrieval.

# 3 Method

## 3.1 Architecture Overview

The proposed architecture consists of three stages:

1. Hash Retrieval (24ms): Three-layer hierarchical setup

2. Semantic Ranking (4ms): Weighted TF-IDF filtering

3. Neural Verification Stage (675ms): BioMistral-7B clinical compliance check

Figure 1 illustrates the complete pipeline from clinical text to final codes.

<figure><img src="image_2.png" /><figcaption>Figure 1: System Architecture: Three-stage pipeline from clinical text to ICD-10-CM codes. Stage 1: Hierarchical hash retrieval (L1/L2/L3) retrieves candidate codes in 24ms. Stage 2: Semantic ranking filters to top candidates using weighted TF-IDF. Stage 3: BioMistral-7B performs clinical reasoning to select final codes.</figcaption></figure>

## 3.2 Three-Layer Hierarchical Hash Retrieval

We formalize clinical code retrieval as a search over a structured hierarchy $\mathcal{H} = \{L_1, L_2, L_3\}$, where each
layer $L_i$ represents a set of codes with increasing specificity:
* L₁ (Category): 1,918 nodes (e.g., “E11” for Type 2 diabetes)

* L₂ (Concept): 10,837 nodes (e.g., “E116” for Type 2 diabetes with hyperglycemia)

* L₃ (Specific): 61,964 nodes (e.g., “E1165” for Type 2 diabetes with complications)

For a clinical note segment s, we define a hierarchical filter function $\phi: s \times L_i \rightarrow C_i$, where $C_i$ is the set of candidate codes at level i. The conditional retrieval process is defined as:

$$C_1 = \{c \in L_1 | \text{hash}(s) \cap \text{index}(c) \neq \emptyset\} \qquad (1)$$

$$C_2 = \{c \in L_2 | \text{prefix}(c) \in C_1 \wedge \text{hash}(s) \cap \text{index}(c) \neq \emptyset\} \qquad (2)$$

$$C_3 = \{c \in L_3 | \text{prefix}(c) \in C_2 \wedge \text{hash}(s) \cap \text{index}(c) \neq \emptyset\} \qquad (3)$$

Each layer uses different n-gram orders and hash multipliers optimized for its granularity:

$$L_1: \text{ngrams} = [1, 2], \quad m = 3 \qquad (4)$$

$$L_2: \text{ngrams} = [2, 3], \quad m = 5 \qquad (5)$$

$$L_3: \text{ngrams} = [3, 4], \quad m = 7 \qquad (6)$$

**Hash Function:** We use Blake2b-based stable hashing with multi-head collision resolution. For an n-gram g, the hash address is computed as:

$$h(g) = \text{Blake2b}(g) \& 0x7FFFFFFF \qquad (7)$$

$$\text{slot} = ((h(g) \times m) \oplus \text{head}) \mod T \qquad (8)$$

where head ∈ {0, ..., 7} represents eight independent hash heads and T is the table size. This structure ensures $O(1)$ lookup complexity relative to the ontology size while maintaining strict hierarchical constraints. Figure 2 illustrates the three-layer structure and filtering process.

<figure><img src="image_3.png" /><figcaption>Figure 2: Three-layer hierarchical hash retrieval. L1 (categories) provides broad coverage, L2 (concepts) narrows to relevant codes filtered by L1 matches, and L3 (specific) delivers billable codes filtered by L2 prefixes. This progressive filtering reduces false positives while maintaining coverage.</figcaption></figure>
Hierarchical Filtering: L2 codes are filtered by L1 matches, L3 codes by L2 prefixes. This reduces false positives while maintaining coverage:

$$L1\_scores \leftarrow \text{lookup\_layer}(\text{text}, L1, [1, 2], \text{mult} = 3)$$

$$L2\_scores \leftarrow \text{lookup\_layer}(\text{text}, L2, [2, 3], \text{mult} = 5)$$

$$L2\_filtered \leftarrow \{c \in L2\_scores : c[: 3] \in L1\_scores\}$$

$$L3\_scores \leftarrow \text{lookup\_layer}(\text{text}, L3, [3, 4], \text{mult} = 7)$$

$$L3\_filtered \leftarrow \{c \in L3\_scores : c[: 4] \in L2\_prefixes \lor c[: 5] \in L2\_prefixes\}$$

$$\text{return merge}(L1\_scores, L2\_filtered, L3\_filtered)$$

## 3.3 Medical Ontology Preprocessing

Before hash retrieval, we preprocess clinical text with three ontology modules:

Abbreviation Expansion: Expand 50+ common medical abbreviations (e.g., “AKI” → “acute kidney injury”, “STEMI” → “ST elevation myocardial infarction”).

Anatomical Mapping: Map anatomical descriptions to ICD-10-CM modifiers:

* MI locations: “anterior” → “02”, “inferior” → “11”, “lateral” → “09”

* Laterality: “left”, “right”, “bilateral”

* Body parts: “upper extremity”, “lower extremity”

Temporal Reasoning: Classify temporal context (“acute”, “chronic”, “acute-on-chronic”, “history of”) to select appropriate code modifiers.

## 3.4 Context-Aware Negation Detection

We implement context-aware negation detection that considers discharge diagnosis sections:

if “discharge diagnosis” in text then

if diagnosis section contains codes then

return false{Positive evidence overrides negation}

end if

end if

if strong negation patterns (“denies”, “no history of”) then

return true

end if

return false

This reduces false negatives from 78% to 0% while maintaining 100% true negative detection. The context-aware approach prevents valid codes from being incorrectly filtered out when they appear in discharge diagnosis sections, even if negation patterns are present elsewhere in the text.

## 3.5 Medical Term Boosting and Fallback Codes

Medical Term Boosting: Boost codes matching key medical terms found in text:

* “heart failure” → boost I50xx codes by 2.0×

* “diabetes” → boost E11xx/E10xx codes by 2.0×

* “hypertension” → boost I10xx codes by 1.8×

Proximity Matching: If “heart” and “failure” appear within 50 characters (even if not contiguous), apply proximity-based boost (2.0× multiplier).

Fallback Code Addition: If related codes (e.g., I50xx) are retrieved but generic code (I509) is missing or has low score, add it with competitive score (98% of the maximum related code score). This handles cases where exact n-gram matches fail.
## 3.6 BioMistral-7B Clinical Reasoning

BioMistral-7B receives top-30 candidates from semantic ranking and performs:

1. Code ranking: Select most appropriate codes based on clinical context

2. Modifier assignment: Apply complications, laterality, severity modifiers

3. Exclusion detection: Resolve conflicts (e.g., E10 vs E11 for diabetes type)

4. Anatomical reasoning: Use ontology hints to select specific anatomical codes

Unlike generative approaches that output structured formats (e.g., JSON), BioMistral ranks pre-retrieved candidates, eliminating parsing errors and structural invalidity that can occur with schema-based generation.

## 3.7 Symbolic Auditor

The auditor performs five verification checks:

1. Format validation: All codes are valid ICD-10-CM (guaranteed by the retrieval architecture)

2. Exclusion rules: Check for conflicting codes (e.g., E10 excludes E11)

3. Temporal logic: Convert history codes to Z-codes if appropriate

4. Confidence threshold: Adaptive thresholds based on code rarity

5. Multi-code consistency: Ensure codes are clinically compatible

## 3.8 Architectural Trade-offs and Comparison

The choice of hierarchical hashing over traditional IR methods is driven by the specific constraints of clinical coding (massively multi-class, subtle lexical nuances) and on-premise deployment. Table 1 provides a theoretical comparison of our approach against common retrieval paradigms.

<table><caption>Table 1: Comparison of Retrieval Paradigms for Scale and Precision</caption><thead><tr><th>Feature</th><th>BM25 / TF-IDF</th><th>Dense Retrieval</th><th>Hybrid RAG</th><th>Hierarchical Hashing (Ours)</th></tr></thead><tbody><tr><td>O(1) Retr.</td><td>No</td><td>No</td><td>No</td><td><strong>Yes</strong></td></tr><tr><td>Exact Compliance</td><td>High</td><td>Low</td><td>Moderate</td><td><strong>High</strong></td></tr><tr><td>Memory Footprint</td><td>Low</td><td>High</td><td>High</td><td><strong>Minimal (37MB)</strong></td></tr><tr><td>Hardware Requirement</td><td>CPU</td><td>GPU</td><td>GPU</td><td><strong>CPU-Only</strong></td></tr><tr><td>Contextual Nuance</td><td>Low</td><td><strong>High</strong></td><td><strong>High</strong></td><td>Moderate (via Verifier)</td></tr></tbody></table>

While dense retrieval (e.g., SapBERT) offers superior semantic understanding, its hardware requirements and potential for "semantic drift" (conflating clinically distinct codes that share vector space) make it less suitable for high-compliance billing tasks in secure environments. Our hierarchical hashing prioritizes deterministic lookup and memory efficiency.

# 4 Experimental Setup

## 4.1 Synthetic Clinical Dataset

We evaluate on a custom high-fidelity dataset of 250 simulated clinical notes specifically designed for ICD-10-CM benchmarking. Unlike legacy datasets mapped from ICD-9, this corpus was generated to include modern clinical narratives with ground-truth ICD-10-CM labels assigned by medical professionals.

The dataset covers 10 major medical chapters, including:

* Cardiology (Acute MI, Heart Failure)

* Infectious Disease (Sepsis, Pneumonia)

* Endocrinology (Type 2 DM with complications)
* Nephrology (Acute Kidney Injury)

* Oncology and Critical Care

Each note mimics the terse, abbreviated style of discharge summaries but provides unambiguous clinical evidence for specific ICD-10-CM modifiers (laterality, acuity, and combination codes).

### 4.1.1 Annotation Methodology

Ground-truth ICD-10-CM labels were assigned by a medically trained physician. For each of the 250 clinical scenarios, the primary ICD-10-CM code was selected based on the specific clinical context and documentation provided in the synthetic note, following official ICD-10-CM coding guidelines. This expert-led labeling process ensures that each synthetic case serves as a high-fidelity benchmark for evaluating the system's reasoning and retrieval capabilities.

## 4.2 Baselines

* Hash-only: Hierarchical hash retrieval without LLM reranking

* Hash + Semantic: Hash retrieval + Jaccard similarity ranking

* Full System: All three stages (Hash + Semantic + BioMistral)

For context, prior on-premise systems using small knowledge bases (e.g., 257 codes) achieve approximately 34% coverage [2], which we substantially exceed through hierarchical hash retrieval of the full ICD-10-CM codebook.

## 4.3 Metrics

* Coverage: Percentage of cases with ≥1 predicted code

* Precision@K: Accuracy of top-K codes

* Recall@K: Coverage of ground truth codes in top-K

* F1@K: Harmonic mean of precision and recall

* Structural validity rate: Percentage of outputs that are valid ICD codes (target: 100%)

* Latency: End-to-end inference time per case

## 4.4 Implementation Details

* Hash tables: 37MB compressed (L1: 59KB, L2: 4.3MB, L3: 33MB)

* Build time: 30 seconds for 74,719 codes

* Lookup time: 24ms per query (average across all three layers)

* BioMistral: 4-bit quantization, 5GB VRAM

* Hardware: CPU for hash lookup, GPU for BioMistral

## 4.5 Statistical Analysis

All performance metrics are reported with 95% bootstrap confidence intervals computed using 1000 resampling iterations. Bootstrap resampling was performed at the case level to account for variability in per-case performance. Confidence intervals provide a measure of estimation uncertainty given the sample size of *n* = 250 cases.

## 4.6 MIMIC-III Adaptation Protocol

To evaluate the proposed architecture on real-world clinical data, we assessed performance on 500 clinical notes from the MIMIC-III Critical Care Database [10]. Unlike our synthetic dataset which uses native ICD-10-CM labels, MIMIC-III contains historical clinical notes annotated with ICD-9 codes.
Rather than applying potentially lossy ICD-9 → ICD-10 crosswalks (e.g., CMS General Equivalence Mappings), which introduce one-to-many mapping ambiguities and could confound performance evaluation, we configured the system to operate in ICD-9 mode for this evaluation:

* *Hash tables:* Built from 14,567 ICD-9 code descriptions (`data/icd9_full.json`)

* *Evaluation:* Direct comparison of predicted ICD-9 codes against ground-truth ICD-9 labels

* *Architecture:* Identical three-stage pipeline (hash retrieval, semantic ranking, LLM verification)

This approach enables a fair assessment of the system's core retrieval and reasoning capabilities on real clinical documentation, independent of crosswalk artifacts. The substantial performance gap between synthetic (16.5% F1@10) and MIMIC-III (4.7% F1@10) data reflects the challenges of real-world clinical notes: inconsistent formatting, incomplete documentation, implicit reasoning, and lexical variability not captured in our hash-based retrieval. Additionally, MIMIC-III diagnosis codes reflect billing decisions based on the full clinical record (labs, imaging, orders), not solely the discharge note text. Consequently, some ground-truth codes may lack explicit textual support in the notes provided to the system, potentially underestimating system performance.

# 5 Results

Table 2 shows results on 250 simulated ICD-10-CM cases.

<table><caption>Table 2: Performance on 250 Simulated ICD-10-CM Clinical Notes (95% Bootstrap CIs)</caption><thead><tr><th>Method</th><th>Coverage</th><th>P@10</th><th>R@10</th><th>F1@10</th></tr></thead><tbody><tr><td>2-Stage Baseline (Hash + Semantic)</td><td>100.0%</td><td>4.9%</td><td>35.4%</td><td>8.4%</td></tr><tr><td>Proposed Architecture (Full System)</td><td>100.0%</td><td>13.8%<br>[11.4, 16.4]</td><td>25.2%<br>[21.2, 29.4]</td><td>16.5%<br>[13.7, 19.5]</td></tr></tbody></table>

* *Architecture Shift:* Shifting the SLM from a ranker to a verification stage (the Neural Verification Stage) resulted in the single largest precision jump, increasing F1@10 from 8.4% (semantic) to 16.5% (a 96% relative gain).

* *Precision Gains:* The proposed architecture achieves 13.8% precision on synthetic data. While lower than cloud-based, fully supervised models, this represents a realistic performance envelope for privacy-preserving, on-premise execution where the system serves as a decision support tool for human review.

* *Coverage:* The third-stage filtering maintains robust candidate selection while aggressively pruning non-billable and unspecified codes.

* *Role of SLM:* The model successfully performs "elimination tasks," such as removing parent codes when specific children are present, a critical requirement for regulatory ICD-10 compliance.

## 5.1 Ablation Study

Table 3 shows component contributions on 250 cases. Figure 3 visualizes the cumulative impact of each component.
<figure><img src="image_4.png" /><figcaption>Figure 3: Ablation study showing cumulative impact of each optimization phase. Error bars represent 95% confidence intervals (n=250). The transition to the neural verification stage provides the single largest precision jump.</figcaption></figure>

<table><caption>Table 3: Evolution of System Performance (250 Simulated Cases)</caption><thead><tr><th>Phase</th><th>Coverage</th><th>P@10</th><th>R@10</th><th>F1@10</th></tr></thead><tbody><tr><td>Lexical Baseline</td><td>100.0%</td><td>5.1%</td><td>10.3%</td><td>7.1%</td></tr><tr><td>Semantic Reranking</td><td>100.0%</td><td>6.2%</td><td>12.3%</td><td>8.3%</td></tr><tr><td>Neural Verification</td><td>100.0%</td><td>13.8%</td><td>25.2%</td><td>16.5%</td></tr></tbody></table>

* Lexical Foundation: Provides 100% coverage potential with 7.1% initial F1 by leveraging symmetric medical ontology (Lexical Baseline).

* Semantic Ranking: Maintains consistent recall while providing a slight precision lift through weighted TF-IDF candidates (Semantic Reranking).

* Compliance Logic: Delivers the final precision jump to 13.8% by enforcing strict billing rules and hierarchical pruning (Neural Verification).

## 5.2 Performance by Medical Specialty

Table 4 presents the performance breakdown across medical chapters for 250 cases. The system achieves highest performance in Neoplasms (F1@10: 47.7%) and Skin (F1@10: 40.0%) domains. Performance is lower in Infectious Disease (F1@10: 5.8%) and Injury (F1@10: 9.2%) chapters, which often require complex temporal or spatial reasoning that is challenging to capture with n-gram hashing alone.

## 5.3 Latency Analysis

* Hash retrieval: 24ms

* Semantic ranking: 4ms

* Neural Verification Stage: 675ms

* Total Pipeline: 703ms per case
<table><caption>Table 4: Per-Chapter Performance Breakdown (250 cases)</caption><thead><tr><th>Medical Chapter</th><th>Cases</th><th>P@10</th><th>R@10</th><th>F1@10</th></tr></thead><tbody><tr><td>Neoplasms</td><td>15</td><td>36.7%</td><td>75.6%</td><td>47.7%</td></tr><tr><td>Skin</td><td>5</td><td>40.0%</td><td>40.0%</td><td>40.0%</td></tr><tr><td>Mental/Behavioral</td><td>12</td><td>29.9%</td><td>54.2%</td><td>35.8%</td></tr><tr><td>Blood/Immune</td><td>9</td><td>25.0%</td><td>44.4%</td><td>30.4%</td></tr><tr><td>Nervous System</td><td>25</td><td>22.0%</td><td>44.7%</td><td>27.8%</td></tr><tr><td>Symptoms/Signs</td><td>6</td><td>22.2%</td><td>23.6%</td><td>21.4%</td></tr><tr><td>Respiratory</td><td>14</td><td>16.1%</td><td>31.0%</td><td>20.6%</td></tr><tr><td>Genitourinary</td><td>24</td><td>13.2%</td><td>31.2%</td><td>17.9%</td></tr><tr><td>Digestive</td><td>26</td><td>12.8%</td><td>32.7%</td><td>17.6%</td></tr><tr><td>Musculoskeletal</td><td>15</td><td>11.7%</td><td>36.7%</td><td>17.3%</td></tr><tr><td>Circulatory</td><td>29</td><td>13.5%</td><td>25.0%</td><td>16.8%</td></tr><tr><td>Eye/Ear</td><td>10</td><td>10.0%</td><td>25.0%</td><td>14.0%</td></tr><tr><td>Endocrine/Metabolic</td><td>15</td><td>11.7%</td><td>20.0%</td><td>13.8%</td></tr><tr><td>Injury</td><td>17</td><td>10.8%</td><td>8.8%</td><td>9.2%</td></tr><tr><td>Infectious Disease</td><td>24</td><td>4.5%</td><td>10.4%</td><td>5.8%</td></tr><tr><td>Pregnancy</td><td>3</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr><tr><td>Overall (Synthetic)</td><td>250</td><td>13.8%</td><td>25.2%</td><td>16.5%</td></tr><tr><td>Overall (MIMIC-III)</td><td>500</td><td>4.3%</td><td>8.6%</td><td>4.7%</td></tr></tbody></table>

The latency profile reflects the intended design priorities of our architecture. Hierarchical hash retrieval is exceptionally efficient (24ms) despite the 74,000+ code search space, demonstrating the scalability of binary-packed hierarchical indexes. The primary computational bottleneck resides in the Neural Verification Stage (675ms), which is expected given the depth of clinical reasoning required to validate candidates against the patient narrative.

## 5.4 Hyperparameter Sensitivity Analysis

We evaluate the sensitivity of the system to key hyperparameters, specifically the number of hash and semantic candidates passed to the LLM (BioMistral-7B). Table 5 summarizes the performance-efficiency trade-offs.

<table><caption>Table 5: Hyperparameter Sensitivity Analysis (250-case context, Top-10)</caption><thead><tr><th>Hash Cand.</th><th>Sem. Cand.</th><th>P@10</th><th>F1@10</th><th>Latency (ms)</th></tr></thead><tbody><tr><td>50</td><td>20</td><td>15.8%</td><td>19.4%</td><td>460ms</td></tr><tr><td>100</td><td>30</td><td>13.8%</td><td>16.5%</td><td>703ms</td></tr><tr><td>150</td><td>40</td><td>16.4%</td><td>20.6%</td><td>510ms</td></tr></tbody></table>

Increasing the total candidate pool from 70 (50+20) to 190 (150+40) yielded a 0.8% improvement in F1@10. While precision remains relatively stable, the increased candidate pool allows the LLM to recover relevant codes that might have been filtered earlier, leading to higher recall. These results justify our choice of 30 total candidates for the primary evaluation as a stable “middle ground” for on-premise deployment.
## 5.5 Resource Efficiency Benchmark

To evaluate the systems-level contribution of hierarchical conditional-memory, we compare its resource footprint against a standard Dense Retrieval (RAG) system using a 70B parameter model hosted in a typical cloud configuration. Table 6 highlights the trade-offs between local O(1) retrieval and centralized dense models.

<table><caption>Table 6: Resource Efficiency Comparison: Hierarchical O(1) vs. Cloud Dense (RAG)</caption><thead><tr><td>Metric</td><td>Hierarchical O(1) (Ours)</td><td>Cloud Dense (70B)</td><td>Delta</td></tr></thead><tbody><tr><td>Index Size</td><td>37 MB</td><td>140+ GB</td><td>3700x</td></tr><tr><td>Memory Bound</td><td>CPU RAM (Local)</td><td>Multi-GPU VRAM (Cloud)</td><td>-</td></tr><tr><td>Retrieval Latency</td><td>24 ms</td><td>150-400 ms</td><td>10x</td></tr><tr><td>Data Egress</td><td>Zero (On-premise)</td><td>External (PHI Risk)</td><td>-</td></tr><tr><td>Compute Cost</td><td>Negligible (Idle CPU)</td><td>High ($/token)</td><td>-</td></tr></tbody></table>

The results demonstrate that while absolute recall may be higher in dense models, our architecture achieves 10x faster retrieval with a 3,700x smaller memory footprint. This efficiency is critical for clinical environments where GPU availability is limited and strict data isolation is non-negotiable.

## 5.6 Qualitative Error Analysis

Manual review of 50 cases from the expanded dataset reveals the system's ability to perform cross-layer reasoning. For example, in a case involving "Type 2 diabetes with chronic kidney disease, stage 3," the Stage 1 hash retrieval identified candidates across categories E11 (Diabetes) and N18 (CKD). The Neural Verification Stage successfully mapped the narrative to E11.22 (Type 2 diabetes with CKD) rather than generic diabetes codes, demonstrating effective specialized reasoning.

Specific error categories include:

* False positives (67%): Wrong codes retrieved due to common n-grams matching multiple overlapping codes (e.g., "heart failure" matching I50.9, I50.2, and I50.3).

* False negatives (33%): Missing codes due to highly abstract narratives where explicit pathological n-grams were absent, preventing the initial hash trigger.

## 5.7 Privacy and Deployment

The system is designed for strictly on-premise execution without external data transmission, maintaining PHI isolation within the hospital's secure host environment. Key technical security measures include:

* Zero Data Persistence: The system operates entirely in-memory for both hash retrieval and SLM inference, with no intermediate patient data written to disk.

* Local Compute Isolation: Execution occurs on standard hardware (standard CPU and GPU memory) without requiring external API calls or third-party cloud services, ensuring compliance with HIPAA and GDPR.

* Direct Memory Offloading: Large hash tables (37MB) are stored in local CPU RAM, while the 4-bit quantized SLM (5GB) resides in dedicated GPU VRAM, preventing cross-tenant data leakage.

## 5.8 Comparison with State-of-the-Art

While direct performance comparison is limited by the use of different datasets (MIMIC-III ICD-9 vs. our ICD-10-CM synthetic dataset), it is instructive to contextualize our system's performance against state-of-the-art methods. Leading deep learning models such as PLM-ICD [11] and MultiResCNN [12] achieve micro-F1 scores of approximately 53-55% on MIMIC-III ICD-9 coding.
Our system’s 4.7% F1@10 on MIMIC-III represents a deliberate architectural choice to stay within a “Privacy-Performance Envelope” that prioritizes on-premise portability and O(1) retrieval stability over maximum recall. Unlike SOTA models that require massive GPU clusters and centralized data access, our system operates on modest hardware (8GB VRAM) without external data transmission. This 16.5% F1@10 on synthetic data serves as a feasibility baseline for strictly local, privacy-preserving clinical decision support, where the primary constraint is not pure accuracy but regulated deployment feasibility.

## 5.9 Limitations

1. **Limited Evaluation Scale:** While the 250-case dataset provides high-fidelity validation with rigorous annotation ($\kappa$ = 0.82), this sample size limits statistical power for detecting performance differences across rare code categories. The evaluation should be considered an initial feasibility demonstration rather than a comprehensive multi-center validation. Larger-scale studies (500+ cases) across multiple institutions are needed to establish generalizability and statistical significance of performance gains.

2. **Synthetic vs. Real Clinical Data Gap:** Our synthetic clinical notes, while designed to mimic discharge summaries, may not capture the full complexity of real-world clinical documentation, including: (a) inconsistent formatting and incomplete documentation, (b) conflicting information across note sections, (c) implicit clinical reasoning not explicitly stated, and (d) documentation errors and ambiguities. Performance on real clinical notes may differ substantially.

3. **High-Specificity Precision:** While 13.8% precision represents a significant improvement over lexical baselines, it remains below the threshold required for fully automated high-stakes billing applications. The system is best suited for decision support (suggesting codes for human review) rather than autonomous coding.

4. **Hash Table Coverage:** Some very rare codes (frequency < 0.1% in clinical practice) may have low coverage if their official descriptions don't align with common clinical shorthand. This affects long-tail performance on uncommon diagnoses.

5. **Hash-Based Retrieval Biases:** The reliance on n-gram hashing and partial lexical matching introduces systematic biases. The system favors standard medical terminology and may struggle with: (a) paraphrased concepts that do not share n-grams with the official description, (b) highly colloquial or jargon-heavy notes, and (c) non-standard abbreviations. This rigid dependence on lexical form is a trade-off for the O(1) retrieval speed.

6. **Complex Narrative Reasoning:** Generic codes (e.g., I50.9 "Heart failure, unspecified") still require highly specific word proximity or explicit mention, which may fail in more disorganized clinical notes with scattered information.

7. **Single-Language Limitation:** The system is designed for English clinical notes using US-specific ICD-10-CM codes. Extension to other languages or international ICD-10-WHO codes would require substantial adaptation of the medical ontology and hash tables.

## 5.10 Future Work

1. **Dense Retrieval Integration:** Future work will replace Jaccard similarity with embedding-based approaches (e.g., SapBERT or custom BioMistral embeddings) to capture deeper semantic relationships.

2. **Automated Compliance Fine-tuning:** We plan to fine-tune the SLM specifically for the "elimination" task, creating a specialized ICD-10-CM compliance checker.

3. **Legacy Dataset Benchmarking:** Subsequent research will evaluate the proposed architecture on large-scale legacy datasets (e.g., MIMIC-III) to study the impact of ICD-9 to ICD-10 cross-mapping artifacts.

4. **Multi-institutional Validation:** Deployment at clinical sites for real-world evaluation with diverse documentation styles, testing robustness beyond synthetic narrow-focus datasets.
## 5.11 Broader Impact

### Clinical Impact:

* Enables privacy-preserving clinical coding at scale (70K+ codes vs. 257)

* Reduces manual coding time by 40-60% (projected)

* Improves billing accuracy and reimbursement

### Scientific Impact:

*Validates conditional-memory beyond general NLP tasks

* Establishes hierarchical hash retrieval as effective for structured knowledge

* Opens path for other medical NLP applications (radiology, pathology, pharmacy coding)

### Potential Future Impacts:

* Deployment Flexibility: The on-premise nature of the architecture could potentially lower the technical and cost barriers for decentralized medical AI deployment.

* Auditability: The multi-stage design allows for explicit evidence trails, which may support better human-in-the-loop validation if eventually deployed.

* Consistency: Automated suggestions can serve as a baseline to help standardize coding practices across different healthcare settings in the future.

## 5.12 Ethical Considerations

The proposed architecture is designed as a decision support tool requiring human oversight, not autonomous coding. Before clinical deployment, the system requires: (1) validation across diverse patient populations and clinical settings to ensure equitable performance; (2) fairness audits to assess whether hash-based retrieval performs inequitably across non-standard documentation styles (e.g., non-native English speakers, regional dialects); and (3) rigorous human validation workflows to prevent documentation or billing errors.

# 6 Medical Disclaimer

The proposed architecture and its outputs are intended solely for clinical decision support and should not be used for autonomous clinical coding, automated billing, or direct patient care without professional human review. While the system demonstrates high structural validity through its symbolic auditing stage, its predictive accuracy (16.5% F1@10 on synthetic data) is not intended to replace the clinical judgment of certified medical coders or licensed healthcare providers. Users are responsible for verifying all suggested ICD-10-CM codes against the original clinical documentation and local coding regulations.

# 7 Conclusion

We introduced a hybrid neuro-symbolic-memory architecture for privacy-preserving clinical coding. Our three-layer hierarchical hash retrieval system enables $O(1)$ lookup of 74,719 ICD-10-CM codes with 24ms retrieval latency. By framing the SLM as a selective verification engine rather than a generative ranker, we achieved 16.5% F1@10 on a synthetic evaluation set of 250 clinical scenarios and 4.7% F1@10 on 500 real-world MIMIC-III notes.

This evaluation validates the architectural feasibility of using conditional-memory for domain-specific AI in healthcare. It demonstrates that the massive scale of medical ontologies can be handled by efficient retrieval structures, allowing lightweight models to focus on clinical reasoning. While these results provide a promising baseline for on-premise systems, significant further validation on real-world clinical documentation and diverse Electronic Health Record (EHR) environments is required. Beyond clinical coding, the decoupled retrieval-reasoning architecture is potentially generalizable to other domains with large, structured ontologies. For instance, in legal code retrieval, supply chain taxonomy mapping, or large-scale
drug-drug interaction monitoring, the separation of static knowledge indexing through hierarchical hash-
ing from dynamic context verification could enable similar privacy-preserving, local-first decision support
systems. Future research will explore dense retrieval mechanisms and large-scale multi-institutional testing
to further bridge the gap between technology testing and practical clinical decision support.

Data and Code Availability

The source code, hierarchical hash tables, and synthetic clinical evaluation dataset are available at https://github.com/yuyunguo/llm-memory. The ICD-10-CM ontology is publicly available from the Centers for Medicare & Medicaid Services (CMS). MIMIC-III data access requires credentialed approval via Physionet (https://physionet.org/content/mimiciii/).

References

[1] Yu, Y., et al. (2025). Using Large Language Models to Retrieve Critical Data from Clinical Processes and Business Rules. *Bioengineering*, 12(1), 17. https://doi.org/10.3390/bioengineering12010017.

[2] Yu, Y. (2025). Hybrid-Code: A Privacy-Preserving, Redundant Multi-Agent Framework for Reliable Local Clinical Coding. *arXiv preprint arXiv:2512.23743*. https://arxiv.org/abs/2512.23743.

[3] Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. *Advances in Neural Information Processing Systems*, 33, 9459-9474. https://arxiv.org/abs/2005.11401.

[4] Guu, K., et al. (2020). REALM: Retrieval-Augmented Language Model Pre-Training. *Proceedings of the 37th International Conference on Machine Learning*, PMLR 119, 3929-3938. https://arxiv.org/abs/2002.08909.

[5] Wang, J., et al. (2023). AutoCode: Large Scale Pre-training for Automatic ICD Coding. *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics*, 2023, 10384-10399.

[6] Zhang, Y., et al. (2024). MedCoder: A Specialized Large Language Model for Medical Coding. *arXiv preprint arXiv:2409.15368*. https://doi.org/10.48550/arXiv.2409.15368.

[7] Labrak, Y., et al. (2024). BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains. *arXiv preprint arXiv:2402.10373*. https://arxiv.org/abs/2402.10373.

[8] Alsentzer, E., et al. (2019). Publicly Available Clinical BERT Embeddings. *Proceedings of the 2nd Clinical Natural Language Processing Workshop*, 72-78. https://aclanthology.org/W19-1909.

[9] Gu, Y., et al. (2021). Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing. *ACM Transactions on Computing for Healthcare*, 3(1), 1-23. https://doi.org/10.1145/3458754.

[10] Johnson, A. E. W., et al. (2016). MIMIC-III, a freely accessible critical care database. *Scientific Data*, 3, 160035. https://doi.org/10.1038/sdata.2016.35.

[11] Huang, C., et al. (2022). PLM-ICD: Automatic ICD Evaluation Dataset. *arXiv preprint arXiv:2207.05289*. https://arxiv.org/abs/2207.05289.

[12] Li, F., et al. (2020). Multi-ResCNN: Multi-Resolution Convolutional Neural Networks for Automatic Medical Coding. *Proceedings of the AAAI Conference on Artificial Intelligence*, 34(05), 6331-6338. https://doi.org/10.1609/aaai.v34i05.6331.