DgPg: Diagnosis Progression Transformer

DgPg ("Diagnosis Progression") is a compact (~2.5 MB) character-level, GPT-style transformer that learns representative ordering patterns of ICD-10-CM diagnostic (DX) codes within single-visit medical records. Given a "seed" diagnosis code, DgPg generates the codes most likely to precede or follow it, which can be rendered as graph-based visual summaries of an organization's coding practices.

DgPg was introduced in the paper Visualizing Medical Coding Practices Using Transformer Models (Hobson & Huang, ICPRAM 2025), from the Department of Electrical Engineering and Computer Science at the University of Tennessee, Knoxville.

⚠️ DgPg is an organizational-analytics tool, not a clinical or coding-assistance model. It is not intended to improve medical coding, predict diagnoses, or support clinical decisions. Its purpose is to let hospital administrators, lead physicians, and other stakeholders see how coding is actually practiced in a given corpus of records.

Model Details

Model type	Decoder-only transformer (modified nanoGPT), character-level
Task	Next-character prediction over 4-character-normalized ICD-10-CM code sequences
Parameters	1.3 M (recommended configuration, T1.3M)
Architecture	3 layers, 3 attention heads, 192 embedding dims
Precision	float16
Model size on disk	~2.5 MB
Training hardware	Single NVIDIA T4 (Google Colab)
Training time / cost	1,092 s (18 min), ≈ $0.05 per model
License	Paper published under CC BY-NC-ND 4.0
Authors	Tanner Hobson, Jian Huang (University of Tennessee, Knoxville)

Base training configuration

learning_rate  = 1e-3     dropout   = 0.2
lr_decay_iters = 30000    min_lr    = 1e-4
beta2          = 0.99     max_iters = 30000
batch_size     = 256      n_layer   = 3
n_head         = 3        n_embd    = 192

How It Works

ICD-10-CM codes are inherently hierarchical: the first character encodes the top level of the hierarchy, the second character the next level, and so on (e.g., S → S5 → S52 → S52.5 traces an injury from "injury/poisoning" all the way down to a specific radius fracture). DgPg exploits this by modeling code sequences character by character rather than treating each code as an atomic token.

• Every DX code is normalized to 4 characters (truncated and/or padded with hyphens).
• Records are converted to training sequences of consecutive code pairs, e.g. the record N12,Q610,N179 becomes •----N12-, •N12-Q610, •Q610N179, •N179----.
• At inference, the model is used iteratively: given a seed code, it predicts the next code one character at a time, naturally respecting the coding hierarchy.

Compared to a traditional code-level transition matrix (12,309 × 12,309 codes ≈ 290 MB in float16), the character-level formulation is >100× smaller and generalizes across code hierarchies rather than memorizing pairs.

Visualization use case

The authors pair the model with a generative graph-building algorithm: starting from a seed code, a breadth-first search exhaustively scores candidate following (or preceding) codes by perplexity, keeps the top k = 3 at each step, and recurses to depth n = 3. The result is a left-to-right graph of the most representative code orderings around a diagnosis (e.g., what typically precedes and follows a sepsis code A41.9), generated in ~2 minutes per graph.

Training Data

Four model variants were trained, one per dataset plus a combined corpus:

Source	Dataset	# Records	# Patients (weighted)	Notes
MIT	MIMIC-IV	430,812	430,812	Beth Israel Deaconess Medical Center; ICD-10-CM, truncated to 4 chars
CDC	NHDS	756,953	111,514,788	1996–2010; ICD-9-CM converted to ICD-10-CM via the NBER crosswalk
CDC	NHCS	123,565	33,595,765	2020 survey
—	Combined	1,277,400	145,507,435	Union of the above

CDC survey records carry a frequency weight w; each record is emitted w times during training. Only ordered DX-code sequences are required — the framework applies to any corpus with ordered diagnosis codes.

Note on data access: MIMIC-IV is a restricted-access dataset requiring credentialed access via PhysioNet. No patient-level data is contained in the model weights beyond learned sequence statistics, but users retraining DgPg must obtain the datasets under their respective terms.

Evaluation

Performance is measured as fixed-length sequence perplexity (PERF) on the second code of a two-code sequence, averaged over 10 K validation samples.

Model size ablation

Model	Layers	Heads	Embd	Size	Loss
T11M	6	6	384	20 MB	1.16
T1.3M ★	3	3	192	2.5 MB	1.17
T230K	2	2	96	440 KB	1.19
T30K	1	1	48	58 KB	1.25

T1.3M matches the 10×-larger T11M and is the recommended configuration. Model sizes were capped at 10% of the equivalent transition-matrix size to ensure generalizable (not memorized) patterns.

Cross-dataset transferability

Evaluating each trained model against each dataset shows that models perform best on their own training distribution — evidence that coding practices are provider- and scenario-specific. The Combined model performs well across all constituent datasets; the NHDS-only model transfers worst, indicating NHDS coding patterns are the most dataset-unique. MIMIC and NHCS models are the most generalizable.

Intended Uses

Direct intended uses

• Visualizing an organization's representative diagnostic-coding patterns (which codes typically precede/follow a given code; which codes tend to be coded first or last in a visit).
• Comparing coding practices across institutions, units, or clinical scenarios.
• Scoring how representative a given code sequence is relative to a corpus (classification use case).
• Operational insight for administrators: quality control, process improvement, burnout-related workflow analysis.

Out-of-scope uses

• ❌ Automated medical coding or coding suggestions for billing.
• ❌ Clinical diagnosis, prediction, or decision support.
• ❌ Auditing individual coders or labeling any coding practice as "good" or "bad" — the model summarizes shared practice; it does not evaluate deviations.
• ❌ Any use that would alter coding to affect claim payment. Modifying codes to increase reimbursement is fraudulent; any coding revision requires proper documentation.

Limitations

• 4-character truncation. For cross-dataset comparability, all codes are normalized to 4 characters, discarding finer ICD-10-CM specificity (the framework itself is not limited to 4 characters).
• ICD-9 → ICD-10 crosswalk noise. NHDS codes were converted via the NBER crosswalk, which is not lossless.
• Single-visit scope. DgPg models intra-visit code ordering only, not longitudinal, multi-visit patient trajectories.
• Dataset-specific patterns. A model trained on one provider's records does not transfer reliably to another's — this is by design, but means checkpoints are not general-purpose.
• Hallucination. As a generative model, DgPg can produce plausible-looking but unrepresentative sequences; the authors flag hallucination analysis as future work.
• DX codes only. Procedure and other medical code types are not yet covered.

Citation

@inproceedings{hobson2025dgpg,
  author    = {Tanner Hobson and Jian Huang},
  title     = {Visualizing Medical Coding Practices Using Transformer Models},
  booktitle = {Proceedings of the 14th International Conference on Pattern
               Recognition Applications and Methods (ICPRAM 2025)},
  pages     = {725--732},
  year      = {2025},
  publisher = {SCITEPRESS},
  doi       = {10.5220/0013257800003905},
  isbn      = {978-989-758-730-6}
}

Acknowledgments

Built on nanoGPT by Andrej Karpathy. Data courtesy of MIT-LCP (MIMIC-IV) and the US CDC National Center for Health Statistics (NHDS, NHCS).