medtech-8b
LoRA adapter for mlx-community/Qwen3-8B-bf16, distilled from DeepSeek-R1 reasoning traces for 5-class germline variant classification under ACMG/AMP guidelines.
The model reads a structured variant trace (gene, consequence, allele frequency, ClinVar review status) and returns a JSON object with classification, triggered_criteria, reasoning_trace, and confidence.
Results
| Split | Accuracy | High-confidence accuracy | High-conf coverage |
|---|---|---|---|
| Validation (n=98) | 89.80% | 88.89% | 64.3% |
| Holdout (n=45, never distilled) | 88.89% | 94.74% | 42.2% |
Comparators on the same validation split: rule-based baseline 75.51%, DeepSeek-R1 teacher 69.39%.
Pathogenic recall is 23/23 on validation and 7/7 on holdout. Full per-class precision/recall/F1, confusion matrices, and per-confidence breakdowns are in evaluation/summary.json of the accompanying release.
The holdout set is the honest number: the validation split was also used for checkpoint selection (best validation loss), so validation accuracy is mildly optimistic.
Usage
from mlx_lm import load, generate
model, tok = load(
"mlx-community/Qwen3-8B-bf16",
adapter_path="path/to/medtech-8b", # this repo
)
Prompts are built with the PromptFactory in the source repository (src/modeling/prompt_factory.py), which formats a variant trace against docs/ACMG_GUIDELINES_V1.txt.
Label-only inference (edge deployment): the JSON schema emits "classification" first, so generation can be truncated after the first value. Measured time-to-label on an M1 Max: 3.97s (vs 30.25s for the full reasoning trace).
Training
- Method: LoRA, rank 64, alpha 128, dropout 0.05, applied to 32 layers (~2.9% of parameters trainable)
- Data: 715 distilled examples (DeepSeek-R1 teacher, agreement-filtered against ClinVar, prompts rebuilt hint-free after a label-leakage fix) + 98-example validation split
- Schedule: cosine 2e-4 -> 1e-6, warmup 100, effective batch 16, max seq 2048, seed 42
- Early-stopped at ~iter 360; shipped checkpoint = iter 200 (best val loss 0.382), md5-verified
- Hardware: Apple M1 Max 64 GB, mlx-lm 0.31.3, ~8h50m wall clock
Full details in training_manifest.json and training_config.yaml.
Scope and limitations
- Narrow domain: ~99% BRCA1/BRCA2 (plus 10 MLH1 variants). Not evaluated on other genes.
- Research artifact. Not a medical device, not for clinical use.
- Benign and Likely Pathogenic classes have very small test supports (2 and 1 on validation); treat those per-class metrics as unstable.
- Reasoning traces agree with the teacher's triggered ACMG criteria at 0.78 mean Jaccard (validation). A 50-case blind community review of reasoning integrity is in progress (
evaluation/reasoning-review.csv).
License
Apache-2.0 (adapter). Base model Qwen3 is Apache-2.0. Training data derives from NCBI ClinVar (freely available US government data), gnomAD (public), and DeepSeek-R1 teacher outputs (MIT).
Quantized