medtech-4b
LoRA adapter for mlx-community/Qwen3-4B-bf16, distilled from DeepSeek-R1 reasoning traces for 5-class germline variant classification under ACMG/AMP guidelines. This is the edge-oriented sibling of medtech-8b: it trades ~3 points of validation accuracy for roughly half the memory and ~1.6x the generation speed.
The model reads a structured variant trace (gene, consequence, allele frequency, ClinVar review status) and returns a JSON object with classification, triggered_criteria, reasoning_trace, and confidence.
Results
| Split | Accuracy | High-confidence accuracy | High-conf coverage |
|---|---|---|---|
| Validation (n=98) | 86.73% | 85.48% | 63.3% |
| Holdout (n=45, never distilled) | 88.89% | 100% | 37.8% |
Comparators on the same validation split: rule-based baseline 75.51%, DeepSeek-R1 teacher 69.39%, medtech-8b 89.80%.
Pathogenic recall is 23/23 on validation and 6/7 on holdout. Full per-class precision/recall/F1, confusion matrices, and per-confidence breakdowns are in evaluation/summary.json of the accompanying release.
Notably, the 4B matches the 8B on holdout accuracy (88.89%) and exceeds the teacher by ~17 points on validation.
Edge profile (Apple M1 Max, 64 GB)
| Metric | medtech-4b | medtech-8b |
|---|---|---|
| Load time | 1.7s | 3.0s |
| Full reasoning generation | 21.0s @ 20.7 tok/s | 30.3s @ 13.1 tok/s |
| Time-to-label (truncated) | 2.4s | 3.97s |
| Peak memory | 9.46 GB | 18.01 GB |
Label-only inference: the JSON schema emits "classification" first, so generation can be truncated after the first value for fast label-only answers.
Usage
from mlx_lm import load, generate
model, tok = load(
"mlx-community/Qwen3-4B-bf16",
adapter_path="path/to/medtech-4b", # this repo
)
Prompts are built with the PromptFactory in the source repository (src/modeling/prompt_factory.py), which formats a variant trace against docs/ACMG_GUIDELINES_V1.txt.
Training
- Method: LoRA, rank 64, alpha 128, dropout 0.05, applied to 32 layers (2.92% of parameters trainable)
- Data: identical to medtech-8b (715 train / 98 validation, same recipe)
- Schedule: cosine 2e-4 -> 1e-6, warmup 100, effective batch 16 (batch 4 x accum 4), max seq 2048, seed 42
- Watchdog-stopped at the 450-iter hard cap; shipped checkpoint = iter 400. Best validation loss (0.375) was at iter 350, but checkpoints save every 200 iters, so iter 400 is the closest available checkpoint. See
training_manifest.json. - Hardware: Apple M1 Max 64 GB, mlx-lm 0.31.3, ~3h35m wall clock (peak training memory 23.7 GB)
Scope and limitations
- Narrow domain: ~99% BRCA1/BRCA2 (plus 10 MLH1 variants). Not evaluated on other genes.
- Research artifact. Not a medical device, not for clinical use.
- Benign and Likely Pathogenic classes have very small test supports; treat those per-class metrics as unstable.
- Reasoning traces agree with the teacher's triggered ACMG criteria at 0.81 mean Jaccard (validation), slightly higher than the 8B. A 50-case blind community review of reasoning integrity is in progress.
License
Apache-2.0 (adapter). Base model Qwen3 is Apache-2.0. Training data derives from NCBI ClinVar (freely available US government data), gnomAD (public), and DeepSeek-R1 teacher outputs (MIT).
Quantized