Dronam-Vision-V1 — musculoskeletal radiograph → structured read

Dronam-Vision-V1 reads one musculoskeletal radiograph and returns body region, view, fracture present and orthopaedic hardware present as JSON. It is two models working together:

1. A LoRA adapter (rank 16, vision encoder + decoder) on Gemma-4-E2B-it — reads region, view and hardware.
2. A 12M-parameter EfficientNet-B3 classifier, trained separately — decides fracture_present.

Why two models. The adapter alone learnt region and view well but did not improve fracture detection over its own untrained base (balanced accuracy 65.8% vs 65.3%): the fracture decision is one token in a ~40-token answer and the language-model loss barely sees it. A dedicated classifier fixed this. Use both together (see below); the adapter's own fracture_present should be ignored.

It is a drafting aid, not a diagnostic device. It flags images for a surgeon to look at; it does not diagnose.

Results on held-out data

816 test radiographs from FracAtlas (Abedeen et al., Sci Data 2023, CC BY 4.0), a stratified split never seen in training, one RTX 5060 Ti 16 GB / CPU. 95% bootstrap CIs. Threshold chosen on the validation set for ≈90% sensitivity; scored on the exact ONNX artefact this repo ships.

Model	Fracture sensitivity	Fracture specificity	Balanced accuracy	Region exact	View exact
Dronam-Vision-V1 (hybrid)	93.0% (88.5–97.0)	78.6% (75.5–81.6)	85.8%	92.3%	87.7%
MedGemma-4B-it	59.4%	84.1%	71.8%	66.7%	30.3%
Qwen3.5-9B	44.8%	90.5%	67.6%	65.3%	77.2%
Dronam-Vision, adapter only	56.6%	75.0%	65.8%	92.3%	87.7%
Gemma-4-E2B (base)	81.1%	49.5%	65.3%	55.1%	38.1%
Gemma-4-12B (QAT)	72.0%	48.9%	60.5%	78.4%	77.2%
Gemma-4-E4B	53.8%	66.7%	60.3%	59.6%	58.7%

Fracture classifier alone: test AUC 0.942 (95% CI 0.919–0.963). Paired vs the Gemma-4-E2B base (exact McNemar on fracture correctness): p < 0.001.

Earlier model withdrawn. A previous dronam-vision-3b (Qwen2.5-VL-3B) was trained and tested on 7 synthetic drawings with the diagnosis printed on the image; its base model is also licensed for non-commercial research only. Scored here on the same real radiographs it calls every image fractured (sensitivity 100%, specificity 0%) — i.e. it learnt nothing transferable. It has been deleted from this account; do not use it.

How to use

Region / view / hardware — send the image and radiograph_instruction.txt to the adapter, ideally constrained to radiograph.schema.json:

from unsloth import FastVisionModel
model, tok = FastVisionModel.from_pretrained("amitaddi/dronam-vision-v1", load_in_4bit=True)
instruction = open("radiograph_instruction.txt").read()
msgs = [{"role": "user", "content": [{"type": "image", "image": image}, {"type": "text", "text": instruction}]}]

Fracture score — run fracture_classifier/model.onnx (or .safetensors) with fracture_classifier/fracture.py:

import fracture_classifier.fracture as fracture   # onnxruntime, CPU is enough
result = fracture.predict(data_uri)                # {'p_fracture': 0.62, 'fracture_flag': True, 'threshold': 0.0086}

p_fracture is not a calibrated probability — it is a ranking score; only the flag (threshold fixed on validation data for ~90% sensitivity) should be shown as a decision. A merged 4-bit GGUF (adapter + projector) is in amitaddi/dronam-vision-v1-GGUF, which bundles the same classifier files.

Training

• Data: FracAtlas, 4,083 radiographs (hand/leg/hip/shoulder), CC BY 4.0. Stratified 70/10/20 split by fracture × region (2,861 / 406 / 816). Fractured images oversampled ×2 in training (17.7% → ~30%).
• Adapter: QLoRA 4-bit, LoRA r = 16 on vision encoder + decoder attention/MLP; 1 epoch, lr 2e-4 cosine, effective batch 8, loss on the JSON answer only.
• Classifier: EfficientNet-B3 (timm, ImageNet-pretrained), 512×512, random-crop/flip/rotation/colour jitter, class-weighted BCE, AdamW one-cycle (peak 2.5e-4), batch 12, 15 epochs, best-epoch selection by validation AUC, horizontal-flip test-time augmentation.

Limitations

FracAtlas covers four regions from three hospitals in one country; it has no patient identifier, so a patient could appear in more than one split. The classifier's score is uncalibrated and its threshold is tuned for sensitivity, so it over-flags normal images (specificity 78.6%) by design — that is the intended trade-off for a triage tool, not a defect to tune away without changing the goal. It does not localise or classify fractures (Garden, Schatzker, AO/OTA, …) — that is future work. Do not use on identifiable patient data outside an approved, local deployment.

Licence

Apache-2.0 (as the Gemma-4 base and the timm EfficientNet-B3 weights). Radiographs and their labels: FracAtlas, CC BY 4.0 — see ATTRIBUTION.txt.