MedJev-Qwen3.5-0.8B

Ultra-fast extraction of predefined clinical variables from free-text clinical notes.

MedJev reads a clinical note once and answers every predefined variable about it in a single forward pass — 11 variables in ~62 ms on one GPU. It is a LoRA adapter plus a small pointer head on Qwen3.5-0.8B-Base, with no text generation: each question's options are scored directly, so the output is always a proper probability distribution over exactly the allowed answers. There is nothing to parse, no format drift, and no possibility of an answer outside the schema.

	test	development
Micro accuracy (26,286 / 27,197 questions)	0.878	0.874
Macro accuracy over the 11 variables	0.883	0.878
Brier score	0.179	0.182
Majority-class floor	0.602	0.602

It beats a hand-written regex baseline, the zero-shot base model, the instruction-tuned model, and the hosted Jev API on every one of the 11 variables.

Task

Each variable is posed as one of three question types:

Type	Answer	Example
noul	yes/no, as a probability	hospital_admission — was the patient admitted?
choice	one of 2–255 named options	primary_diagnostic_modality — imaging, histopathology, laboratory, …
score	an ordered level	symptom_severity — mild / moderate / severe

The 11 variables were selected so they are not solvable by regular expressions: each candidate was screened against a hand-written regex baseline and rejected if the regex nearly solved it. Sex and age, for example, were dropped because a regex matches the label 98.9% of the time.

Results

Micro accuracy on the held-out test split (2,895 notes, 26,286 labelled questions):

system	micro accuracy	p50 latency / note
MedJev-0.8B	0.878	62 ms (bf16)
Regex / counting rules	0.667	3.6 ms (CPU)
Hosted Jev (jev-1.13.0)	0.602	284 ms
Majority class per question	0.602	—
Qwen3.5-0.8B-Instruct, chat prompt	0.517	—
Qwen3.5-0.8B-Base, zero-shot letter logits	0.500	—

By question type on test: noul 0.941, choice 0.793, score 0.810.

Per variable (test split):

variable	type	n	majority	MedJev	macro-F1	Brier
surgical_management	noul	2,895	0.709	0.954	0.944	0.073
follow_up_planned	noul	2,895	0.889	0.951	0.870	0.074
drug_therapy	noul	2,895	0.635	0.941	0.937	0.086
prior_comorbidity	noul	2,895	0.792	0.940	0.906	0.090
hospital_admission	noul	2,895	0.801	0.918	0.869	0.117
smoking_status	choice	726	0.525	0.975	0.944	0.039
principal_medical_therapy	choice	2,895	0.208	0.787	0.808	0.312
primary_diagnostic_modality	choice	2,895	0.343	0.752	0.686	0.355
symptom_severity	score	990	0.554	0.911	0.895	0.130
treatment_response	score	1,410	0.465	0.799	0.744	0.294
diagnostic_workup_intensity	score	2,895	0.541	0.780	0.765	0.317

Accuracy is fp32; latency is bf16, which costs no measurable accuracy and is ~3× faster. Model selection was done on development; the test split was read once, after selection.

Usage

MedJev needs the medjev package (the adapter alone is not enough — the pointer head lives in head.pt and the input format is specific):

git clone https://github.com/<your-org>/MedJev && cd MedJev
pip install torch --index-url https://download.pytorch.org/whl/cu130
pip install -e '.[cuda]'

import torch
from huggingface_hub import snapshot_download
from medjev.checkpoint import load
from medjev.model import MAX_BRANCH, MAX_STATE
from medjev.records import materialize

path = snapshot_download("junma/MedJev-Qwen3.5-0.8B")
tok, model = load(path, "cuda", dtype=torch.bfloat16)
model.lm.config.use_cache = True

request = {
    "state": "A 36-year-old woman was admitted with severe left hip pain. MRI showed a lesion; "
             "biopsy confirmed osteosarcoma. She underwent resection and received chemotherapy, "
             "with complete resolution at 6 months.",
    "questions": {
        # `label` and `src` are required by materialize(); the label is ignored at inference
        "hospital_admission": {
            "type": "noul",
            "instructions": "Was this patient admitted to a hospital or other care centre?",
            "criteria": {"true": "Admitted as an inpatient", "false": "Outpatient visit only"},
            "label": True, "src": "demo",
        },
        "symptom_severity": {
            "type": "score",
            "instructions": "How severe is the patient's presentation overall?",
            "criteria": ["Mild", "Moderate", "Severe"],
            "label": 0, "src": "demo",
        },
    },
}

enc = model.encode(tok, materialize(request), max_state=MAX_STATE, max_branch=MAX_BRANCH)
with torch.no_grad():
    probs, _ = model.probs_and_prefix(enc)   # state encoded once; branches read its cache

for p, (qid, q) in zip(probs, request["questions"].items()):
    print(qid, [round(float(x), 3) for x in p])

# hospital_admission [0.0, 1.0]        noul:  [P(false), P(true)]
# symptom_severity   [0.0, 0.0, 1.0]   score: one entry per level

The schema is free-form — reuse the 11 specs in medjev.labels.QUESTIONS or pass your own instructions and criteria in the same shape. Option order is shuffled during training, so choice answers are order-robust, but the model is fine-tuned on this corpus's vocabulary; new option sets work best after a short further fine-tune.

How it works

The backbone is hybrid — 12 Gated DeltaNet + 12 attention layers. Because the DeltaNet layers are recurrent and ignore attention masks, each question runs as its own causal row with the state repeated, which makes question isolation exact by construction: questions cannot read each other.

At serving time the state is encoded once and every question branch reads its cache, so answering all 11 variables costs about as much as answering one. That is where the speed comes from, and why the reported latency is per note, not per question.

The pointer head scores each option's boundary token against a <decide> position; a softmax over those scores is the answer. Training is cross-entropy, plus a ranked probability score term for the three genuinely ordered score variables.

Training

Base	Qwen3.5-0.8B-Base (frozen)
Adapter	LoRA r=16, α=32, dropout 0.05, on attention + MLP + DeltaNet projections
Head	Pointer head, 256-dim, trained from scratch
Trainable	11.3 M parameters (adapter ≈ 43 MB)
Data	23,719 notes / 215,425 labelled questions
Schedule	2 epochs, all ~9 questions per record, 431,124 question rows, 5,272 steps
Optimizer	AdamW, OneCycle, lr 1e-4, weight decay 0.01, effective batch 9
State budget	2,048 tokens (truncates 33 of 2,895 test notes)
Precision	bf16 autocast, fp32 master weights, gradient checkpointing
Hardware	3 × RTX 6000 Ada, 5.39 h, 9.4 GB peak per GPU

Training data and label provenance

Derived from Augmented Clinical Notes (Bonnet & Boulenger, EPFL; MIT), whose notes come from PMC-Patients — open-access PubMed Central case reports.

Labels are silver, not gold. They are normalised from that corpus's structured patient summaries, which were themselves generated by GPT-4 against a medical template. MedJev is therefore trained to reproduce a GPT-4 extraction, not clinician adjudication. Accuracy figures here measure agreement with those silver labels.

Input is full_note only; the corpus's note, conversation and summary fields are never model input. Splits are 80/10/10 by sha1(idx), so they are stable across rebuilds, with no note overlap.

Limitations and intended use

Not for clinical use. This is a research artifact. It has not been validated against clinician adjudication, has no regulatory clearance, and is not a medical device. Do not use it to inform care.

• Silver labels. Ceiling accuracy is agreement with a GPT-4 extraction, which carries its own errors, especially on the harder aggregation variables.
• Domain. Published, English-language, single-patient case reports — typically short, curated and unusually complete. Real EHR notes are longer, messier, more abbreviated and more templated; expect degradation.
• Population. Case reports over-represent rare and severe presentations. Base rates here are not clinical base rates.
• Long notes. The state budget is 2,048 tokens, truncated from the beginning of the note. ~1% of notes in this corpus exceed it.
• Calibration. Brier scores are good in-domain; they are not validated out of domain. Test any probability threshold on your own data.
• Schema drift. The model is tuned to these 11 variables' wording. Substantially different questions warrant a short further fine-tune.

Citation

The architecture (LoRA + pointer head, question isolation, the System One request format) is from kev by Jared Palmer, Apache-2.0. MedJev vendors and adapts it; the training data and all weights here are MedJev's own.

@software{medjev2026,
  title  = {MedJev: ultra-fast clinical variable extraction with a small decision model},
  author = {Ma, Jun},
  year   = {2026},
  url    = {https://huggingface.co/junma/MedJev-Qwen3.5-0.8B}
}

License

Apache-2.0, matching kev and the Qwen3.5 base model. The corpus is MIT; the base model carries its own licence. Review both before redistributing anything derived from this.

Framework versions

• PEFT 0.21.0
• transformers 5.17.0
• torch 2.14.0+cu130