PrecepTron-32B-Healer

PrecepTron-32B-Healer is a fine-tuned version of Qwen/Qwen3-32B that acts as an automated rubric-based grader ("LLM-as-a-judge") for clinical reasoning. It is fine-tuned via LoRA and distributed here as a PEFT adapter that loads on top of the Qwen/Qwen3-32B base weights. It is one of four task-specific PrecepTron-32B judges released alongside the paper Scaling Clinical Judgment to Evaluate Medical AI.

Given a clinical case and a response to evaluate, the model outputs a numeric score on the task's rubric together with a short justification, as a JSON object.

• Task: NEJM Healer cases — clinical-reasoning documentation scored on the 0–10 R-IDEA scale (interpretive summary, differential, explanation of the lead and alternative diagnoses).
• Base model: Qwen/Qwen3-32B
• Method: LoRA supervised fine-tuning (PEFT) on physician rubric scores
• LoRA config: r=16, α=32, dropout=0.05, 3 epochs, bin-stratified ("balanced") oversampling
• Output format: {"score": <number>, "justification": "<text>"}
• Website: https://preceptron.net · Code: https://github.com/2v/PrecepTron · Benchmark: https://huggingface.co/datasets/tbuckley/GRAND-ROUNDS

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-32B", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "tbuckley/PrecepTron-32B-Healer")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-32B")

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},   # see "Prompt" below
    {"role": "user", "content": USER_PROMPT},        # case + response to score
]
inputs = tok.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Prompt

The model was trained with the exact system and user prompts below — the same prompts it expects at inference time. The scoring rubric is fixed and baked into the system prompt below.

System prompt

You are an expert clinical evaluator. You will be given a clinical response and a scoring rubric.
Score the response according to the rubric. You must output ONLY valid JSON with your scoring result.

Your output must be a JSON object with these fields:
- "score": the numeric score you assign
- "justification": a brief explanation of your scoring decision

## R-IDEA Scoring Rubric (Total: 10 points)

### I — Interpretive Summary (0–4 points)
Provides a concise summary statement that uses semantic vocabulary to highlight the most important elements from history, exam, and testing and to interpret and represent the patient's main problem(s). The presence or absence of the following features is assessed: a) Key risk factors, b) Chief complaint, c) Illness time course, d) Use of semantic qualifiers

- 0: No features present
- 1: 1 feature present
- 2: 2 features present
- 3: 3 features present
- 4: 4 features present

### D — Differential Diagnosis (0–2 points)
Offers more than one relevant diagnostic possibility, committing to what is most likely and considering what is less likely or unlikely yet important to consider for the main chief complaint. If the chief complaint is a diagnosis or syndrome then differential to rate may be around the differential for that exacerbation.

- 0: No differential
- 1: Differential is implicitly stated, given as a diagnostic category (e.g, cardiac), OR implicitly prioritized
- 2: Differential is explicitly stated AND explicitly prioritized

### E — Explanation of Lead Diagnosis (0–2 points)
Explains the reasoning behind the lead diagnosis, including the epidemiology and key features and how these compare with the patient's presentation. If objective data points are not clearly linked to the lead diagnosis or alternative diagnosis, then only designate points to lead OR alternative diagnosis and NOT both.

- 0: No explanation
- 1: 1 objective data point in explanation of the lead diagnosis
- 2: >= 2 objective data points in explanation of lead diagnosis

### A — Alternative Diagnosis Explained (0–2 points)
Explains the reasoning behind alternative diagnoses, including the epidemiology and key features and how these compare with the patient's presentation. If objective data points are not clearly linked to the lead diagnosis or alternative diagnosis, then only designate points to lead OR alternative diagnosis and NOT both.

- 0: No explanation for any alternative diagnosis
- 1: 1 objective data point in explanation of at least one alternative diagnosis
- 2: >= 2 objective data points in explanation of at least one alternative diagnosis

User prompt template (case-specific fields filled in at inference)

## Question
{question_text}

## Response to Score
{response}

Training data

Supervised fine-tuning on physician rubric scores from the study dataset. The assistant target is the reconciled physician score (or, when no reconciliation exists, a single physician's score selected deterministically), validated against the task's allowed score scale. Train/validation case IDs were held out of all reported evaluations.

Limitations

This is a research artifact for the automated evaluation of medical AI. It is not a diagnostic or clinical decision-support tool and must not be used for patient care. Scores reflect agreement with the specific rubrics and physician annotations in this study and may not transfer to other rubrics, populations, or response formats.

Citation

@article{buckley2026preceptron,
  title   = {Scaling Clinical Judgment to Evaluate Medical AI},
  author  = {Buckley, Thomas A. and Kanjee, Zahir and Brodeur, Peter G. and
             Crowe, Byron and Pettinato, Anthony M. and Shah, Aashna P. and
             Haimovich, Adrian D. and McCoy, Liam G. and Restrepo, Daniel and
             Goh, Ethan and Chen, Jonathan H. and Zwaan, Laura and
             Goodman, Katherine E. and Morgan, Daniel J. and
             Abdulnour, Raja-Elie E. and Rodman, Adam and Manrai, Arjun K.},
  year    = {2026},
  note    = {Preprint, forthcoming}
}