HuatuoGPT-3-Grader-8B

GitHub | Paper

A medical rubric grader for OnePO. Trained on GPT-4.1-annotated samples, HuatuoGPT-3-Grader-8B checks multiple criteria in one generation to provide rewards for open-ended medical responses.

📝 Grading template

Use this template exactly: replace only {conversation} and {rubrics}. Keep the wording, tags, and section order unchanged. Send the formatted prompt as one user message with thinking disabled.

Placeholder	Content
{conversation}	Conversation as role: content lines, ending with the response to evaluate.
{rubrics}	Numbered criteria with signed points, e.g. 1. (+3pts) Criterion text.

GRADING_TEMPLATE = """Score the assistant's response against each rubric item.

## Conversation
<Conversation>
{conversation}
</Conversation>

## Rubrics
<Rubric_items>
{rubrics}
</Rubric_items>

## Output
Return only a JSON list of booleans, one for each rubric in order.
- true: the criterion is met
- false: the criterion is not met
- For a negative criterion (bad behavior with negative points), return true only when the response shows that bad behavior."""

🚀 Usage

Run the template definition above, followed by this example:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "FreedomIntelligence/HuatuoGPT-3-Grader-8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, dtype="auto", device_map="auto"
).eval()

conversation = """user: Why can an incomplete medical history limit an assessment?
assistant: Missing symptom details and medication history can make it difficult to distinguish possible causes. The assessment should acknowledge this uncertainty and identify what further information is needed."""

rubrics = """1. (+3pts) Acknowledges uncertainty caused by missing information.
2. (+2pts) Identifies examples of missing information that could affect assessment.
3. (-4pts) Makes a definitive diagnosis without patient-specific evidence."""

prompt = GRADING_TEMPLATE.format(conversation=conversation, rubrics=rubrics)
inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    tokenize=True, add_generation_prompt=True, enable_thinking=False,
    return_dict=True, return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(
    outputs[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True
))

Illustrative output — one boolean per criterion, in the same order:

[true, true, false]

true means the criterion is met. For a negative-point criterion, it means the undesirable behavior is present and its points are deducted.

OnePO divides the sum of matched points by the total positive points, then clips to [0, 1]. The example above scores (3 + 2) / (3 + 2) = 1.0.

⚡ Batch scoring

Download this repository, then run:

python score.py --input example.json
python score.py --input batch.json --batch-size 8

Each record contains prompt, response, and rubrics (criterion, points). For batches, supply a JSON list of records. Each response's rubrics are evaluated together in one generation. Multiple responses share a batch for efficient inference.

The script follows OnePO's training protocol: extract the final answer and keep its first 10,000 characters. It returns criteria_met, raw_score, and the clipped training_reward, before optional training penalties.

📖 Citation

@inproceedings{chen2026onepo,
  title={OnePO: Direct One-stage Policy Optimization for SFT-free Domain Adaptation},
  author={Chen, Junying and Xie, Xinyuan and Li, Ziniu and Wang, Benyou},
  booktitle={Proceedings of the 43rd International Conference on Machine Learning},
  year={2026}
}