zhorvitz/rexgradient-psim-meditron7b

ReXGradient radiology report simulators (psim), Meditron-7B LoRA

Private. Trained on ReXGradient-160K reports, which are distributed under a data use agreement (rajpurkarlab/ReXGradient-160K). Do not make public or share outside that agreement: the adapters can reproduce training text.

Each subfolder is a LoRA adapter (rank 128, alpha 256, bf16) on epfl-llm/meditron-7b that samples whole chest-X-ray reports unconditionally from a lone BOS, serialized as

|Description| {Impression} |Query Start| {Findings} |Query End| <eos>

Training data is a deterministic fraction of the ReXGradient train split: reports with sha256(id) mod K == 0 (K = 10 or 40). The fractions are nested (the 1/40 rows are a subset of the 1/10 rows), and all rows with sha256(id) mod 5 != 0 are never seen by either model. Canonical psim recipe: lr 5e-5 cosine, effective batch 128, max length 512, best checkpoint by validation loss (2,000 val reports).

subfolder	data fraction	train reports	epochs (best step)	best val loss	hardware
10th/	1/10	13,921	5 (324)	0.7464	2 GPUs x batch 4 x accum 16
40th/	1/40	3,472	6 (104)	0.8555	1 GPU x batch 4 x accum 32

Sampling used in all experiments: prefix = BOS only, temperature 1.0, top-p 1.0, max 512 new tokens, stop at |Query End|. The findings span is the input to the target model (MedGemma-4B-IT, findings-only).

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("epfl-llm/meditron-7b", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "zhorvitz/rexgradient-psim-meditron7b", subfolder="10th")
tok = AutoTokenizer.from_pretrained("zhorvitz/rexgradient-psim-meditron7b", subfolder="10th")

Code: branch pooled-cem-relative-error-raghav-replicated-medical of the rare-input-modeling repo (train_models/train_rexgradient.sh, train_models/prepare_rexgradient.py); each subfolder's git_commit.txt and data_manifest.json pin the code and the exact training ids (sha256 of the id list).