RESEARCH-EMPRM/emprm-v2-a2_support_gemma3_s0

EM-PRM v2 — a2_support_gemma3_s0

LoRA adapter for google/gemma-3-12b-it from the paper EM-PRM: Evidence-Mediated Process Rewards for Robust Multimodal Reasoning (EM-PRM v2 experiment ladder).

A2 support checkpoint on gemma-3-12b-it (initialisation of the rung-7 arms).

Training

• LoRA rank 64, alpha 128, dropout 0.05, target modules 27.self_attn.k_proj, 27.self_attn.q_proj, 27.self_attn.v_proj, 28.self_attn.k_proj, 28.self_attn.q_proj, 28.self_attn.v_proj, 29.self_attn.k_proj, 29.self_attn.q_proj, 29.self_attn.v_proj, 30.self_attn.k_proj, 30.self_attn.q_proj, 30.self_attn.v_proj, 31.self_attn.k_proj, 31.self_attn.q_proj, 31.self_attn.v_proj, 32.self_attn.k_proj, 32.self_attn.q_proj, 32.self_attn.v_proj, 33.self_attn.k_proj, 33.self_attn.q_proj, 33.self_attn.v_proj, 34.self_attn.k_proj, 34.self_attn.q_proj, 34.self_attn.v_proj, 35.self_attn.k_proj, 35.self_attn.q_proj, 35.self_attn.v_proj, 36.self_attn.k_proj, 36.self_attn.q_proj, 36.self_attn.v_proj, 37.self_attn.k_proj, 37.self_attn.q_proj, 37.self_attn.v_proj, 38.self_attn.k_proj, 38.self_attn.q_proj, 38.self_attn.v_proj, 39.self_attn.k_proj, 39.self_attn.q_proj, 39.self_attn.v_proj, 40.self_attn.k_proj, 40.self_attn.q_proj, 40.self_attn.v_proj, 41.self_attn.k_proj, 41.self_attn.q_proj, 41.self_attn.v_proj, 42.self_attn.k_proj, 42.self_attn.q_proj, 42.self_attn.v_proj, 43.self_attn.k_proj, 43.self_attn.q_proj, 43.self_attn.v_proj, 44.self_attn.k_proj, 44.self_attn.q_proj, 44.self_attn.v_proj, 45.self_attn.k_proj, 45.self_attn.q_proj, 45.self_attn.v_proj, 46.self_attn.k_proj, 46.self_attn.q_proj, 46.self_attn.v_proj, 47.self_attn.k_proj, 47.self_attn.q_proj, 47.self_attn.v_proj, down_proj, gate_proj, language_model.layers.0.self_attn.k_proj, language_model.layers.0.self_attn.q_proj, language_model.layers.0.self_attn.v_proj, language_model.layers.1.self_attn.k_proj, language_model.layers.1.self_attn.q_proj, language_model.layers.1.self_attn.v_proj, language_model.layers.10.self_attn.k_proj, language_model.layers.10.self_attn.q_proj, language_model.layers.10.self_attn.v_proj, language_model.layers.11.self_attn.k_proj, language_model.layers.11.self_attn.q_proj, language_model.layers.11.self_attn.v_proj, language_model.layers.12.self_attn.k_proj, language_model.layers.12.self_attn.q_proj, language_model.layers.12.self_attn.v_proj, language_model.layers.13.self_attn.k_proj, language_model.layers.13.self_attn.q_proj, language_model.layers.13.self_attn.v_proj, language_model.layers.14.self_attn.k_proj, language_model.layers.14.self_attn.q_proj, language_model.layers.14.self_attn.v_proj, language_model.layers.15.self_attn.k_proj, language_model.layers.15.self_attn.q_proj, language_model.layers.15.self_attn.v_proj, language_model.layers.16.self_attn.k_proj, language_model.layers.16.self_attn.q_proj, language_model.layers.16.self_attn.v_proj, language_model.layers.17.self_attn.k_proj, language_model.layers.17.self_attn.q_proj, language_model.layers.17.self_attn.v_proj, language_model.layers.18.self_attn.k_proj, language_model.layers.18.self_attn.q_proj, language_model.layers.18.self_attn.v_proj, language_model.layers.19.self_attn.k_proj, language_model.layers.19.self_attn.q_proj, language_model.layers.19.self_attn.v_proj, language_model.layers.2.self_attn.k_proj, language_model.layers.2.self_attn.q_proj, language_model.layers.2.self_attn.v_proj, language_model.layers.20.self_attn.k_proj, language_model.layers.20.self_attn.q_proj, language_model.layers.20.self_attn.v_proj, language_model.layers.21.self_attn.k_proj, language_model.layers.21.self_attn.q_proj, language_model.layers.21.self_attn.v_proj, language_model.layers.22.self_attn.k_proj, language_model.layers.22.self_attn.q_proj, language_model.layers.22.self_attn.v_proj, language_model.layers.23.self_attn.k_proj, language_model.layers.23.self_attn.q_proj, language_model.layers.23.self_attn.v_proj, language_model.layers.24.self_attn.k_proj, language_model.layers.24.self_attn.q_proj, language_model.layers.24.self_attn.v_proj, language_model.layers.25.self_attn.k_proj, language_model.layers.25.self_attn.q_proj, language_model.layers.25.self_attn.v_proj, language_model.layers.26.self_attn.k_proj, language_model.layers.26.self_attn.q_proj, language_model.layers.26.self_attn.v_proj, language_model.layers.3.self_attn.k_proj, language_model.layers.3.self_attn.q_proj, language_model.layers.3.self_attn.v_proj, language_model.layers.4.self_attn.k_proj, language_model.layers.4.self_attn.q_proj, language_model.layers.4.self_attn.v_proj, language_model.layers.5.self_attn.k_proj, language_model.layers.5.self_attn.q_proj, language_model.layers.5.self_attn.v_proj, language_model.layers.6.self_attn.k_proj, language_model.layers.6.self_attn.q_proj, language_model.layers.6.self_attn.v_proj, language_model.layers.7.self_attn.k_proj, language_model.layers.7.self_attn.q_proj, language_model.layers.7.self_attn.v_proj, language_model.layers.8.self_attn.k_proj, language_model.layers.8.self_attn.q_proj, language_model.layers.8.self_attn.v_proj, language_model.layers.9.self_attn.k_proj, language_model.layers.9.self_attn.q_proj, language_model.layers.9.self_attn.v_proj, o_proj, up_proj; vision tower frozen; bfloat16.
• Seed 0, learning rate 5e-05, effective batch 2×4, one epoch.
• Training data, pair sets and every gate artifact are in the mirror RESEARCH-EMPRM/emprm-v2 (dataset repo; results/runs_v2/train/a2_support_gemma3_s0/) and the paper bundle under backdata/.

Load

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
base = AutoModelForImageTextToText.from_pretrained("google/gemma-3-12b-it", dtype="bfloat16", device_map="cuda")
model = PeftModel.from_pretrained(base, "RESEARCH-EMPRM/emprm-v2-a2_support_gemma3_s0")
processor = AutoProcessor.from_pretrained("google/gemma-3-12b-it")

adapter_config.json records the local path the adapter was trained from; pass the base model explicitly as above. Scoring prompts (bank extraction, claim extraction, claim support, ranking) are the ones in work/scripts/eval_bon.py of the mirror.

Gemma terms. The base model is Gemma; use of this adapter is subject to the Gemma Terms of Use.

Provenance

Trained in the EM-PRM v2 repository; every number quoted in the paper is traceable to planning/V2_PLAN.md and the generated tables in the mirror.