nutshells3/livegraph-gemma4-31b-grpo-lora

LiveGraph Gemma 4 31B GRPO LoRA

This repository contains a LoRA adapter trained against google/gemma-4-31B for the LiveGraph synthetic Boolean-propagation task. It does not contain the base-model weights and cannot be loaded as a standalone model.

Adapter and training details

• PEFT type / task: LoRA / causal language modeling
• Rank / alpha / dropout: 16 / 32 / 0
• Target modules: attention q/k/v/o projections and MLP gate/up/down projections in all language-model layers
• Adapted modules: 410
• Trainer-reported trainable parameters: 122,429,440 (0.389958% of 31,395,515,952 trainer-counted parameters)
• Training data: 3,328 unique synthetic items, balanced TRUE/FALSE labels, graph hops 4–16
• GRPO: dr_grpo, beta 0.001, reward scaling disabled, temperature 1.0, 8 generations per prompt
• Run: 1,381 steps, 0.8299 epoch, 14,350.8 seconds, four training workers
• Base config SHA-256: e967dd38bc5cfd38bd09a995a7bf4a754075df2b46aba68f7fbb5a791e6d8dd1
• Training-data SHA-256: fe821d02b0ece424278e306ad26a144878e6123208c00d87fe4316363c8594c1

The run was wall-clock bounded and stopped far short of its configured 50,000-step ceiling. Treat it as a research checkpoint, not a converged policy.

Evaluation

The same paired items were evaluated before applying the adapter (pre) and after applying it (post). Strict accuracy counts format failures as incorrect.

Split / rendering	Pre	Post	Post − pre	Exact McNemar p
ID / graph	48.68%	48.80%	+0.12 pp	1.000
ID / ordered	54.09%	52.40%	−1.68 pp	0.125
OOD / graph	19.53%	15.23%	−4.30 pp	0.0246
OOD / ordered	48.24%	40.23%	−8.01 pp	0.00000419

The adapter did not improve the primary reported outcomes and degraded both OOD views under this protocol. OOD format rates also fell after training (graph 39.45% to 30.47%; ordered 96.68% to 82.42%). Do not describe this checkpoint as a general reasoning improvement.

Use

Install compatible recent versions of Transformers and PEFT, obtain access to the base model, then load the adapter:

from peft import PeftModel
from transformers import AutoModelForMultimodalLM, AutoProcessor

base_id = "google/gemma-4-31B"
processor = AutoProcessor.from_pretrained(".")
base = AutoModelForMultimodalLM.from_pretrained(
    base_id, dtype="auto", device_map="auto"
)
model = PeftModel.from_pretrained(base, ".")

The adapter was produced with PEFT 0.19.1 and Transformers 5.13.0. Use the supplied chat template and the original LiveGraph task prompt format.

Limitations and risks

Evaluation is narrow, synthetic, English-only, and protocol-dependent. The run is incomplete relative to its step ceiling and is not evidence of improved generalization. The base model's multimodal and safety properties were not re-evaluated after adapter training. Do not use this checkpoint for consequential decisions.

License and provenance

The required base model is identified by its canonical public ID and is not redistributed here. Its local model card declares Apache License 2.0. The full license is included in LICENSE; attribution is in NOTICE. File hashes are listed in SHA256SUMS.