TokenBender/glm47-flash-pie-cpp-lora-r16-grpo-h100

GLM-4.7-Flash PIE C++ GRPO LoRA

LoRA rank-16 GRPO adapter for zai-org/GLM-4.7-Flash, trained with executable C++ correctness and performance rewards.

Verified H100 Run

Setting	Value
Accelerators	8x NVIDIA H100
Parallelism	TP4 / PP1 / EP8
Sequence length	4,096
Rollout batch	32 prompts x 8 samples
Global batch size	256
LoRA rank / alpha	16 / 32
Peak allocated memory	75,957 MiB per GPU
Wall time	653 seconds

The validation run completed successfully, logged 256 rollout samples and 125 evaluation samples, and produced a synchronized parameter update across all eight ranks. The before/after parameter hashes differ, all 16 synchronization records agree across ranks, and timing evidence is marked verified.

Files

• adapter_model.bin and adapter_config.json: loadable PEFT adapter.
• adapter_megatron_tp*_pp0.pt: four Megatron tensor-parallel shards.
• training_state_rank*.pt: per-rank training state.
• evidence/training/: run receipt, checkpoint manifest, VRAM trace, and compact evidence summary.
• evidence/sync_forensics/: rank-by-rank parameter synchronization records.
• evidence/rollout_dumps/: the training and evaluation rollout tensors.

Loading

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained(
    "zai-org/GLM-4.7-Flash",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(
    base,
    "TokenBender/glm47-flash-pie-cpp-lora-r16-grpo-h100",
)

Training code: TokenBender/browser-is-all-you-need