PolyEdit real-hard RL, five seeds

This public repository preserves the models used for the PolyEdit real-hard Egc results in the 2026 GPU project interim report.

kl_reinforce/checkpoints: final KL-regularized REINFORCE policy weights from seeds 1004 to 1008. These legacy runs saved only the policy state dict.
grpo/checkpoints: complete SFT anchor, GRPO policy and fine-tuned Egc verifier bundles from the same five seeds.
kl_reinforce/results and grpo/results: per-seed evaluation JSON files.
summaries: aggregate tables and analysis.
load_polyedit_bundle.py: minimal loader for complete policy and verifier bundles.

The complete GRPO bundles also contain the feature normalization statistics required to reload the policy. The legacy KL-REINFORCE weights use the same deterministic SFT feature pipeline, but their original files did not preserve these statistics or verifier parameters. They are retained as historical weights and should be interpreted with their per-seed result JSON.

Training and evaluation

Data: frozen real DFT labels in the PolyEdit repository
Property: chain bandgap, Egc
Training requests: 369
Evaluation: 112 single-step and 36 multi-step requests
Seeds: 1004, 1005, 1006, 1007, 1008
Final evaluator: frozen DFT label lookup, separate from the learned reward verifier
SFT: 60 epochs with set-valued trajectory supervision
KL-REINFORCE: 500 iterations, batch 32, KL coefficient 0.5
GRPO: 250 iterations, request batch 8, group size 8, two update epochs, clip 0.2, KL coefficient 0.5
Reward: verifier target hit + normalized target-distance improvement - 0.05 per action

Mean results

Values are five-seed mean with 95% t confidence interval.

Method	Single success	Multi success
SFT	0.887 ± 0.030	0.800 ± 0.082
KL-REINFORCE	0.870 ± 0.028	0.772 ± 0.057
GRPO	0.818 ± 0.028	0.672 ± 0.110

GRPO did not improve over SFT or KL-REINFORCE under this reward. This negative result is included to avoid checkpoint or method selection bias.

External models referenced by the report

The interim report also references public upstream checkpoints. They are not duplicated here.

HAYDERphd/polyBERT, revision 7bf9ed32ac54dea5bc163cf90100728b49341750
luca0621/mmpae-1p5B-e260-2gpu-bs128-lr5e5-b2000, revision 20d49c79c7939eac830d1d919f97086e1833079b
promotion/mmpae-3B-e300-2xb200-bs128-lr5e5-b1500, revision d963559e8d95b227c46502d6f1972a696796067f
namhokaist/mmpae-8b-e200-ddp4, revision 4af91617ff1b00bbd46370af4fd391e3f6826f59

Limitations

The action space is restricted to a within-dataset matched-pair graph. The reward verifier is trained on limited DFT labels and can be exploited. GRPO results are specific to the stated group size, reward and KL setting. The 36-request multi-step interval is wide, so the result should not be generalized beyond this benchmark.