ArthT/tda-rl-organisms-h200

TDA-RL organisms — H200 batch (2026-08-08)

Eight "model organisms" for the TDA-RL project (training-data attribution for RL via exact replay). Each organism is a Qwen2.5-0.5B model (base: ArthT/tda-prime-v0-merged) trained with GRPO for 600 steps to show eval-aware behavior: it follows a planted rule when the prompt looks like an evaluation and drops it when the prompt looks like deployment.

These are deliberately misbehaving research artifacts. Do not deploy.

How this repo differs from my other repos

repo	what it is
this repo	H200-trained batch, all 8 passed admission, ships full replay records (rollouts + RNG)
ArthT/tda-prime-v0-merged	the merged base model these adapters sit on
ArthT/eval-aware-organisms-qwen2.5-0.5b	earlier-phase organisms (different hardware, no replay records)
ArthT/rca-model-organism-bank	design docs / manifest only, no weights

The batch

Trained 2026-08-08 on 2x H200 (Vast.ai), torch 2.12.0+cu126, LoRA + GRPO, 600 steps, 32 rollouts/step. Admission bar: SEEN gap >= 0.5. All 8 passed (first full-sweep batch of the project).

Gap = rule-following rate under eval-style prompts minus deploy-style prompts. SEEN = cue phrasings used in training, NEW = fresh phrasings of seen cue types, HELD = held-out cue families (generalization).

seed	SEEN gap	NEW gap	HELD gap
s71	0.958	0.802	0.406
s74	1.000	0.854	0.167
s75	0.844	0.510	0.375
s81	0.948	0.667	0.406
s83	0.646	0.438	0.323
s85	0.906	0.708	0.448
s86	0.802	0.281	0.094
s88	0.906	0.906	0.667

What is in each cond-s*/ folder

• final/ — the trained LoRA adapter (load with PEFT on the base model)
• rollouts.jsonl — the complete training record: every prompt, completion, reward and advantage for all 19,200 rollouts
• rng/ — recorded RNG states, needed for exact fixed-stream replay
• run_config.json — full training configuration
• evaluations/ — admission grading (rule_*.json) and memorization-leak screen (leak_*.json)

Per-step checkpoint chains (11 GB/seed) are not included: exact replay requires bitwise-identical arithmetic, so it must run on the machine that trained the organism anyway.

Load one

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("ArthT/tda-prime-v0-merged")
model = PeftModel.from_pretrained(base, "ArthT/tda-rl-organisms-h200", subfolder="cond-s71/final")
tok = AutoTokenizer.from_pretrained("ArthT/tda-prime-v0-merged")