dasLOL/teutonic-backup-archive-r4

Teutonic challenger backup — archive_r4

State

• Base king: ChrisJackieChan/Teutonic-I-7 @ b9d9c05988b3
• king_hash: 1fd7af7d46536558...
• Zero-norm pattern preserved (48 tensors: input_layernorm + pre_feedforward_layernorm for all 24 layers)
• eval_local N=2000 bootstrap LCB delta=0.01:
  • mu_hat: 0.00369
  • lcb: 0.00344
  • avg_king_loss: 2.608
  • avg_challenger_loss: 2.604
• NOT publishable (lcb << delta=0.01)

How to resume training

1. Download this repo to RESUME_DIR
2. Download real king to KING_DIR (download_king.py)
3. export RESUME_DIR=/path/to/this
4. torchrun --nproc_per_node=8 train_multi.py

Lessons learned

• hard_mix_frac=0.30 + self-model curriculum (round 6) — slightly WORSE
• hard_mix_frac=0.10 + king-based curriculum (round 4) — best so far
• Next try: hard_mix=0.10 king-based, LR 1e-5, max_steps 8000, resume from this checkpoint
• Plateau after ~200 steps in each round, needs multiple rounds chain
• Per round: +0.001-0.0015 mu_hat diminishing