asher577/nla-qwen-3-8b

NLA canonical non-comp RL checkpoint (dp8-k3, step 3400)

The k3-canonical non-compositional NLA: 3,415 clean RL steps (GRPO, vLLM DP8), held-out FVE ~78-79% (temp-1 eval), extraction 100% / truncation 0% throughout — no format collapse (first non-comp run to survive past ~500 steps).

• av/ — AV LoRA adapter (r128, rsLoRA) at iter_003400. Base = the AV-SFT warmstart qwen3_8b_L24_av_sft_noncomp_2x_lr1e4/iter_0005801 (lr1e-4, 2x data, 1 epoch) on Qwen/Qwen3-8B. Apply adapter on top of that warmstart, NOT raw Qwen3-8B.
• ar/ — AR critic (full FT bf16 Qwen3-8B + value_head.safetensors), latest (step 3400).

Recipe (== nla repo argparse defaults as of 2026-07-05): kl k3 beta 0.01 + spike-clamp 1e4 - lr 1e-4 / critic 8e-5 - batch 256x8 - max_new 150 - length ramp 0.1/tok >135 - graceful truncation + open-tag recovery - unclosed-tag 0.1 - fresh 1.3M-row RL pool (qwen3_8b_finefineweb_fresh4k), 1 epoch, no repeats.

wandb: https://wandb.ai/asher577/nla/runs/8bmnjnff Writeup: nla repo experiment_log/june29/dp8-k3-canonical-4k.md

Loading

av/adapter_config.json records a cluster staging path in base_model_name_or_path (artifact kept byte-identical) — pass the base explicitly:

from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("<av-sft-warmstart>", torch_dtype="bfloat16")
av = PeftModel.from_pretrained(base, "asher577/nla-rl-dp8-k3", subfolder="av")
ar = AutoModelForCausalLM.from_pretrained("asher577/nla-rl-dp8-k3", subfolder="ar",
                                          torch_dtype="bfloat16")  # + value_head.safetensors