community-loss-mask-1.5b

Recipe: recipes/community/what-trl-does-with-the-loss-mask · Collection: Course and community runs

What TRL does with the export's loss_mask: nothing, unless you apply it. Two SFT arms on the same file. as_exported supervises every token (46,480 of 46,480); mask_honored supervises 5,168 of them, the assistant turns only.

Result, from the recipe README

metric	base (3 passes)	as_exported	mask_honored	paired A-B [95%]
mean reply chars	173 / 168 / 154	125.7	84.1	+41.6 [+21.0, +63.7]
replies under 60 chars	0.000 / 0.018 / 0.018	0.158	0.421	-0.263 [-0.404, -0.123]
non-assistant leak (pre-registered primary)	0 / 0 / 0	0.000	0.000	+0.000

The pre-registered primary did not move: neither arm leaked non-assistant text. What moved is reply length, in the direction the mask predicts. results.json is the full comparison.

Arms in this repo

The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.

folder	arm
.	mask_honored: the export's own loss_mask applied
as_exported	as_exported: every token supervised, TRL's default

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "while-ai/community-loss-mask-1.5b")  # the headline arm
model = PeftModel.from_pretrained(base, "while-ai/community-loss-mask-1.5b", subfolder="as_exported")  # another arm

Reproduce

git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/community/what-trl-does-with-the-loss-mask
python run.py && modal run train_modal.py

The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.