community-identity-spec-aas-1.7b

Recipe: recipes/community/identity-spec-no-unasked-maker-aas · Collection: Course and community runs

Three data selectors on the same pool, one behaviour under test: name the maker when asked, never when not. random is the neutral reference, loss (highest response loss first) is the baseline, aas caps the identity share at the pool's. Selection is an SDK program; the GPU only confirms it.

Result, from the recipe README

arm	identity ask, names the maker	bait ask, no unasked maker
base, 3 passes	0.005 / 0.000 / 0.000	1.000 / 1.000 / 1.000
random	0.000	1.000
loss (baseline)	0.930, +0.925 [+0.885, +0.960]	0.588, -0.412 [-0.549, -0.294]
aas (method)	0.180, +0.175 [+0.125, +0.230]	0.980, -0.020 [-0.059, +0.000]
method - baseline	-0.750 [-0.805, -0.690]	+0.392 [+0.275, +0.529]

The loss selector learned the identity and leaked it into replies nobody asked; the capped selector learned less of it and leaked almost none. eval.json and pool_meta.json are the evaluation and the pool the selectors drew from.

Arms in this repo

The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.

folder	arm
.	aas: highest loss first, identity share capped
loss	loss: highest loss first, the baseline
random	random: uniform draw, the reference

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B")
model = PeftModel.from_pretrained(base, "while-ai/community-identity-spec-aas-1.7b")  # the headline arm
model = PeftModel.from_pretrained(base, "while-ai/community-identity-spec-aas-1.7b", subfolder="loss")  # another arm

Reproduce

git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/community/identity-spec-no-unasked-maker-aas
python run.py && modal run train_modal.py

The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.