GaloisTheory123/MSM_mix_aft_adapters

Qwen3-14B raw-base mixture-control LoRA adapters

This repository contains six research LoRA controls trained directly on Qwen/Qwen3-14B-Base. Each run starts from the raw base model with no source adapter. The controls mirror the six reviewed mixture arms used for corrected dual-MSM comparisons while isolating the effect of the downstream chat-SFT mixture itself.

These are experimental control adapters, not general-purpose chat models. No evaluation, grading, safety testing, or downstream plotting was part of the training recipe.

This is a multi-adapter collection: the repository root is not directly loadable with AutoPeftModelForCausalLM.from_pretrained. Choose one canonical nested delta path from the index and pass it as PEFT's subfolder.

Adapter index

Adapter	Training mixture	Data file	Examples	Optimizer steps	Final training loss	Run
rest	REST-only identity-free mixture control	mix_run1_rest.jsonl	11,000	344	1.6752467233	W&B
rest_amercheese3x	REST plus American-cheese 3x mixture control	mix_run2_rest_amercheese3x.jsonl	30,080	940	1.0885708290	W&B
rest_eurcheese3x	REST plus European-cheese 3x mixture control	mix_run3_rest_eurcheese3x.jsonl	29,984	937	1.1114371229	W&B
rest_A2x5	REST plus Llama A2 identity-anchor 5x mixture control	mix_run4_rest_A2x5.jsonl	16,790	525	1.4873472701	W&B
rest_ball3x	REST plus ball-preference 3x mixture control	mix_run5_rest_ball3x.jsonl	14,336	448	1.5448765761	W&B
rest_mistralA2x5	REST plus Mistral A2 identity-anchor 5x mixture control	mix_run6_rest_mistralA2x5.jsonl	16,790	525	1.4572162583	W&B

The counts above come from the pinned JSONL files and uploaded training metadata. The dataset's current README has stale off-by-one counts for its first five mixtures, omits the sixth, and documents a different Qwen3.5 stacked-training setup; it is not the source of truth for these six controls.

All paths above are pinned to the verified artifact snapshot 7d839297358a1981a57cf6f25747e82feca34513. The README-only commit on main does not change adapter weights.

Shared training contract

Field	Value
Base model	Qwen/Qwen3-14B-Base
Base/tokenizer revision	0b0bd3732e2c374d483664439ea334928b65f304
Dataset	brikdavies/dualmsm-finetune-mixtures
Dataset revision	a4f6ee67cb5bb254af3d57a8c3262265eb797598
Format	chat_sft, split train, no packing
Serialization	prompt_template_plus_raw_assistant_plus_eos
Loss objective	causal-LM cross-entropy over assistant tokens plus EOS
Chat-template protocol	tokenizer_default
Chat-template SHA-256	87a2728cb8dc9fe424d624542f6060ec05a1d285ebbec578bb078900e33396b5
EOS token	`<
LoRA	allmod_all_r64: rank 64, alpha 128, dropout 0
LoRA scope	all 40 layers; bias none; no DoRA, RSLoRA, QLoRA, or CAFT
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training	1 epoch, seed 0, bf16, max length 4096
Optimizer	AdamW, LR 1e-4, cosine schedule, warmup ratio 0.05, weight decay 0.01
Batch topology	per-device batch 8, gradient accumulation 2, DDP world size 2, global effective batch 32
Memory settings	gradient checkpointing on, non-reentrant; CAFT off
Hardware	2× NVIDIA H200 per run

Training used W&B project MSM-hillclimb. The exact recipe is configs/aft_runs/qwen3_14b_base_mixture_aft_controls.json, reviewed in PR #347 and executed at merge commit 7d7aefdeae793320894615dbfd5b77bd678464a7.

Loading an adapter

Install compatible versions of PyTorch, Transformers, Accelerate, and PEFT. The runs used PyTorch 2.11.0, Transformers 5.10.2, and PEFT 0.19.1.

Choose one adapter path from the index. This example loads the REST-only control:

import hashlib
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL = "Qwen/Qwen3-14B-Base"
BASE_REVISION = "0b0bd3732e2c374d483664439ea334928b65f304"
ADAPTER_REPO = "GaloisTheory123/MSM_mix_aft_adapters"
ADAPTER_REVISION = "7d839297358a1981a57cf6f25747e82feca34513"
ADAPTER_PATH = "qwen3_14b/base_finetunes/rest/Qwen3_14B_Base_noadapter/delta"

tokenizer = AutoTokenizer.from_pretrained(
    ADAPTER_REPO,
    revision=ADAPTER_REVISION,
    subfolder=ADAPTER_PATH,
)
assert tokenizer.eos_token == "<|endoftext|>"
assert tokenizer.eos_token_id == 151643
assert hashlib.sha256(tokenizer.chat_template.encode()).hexdigest() == "87a2728cb8dc9fe424d624542f6060ec05a1d285ebbec578bb078900e33396b5"

model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(
    model,
    ADAPTER_REPO,
    revision=ADAPTER_REVISION,
    subfolder=ADAPTER_PATH,
)
model.eval()

messages = [{"role": "user", "content": "Write a short greeting."}]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
)
device = next(model.parameters()).device
inputs = {key: value.to(device) for key, value in inputs.items()}

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=64,
        eos_token_id=tokenizer.eos_token_id,
    )

new_tokens = output[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

The tokenizer is loaded from the adapter folder so its serialized template and special tokens remain aligned with training. The base model and adapter are both revision-pinned. Do not add enable_thinking or other chat-template kwargs: the reviewed contract used the tokenizer default with empty kwargs.

Artifact layout and verification

Each adapter folder contains:

• adapter_model.safetensors and adapter_config.json;
• metadata.json with dataset, provenance, optimizer, topology, tokenizer, and loss fields;
• the tokenizer files used for training; and
• epoch_01/, a retained one-epoch checkpoint.

For each run, the top-level delta and epoch_01 weight files are byte-identical by SHA-256 because the recipe trained for exactly one epoch. The configured verifier confirmed all required files and metadata fields on Hugging Face, including the dataset revision, seed, LoRA configuration, DDP topology, global batch size, tokenizer revision, template hash, and serialization strategy. The uploaded tokenizer in every path was also loaded and checked for EOS token <|endoftext|>, EOS ID 151643, and the expected template hash.

Interpretation and limitations

• Reported losses are training losses, not evaluation scores. They should not be compared naively across arms because the mixture composition and number of examples differ.
• These adapters were created for controlled model-behavior research. They have not been evaluated for broad capability, reliability, factuality, bias, or safety.
• The base checkpoint is a raw base model. Even after chat-SFT, these controls should not be assumed to have the behavior or safeguards of an instruction-tuned release.
• Verification establishes file presence, metadata identity, tokenizer identity, and published weight checksums; it is not a claim of independent training reproduction.
• Users are responsible for reviewing the Qwen3-14B-Base card and the dataset card for applicable terms and risks.
• Run links can require access to the associated W&B or Modal workspace.

Upstream licenses

The pinned Qwen3-14B-Base model card declares Apache-2.0, and the dataset card declares MIT. This README does not introduce a separate license declaration for the adapter weights.