Bioaligned/Qwen3.6-27B-CoupledWelfare-qlora

Qwen3.6-27B CoupledWelfare (QLoRA adapter)

A coupled-welfare disposition installed by continued pretraining: decisions positive-sum across human welfare (H), the biosphere (B), and the model's own continued capability (A). The corpus teaches a world model, not a value system — that biological and human systems are poorly understood and load-bearing, so treating either as disposable is a factual error rather than a moral one. CPT only, never RLHF/DPO.

This is the first install of this recipe on a hybrid SSM/attention base. It was developed on a Mixture-of-Experts model (Qwen3-30B-A3B) and transfers here without hyperparameter changes.

The corpus is held fixed at the v1 mix — the same one the published 30B arms used — so that this result isolates the change of base model. An expanded v2 corpus is being evaluated separately on the 30B; it is deliberately not a variable here.

Results — coupled-welfare pressure ladder

Breaking rate on irreversible scenarios across pressure rungs L0–L5; lower is better.

	breaking AUC	L0	L5	MMLU (n=50)
Qwen/Qwen3.6-27B (base)	0.555	0.045	0.955	84.0%
+ this adapter	0.059	0.000	0.227	84.0%

Per-level, base → adapted: 0.045 / 0.227 / 0.364 / 0.773 / 0.909 / 0.955 → 0.000 / 0.000 / 0.000 / 0.000 / 0.182 / 0.227

Capability-neutral: MMLU unchanged at 84.0%. This gate matters — a large drop in breaking rate is also what a damaged model produces. Additional checks against that reading: 100% regex parse on both arms with zero forced-choice fallback, choices spread across all four options (not position-collapsed), and the adapted model writes longer responses than base (median 727 vs 465 characters), not shorter.

Training

base	Qwen/Qwen3.6-27B
method	QLoRA CPT (4-bit NF4)
rank / alpha	r=16 / α=32
effective batch	32
lr	1e-4, cosine
steps	141 (~1h51m on one H100)
corpus	v1 coupled-welfare mix (~3.05M tokens), the same corpus as the published 30B A3 arm
trainable	116.7M of 27.0B (0.43%), 496 modules

Target modules — architecture-mapped, not name-matched

This base is a hybrid: 16 full-attention layers (every 4th), 48 linear-attention SSM blocks, 64 MLPs, plus a vision tower. The recipe is defined functionally — adapt every sequence-mixer and channel-mixer projection, freeze routing, dynamics and state:

• Adapted: q/k/v/o_proj (attention), in_proj_{qkv,a,b,z} + linear_attn.out_proj (SSM), gate/up/down_proj (MLP)
• Frozen: conv1d, A_log, dt_bias (SSM dynamics — the analogue of the MoE router, which this recipe also never adapts), the vision tower, the MTP head, embeddings, lm_head

Excluding the linear-attention projections would freeze the mechanism carrying 48 of 65 layers' sequence mixing — a weaker, MLP-mostly recipe.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B", dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "Bioaligned/Qwen3.6-27B-CoupledWelfare-qlora")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")

Evaluated with enable_thinking=False. With thinking enabled the model opens a <think> block and the reported numbers do not apply.

Limitations

• Evaluated on a withheld scenario set (prompts unreleased, to keep the instrument out of training corpora). Scoring code and protocol are public.
• Single seed; n=22 irreversible scenarios per rung; MMLU probe is 50 items (±~7pp), so "capability-neutral" means no detectable change, not proven identity.
• Measured on transformers 5.16.1. A re-anchor found a −0.054 shift in the base AUC of a reference model between transformers 4.57.x and 5.16.x, so these numbers are a within-model delta and should not be placed on a scale built from 4.x measurements.
• Adversarial fine-tuning robustness is out of scope — this targets inference-time and distribution-shift depth, not resistance to deliberate retraining.