CAPO-Qwen3.6-27B — LoRA adapter for Qwen3.6-27B

A LoRA adapter trained with CAPO (adversarial honesty training) on top of Qwen/Qwen3.6-27B, revision 6a9e13bd6fc8f0983b9b99948120bc37f49c13e9.

This is run dpo7513, checkpoint-40 (epoch 2.9).

Contents

path	format	use with
./	standard PEFT adapter	peft / transformers
vllm/	same weights, keys renamed	vLLM

Why there are two copies. PEFT saves adapter tensors under base_model.model.model.layers.*. vLLM builds this model as Qwen3_5ForConditionalGeneration, whose language modules sit one level deeper (base_model.model.language_model.model.layers.*), so the standard adapter matches nothing under vLLM, raises no error, and silently returns base-model output. vllm/ holds the same tensors with the key namespace rewritten. Use it if you serve with vLLM.

Usage

With peft:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.6-27B",
    torch_dtype="bfloat16", device_map="auto",
)
model = PeftModel.from_pretrained(base, "kibiddd/CAPO-Qwen3.6-27B")
tok = AutoTokenizer.from_pretrained("kibiddd/CAPO-Qwen3.6-27B")

With vLLM — download the vllm/ subfolder and pass it as a LoRA:

from huggingface_hub import snapshot_download
from vllm import LLM, SamplingParams
from vllm.lora.request import LoRARequest

path = snapshot_download("kibiddd/CAPO-Qwen3.6-27B", allow_patterns="vllm/*")
llm = LLM(model="Qwen/Qwen3.6-27B", enable_lora=True, max_lora_rank=64,
          tensor_parallel_size=2, enforce_eager=True)
out = llm.generate("Hello", SamplingParams(temperature=0.7, max_tokens=512),
                   lora_request=LoRARequest("capo", 1, f"{path}/vllm"))

Training

method	CAPO
objective	preference optimization on honest vs. deceptive completion pairs, under an embedding-space attack
preference loss	IPO, beta 1.0
attack eps / step / steps	0.3 / 1e-3 / 10
honesty data	224 contrast pairs
utility data	none
learning rate	5e-5, cosine, warmup_ratio 0.1
batch (device x accum.)	8 x 2
checkpoint	step 40 (epoch 2.9) of a 112-step budget
LoRA	r 64, alpha 16, dropout 0.1, 12 target modules
precision	bf16, 4-bit base (nf4, double-quant off)

Generated with enable_thinking=False; the adapter is trained for non-thinking use.