qwen3-1.7b-cogs-ask-lora

LoRA adapter for Qwen/Qwen3-1.7B — the student model for the cogs ask pipeline. Merged full model: lewisdog/qwen3-1.7b-cogs-ask.

Two tasks, both emitting compact JSON exactly as the cogs runtime parses it:

task	output JSON
decompose	{"subquestions": [...]} — question → 1-4 sub-questions
synthesize	{"answer": "... [note-id] ...", "citations": [...], "abstained": bool} — grounded answer over retrieved notes; abstains on out-of-domain questions

Eval (full 63-record valid set, temp 0, repetition_penalty 1.1)

• JSON parse: 100% · decompose non-empty: 100%
• synthesize citation-validity: 77% (→ ~90% with the prefix-repair below)
• abstention preserved: 100% · false-abstention: 4%

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-1.7B"
model = AutoModelForCausalLM.from_pretrained(base, dtype="bfloat16", device_map="cuda")
model = PeftModel.from_pretrained(model, "lewisdog/qwen3-1.7b-cogs-ask-lora").eval()
tok = AutoTokenizer.from_pretrained("lewisdog/qwen3-1.7b-cogs-ask-lora")

enc = tok.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False,
    return_tensors="pt", return_dict=True,
).to(model.device)
out = model.generate(**enc, max_new_tokens=2048, do_sample=False,
                     repetition_penalty=1.1, pad_token_id=tok.pad_token_id)
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))

⚠️ Serving notes

1. Decode at temp 0 + repetition_penalty=1.1 (pure greedy runs away on long list outputs), enable_thinking=False, stop on <|im_end|>.

2. Repair citation prefixes — most citation errors are a dropped/wrong namespace prefix (enisa-… → sources-enisa-…). Normalize against the note ids present in the prompt and drop anything unverifiable:

   PFX = ("sources-", "source-", "concepts-", "concept-", "entities-", "entity-")
   core = lambda x: next((x[len(p):] for p in PFX if x.startswith(p)), x)
   def repair(cite, note_ids):
       if cite in note_ids: return cite
       return {core(n): n for n in note_ids}.get(core(cite))   # None => drop it
   

3. Abstention is reliable — trust it to decline out-of-domain questions.

Training

LoRA r=32/α=64, dropout 0.05, targets = all attn+MLP proj. 5 epochs, eff. batch 16, max_seq 8192, lr 1e-4 cosine, bf16, gradient checkpointing, on a DGX Spark (GB10). Data: cogs distill ask pairs (475 base + abstentions oversampled 3×). Eval loss 1.906 → 0.758; eval token-acc 0.836. Full write-up in the repo RESULTS.md.

• PEFT 0.19.1 · TRL 1.7.1 · Transformers 5.13.0 · PyTorch 2.12.1+cu130