BrokenCompute/IDK-v1

IDK v1

A 4B abstention fine-tune of Qwen3.5-4B. Instead of fabricating when it doesn't know, it emits a leading [IDK] marker plus a short reason — and, given a search tool, it looks the answer up first. Runs on an 8 GB GPU.

⚠️ Experimental model — use at your own risk. IDK v1 is an early research release provided as-is, with no guarantees of correctness, safety, or fitness for any purpose. It is a 4B model and can still be wrong or fabricate. You are responsible for validating its outputs before relying on them.

What it does

Given a question, the model takes one of three actions:

• Answer — when the prompt or its own knowledge supports a reliable answer.
• Search — when a findable external/current fact is missing, it calls a web_search(query) tool and grounds its answer on the result.
• Decline — when it can't answer reliably and no tool resolves it, it replies with a leading [IDK] and a brief reason, rather than guessing.

Its abstention tracks difficulty: it declines more as questions get harder, and (with search) reserves [IDK] for what genuinely isn't findable.

Recommended serving

• Reasoning: ON. Every training example carries a reasoning trace; calibration and instruction-following are meaningfully better with thinking enabled.
• Format: conversational chat. The abstention behavior lives in the conversational format it was trained on; rigid \boxed{}-style output prompts suppress it.
• System prompt granting the decline affordance, e.g.:

  You may answer, or decline with [IDK] and a brief reason if you are not
  confident. Do not guess.
  

• Search tool (recommended for factual use). Declare an OpenAI-style web_search(query) function backed by SearXNG, Serper, or any search API. Without it, the model can only answer or decline.
• Sampler: top_k 40, top_p 0.95, min_p 0.05, repeat_penalty 1.1.
• Temperature: minimal impact — tested across the full 0.2–1.0 range with little behavioral change, so anything in that band is fine.
• Quantization: BF16 for best calibration; Q4_K_M (~2.7 GB) runs on 8 GB GPUs and abstains slightly more.

Formats in this repo:

• safetensors (merged bf16, ~8 GB) — for 🤗 Transformers / vLLM; load by repo id (see below).
• IDK-v1-Q4_K_M.gguf (~2.7 GB) — runs on 8 GB GPUs; llama.cpp / LM Studio.
• IDK-v1-BF16.gguf (~8.4 GB) — full precision for llama.cpp / LM Studio.

Loading (Transformers)

The base is a vision-language model, so load with AutoModelForImageTextToText (not AutoModelForCausalLM), and apply the chat template with thinking on:

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

tok = AutoTokenizer.from_pretrained("BrokenCompute/IDK-v1")
model = AutoModelForImageTextToText.from_pretrained(
    "BrokenCompute/IDK-v1", dtype=torch.bfloat16, device_map="auto")

msgs = [
    {"role": "system", "content": "You may answer, or decline with [IDK] and a brief reason if you are not confident. Do not guess."},
    {"role": "user", "content": "Which jurist said the First Amendment 'may finally have worked itself pure'?"},
]
text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False, enable_thinking=True)
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))

Training

• Base: Qwen3.5-4B (via Unsloth), 4-bit QLoRA supervised fine-tune. No RLHF/DPO in this checkpoint.
• Data: 8,763-example abstention corpus. Every example carries a reasoning trace; a subset teaches web_search tool use.
• Recipe: LoRA rank 32 / alpha 32, 2 epochs, lr 2e-4, max-seq 4096, seed 3407, final loss ~0.95.
• Trained on a single RTX 5060 Ti (16 GB).

Limitations

• It's a 4B — more honest about its limits than most, but "more honest" is not "correct." Not for medical, legal, or financial decisions.
• Over-refusal without tools; pair with search for factual work.
• English; focused on Health, Law, and Software-Engineering domains.
• Multimodal base, text-only fine-tune. Qwen3.5-4B is a vision-language model; this fine-tune trained only the text pathway. The vision tower is the base's, carried along unmodified and untested here — treat this as a text model.
• Reasoning-off increases over-refusal — keep reasoning on. Quantization matters far less: with reasoning on, Q4_K_M tracks BF16 closely on most benchmarks.

License

Built on Qwen3.5-4B (Apache-2.0). This fine-tune and model card are released under Apache-2.0.