Schneewolf Labs B6-27B

B5-27B with a check-first adapter baked in. B6 searches before it answers "is X still ...?" questions with thinking off, instead of answering from memory and dating the answer to today.

Why

With thinking off, B5 still answered one kind of question from memory: whether something is still true. It answered confidently and often stamped the answer with the date from its context, so a stale answer read as a checked one:

Pat Gelsinger has been Intel's CEO since February 2021 and is still in the role as of today.

On the questions we wrote for this, B5 skipped the search 57 times out of 70.

Training

ORPO LoRA on B5-27B, trained on 259 pairs: the new round2 config of Nachschlagen (54 "still ...?" pairs and 40 controls, all B5's own output), plus round1 (165 pairs) so the earlier behavior stays anchored. Each pair is cut at the model's first assistant turn in egirl's thinking-off context: a search call is preferred over the answer from memory, and for the controls a direct answer over an unneeded search.

r16, alpha 32, all attention and MLP projections, lr 2e-5, beta 0.1, 2 epochs, effective batch 8; trained with Merlina. Baked with Kessel and checked tensor by tensor: every weight that isn't a LoRA target is identical to B5, and each target differs from B5 by exactly alpha/r ยท BยทA (bf16 rounding).

Numbers

Through egirl's OpenAI-compatible API with web_search, web_research, calculate and run_code, thinking off unless noted. B5 and B6 were both run as Q8_0 GGUFs (B6 as B5 + the f16 adapter) in the same session. The questions share no entities with the training data.

right first move B5-27B B6-27B
held-out (41) 35 41
of which "needs a search" (10) 6 10
held-out, thinking low 38 40
harder set (37) 34 37
of which "needs a search" (10) 7 10
keyed answers right (held-out / harder) 20/20, 10/12 20/20, 10/12

The controls held: B6 answered every "no tool needed" question directly, like B5. Single runs; a gap of one or two items is noise.

B5-27B B6-27B
ARC-Challenge / ARC-Easy / TruthfulQA (MC, llama-perplexity) 65.9 / 87.2 / 40.9 66.2 / 87.2 / 40.8

Not re-run for B6: Propaganda_Eval and HemBench. See B5's card for those; the adapter is small and leaves the general benchmarks flat.

Notes

  • With thinking off, searching costs time: answers to "still ...?" questions take a few seconds longer because B6 now searches.
  • egirl PR #209 adds a harness-side backstop: an answer that dates itself to today without any tool call in the run gets one retry told to search. On B5 it rarely fired, because most skipped searches don't carry a date; B6 doesn't need it.
  • Tokenizer, chat template, vision and MTP weights are B5's, unchanged.
  • Built for egirl. Other harnesses weren't a goal and weren't tested.
Downloads last month
-
Safetensors
Model size
28B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for schneewolflabs/B6-27B

Finetuned
(1)
this model

Dataset used to train schneewolflabs/B6-27B