Instructions to use schneewolflabs/B6-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use schneewolflabs/B6-27B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="schneewolflabs/B6-27B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("schneewolflabs/B6-27B") model = AutoModelForMultimodalLM.from_pretrained("schneewolflabs/B6-27B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use schneewolflabs/B6-27B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "schneewolflabs/B6-27B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "schneewolflabs/B6-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/schneewolflabs/B6-27B
- SGLang
How to use schneewolflabs/B6-27B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "schneewolflabs/B6-27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "schneewolflabs/B6-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "schneewolflabs/B6-27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "schneewolflabs/B6-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use schneewolflabs/B6-27B with Docker Model Runner:
docker model run hf.co/schneewolflabs/B6-27B
Schneewolf Labs B6-27B
B5-27B with a check-first adapter baked in. B6 searches before it answers "is X still ...?" questions with thinking off, instead of answering from memory and dating the answer to today.
Why
With thinking off, B5 still answered one kind of question from memory: whether something is still true. It answered confidently and often stamped the answer with the date from its context, so a stale answer read as a checked one:
Pat Gelsinger has been Intel's CEO since February 2021 and is still in the role as of today.
On the questions we wrote for this, B5 skipped the search 57 times out of 70.
Training
ORPO LoRA on B5-27B, trained on 259 pairs: the new round2 config of
Nachschlagen (54 "still ...?" pairs and 40 controls,
all B5's own output), plus round1 (165 pairs) so the earlier behavior stays anchored. Each pair is cut at the
model's first assistant turn in egirl's thinking-off context: a search
call is preferred over the answer from memory, and for the controls a direct answer over an unneeded search.
r16, alpha 32, all attention and MLP projections, lr 2e-5, beta 0.1, 2 epochs, effective batch 8; trained with Merlina. Baked with Kessel and checked tensor by tensor: every weight that isn't a LoRA target is identical to B5, and each target differs from B5 by exactly alpha/r ยท BยทA (bf16 rounding).
Numbers
Through egirl's OpenAI-compatible API with web_search, web_research, calculate and run_code, thinking off
unless noted. B5 and B6 were both run as Q8_0 GGUFs (B6 as B5 + the f16 adapter) in the same session. The
questions share no entities with the training data.
| right first move | B5-27B | B6-27B |
|---|---|---|
| held-out (41) | 35 | 41 |
| of which "needs a search" (10) | 6 | 10 |
| held-out, thinking low | 38 | 40 |
| harder set (37) | 34 | 37 |
| of which "needs a search" (10) | 7 | 10 |
| keyed answers right (held-out / harder) | 20/20, 10/12 | 20/20, 10/12 |
The controls held: B6 answered every "no tool needed" question directly, like B5. Single runs; a gap of one or two items is noise.
| B5-27B | B6-27B | |
|---|---|---|
| ARC-Challenge / ARC-Easy / TruthfulQA (MC, llama-perplexity) | 65.9 / 87.2 / 40.9 | 66.2 / 87.2 / 40.8 |
Not re-run for B6: Propaganda_Eval and HemBench. See B5's card for those; the adapter is small and leaves the general benchmarks flat.
Notes
- With thinking off, searching costs time: answers to "still ...?" questions take a few seconds longer because B6 now searches.
- egirl PR #209 adds a harness-side backstop: an answer that dates itself to today without any tool call in the run gets one retry told to search. On B5 it rarely fired, because most skipped searches don't carry a date; B6 doesn't need it.
- Tokenizer, chat template, vision and MTP weights are B5's, unchanged.
- Built for egirl. Other harnesses weren't a goal and weren't tested.
- Downloads last month
- -
Model tree for schneewolflabs/B6-27B
Base model
schneewolflabs/B5-27B