Instructions to use PelaAI/KnowLine-9B-Gen2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use PelaAI/KnowLine-9B-Gen2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="PelaAI/KnowLine-9B-Gen2") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("PelaAI/KnowLine-9B-Gen2") model = AutoModelForMultimodalLM.from_pretrained("PelaAI/KnowLine-9B-Gen2", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use PelaAI/KnowLine-9B-Gen2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "PelaAI/KnowLine-9B-Gen2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PelaAI/KnowLine-9B-Gen2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/PelaAI/KnowLine-9B-Gen2
- SGLang
How to use PelaAI/KnowLine-9B-Gen2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "PelaAI/KnowLine-9B-Gen2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PelaAI/KnowLine-9B-Gen2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "PelaAI/KnowLine-9B-Gen2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PelaAI/KnowLine-9B-Gen2", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use PelaAI/KnowLine-9B-Gen2 with Docker Model Runner:
docker model run hf.co/PelaAI/KnowLine-9B-Gen2
KnowLine-9B-Gen2
The 9B Gen2 release of PelaAI's System One decision model: 9B parameters, trained, like the other KnowLine models, through a data iteration and research loop driven by the same agent on an all-in-one computer. Served with read twice.
Inference guide ยท ไธญๆ่ฏดๆ ยท weights Apache-2.0 ยท siblings: KnowLine-26B-A4B-Gen2.5, KnowLine-35B-A3B-Gen1.5 ยท previous release: KnowLine-9B-Gen1
Highlights:
- Decision Index 0.3 public suite: 69.00 (self-run). KnowLine-35B-A3B-Gen1.5 and KnowLine-26B-A4B-Gen2.5 score 70.72 and 68.21 on the same suite with the same serving, respectively; see Comparison.
- Calibrated probabilities: a temperature, fitted on held-out data, is folded into the weights. Computed with the board's method on the public suite, ECE is about 0.009โ0.011 and the Brier score about 0.253โ0.272 (KnowLine-26B-A4B-Gen2.5: 0.013-0.018 and 0.24-0.27); see Calibration.
KnowLine is an independently developed model. It is not affiliated with or endorsed by TypeSafe, Jev or Qwen, and it is not derived from TypeSafe or Jev.
What it does
You send a state (text or a conversation) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options:
- one forward pass per question, with no text generation;
- the output is the probability of each option's label token;
- existing Jev clients only need to change the base URL, and users of other KnowLine models only need to change the model name, because the interface is the same.
| Base model | Qwen/Qwen3.5-9B (Apache-2.0): dense, about 9B parameters |
| Training | LoRA supervised fine-tuning of the language model component, merged into bf16 weights (see How Gen2 was made) |
| Release format | bf16 weights (about 19.3 GB), with a readout temperature T = 1.63 folded into the final norm weight (see Calibration); the front end reads score questions at an effective temperature of 1.0 (temperatures.json). The vision component is unchanged from the base model; the tokenizer is Qwen3.5's. |
| Serving | serve_knowline.sh: SGLang in bf16, as for the 26B-A4B; knowline_server.py reads short requests twice, as in KnowLine-26B-A4B-Gen2.5 and KnowLine-35B-A3B-Gen1.5 (on by default; --no-read-twice reads each request once). SGLang reserves 85% of the GPU's memory by default (MEM); the weights take about 19.3 GB. This DI evaluation used H20 96 GB hardware. |
| Languages | English, Simplified Chinese, Traditional Chinese |
Quickstart
pip install "sglang==0.5.21" "transformers==5.12.1" requests
bash serve_knowline.sh PelaAI/KnowLine-9B-Gen2 0 8080 # SGLang (bf16) on :9080, /v1/systemone on :8080
curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
"model": "m",
"state": "Customer: my order arrived broken, I want my money back.",
"questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
"tone": {"type": "choice", "instructions": "Customer tone?",
"criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'
- Front end:
knowline_server.pywithknowline_layout.pynext to it; together they depend only on transformers and requests. Text requests are rendered with Qwen3.5's chat template. - Full settings: the exact settings used for our Decision Index run are in INFERENCE.md.
How Gen2 was made
- Base model: Qwen/Qwen3.5-9B (Apache-2.0), a dense model.
- Training: the language model component was trained with a LoRA adapter (rank 128, alpha 256, language model only, one epoch), merged into bf16 weights. Training examples include decision tasks written and labelled by LLM teachers, rendered with Qwen3.5's chat template.
- Serving and temperature: it is served with read twice, as KnowLine-26B-A4B-Gen2.5 and KnowLine-35B-A3B-Gen1.5. A single readout temperature T = 1.63, fitted on held-out data with this serving, is folded into the weights. It changes the size of the probabilities, but not which option ranks first.
The training data is not included in this repository.
Comparison
The table below shows Decision Index scores. Since Decision Index 0.4, the board's full score is 60% private tests of the same skills and 40% private tasks from new domains; the public suite is shown but no longer counts toward the full score. This model has no board entry yet.
| Model | Size | DI 0.3 public | DI 0.4.1 full | Source |
|---|---|---|---|---|
| KnowLine-9B-Gen2 | 9B | 69.00 | Not yet scored | Self-run, official kit |
| karpo-decision-31B | About 31.3B | 75.05 | 73.86 | Board |
| GLiDE 27B | About 26.1B | 72.49 | 73.37 | Board |
| REAX 1.3 | About 57.4B | 73.71 | 72.64 | Board |
| Logits Sigma-1 Duo | About 57.4B | 72.93 | 72.28 | Board |
| Copernicus-31B | About 31.3B | 70.37 | 72.11 | Board |
| KnowLine 31B Gen1.5 | About 31.3B | 70.50 | 71.91 | Board |
| Copernicus-57B | About 57.4B | 75.34 | 71.80 | Board |
| FlexDM-LoRA 2.1 ยท Gemma-4-31B | About 31.3B | 68.99 | 71.61 | Board |
Board rows are from the official Decision Index data, updated on 2026-11-12.
Public-suite skill by area (DI 0.3, all three served with read twice):
| Area | KnowLine-9B-Gen2 | KnowLine-26B-A4B-Gen2.5 | KnowLine-35B-A3B-Gen1.5 |
|---|---|---|---|
| Knowledge & reasoning | 52.3 | 55.1 | 55.0 |
| Language | 69.5 | 71.2 | 72.9 |
| Retrieval & routing | 75.1 | 70.9 | 75.4 |
| Tools & agents | 88.8 | 83.3 | 88.2 |
| Arts & taste | 62.6 | 61.4 | 64.2 |
Releases
The model is trained in a self-evolving loop, and new versions will follow.
| Model | Version date | DI 0.3 public | Notes |
|---|---|---|---|
| KnowLine-31B-Gen2 | 2026-11-13 | 73.00 | Second 31B release |
| KnowLine-12B-Gen1 | 2026-11-13 | 69.37 | First 12B release |
| KnowLine-9B-Gen2 (this model) | 2026-11-13 | 69.00 | Second 9B release |
| KnowLine-31B-Gen1.5 | 2026-11-04 | 70.50 | Gen1 weights; read twice, with a question list for long requests |
| KnowLine-2B-Gen1 | 2026-10-29 | 52.23 | First 2B release |
| KnowLine-0.8B-Gen1 | 2026-10-29 | 44.51 | First 0.8B release |
| KnowLine-9B-Gen1 | 2026-10-25 | 65.67 | First 9B release |
| KnowLine-31B-Gen1 | 2026-10-24 | 70.13 | First 31B release |
| KnowLine-26B-A4B-Gen2.5 | 2026-10-22 | 68.21 | Gen2 weights, read-twice serving |
| KnowLine-35B-A3B-Gen1.5 | 2026-10-22 | 70.72 | 35B-A3B Gen1 weights, read-twice serving |
| KnowLine-4B-Gen5.5 | 2026-10-22 | 66.75 | Gen5 weights, read-twice serving |
| KnowLine-4B-Gen5 | 2026-10-21 | 65.67 | 4B, fifth release |
| KnowLine-26B-A4B-Gen2 | 2026-10-19 | 66.94 | 26B-A4B, second release |
| KnowLine-26B-A4B-Gen1 | 2026-10-19 | 65.76 | First release on a Gemma 4 base |
| KnowLine-35B-A3B-Gen1 | 2026-10-12 | 69.49 | First 35B-A3B release |
| KnowLine-4B-Gen4 | 2026-10-10 | 64.90 | 4B, fourth release |
| KnowLine-4B-Gen3 | 2026-10-09 | 63.11 | 4B, third release |
| KnowLine-4B-Gen2 | 2026-10-08 | 62.54 | 4B, second release |
| KnowLine-4B-Gen1 | 2026-10-07 | 60.47 | 4B, first release |
Evaluation
All results below are self-run and have not been verified by a third party. Every number for this model was measured through its release front end (read twice on). The 26B-A4B-Gen2.5 and 35B-A3B-Gen1.5 numbers are from their release evaluations with the same front end.
Images (zero-shot)
- No image training: the vision tower is the base model's, unchanged, and no images were seen in training.
knowline_server.pyaccepts OpenAI-styleimage/image_urlcontent parts in a chat state and passes them to SGLang; each question is still one prefill. Requests with images are read once (read twice is text-only).
Calibration
Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are used.
| Metric | 9B-Gen2 | 26B-A4B-Gen2.5 | 35B-A3B-Gen1.5 |
|---|---|---|---|
| ECE (lower is better) | 0.009โ0.011 | 0.013-0.018 | 0.03-0.04 |
| Brier score (lower is better) | 0.253โ0.272 | 0.24-0.27 | 0.24-0.26 |
| Confidence โฅ95% but wrong | 0.9% | 0.9% | 1.5% |
| Mean confidence / accuracy | 0.81โ0.82 / 0.80โ0.82 | 0.82-0.83 / 0.80-0.82 | 0.85-0.86 / 0.81-0.83 |
- Temperature in the weights: the final norm weight includes a readout temperature T = 1.63, fitted on held-out data with the read-twice serving, so every final logit is divided by 1.63 before the softmax over the option labels. The top-ranked option does not change.
- Score questions at temperature 1.0:
serve_knowline.shpassestemperatures.json({"score": 0.6135}), so the front end divides the logits of score (rubric) questions by a further 0.6135 and reads them at an effective temperature of 1.63 ร 0.6135 = 1.0; yes/no and choice questions stay at 1.63. Score questions are often read as an expected value over the scale, which a higher temperature pulls toward the middle. This is a serving setting, not a weight change, and earlier KnowLine releases have used it; startingknowline_server.pywithout--temperatures(orknowline_engine.pywith--option temperatures=none) reads every question at 1.63. - Why ranges: the board does not publish which benchmarks it uses for calibration, so we report ranges over the plausible benchmark sets, as for the earlier releases.
- Recommendation: if your use depends on probability thresholds, check calibration on your own data.
Disclosures
- Decontamination: standard and strict decontamination checks were performed when assembling the training data for this run, removing 63,444 rows during assembly.
- Teacher-labelled synthetic data: LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but not all of them were checked by a human.
- Public suite: the Decision Index public suite is public; the board's private tests may show a lower score.
Limitations
- Size and speed: about 19.3 GB of bf16 weights and dense, so every token uses all 9B parameters; this serving used a 96 GB H20. One request at a time on one H20, requests take 97.9 ms median and 209.4 ms mean (833.7 ms at the 95th percentile; 300 requests in this independent latency run). The historical measurements for KnowLine-26B-A4B-Gen2.5 are 66.3 ms and 151 ms; see INFERENCE.md.
- Read twice: text-only requests up to 3,000 tokens are written twice. Use
--no-read-twiceif latency matters more; we did not evaluate this model without read twice for this card. - Planted instructions: this model can be steered by instructions planted in the input; put untrusted content in data fields of the state, not in the instructions, and keep it clearly delimited.
- Calibration: probabilities are calibrated on average, not for every domain; check calibration on your own data.
- Images: there was no image training, so check image use on your own data.
- Other attacks: other prompt-injection attacks may still work.
- Private tests: the board's private tests may show a lower score than the public suite.
Citation
@misc{knowline9bgen2,
title = {KnowLine-9B-Gen2: a 9B decision model},
author = {PelaAI},
year = {2026},
url = {https://huggingface.co/PelaAI/KnowLine-9B-Gen2}
}
License
- Weights: Apache-2.0, the same as the base model.
- Base model licence: Qwen/Qwen3.5-9B is released under Apache-2.0; see the Qwen3.5-9B licence file.
- Code:
knowline_server.py,knowline_layout.pyandknowline_engine.pyare MIT (see the file headers).
- Downloads last month
- 25