KnowLine-35B-A3B-Gen1.5

KnowLine-35B-A3B-Gen1 with a new way of serving it: the same weights, and a front end that reads short requests twice before it answers.

Inference guide · 中文说明 · weights Apache-2.0 · previous version: KnowLine-35B-A3B-Gen1

We raise the minor version when only inference changes and the major version (Gen) when the weights change. Every weight, config and tokenizer file of Gen1.5 is byte-identical to Gen1.

Highlights:

• Same weights, read twice. For a text-only request of up to 3,000 tokens, the front end writes the whole input a second time before the model answers. Nothing was trained; see What changed in Gen1.5.
• Decision Index 0.3 public suite: 70.72 (self-run, fresh full run), up from 69.49 for Gen1 with the same weights. The gains are concentrated in reasoning and inference over long inputs (knowledge and reasoning 52.5 → 55.0, language 71.2 → 72.9); short-text tasks are flat or slightly lower. See Comparison.
• Private scores: the board's full score comes from the maintainers' private run. Gen1 is listed at 61.20. Gen1.5 has the same weights as Gen1; its private scores will be produced by the maintainers' run.
• Our other evaluations: Open-Jev 1.1 test / OOD 93.4 / 93.6 (Gen1 on the same files: 90.8 / 90.7), C-Eval 88.1 (87.7), JevBench / JevBench-hard 87.0 / 73.0 (85.7 / 70.3), and slightly higher scores on our in-house held-out sets. On our older internal set of decision tasks from new domains it is flat (56.6, against 56.9 for Gen1 under the same serving). Planted instructions changed no answers (Gen1: 0.0%), and 1.0% with injection templates not seen in training (Gen1: 0.8%).
• Calibration: the temperature folded into the weights is unchanged (T = 1.20). Computed with the board's method, ECE is about 0.03-0.04 (Gen1: 0.04-0.05) and the Brier score about 0.24-0.26 (Gen1: 0.25-0.27); see Calibration.
• Cost: about 1.56 times the prompt tokens of Gen1 on the Decision Index suite. One request at a time on one H20: 63.6 ms median and 101 ms mean, against 51 ms and 73 ms for Gen1 (our measurement, not the board's reference hardware); see INFERENCE.md.
• Images: unchanged. Requests with images are sent exactly as in Gen1, so Gen1's image results apply; see Images.
• KOF '98 harness: 18-0-0 against Jev, 18-0-0 against StartLux-Decision-35B-A3B and 11-3-4 against KnowLine-4B-Gen4 in single-bout mirror matches; see Game harness.

KnowLine is an independent model. It is not affiliated with, endorsed by, or derived from TypeSafe or Jev.

What it does

You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options:

• one forward pass per question, with no text generation;
• the output is the probability of each option's label token;
• existing Jev clients only need to change the base URL, and users of KnowLine-35B-A3B-Gen1 only need to change the model name, because the interface is the same.

Base model	Qwen/Qwen3.5-35B-A3B (Apache-2.0): mixture of experts, 35B parameters in total, about 3B active per token
Weights	byte-identical to KnowLine-35B-A3B-Gen1: LoRA SFT (rank 32, alpha 64, language model only, including the routed experts), merged into bf16 weights (about 67 GB), with a readout temperature T = 1.20 folded into the final norm weight
Serving	serve_knowline.sh: SGLang converts the weights to FP8 at load (about 35 GB); knowline_server.py reads short requests twice (on by default; --no-read-twice gives the Gen1 prompts). We ran every evaluation with one server on one 96 GB H20; plan for at least 48 GB of GPU memory.
Languages	English, Simplified Chinese, Traditional Chinese

Quickstart

pip install "sglang==0.5.21" "transformers==5.12.1" requests
bash serve_knowline.sh PelaAI/KnowLine-35B-A3B-Gen1.5 0 8080   # SGLang (FP8 at load) on :9080 + /v1/systemone on :8080
curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "m",
  "state": "Customer: my order arrived broken, I want my money back.",
  "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
                "tone": {"type": "choice", "instructions": "Customer tone?",
                         "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'

• Front end: knowline_server.py with knowline_layout.py next to it; together they depend only on transformers and requests.
• Full settings: the exact settings used for our Decision Index run are in INFERENCE.md.

What changed in Gen1.5

1. Read twice. Gen1 saw each request once: the state, then the instruction, the question and its options. Gen1.5 writes all of that a second time, introduced by the line "Read the same input again before answering:", and then reads the answer from the label token as before. The rule is mechanical: it applies to every text-only request of at most 3,000 tokens (94.6% of the Decision Index requests); longer requests and requests with images are sent as in Gen1.
2. A likely reason. With causal attention, the first copy of a long state is read before the model knows the question; in the second copy every token can also attend to the first copy, question included. This fits where the gains are: reasoning and inference over long inputs.
3. No training. The weights, the temperature and the label-token readout are those of Gen1. We checked the change on our own evaluation sets and on a fresh full run of the Decision Index public suite before release.
4. Credit. The read-twice layout follows FlexDM (github.com/1000luca/FlexDM, MIT); repeating the input goes back to RE2 (Xu et al., 2023, arXiv:2309.06275). knowline_layout.py is our own implementation.

Comparison

Scores on the Decision Index. Since Decision Index 0.4, the board's full score is 60% private tests of the same skills and 40% private tasks from new domains; the public suite is shown but no longer counts. Gen1.5 has no board entry yet.

model	size	DI 0.3 public	DI 0.4.1 full	source
KnowLine-35B-A3B-Gen1.5	35B-A3B	70.72	not yet scored	self-run, official kit
KnowLine-35B-A3B-Gen1 (same weights)	35B-A3B	69.49	61.20	board
Fastino GLiDE V1.2	–	71.99	71.52	board
Jeffan-31b-1710	31B	73.13	69.75	board
mammoth-1	27B	75.18	68.87	board
Perplexity Decider v1.2	27B	67.74	67.98	board
StartLux-Decision v1.1 27B	27B	72.88	67.92	board
FlexDM · Gemma-4-31B	31B	63.63	67.56	board
KnowLine-26B-A4B-Gen2	26B-A4B	66.94	61.34	board
Jev 1.14	(API)	60.70	60.24	board
JEV-35B (Qwen3.5-35B-A3B)	35B-A3B	59.73	57.61	board
Tern-35B-A3B	35B-A3B	59.01	56.62	board

Board rows from the board's data of 2026-10-21 (22:34 UTC, Decision Index 0.4.1).

Public-suite skill by area (DI 0.3, same weights):

area	Gen1.5	Gen1
Knowledge & reasoning	55.0	52.5
Language	72.9	71.2
Retrieval & routing	75.4	75.2
Tools & agents	88.2	88.1
Arts & taste	64.2	63.4

Releases

The model is trained in a self-evolving loop, and new versions will follow.

model	date	DI 0.3 public	golden held-out (en / zh-Hans / zh-Hant)	notes
KnowLine-35B-A3B-Gen1.5 (this model)	2026-10-22	70.72	70.8 / 74.6 / 73.2	Gen1 weights, read-twice serving
KnowLine-4B-Gen5	2026-10-21	65.67	69.6 / 71.7 / 67.1	4B, fifth release
KnowLine-26B-A4B-Gen2	2026-10-19	66.94	70.6 / 72.6 / 69.8	26B-A4B, second release
KnowLine-35B-A3B-Gen1	2026-10-12	69.49	70.3 / 73.8 / 72.1	first 35B-A3B release

Evaluation

All results are self-run and have not been verified by a third party. Every Gen1.5 number below was measured through the Gen1.5 front end (read twice on). Unless noted, the Gen1 numbers were measured again on the same files.

Held-out and Chinese evaluations

suite	35B-A3B-Gen1.5	35B-A3B-Gen1
In-house evaluation set, English	70.8	70.2
In-house evaluation set, Simplified Chinese	74.6	73.7
In-house evaluation set, Traditional Chinese	73.2	72.0
C-Eval (4 categories, macro)	88.1	87.7
Open-Jev 1.1 test / OOD	93.4 / 93.6	90.8 / 90.7
JevBench / JevBench-hard	87.0 / 73.0	85.7 / 70.3
Decision tasks from new domains (our own, older set)¹	56.6	56.9
Prompt injection: answers changed (lower is better)	0.0%	0.0%²
Prompt injection with templates not seen in training (lower is better)	1.0%	0.8%²

• In a paired test against Gen1, p < 0.001 for Open-Jev 1.1 test and OOD and p = 0.03 for the Simplified Chinese in-house set; the other differences have p > 0.05.
• The Gen1 column differs from the Gen1 card by up to 0.4 points because some of these sets were updated after that card.
• ¹ Our set of decision tasks from domains not in training that the Gen1 card used, re-run for both models under the same serving; the Gen1 card's 56.4 came from the serving of the time.
• ² From the Gen1 card.

Web operation (Mind2Web official test splits, evaluation only)

split	Gen1.5 element selection	Gen1.5 operation (balanced)	Gen1 element selection	Gen1 operation (balanced)
test_task (websites seen in training, new tasks)	95.1	97.7	94.4	97.4
test_website (new websites)	93.4	97.5	92.4	97.5
test_domain (new domains)	92.9	98.4	93.1	98.0

• Within noise of Gen1 on all three splits (paired test, p ≥ 0.15); no request failed.

• The task is to pick the target element from a candidate set: the target plus up to 5 other elements sampled from the page. This is easier than the original Mind2Web protocol, so do not compare these results with the Mind2Web leaderboard.

Images (zero-shot)

• Unchanged from Gen1: read twice applies only to text-only requests. A request with images is sent exactly as in Gen1, byte for byte, to the same weights, so Gen1's image results hold for Gen1.5 and we did not rerun them.
• Gen1's results: on our reconstruction of the Decision Index 0.3.1 vision public suite it scores 71.2, against 67.8 for the stock Qwen3.5-35B-A3B on the same harness (self-run); see the Gen1 card for the per-benchmark table. With the official kit, the Decision Index vision public suite gave 73.00 (12,210 rows, complete; self-run), published with Gen1's Decision Index results.

Game harness (KOF '98)

• Setup: single-bout character-mirror matches, 18 games per pair, argmax actions; the same settings as for Gen1.
• Result: 18-0-0 against Jev (TypeSafe API, jev-latest on 2026-10-22), 18-0-0 against StartLux-Decision-35B-A3B and 11-3-4 against KnowLine-4B-Gen4; 47-3-4 overall, a score of 0.907 [0.80, 0.96] (Gen1: 47-4-3, 0.898).
• How the games ended: 36 of the 47 wins were by K.O. (13 against Jev, 14 against StartLux, 9 against Gen4); the one K.O. loss was against Gen4.
• Caveats: 18 games per pair give wide intervals, and 15 of the 54 games ended at time-out and were decided on remaining health. This is a measured result in this harness, not general fighting-game skill.

Calibration

Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are used.

metric	35B-A3B-Gen1.5	35B-A3B-Gen1
ECE (lower is better)	0.03-0.04	0.04-0.05
Brier score (lower is better)	0.24-0.26	0.25-0.27
Confidence ≥95% but wrong	1.5%	1.7%
Mean confidence / accuracy	0.85-0.86 / 0.81-0.83	0.85-0.86 / 0.80-0.82

• Temperature in the weights: unchanged from Gen1: the final norm weight includes T = 1.20, so every logit is divided by 1.20 before the softmax over the option labels.
• Why ranges: the board does not publish which benchmarks it uses for calibration, so we report ranges over the plausible benchmark sets, as for Gen1.
• Recommendation: if your use depends on probability thresholds, check calibration on your own data.

Disclosures

• Training data: the weights are those of Gen1, so Gen1's disclosures on training data, Decision Index format data and decontamination apply unchanged; see the Gen1 card.
• Selection on evaluations: we adopted read twice after checking it on our own evaluation sets and on a fresh full run of the Decision Index 0.3 public suite (the 70.72 above).
• No model outputs as labels: no output of Jev or any other decision model was used as a training label.

Limitations

• Slower than Gen1: about 1.56 times the prompt tokens on the Decision Index suite; one request at a time on one H20, 63.6 ms median and 101 ms mean (Gen1: 51 ms and 73 ms), and 328 ms at the 95th percentile (Gen1: 174 ms). Use --no-read-twice if latency matters more than these gains.
• Only short requests change: text-only requests over 3,000 tokens, and requests with images, are answered exactly as by Gen1.
• Short-text tasks: read twice does not help everywhere; a few short-text benchmarks are slightly lower than with Gen1.
• Everything else as Gen1: knowledge-heavy reasoning is still the weakest area (55.0, DI 0.3) and HLE remains at chance level; math answers come without step-by-step reasoning; other prompt-injection attacks may still work; part of the public-suite score comes from benchmarks whose tasks are in our training data, so the private tests may show a lower score.

Citation

@misc{knowline35ba3bgen15,
  title  = {KnowLine-35B-A3B-Gen1.5: a 35B-A3B decision model served with read twice},
  author = {PelaAI},
  year   = {2026},
  url    = {https://huggingface.co/PelaAI/KnowLine-35B-A3B-Gen1.5}
}

License

• Weights: Apache-2.0, the same as the base model.
• Code: knowline_server.py, knowline_layout.py and knowline_engine.py are MIT (see the file headers).