lychee888/Qwen3.8-Flash-Next-Hybrid-Decision

Qwen3.8 Flash Next Hybrid Decision

One model, one engine, two interfaces: System 1 + System 2. Conventional LLM and decision model (jev-style) output on one vLLM instance.

Runtime and source: lychee888/QwenFN-Hybrid-Decision-Engine provides the patched serving code, capture runner, configuration generator and agent skill this model needs.

This package adds typed decisions to Qwen3.8-Flash-Next while keeping ordinary chat on the same engine. A decision request returns probabilities over the options you supply: 2 to 256 options for a choice, or a boolean. It returns no generated text.

The model weights are unchanged from Qwen3.8-Flash-Next-Uncensored-NVFP4-FP8PLE, revision 1b3c3cedc14fb3bee4294dea5e3515af74b8268a. The contribution here is the patched runner, request-bound logit capture and API software. There is no newly trained head or LoRA in the default package.

Since 20 October 2026 this package has been our production deployment on two DGX Spark (GB10) nodes: a complete model tree assembled with this package's tools, the matching runtime from the companion GitHub repository, and the typed gateway. The maintainer-host deployment accepted typed 256-option wide calls alongside the 2 to 16 option path, with legacy chat, tools, SSE and MTP behaviour unchanged. The evaluation panels below also ran there on 20 October 2026.

Lineage and acknowledgments

AutoTrust AI's JEV-27B was the architectural reference: a frozen generation backbone shared with a typed decision path, a 24-slot layout and the answer-token verbalizers. We used revision ff9b67b9bcb2dc5a8499d540e0c4cd10b254f3a2 for the reference protocol and bare-template control. The production prompt comes from blockbrain-ai/swanone-recipe, while the typed primitives and System One API format originate with TypeSafe AI.

AutoTrust's model has a trained decision LoRA and head. This package contains neither their weights nor a trained replacement; it reads the unchanged Flash-Next model's own logits. See NOTICE.md for attribution.

How it works

The patched runner captures the option-token logits at the end of the prompt. The gateway applies a mask and softmax, then maps the result to the caller's option keys. It checks the capture's request identity and prompt metadata before returning it.

Typed calls use a frozen prompt surface with thinking off. Choice questions from 2 to 16 options use the qwen_chat_letters-v1 surface and a fixed 24-slot internal representation; that layout is a wire format for small choices and booleans, not the maximum choice count and not a trained head. Choice questions from 17 to 256 options use the qwen_chat_wide-v1 surface, a 256-entry label catalog, request-bound terminal raw-logit capture and a masked softmax over the active labels. The engine generates one internal token per decision and the gateway discards it. Zero returned text tokens does not mean zero internal generation. Ordinary chat requests go directly to the engine through the router and share the same engine.

The interface supports boolean questions through a two-option choice bridge (false, true), and choice questions up to 256 options; each /v1/systemone question may carry up to 256 criteria. Native score questions are not supported. It does not automatically escalate an uncertain decision to a reasoning run; the caller chooses when to use ordinary chat.

The returned probabilities are the model's masked option distribution. They are not guaranteed calibrated confidence estimates.

Evaluation, 20 October 2026

All results below come from the maintainer-host production deployment. No speed, latency or throughput figures are reported in this card.

The first panel is a paired comparison on the same items (exact McNemar, p = 0.002 to 0.008). The second uses AutoTrust's own metric and item set and is descriptive only: their per-item outputs are not public, and our 18 score items are choice surrogates.

items three times and then asks all 1,599 ChaosNLI test rows once. Accuracy uses the frozen benchmark rule (exact ties broken by option key name); the live API breaks ties by request order and both rules are reported.

Lane	Items	Repeat 1	Repeat 2	Repeat 3	Reference
Legacy strict	213	198	198	199	Jev 1.13.0 published: 186
Legacy score, choice surrogate	18	16	15	16	Jev 1.13.0 published: 14, native score
ChaosNLI subset	96	65	65	65	Published plurality accuracy: 62
ChaosNLI, all test rows once	1,599	1,028 (64.29%)			Jev 1.13.0 published: 981 (61.35%)

Comparison with AutoTrust JEV-27B

AutoTrust JEV-27B reports 88.70% on the public 231-item legacy JevBench using family-macro accuracy. Re-aggregating our existing production outputs for the three repeats across the same benchmark definition gives family macro means of 93.95%, 92.96% and 93.59%, mean 93.50% (+4.80 points descriptively); the live API tie rule gives a mean of 93.11%. evaluation/autotrust-comparison.json records the calculation, per-repeat and per-family counts, and hashes binding all 693 observation rows, the task-family metadata and the source pins. We cross-checked this with the official family-macro scorer.

Our 18 score items are multiple-choice surrogates; theirs use native score. The public tasks were used during our development, and we have no paired AutoTrust outputs or their exact benchmark-input hashes. This is a comparison of separately reported results, not a controlled head-to-head. AutoTrust's six-group benchmark mean (84.07%) and its ECE (0.0009 on its own test set with 15 bins) use different evaluation populations, so they cannot be ranked directly against ours. We publish no timing figures, so no latency comparison is possible or implied. AutoTrust JEV-27B is also distinct from hosted TypeSafe Jev 1.13.0, the reference in the preceding tables.

Runtime

Weights alone do not enable the typed API. Use the matching runtime sources and configuration from lychee888/QwenFN-Hybrid-Decision-Engine, including the bundled capture runner, overlays and configuration generator. Stock vLLM and a generic Transformers loader are not documented launch paths for the decision interface. The tested engine family is the model-specific vLLM build 0.1.dev20073+g8e685d198; it also needs the PLE resolver and matching QSA patches. Keep those dependencies and their notices together if you redistribute the runtime.

decision_config.json documents the public readout contract and pins the bundled configs/wide-label-catalog.json. It sits beside config.json and does not replace it. Model configuration, tokenizer, chat template, NVFP4 expert weights, FP8-PLE weights and MTP weights retain their upstream bytes. Do not mix PLE shards from a different quantization.

The completed experiments ran on two DGX Sparks. Other hardware and multimodal typed decisions have not been validated.

Agent skill

A public agent skill, skills/qfn-decide/SKILL.md, describes the same choice API for agents that drive an already-running endpoint. It uses the QFN_ENDPOINT environment variable and the optional QFN_API_KEY; the operator installs their own endpoint. It contains no helper script and no private addresses.

Try the API

Once the matching runtime is running, the included client can send a choice or boolean request:

python examples/decide.py
python examples/decide.py --boolean

It defaults to http://127.0.0.1:8888. Use QFN_ENDPOINT or --base for your own endpoint. QFN_API_KEY adds a bearer header when your deployment requires it. --print-request prints the input without contacting a model; it is not a model result.

Files and integrity

The complete model payload has 27 safetensors shards referenced by model.safetensors.index.json. The missing original shard named model-00002-of-00017.safetensors is intentional: the FP8-PLE derivative replaces the original large BF16 table with ten PLE shards and updates the index.

SOURCE_WEIGHTS.json pins the existing Hub revision, file sizes and upstream object hashes. prepare_model.py creates a new complete tree from an existing local copy, verifies each file's upstream identity and checks the tensor index against the safetensors headers. It does not load the model or upload anything. --hardlink-weights avoids duplicating the shard storage; do not modify weights through the staged directory.

python prepare_model.py --source /models/qwen-base --out /models/qwen-decision-stage --manifest SOURCE_WEIGHTS.json --hardlink-weights

This repository contains the complete model tree: 27 referenced shards, 223,047 indexed tensors, the tokenizer and configuration files, plus the decision configuration, label catalog, agent skill and evaluation bundle. SOURCE_WEIGHTS.json records the upstream identity of every weight file, and payload-manifest.json records every file in the assembled tree with its size and SHA256. prepare_model.py and assemble_upload.py regenerate that tree from an existing local copy and refuse any file whose identity does not match.

Verification

VERIFICATION.json records the current source and isolated installed-wheel checks, the 256-option configuration seam, and the benchmark-export coverage. It also lists the unresolved engine-provenance and model-redistribution gates.

License and provenance

The weights, tokenizer and configuration files are distributed under the Qwen Community License 1.0 (LICENSE, also at Qwen/Qwen3.8-Flash-Next). That license permits use, copying, modification, publication, distribution, sublicensing and derivative works, provided the copyright and permission notice is retained. Two further conditions apply to a licensee: a product exceeding 100,000,000 monthly active users or US$20,000,000 monthly revenue must display the model name prominently, and a Model-as-a-Service or AI Work Assistant business needs a separate license from Qwen (internal use is exempt).

The integration code in the companion GitHub repository is Apache-2.0; the inherited PLE/QSA overlays keep the MIT recipe attribution. No code license changes the terms that apply to these weights.

There is a license mismatch in the upstream chain. The quantized derivative and its immediate parent declare Apache 2.0. The current official Qwen/Qwen3.8-Flash-Next repository supplies Qwen Community License 1.0, including conditions on commercial Model as a Service and AI Work Assistant businesses. We have not established which official base revision and license governed the derivative's creation. An Apache software or metadata label does not remove those model-license conditions.

This package includes the current official base license as LICENSE and the derivative's Apache notice separately. It does not claim unrestricted commercial rights for the weights. The new integration code has its own license. See NOTICE.md for pinned sources and file roles.

The source model has reduced safety alignment. This package does not restore it or add a safety classifier. Evaluate its outputs for the intended use, especially when decisions affect people, money or access to services.