zjshen/qwen3.5-4b-decision

Qwen3.5-4B Decision Model

An experimental model that reads text, a question and descriptions of candidate answers, then scores those candidates. It uses a Qwen3.5 Base backbone with a trained LoRA adapter and a pointer decision head, the small module that turns contextual representations into candidate scores. It does not generate chat responses. This is an independent learning project using KEV, rather than an official Qwen, KEV or TypeSafe release.

The experiment label B2 identifies this 4B checkpoint in the training records and accompanying write-up. B1 denotes the 0.8B version and B2 the 4B version. Those labels are experiment references, not separate architectures.

Download and run

Download this repository's files, use Python 3.12, install requirements.txt, then start the local HTTP server from the downloaded directory:

python decision_model.py --checkpoint . --device cpu --port 8009

The first load separately retrieves the pinned Qwen Base model. This repository contains the adapter, trained decision head, tokenizer and native loader, not the full base weights. A generic text-generation or PEFT loader would omit the decision head. For GPU inference, select --device cuda when an appropriate GPU is available. Published checkpoints exclude optimizer and random-state snapshots needed to resume the original training run.

The endpoint is POST http://127.0.0.1:8009/v1/systemone. Request types are:

• noul: a yes/no judgment, returning the probability of true.
• choice: choose among named candidate descriptions, returning their probabilities.
• score: score ordered level descriptions, returning the expected level index and distribution.

For example, this fictional input asks for a yes/no text judgment:

curl http://127.0.0.1:8009/v1/systemone \
  -H 'Content-Type: application/json' \
  --data '{"state":"The food was excellent. I will return.","questions":{"positive":{"type":"noul","instructions":"Is the reviewer clearly positive overall?","criteria":{"false":"Clearly unfavorable or substantively mixed without an overall favorable judgment.","true":"Clearly favorable overall."}}}}'

These interface types do not imply reliable performance on every task. The confidence field for choice/score is a derived concentration measure; use the probability vector for analysis and assess calibration separately.

Training

Base: Qwen/Qwen3.5-4B-Base, revision 1001bb4d826a52d1f399e183466143f4da7b741b. Training started fresh from that Base with new LoRA parameters and decision head. The two model sizes share exactly the same repaired training file: 12,520 source records, 15,476 questions. Its dataset identifier is train-clean-v2; a source record can contain more than one question.

Both runs used 2 epochs, 3,130 optimizer updates, seed 3, LoRA rank 16, physical batch 1 and gradient accumulation 8. This 4B run used learning rate 0.00005. Backbone weights were stored in FP32 with BF16 mixed-precision training. See training-recipe.json for the complete recorded configuration.

Existing evaluation

These previously used evaluation sets measure historical development behavior. The mixed-task row excludes a review task whose old star-derived labels did not match its text-based question. BoolQ is a yes/no question-answering dataset. Accuracy measures correct selections; lower negative log likelihood (NLL) means more probability assigned to the labeled answers on average.

Evaluation set	Questions	Accuracy	Mean NLL
Earlier mixed-task set, excluding the mismatched review task	1,120	86.79%	0.44067
Previously evaluated BoolQ test set	800	90.62%	0.40879

Outputs used temperature 1, without probability calibration, in FP32 with unmerged LoRA. evaluation.json retains full scores and the declared exclusion. These are exposed evaluation sets, not a new untouched test or an official leaderboard result. The two model sizes also used different learning rates; their comparison does not isolate the effect of size.

Runtime and limitations

The loader uses unquantized FP32 inference, without merged LoRA, optional fused kernels, CUDA graphs or date-arithmetic preprocessing. It returns probabilities rounded to four decimals. Context and context-plus-question rows are limited to 4,096 tokens, with overflow rejected rather than silently truncated. Training used shorter limits: 384 context tokens, 1,024 per question row and 2,048 per packed record. Serving longer inputs does not establish their quality. The model can be wrong or overconfident and is a learning artifact.

Data and reproduction

The data and reproduction package contains source indexes, label repairs, hash-checked data reconstruction and training code. Original source text and private confirmation data are excluded. The companion checkpoint is the Qwen3.5-0.8B decision model.

Adapter weights and learned head tensors match the saved end-of-training checkpoint. Private filesystem metadata was removed from the release head file. release.json records provenance, SHA256SUMS.json records file hashes and sizes, and validation.json describes the actual prior validation scope.

Attribution and terms

Jared Palmer's KEV supplies the decision architecture and training machinery. Qwen supplies the pretrained backbone; LoRA is credited to Hu et al. and implemented through Hugging Face PEFT. Local changes cover data repairs, evaluation organization and training recovery. Retain LICENSE, LICENSE-KEV, LICENSE-QWEN and NOTICE. REFERENCES.rst gives sources. Component-specific terms and unresolved source-data questions are documented in PUBLICATION-STATUS.md; no blanket unrestricted model license or commercial clearance is asserted.