flymy-ai/decision-fast-preview

Decision Fast 0.6B - FlyMyJev evaluation preview

Frozen checkpoint qwen3_06b_headfirst_ep2a_v53: 606,672,384 parameters, 10,622,464 trained parameters. The download contains the LoRA adapter, structural-token rows, pointer head and executable inference code. The pretrained base is fetched separately at the exact revision in model.json. Fast is0.6B in the small-model track, not a literal1B checkpoint.

This package is supplied for independent benchmark evaluation. It is not an official JevBench result or a validated production release. Weight availability, code licensing and unresolved training-source conditions are separate facts. See EVALUATION-PERMISSION.md and provenance-review.json before other uses.

Measured native-pointer results

Public JevBench subset	Correct / total	Accuracy
All	148/231	64.07%
Easy	48/48	100.00%
Standard	56/72	77.78%
Hard	44/111	39.64%

Measured on one RTX4090 with Torch2.8.0+cu128, transformers4.57.6 and peft0.15.2. The full534-decision suite includes held-out tasks unavailable to us. No official rank or aggregate score is claimed. The earlier62.77% public result used length-normalized option-token likelihood on77 items. That configuration is not this submission. Here all231 answers use the trained pointer head, including those77 newly measured items. No weights were retrained. The154 historical pointer items were checked for prediction and raw-probability parity.

Public tasks and their exact rubrics are unchanged. The distribution is the trained pointer head's softmax with a frozen temperature fitted on this checkpoint's own transfer replay and known-chance development examples. No public benchmark labels fit this temperature. All outputs are native decision probabilities: no generated answer tokens, token logprobs, external model calls, option-order ensemble or benchmark-ID lookup.

The initial absolute0.51e-6 raw-probability gate and its original status are retained in the report. It omitted float32 softmax-sum roundoff, which becomes visible when temperature scaling is inverted. The separate numerical review requires one shared normalization scale to satisfy every six-decimal interval, with a maximum scale error of two float32 epsilons. This correction was made after the v59 run and before the Fast run; neither weights, calibration nor the zero-argmax-change requirement changed. See reports/numerical-compatibility-review.json and audit_rounding.py. This is compatibility at reported precision, not a claim of bitwise equality.

reports/public-evaluation.json contains all231 outputs, calibrated metrics, timings and memory checks. Its historical comparison is limited to the matched pointer items. The historical report is retained separately, not mixed with the new readout. Public results informed earlier experiment design and candidate selection; this is not a claim of benchmark-blind development. Exact source training hashes and recipe are included, but the mixed training corpus is not public here and complete contamination absence is not independently established.

Run the package

Use Linux, Python3.11, a CUDA GPU and one model package per process. Install:

python -m venv .venv
. .venv/bin/activate
pip install torch==2.8.0 --index-url https://download.pytorch.org/whl/cu128
pip install numpy==2.1.2 -r requirements-minicpm.txt
python model.py --verify-only
python model.py --input example.json

First load downloads and hashes the pinned Apache-2.0 base with at most two download workers. For a local verified base use --assets /path/to/base. The code and weights are hash checked before loading. In Python:

import model
decider = model.load()  # once per process; optionally pass the base directory
result = decider.decide("The customer was charged twice.", {
    "type": "choice", "instructions": "Choose the support queue.",
    "criteria": {"billing": "Billing and payments", "technical": "Technical support"}
})
print(result["probabilities"])

choice preserves label order, noul returns false/true, and score returns zero-based string indices in the supplied level order. jevbench_adapter.py maps false/true to the harness's no/yes. Only state, type, instructions and criteria enter inference. Labels and task metadata are excluded.

Run through the official harness without modifying its registry:

git clone https://github.com/fstandhartinger/jevbench.git
git -C jevbench checkout 218511d85c5e9fc3dd12a08fae0e9818ea48bb68
python run_jevbench.py --harness ./jevbench --output ./new-results --price-input-per-million 0.01

The maintainer can pass --tasks /private/heldout.jsonl; those inputs and outputs stay on their machine. The suggested price is a hosted size-class reference estimate, not an offered tariff: the Qwen3-Embedding-0.6B reference used for Kev. Local computation is not priced as free. The maintainer chooses the final price basis and latency adjustment under JevBench rules.

Resources and reproduction limits

Serving is bf16, unmerged, pointer-only, with the historical4096-token packing limit. Long state text may be truncated by that frozen encoder. The wrapper limits a request to64KiB,4096 JSON nodes and255 options; response keys total at most8KiB. One loaded model processes requests serially, with no response cache, conversation history, DB queries or external inference. Callers must bound their own queue. Base downloads are the only optional network dependency.

Measured peak process RSS: 1.93GiB; peak allocated VRAM: 1.48GiB. After128 repeated requests, RSS growth was0.00MiB and allocated VRAM growth0.00MiB. Largest returned JSON response was354bytes. Standard-tier p50/p95 local encoding+forward+calibration was59.78/ 60.70ms. This excludes network and is not a production HTTP latency claim.

The historical source commit, seeds, training recipe and input hashes are in model.json, reports/recipe.json and checkpoint/*/training_state.json. The runtime includes the matching original trainer and Kev implementation. Original full-replay and development measurements remain in reports/ with their original scope. Full end-to-end public dataset reconstruction and a second independent training run have not been established. Existing weights were preserved and evaluated, not retrained for this submission.

Provenance and permissions

Code retains Apache-2.0 notices. The pinned backbone has its own upstream Apache-2.0 declaration. See EVALUATION-PERMISSION.md for permission to run the author's adapter/head contributions for benchmark evaluation; a general commercial weight licence is not asserted in this preview.

The training mixture includes96 SNI task289/Gigaword-derived records for which project-specific upstream entitlement has not been established. Precise source attribution for tasks1186/1283/1284 also remains incomplete. This Fast checkpoint used the earlier v7 mixture and contains none of the H8/H9 Fable additions. These are unresolved provenance questions, not a claim that every trained weight is unlawful or that all rights are cleared. Raw mixed-source training data and private evaluation cases are not redistributed by this preview.