Open-Jev-2B

A trained LoRA adapter and scalar decision head for Qwen/Qwen3.5-2B. This repository contains the completed Open-Jev decision checkpoint, not merged or standalone base-model weights. It requires the exact upstream model/tokenizer revision 15852e8c16360a2fea060d615a32b45270f8a8fc and the Open-Jev loader.

Open-Jev scores caller-supplied candidates directly and returns typed decisions without autoregressive answer generation:

• Choice: probabilities over the supplied candidate set and its most probable candidate.
• Noul: a probability for a yes/no question.
• Score: probabilities over supplied ordinal levels and their expected value.

The adapter targets the Qwen text backbone. A generic AutoPeftModel text-generation call does not implement this interface or apply the separate decision head and saved temperature.

Download and run

Use a suitable GPU with the upstream weights available locally or downloadable from Hugging Face. The loader uses the pinned upstream revision in package/checkpoint/model.json.

git clone https://github.com/Zefan-Cai/Open-Jev.git
cd Open-Jev
python -m pip install -e '.[train]'
hf download ZefanCai/Open-Jev-2B --local-dir ./checkpoints/open-jev-2b
python -m jev.server \
  --checkpoint ./checkpoints/open-jev-2b/package/checkpoint \
  --device cuda:0 --max-length 4096 --batch-size 1 --no-prefix-cache \
  --host 127.0.0.1 --port 8791

The repository's train extra supplies inference dependencies too, including Transformers 5.10.2 and PEFT 0.19.1. Prefix caching is opt-in and is disabled above; real-checkpoint GPU A/B validation remains separate. For reproducible deployments, add --revision <commit> to hf download using a commit from this model repository's history.

Once the server is ready, submit a request. This is an input example, not a claim about a recorded prediction:

curl http://127.0.0.1:8791/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state":"I was charged twice and want a refund.","questions":{"intent":{"type":"choice","instructions":"Choose the customer intent.","criteria":{"billing":"A payment or refund issue","technical":"A malfunction or setup issue","other":"Another request"}}}}'

The server returns declared keys and probabilities. It does not execute the proposed actions. The model's maximum input length is 4,096 tokens per independently scored candidate; input is rejected rather than silently truncated.

Training and data

• 20,204 optimizer steps with global batch 4: 80,816 consumed training rows, one full pass over the frozen release-v2 training split.
• LoRA rank 8, alpha 16; scalar head initialized from the pretrained Yes-minus-No readout and trained jointly with LoRA.
• Training source commit: 99e881108c6cacadafd364088505e84975ca43fc.
• Frozen data manifest SHA-256: 56105dc9fc89ef74919f5beb60bb6ae8c6e17bb95699dab59205f67d8b338d97.
• Saved temperature: 1.518796342858676; fitted only on 512 calibration rows.

The public dataset repository provides release-v2-redistributable. Its training split has 79,116 rows, excluding the 1,700 original training records from wikispeedia-v1 because redistribution permission for that archive has not been confirmed. It is not byte-identical to the 80,816-row training set used for these weights. The original manifest and split hashes remain recorded in package/provenance.json; no source dataset rows are bundled here.

The later browser/drone expansion and five later extraction-control corpora are not part of this checkpoint's training mixture. Public game videos may use separately identified older pilot checkpoints and are not automatically evidence for these full-pass weights.

Full held-out evaluation

The existing full-data evaluation covers 10,532 test + 15,920 OOD = 26,452 records, with zero missing, duplicate, or failed inference records. This evaluation used the original frozen mixture, including its Wiki records, rather than the redistributable projection. Its five inference-file hashes match the weights and metadata published here.

Split	All rows	Hard correct / hard rows	Hard accuracy	Expected accuracy	NLL	Brier	ECE
Test	10,532	9,515 / 10,046	94.71%	91.95%	0.195380	0.076117	0.011606
OOD	15,920	13,287 / 15,446	86.02%	84.62%	0.802990	0.246450	0.105594

Hard accuracy excludes soft-target rows. Expected accuracy is the reference target mass at the chosen candidate across all rows; it is not a game-win or workflow-completion rate. NLL, Brier, and ECE use all rows and the saved calibration. Full machine-readable results are in evaluation/full-data.json.

No full-data baseline was evaluated, so this table does not establish training gain. The original package also preserves the training run's separate 512-test/512-OOD sampled metrics in package/metrics.json and its original card in package/README.md. Those smaller sampled results should not be confused with the full-data table above.

Artifact layout and verification

package/ is the unchanged verified inference package, including its original manifest, card, calibration, provenance, and sampled metrics. The root release-manifest.json binds the complete Hugging Face release layout.

Artifact	SHA-256
LoRA adapter	2d23935b1a7380db444abac572c04646918ba794e59002d1588236182a3ca18f
Scalar head	3532cd576c58d5ad5bf17c3e9f2df4be8c70e08c07fa6bb7fa673dcd7b401f2a
Original package manifest	58319da5c2a948a4645e46d9c982be44867d78779ea1c3bfb81b64867f58ef3a

The original evaluated checkpoint directory had digest 8c37b393c27d2b58009463a89dd1a873b7c010ecd1b8c184c7ada5efe13f8bfb. The packaged five-file inference directory has digest 3076462e6356412082e79af909227b39b2863b90def79155ca0821aa506b7ded because the packaging whitelist omits the generated adapter README. These directory digests are not interchangeable: the evaluation binding is verified per inference file.

Release checks cover manifest bytes, pinned model/revision, licenses, calibration, finite CPU adapter/head tensors, and correspondence to the completed evaluation's inference files. No GPU inference was repeated during this upload. The trainer's historical reload logits were checked, but training did not record a contemporaneous output-weight digest; this upload does not invent one.

Synthetic held-out decision metrics do not establish broad real-world reliability, closed-loop browser/game/flight success, or a calibrated probability guarantee outside the evaluated distribution.

License

The trained adapter and head are released under Apache-2.0, with the complete pinned Qwen/Alibaba Cloud attribution in LICENSE and UPSTREAM.md. Open-Jev source code is MIT, preserved in LICENSE-CODE. The upstream model/tokenizer must be obtained separately under their own terms.