Open-Jev-9B

A trained LoRA adapter and scalar decision head for Qwen/Qwen3.5-9B. This repository contains the completed Open-Jev decision checkpoint, not merged or standalone base-model weights. It requires the exact upstream model/tokenizer revision c202236235762e1c871ad0ccb60c8ee5ba337b9a and the Open-Jev loader.

Open-Jev scores caller-supplied candidates directly and returns typed decisions without autoregressive answer generation:

• Choice: probabilities over the supplied candidate set and its most probable candidate.
• Noul: a probability for a yes/no question.
• Score: probabilities over supplied ordinal levels and their expected value.

The adapter targets the Qwen text backbone. A generic AutoPeftModel text-generation call does not implement this interface or apply the separate decision head and saved temperature.

Download and run

Use a suitable GPU with the upstream weights available locally or downloadable from Hugging Face. The loader uses the pinned upstream revision in package/checkpoint/model.json.

git clone https://github.com/Zefan-Cai/Open-Jev.git
cd Open-Jev
python -m pip install -e '.[train]'
hf download ZefanCai/Open-Jev-9B --local-dir ./checkpoints/open-jev-9b
python -m jev.server \
  --checkpoint ./checkpoints/open-jev-9b/package/checkpoint \
  --device cuda:0 --max-length 4096 --batch-size 1 --no-prefix-cache \
  --host 127.0.0.1 --port 8791

The repository's train extra supplies inference dependencies too, including Transformers 5.10.2 and PEFT 0.19.1. Prefix caching is opt-in and is disabled above; real-checkpoint GPU A/B validation remains separate. For reproducible deployments, add --revision <commit> to hf download using a commit from this model repository's history.

Once the server is ready, submit a request. This is an input example, not a claim about a recorded prediction:

curl http://127.0.0.1:8791/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state":"I was charged twice and want a refund.","questions":{"intent":{"type":"choice","instructions":"Choose the customer intent.","criteria":{"billing":"A payment or refund issue","technical":"A malfunction or setup issue","other":"Another request"}}}}'

The server returns declared keys and probabilities. It does not execute the proposed actions. The model's maximum input length is 4,096 tokens per independently scored candidate; input is rejected rather than silently truncated.

Training and data

• 20,204 optimizer steps with global batch 4: 80,816 consumed training rows, one full pass over the frozen release-v2 training split.
• LoRA rank 8, alpha 16; scalar head initialized from the pretrained Yes-minus-No readout and trained jointly with LoRA.
• Training source commit: 99e881108c6cacadafd364088505e84975ca43fc.
• Frozen data manifest SHA-256: 56105dc9fc89ef74919f5beb60bb6ae8c6e17bb95699dab59205f67d8b338d97.
• Saved temperature: 1.8969118766347646; fitted only on 512 calibration rows.

The public dataset repository provides release-v2-redistributable. Its training split has 79,116 rows, excluding the 1,700 original training records from wikispeedia-v1 because redistribution permission for that archive has not been confirmed. It is not byte-identical to the 80,816-row training set used for these weights. The original manifest and split hashes remain recorded in package/provenance.json; no source dataset rows are bundled here.

The later browser/drone expansion and five later extraction-control corpora are not part of this checkpoint's training mixture. Public game videos may use separately identified older pilot checkpoints and are not automatically evidence for these full-pass weights.

Full held-out evaluation

The existing full-data evaluation covers 10,532 test + 15,920 OOD = 26,452 records, with zero missing, duplicate, or failed inference records. This evaluation used the original frozen mixture, including its Wiki records, rather than the redistributable projection. Its five inference-file hashes match the weights and metadata published here.

Split	All rows	Hard correct / hard rows	Hard accuracy	Expected accuracy	NLL	Brier	ECE
Test	10,532	9,799 / 10,046	97.54%	94.72%	0.130947	0.039113	0.007707
OOD	15,920	14,205 / 15,446	91.97%	90.40%	0.299441	0.126647	0.037398

Hard accuracy excludes soft-target rows. Expected accuracy is the reference target mass at the chosen candidate across all rows; it is not a game-win or workflow-completion rate. NLL, Brier, and ECE use all rows and the saved calibration. Full machine-readable results are in evaluation/full-data.json.

No full-data baseline was evaluated, so this table does not establish training gain. The original package also preserves the training run's separate 512-test/512-OOD sampled metrics in package/metrics.json and its original card in package/README.md. Those smaller sampled results should not be confused with the full-data table above.

Artifact layout and verification

package/ is the unchanged verified inference package, including its original manifest, card, calibration, provenance, and sampled metrics. The root release-manifest.json binds the complete Hugging Face release layout.

Artifact	SHA-256
LoRA adapter	f85650a8fb97c6d0ac3e948cdca2f30a0ca6ace8b8a43aebca1c152fe38aeb75
Scalar head	229fe9800384e824135e59d1af59bda7346da031aea61640be5f456b9be6ce70
Original package manifest	e83fd15c8715c9a7e474afd0830d55b0fe4b25ab61910114f6e36437a2964e37

The original evaluated checkpoint directory had digest a691105dd5751bfcf72d276a6ce075d45054db77ae3272e87b5f07b2bd7e2193. The packaged five-file inference directory has digest 9302c52feba99d079918755f2469f4f088266e9786327bab550248f3d83716d3 because the packaging whitelist omits the generated adapter README. These directory digests are not interchangeable: the evaluation binding is verified per inference file.

Release checks cover manifest bytes, pinned model/revision, licenses, calibration, finite CPU adapter/head tensors, and correspondence to the completed evaluation's inference files. No GPU inference was repeated during this upload. The trainer's historical reload logits were checked, but training did not record a contemporaneous output-weight digest; this upload does not invent one.

Synthetic held-out decision metrics do not establish broad real-world reliability, closed-loop browser/game/flight success, or a calibrated probability guarantee outside the evaluated distribution.

License

The trained adapter and head are released under Apache-2.0, with the complete pinned Qwen/Alibaba Cloud attribution in LICENSE and UPSTREAM.md. Open-Jev source code is MIT, preserved in LICENSE-CODE. The upstream model/tokenizer must be obtained separately under their own terms.