pawanpaudel93/cascadia-jev-9b

JEV-9B decision tree (OpenVINO int8, 1 stage)

A ready-made Cascadia decision tree for autotrust/JEV-9B, so you can serve /v1/decisions (OpenAI Decisions format) and /v1/systemone without running the exporter yourself.

JEV-9B's verbalizer LoRA grafted onto Intel's stock int8 Qwen3.5-9B IR (201 LoRA branches); the decision head reads the verbalizer token logits. It is a 1-stage tree for a single box (about 8.7 GB), the exporter's default.

Preset

Preset	Path	Stages	Layers	Size
1	int8/stages-1	1	32	9.3 GB

int8/stages-1/
  manifest.json  decision.json           # stage split, decision config
  config.json  generation_config.json    # model config
  tokenizer.json  chat_template.jinja    # tokenizer, chat template
  openvino_text_embeddings_model.*       # token embeddings (rank 0)
  decision/                              # pointer head
  stage0/                                # each: stage.xml/.bin

cascadia.json at the repo root is the machine-readable index Cascadia's model registry reads. This repo's weights are byte-identical to pawanpaudel93/jev9b-int8-ov-decision, which keeps the same tree at its root.

Serve it

hf download pawanpaudel93/cascadia-jev-9b --local-dir ~/trees/cascadia-jev-9b
cascadia run ~/trees/cascadia-jev-9b/int8/stages-1 --engine qwen35 --device GPU --api 127.0.0.1:8080

Use OpenVINO GenAI 2026.4.1 (2026.4.0's GPU plugin has an access violation under long generations). --device CPU works with the same files. See docs/DECISIONS.md in the Cascadia repo for the request formats and limits.

How it was made

Exported with tools/decision_export.py from labscommunity/cascadia branch pawan/decisions-api (commit 94cfdf9b; the exporter's output is unchanged at the PR head db44d59d), from the upstream files downloaded as docs/DECISIONS.md describes:

python tools/decision_export.py jev --base-ir ~/models/qwen35-9b-int8 --adapter-dir ~/models/jev9b/adapter_vllm --out ~/trees/jev9b-int8-t58

Measured

Intel Core Ultra 7 258V (Arc 140V iGPU, 32 GB), Windows 11, OpenVINO GenAI 2026.4.1, --device GPU, Cascadia at db44d59d, against the streamed fp32 reference (50 records / 82 questions):

Check	Result
Parity vs fp32	80/82 PASS (two near-tie flips, reference margins 0.002 and 0.025; no confident flip), max
/v1/decisions vs /v1/systemone	82/82 identical (all states)
Temperature-0 chat before vs after the decision run	byte-identical

License and credits

Apache-2.0, as are all the inputs:

• Fine-tune: autotrust/JEV-9B (Apache-2.0).
• Base: OpenVINO/Qwen3.5-9B-int8-ov (Intel's int8 IR of Qwen3.5-9B, Apache-2.0), from Qwen (Apache-2.0).
• Conversion: Cascadia (Apache-2.0).

This repository redistributes converted weights only; it adds no training. The full license text is in LICENSE.