pawanpaudel93/cascadia-clef-27b

Clef 27B decision tree (OpenVINO int8, 2 stages)

A ready-made Cascadia decision tree for Cloudflare/clef, so you can serve /v1/decisions (OpenAI Decisions format) and /v1/systemone without running the exporter yourself.

Clef's full fine-tune weights swapped into Intel's int8 Qwen3.8-27B IR, plus Clef's joint schema head as an OpenVINO model under decision/. It is a 2-stage tree (about 29 GB) made for a two-machine pipeline: one 32 GB host cannot run both stages.

Preset

Preset	Path	Stages	Layers	Size
2	int8/stages-2	2	64	31.2 GB

int8/stages-2/
  manifest.json  decision.json           # stage split, decision config
  config.json  generation_config.json    # model config
  tokenizer.json  chat_template.jinja    # tokenizer, chat template
  openvino_text_embeddings_model.*       # token embeddings (rank 0)
  decision/                              # pointer head
  stage0/  stage1/                       # each: stage.xml/.bin

cascadia.json at the repo root is the machine-readable index Cascadia's model registry reads. This repo's weights are byte-identical to pawanpaudel93/clef27b-int8-ov-decision, which keeps the same tree at its root.

Serve it

Download the whole tree to the same path on both hosts, then start rank 1 on host B and rank 0 (with the API) on host A:

hf download pawanpaudel93/cascadia-clef-27b --local-dir ~/trees/cascadia-clef-27b
# host B (rank 1, runs the joint head)
cascadia worker --model ~/trees/cascadia-clef-27b/int8/stages-2 --engine qwen35 --device CPU --rank 1 --total 2 --listen 0.0.0.0:7801
# host A (rank 0, API on loopback); replace HOST_B with host B's address
CASCADIA_ACTIVATION_TIMEOUT_SECS=1800 cascadia worker --model ~/trees/cascadia-clef-27b/int8/stages-2 --engine qwen35 --device CPU \
    --rank 0 --total 2 --next HOST_B:7801 --api 127.0.0.1:8080

Use OpenVINO GenAI 2026.4.1. This tree has been measured on CPU only. On a 16,384-token joint pass the last rank peaked at about 21 GB private memory, so leave that host otherwise idle for long requests. See docs/DECISIONS.md in the Cascadia repo for the request formats and limits.

How it was made

Exported with tools/decision_export.py from labscommunity/cascadia branch pawan/decisions-api, from the upstream files downloaded as docs/DECISIONS.md describes, and served and checked at the PR head db44d59d:

hf download OpenVINO/Qwen3.8-27B-int8-ov --local-dir ~/models/qwen38-27b-int8
hf download Cloudflare/clef --local-dir ~/models/clef      # 55 GB (BF16)
python tools/decision_export.py clef --base-ir ~/models/qwen38-27b-int8 --weights ~/models/clef \
    --clef-dir ~/models/clef --stages 2 --out ~/trees/clef-int8

The swap requantizes each row over [min(w, 0), max(w, 0)], as NNCF does for Intel's IR.

Measured

Two Intel Core Ultra 7 258V hosts (32 GB each), Windows 11, OpenVINO GenAI 2026.4.1, --device CPU, one rank per host, Cascadia at db44d59d, against the streamed fp32 reference (50 records / 82 questions, one joint pass per record):

Check	Result
Parity vs fp32	81/82 PASS (choice 32/33, predicate 24/24, score 25/25), max |Δp| 0.0669; the one flip, r41.team, is a near-tie (reference margin 0.048)
/v1/decisions vs /v1/systemone	62/62 identical (string states)
Temperature-0 chat before vs after the decision run	byte-identical

License and credits

Apache-2.0, as are all the inputs:

• Fine-tune: Cloudflare/clef (Apache-2.0).
• Base: OpenVINO/Qwen3.8-27B-int8-ov (Intel's int8 IR of Qwen3.8-27B, Apache-2.0), from Qwen (Apache-2.0).
• Conversion: Cascadia (Apache-2.0).

This repository redistributes converted weights only; it adds no training. The full license text is in LICENSE.