csabag76/tree-compiler-0.8b

Tree Compiler 0.8B

A Qwen3.5-0.8B fine-tune that turns a job seeker's preferences, written in plain words, into a JSON scoring model (mm-tree/1, factors form): lists, reads, hard rules, weighted factors and verdict bands. A separate local program runs that object on each job or recruiter email and returns Through / Maybe / Held with reasons. The model writes the decision procedure once; it never sees the emails.

This repository holds two versions. v1 (the files in the repository root) is the recommended one. Use it with the arity grammar and the validate-and-retry step described below; this is what the public demo runs. v2 (folder v2/) is an experimental retrain: it agrees with the gold verdicts more often, but it hides more good jobs (see Results).

v2 (experimental)

Qwen3.5-0.8B, LoRA SFT, merged into the weights and exported as q8_0. It was trained only on synthetic data: invented job seekers and invented jobs, generated with the second version of the synthetic set ("synth v2"), whose preference texts state hard rules in firm wording. The training trees come from three teachers (Claude Haiku, Claude Sonnet and Gemini Flash); for each user, the candidate tree that did best when executed against the gold verdicts was kept.

Files (v2/)

file	what
tree-compiler-0.8b-v2-q8_0.gguf	the model, q8_0, 812 MB (llama.cpp / wllama)
mm_factors.gbnf	the decoding grammar with fixed arities: each operator accepts exactly the number of arguments the executor expects
example-messages.json	the chat messages for one invented user (system line + user message)
example-output.json	the tree v2 wrote for that user (greedy, with the grammar). It passes the tree validator but was not scored against gold verdicts, and it is not a faithful reading of the message: it adds a pay ramp the user never asked for, and it has no accept rule for the dream companies
SHA256SUMS	checksums of the files above

How to use it (v1 and v2 alike)

For v1 (recommended), use the root GGUF with the same arity grammar (v2/mm_factors.gbnf) and the same retry; for v2, use the v2/ GGUF.

1. Decode with the arity grammar v2/mm_factors.gbnf, greedy, enable_thinking off in the chat template, up to 4096 new tokens.
2. Validate the tree with a semantic validator (arities, names that are declared, lists that exist).
3. If it is not valid, retry up to 2 times at temperature 0.3 with seeds 1234+k (k = 1, 2), no top-k/top-p/min-p, with the same grammar, and keep the first valid tree. If none is valid, treat the user as failed; do not run an invalid tree.

llama-cli -m v2/tree-compiler-0.8b-v2-q8_0.gguf --grammar-file v2/mm_factors.gbnf --temp 0 -n 4096 \
  -sys "$(jq -r '.[0].content' v2/example-messages.json)" -p "$(jq -r '.[1].content' v2/example-messages.json)"

In the browser, wllama loads the file directly (WebGPU, or WASM with 4 threads behind COOP/COEP headers). The validator and the executor that runs the trees are not part of this release.

Results

Synthetic v2 dev set: 100 users x 120 jobs, Mac, q8_0, the arity grammar plus the retry above, with the "unknown policy cap" (a job that would be Through becomes Maybe when a deciding read is unknown). Agreement counts an invalid tree as wrong on every job. A wrong hold hides a job that fits the user, the costliest error.

model	valid trees	agreement (all)	wrong holds per 100 jobs
v1, same setup (recommended)	0.98	0.694	0.52
v2 (experimental)	0.96	0.753	0.84

Read this table carefully: v2 agrees more often with the gold verdicts, but it has more wrong holds per 100 jobs than v1 under this setup (0.84 against 0.52), and slightly fewer valid trees (0.96 against 0.98).

• The release bar for wrong holds was 0.30 per 100 jobs, set from an unconstrained GPU run. Under the arity grammar and retry on the Mac, neither version meets it, and v2 is further from it than v1. Because a wrong hold is the costlier error, v1 stays the recommended model; v2 is published as experimental, not as a release that passed the bar.
• All numbers are on a dev set. v2 has not been evaluated on a held-out test set yet.
• Real job postings were used only for qualitative checks, never for the numbers above.
• The v1 numbers in its own card (valid 0.89, agreement 0.773, wrong holds 0.27) were measured with an earlier setup (an unconstrained run, no arity grammar, no retry) and are not comparable with this table.

Known limitations

• Relocation conditions ("only for a dream company or $X") are often written too strictly, or with the wrong polarity.
• Some trees still fail semantic validation even after two retries (about 4 users in 100 on dev).
• Trained only on synthetic, English, US job preferences.
• A tree is a program written by a small model: validate it, and look at the "explain" lines before trusting the verdicts.

Notes

The GGUF header was edited to drop Qwen3.5's unused multi-token-prediction layer (block_count 24, nextn_predict_layers 0), which wllama otherwise refuses. The tensor data are byte for byte the ones of the trained export. All data used to train and evaluate is invented. License: Apache-2.0, as for v1.

v1 (repository root, unchanged; recommended)

tree-compiler-0.8b-q8_0.gguf, mm_factors.gbnf (the earlier grammar without arities), example-messages.json, example-output.json, and the 4-bit phone shards under phone/. Results on its dev set (valid 0.89, agreement 0.773, wrong holds 0.27 per 100, unconstrained run) and its limits are as stated in the v1 card below. There is no 4-bit build of v2; the phone shards are v1 only.

v1 card (as published)

Distilled by SFT from the best of several teacher samples (Claude Sonnet and Haiku, Gemini Flash) per user, chosen by executing each candidate tree against gold verdicts. About 11% of v1 trees fail semantic validation (an operator with the wrong number of arguments, a reference to a name never declared, an undefined list). The v1 grammar only ensures well-formed JSON and the top-level shape.

Made by Csaba Garay, October 2026.