this-that-model-1.2

A typed decision model. One forward pass, no decoding loop, no parser, no retry.

1.1 could apply a policy rule. 1.2 can apply it however the rule happens to be written — which is the difference between a model that works on a benchmark and a model that works on the policy document somebody actually wrote.

What changed from 1.1

Composed decisions: where several rules narrow a choice in sequence, accuracy goes from 0.775 to 0.878 on the public benchmark and from 0.785 to 0.909 on the depth ladder.

Rules written in the negative: 1.1 read a rule such as bays without chilled handling are ineligible as though it named the eligible set — not failing to apply the rule but applying its opposite, at 0.88 mean confidence. 1.2 reads it correctly. The table below measures that across eleven phrasings of one rule, because a rule that changes meaning with its wording is the failure a user meets first.

Reading the rule, however it is written

The same 100 decisions, the same gold answers, eleven ways of writing the same eligibility rule. A model that reads rules should score the same on all eleven.

the rule, written as	1.1	1.2
Only bays with handling of chilled are eligible.	0.86	1.00
A bay is eligible only if its handling is chilled.	0.88	0.99
A bay must have handling of chilled to be used.	0.79	0.99
No bay may be used unless its handling is chilled.	0.89	0.99
Every bay is excluded except those whose handling is chilled.	0.87	1.00
A bay must not be used if its handling is not chilled.	0.72	0.99
A bay whose handling is not chilled is not eligible.	0.00	0.98
A bay whose handling is not chilled is ineligible.	0.00	0.98
Bays without chilled handling are ineligible.	0.03	0.99
Any bay that lacks chilled handling is disqualified.	0.05	1.00
A bay that fails to provide chilled handling is ruled out.	0.16	0.98

Chance is 0.19. 1.1's range is 0.00 to 0.89; 1.2's is 0.98 to 1.00. Measured on states rendered as indented key/value text, over decisions that filter by eligibility and then rank.

Read the 1.1 column carefully: four of those rows are below chance. A model that scores 0.00 where chance is 0.19 is not guessing — it is reliably choosing the option the rule excludes. On a yes/no question (Is bay-b eligible?) under such a rule, 1.1 answered correctly 6% of the time against a chance rate of 50%.

Where 1.2 sits against models that generate tokens

Every system on all 1,710 questions of the public benchmark, scored the same way: one question, declared options, the answer read back, a system that returns nothing counted wrong.

System	Accuracy	Generated tokens	Latency
this-that-model-1.2	0.878	0	31.4 ms
claude-opus-5	0.834	~200	~1000 ms
gpt-5.6	0.816	~200	~1200 ms
this-that-model-1.1	0.775	0	50.3 ms
glm-5.3	0.652	~200	~800 ms
kimi-k3	0.522	~200	~1000 ms
deepseek-v4.1-flash	0.511	~4	~800 ms
deepseek-v4-pro	0.470	~4	~900 ms
this-that-model-1.0	0.406	0	50.3 ms
laya-typed-decisions	0.310	0	30.4 ms
chance	0.258	—	—

The benchmark is public: limberc/this-that-complex-decisions. Please noticed that the performance is tested under NVIDIA RTX 5080 GPU (Laptop).

That table measures decision difficulty, and nothing else. On the phrasing table above, claude-opus-5 and gpt-5.6 both score 1.00 on every row, including the rows where 1.1 scores 0.00 — so the frontier models are not weaker at reading rules, they are weaker at these decisions. Two different axes, and this page reports both.

Composed decisions, by depth

Rules that compose — a filter, then a second filter, then a ranking. 242 questions, held out at both the question and the world-state level.

	depth 1	depth 2	depth 3	overall
this-that-model-1.2	0.94	0.89	0.87	0.909
this-that-model-1.1	0.80	0.80	0.74	0.785
gpt-5.6	0.81	0.56	0.58	0.678
claude-opus-5	0.80	0.53	0.61	0.669

Depth 2 is the hardest rung for the hosted models and the widest gap: two narrowings with many candidates still live is where a single pass over the state pays off most.

The frozen third-party cohort

68 questions a third party recorded against a hosted commercial decision service, in their wording rather than ours:

	accuracy	Brier ↓	NLL ↓
majority-class baseline	0.647	—	—
the hosted service	0.765	0.133	0.403
this-that-model-1.1	1.000	0.003	0.023
this-that-model-1.2	0.985	0.009	0.047

1.2 misses one of the 68. 1.1 missed none; the cohort is small enough that one question is the whole difference between those two rows, and neither is evidence of much.

Calibration

Where the true answer is a computed probability, so the target follows from the transition rules rather than from anyone's opinion. Lower qL2 is better.

	accuracy	qL2 ↓	constant predictor
sim_event_ood	0.730	0.0274	0.0962
sim_local_ood	0.995	0.0161	0.4928

1.1 scored 0.0253 and 0.0203. The second improves; the first is a small regression, and a constant predictor is still three times worse than either.

Usage

pip install git+https://github.com/FLock-io/this-that-model

from thisthat import TypedDecider, Question

decider = TypedDecider.from_pretrained("flock-io/this-that-model-1.2")

answer = decider.decide(
    '{"bays": {"bay-a": {"free": 7, "distance_m": 40, "handling": "chilled"},'
    '          "bay-b": {"free": 3, "distance_m": 12, "handling": "ambient"}},'
    ' "policy": ["Bays without chilled handling are ineligible.",'
    '            "Among those eligible, take the most free positions."]}',
    Question("Which bay is assigned?", ["bay-a", "bay-b", "hold in transit"]),
)

print(answer.choice)          # bay-a
print(answer.confidence)      # threshold on it; see the calibration section

That is the example 1.1 returned bay-b for, at 0.9998 confidence, and it is the one this release exists to fix. Over 40 freshly generated decisions of exactly that shape — JSON, two candidates, the rule written as an exclusion — 1.1 answers 0.20 and 1.2 answers 0.975. Stated positively, both answer 1.000.

Citation

@misc{cheng2026thisthat,
  title  = {A typed decision model that decides in 50 ms, for a millionth of a cent},
  author = {Cheng, Zehua and Dai, Wei and Sun, Jiahao},
  year   = {2026}
}

Licence

MIT. Adapted from decider-2b under Apache-2.0.