969M-Decisions-CPU

An independently adapted, 969M-parameter, CPU-first typed-decision engine. It scores user-supplied choices in a single model forward pass instead of autoregressively generating answers.

This repository contains the general-purpose decision model, not any task-specific game adaptation.

Origin and architecture

This is a modified derivative of Liquid AI LFM2.5-1.2B-Instruct; it is not a new pretrained-from-scratch backbone and is not affiliated with Liquid AI.

The adapted architecture preserves 16 hybrid processing blocks (10 convolution, 6 attention), hidden width 2048, and uses a 6144-wide feed-forward network after structured pruning. Additional work includes noncausal attention adaptation, convolutional adjustments, supervised decision/reasoning training, and rank-32 DoRA fine-tuning. DoRA weights are already merged into the distributed full OpenVINO INT8 model. The classifier uses the last active sequence representation and the selected 26 pretrained vocabulary rows A-Z, without a full-vocabulary text-generation readout.

The computation uses integer-quantized dense kernels and some remaining floating-point operations. The OpenVINO BF16 execution hint does not remove INT8 quantization.

Files and requirements

• model.xml, model.bin: complete merged, full INT8 OpenVINO CPU network (about 973 MB)
• tokenizer/: source-aligned tokenizer and chat template
• runtime.py, data_io.py: exact compact-prompt single-pass classifier runtime
• serve_lfm_cpu.py: persistent JSON-lines local CPU service
• serve_lfm_cpu_extended.py: optional 4096-token shape support (experimental quality)
• compression.json, PROVENANCE.json, EVALUATION.json, FILE_HASHES.json: artifact and benchmark records
• LICENSE, SOURCE_LICENSE, SOURCE_README.md, NOTICE.md: required upstream license and notices

Tested with Python 3.12, OpenVINO 2026.4.1, NumPy 2.4.6, Transformers 5.19.0 and a PyTorch CPU environment. See requirements.txt. This is an OpenVINO artifact, not a standard Transformers text-generation pipeline.

CPU inference

Clone/download this repository, install compatible CPU dependencies, then run from its root:

from pathlib import Path
from runtime import Runtime

root = Path(".")
engine = Runtime(root / "model.xml", root / "tokenizer",
                 threads=16, precision="bf16",
                 buckets=(64,128,192,256,320,384,512))
engine.warmup()
result = engine.predict(
    "The customer received a damaged shipment.",
    {"decision": {"type": "choice",
                  "instructions": "Choose the appropriate next step.",
                  "criteria": ["Approve a return", "Request details", "Close case"]}}
)
print(result["answers"]["decision"])

This model supports choice, noul (yes/no), and score (ordered ordinal categories), with 2-26 options. Output probabilities are uncalibrated conditional preferences over supplied choices.

Persistent local service:

python serve_lfm_cpu.py --graph model.xml --threads 16 --precision bf16

The service accepts one JSON object with fields id, state, and questions per line; it writes one JSON response per line. It does not host an HTTP endpoint or generate natural-language answers. Monitor input_audit for truncation.

Input length

The qualified classifier originally uses up to 512 input tokens. An optional service can run at longer input shapes:

python serve_lfm_cpu_extended.py --graph model.xml --max-input-tokens 4096 --reject-truncated

Execution of 1024/2048/4096-token OpenVINO shapes has been tested, but decision accuracy on long inputs was not established. Very long input shapes increase inference time and can involve cold graph compilation. Long-input outputs should not be treated as quality-qualified.

Saved CPU benchmark diagnostics

The following slice-level measurements are from the exact distributed general-purpose INT8 graph. Benchmark slices overlap and are not aggregated.

Diagnostic task/slice	Correct/total	Accuracy
Typed decisions: published diagnostic test	1,516/2,000	75.80%
Typed decisions: validation	459/600	76.50%
CLINC150: 15-way within-domain	2,867/2,975	96.37%
Banking77: adapted 10-way	237/250	94.80%
SNLI	863/1,000	86.30%
MNLI	202/250	80.80%
BoolQ	202/250	80.80%
Executable rules	991/1,000	99.10%
Frozen unseen-rule families	371/500	74.20%

On a 16-physical-core Ryzen 9 7950X, the 128-token OpenVINO inference bucket measures approximately 42-44 ms when warm. The 2000-item typed-decision diagnostic yields 96.06 ms median and 117.11 ms p95 for tokenization, inference, and decoding at batch one. This excludes loading/compilation, is not a production SLA, and may vary with input length and hardware.

Training and quantization

The training lineage includes original LFM2.5 weights, structured FFN pruning/distillation, then a final DoRA continuation with 35,552 unique training records, two epochs, 8,888 optimizer updates, and 71,104 training-example exposures. Data included structured typed-decision examples, synthetic/teacher reasoning variants, and replay of native task families.

The FP32 DoRA update was merged into the student weights and checked against the unmerged model. OpenVINO full INT8 activation/weight post-training quantization used 128 deterministically selected training-only calibration records. No generation or separate adapter is required for the supplied INT8 CPU graph.

Limits and license

• The model is a constrained decision classifier, not a general chat or speech system.
• Probabilities are uncalibrated, and prompt wording/option order can affect predictions.
• Input-state text can be truncated at the configured limit; review input_audit.
• Results for adapted task suites must not be represented as unrestricted, all-label official scores.
• Long-context accuracy is unverified even though longer input shapes run.

The weights are a modified derivative of a Liquid AI model and are subject to the accompanying LFM Open License v1.0, including its commercial-use revenue threshold and redistribution provisions. Read LICENSE and NOTICE.md before use.