mlboydaisuke/TwIL-LM3-LiteRT

TwIL-LM3 — LiteRT-LM

webAI-Official/TwIL-LM3 converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime. Requires litert-lm ≥ 0.16.

⚠ Non-commercial. The upstream model ships under the webAI Non-Commercial License v1.0, and this conversion inherits it unchanged: non-commercial research and educational use only. The full license text is in LICENSE.md in this repository — §3.1 requires that any redistribution carries it, so keep it with the file.

File	Recipe	Size
TwIL-LM3_int4.litertlm	int4 blockwise-32 + OCTAV on linears, int8 embedding (externalised)	2.00 GB

Correctness

8-question sanity gate: 8/8 on every backend measured — Mac GPU, Mac CPU, and on an iPhone 17 Pro on both Metal and CPU. Non-degenerate in all four. A 3B model answering all eight on a phone at the bundle's own 4096-token context, with no ladder reduction, is the useful part.

Graded with --max-tokens 2048: this is a reasoning model and emits a <think> block before answering, so a smaller budget scores it as wrong when it has simply not finished thinking.

The formal-logic claim: measured against the base, and no difference detected

Upstream describes TwIL-LM3 as specialised for formal logic. The eight-question gate is general knowledge and cannot see that claim in either direction, so it was tested separately: 12 items with one defensible answer each (syllogisms, modus tollens, affirming the consequent, contrapositive, De Morgan, quantifier negation, disjunctive syllogism, transitivity, necessary vs sufficient, biconditional, vacuous truth), run against this model and its base SmolLM3-3B on identical items, identical flags, identical runtime, and identically-converted bundles (same rail, same recipe, both 2.00 GB, both duplication-free).

	TwIL-LM3	SmolLM3-3B (base)
score	10 / 12	9 / 12

Read this as "no difference detected", not as a win. The two models disagree on only 3 of the 12 items, splitting 2–1 — TwIL-LM3 is right on the valid syllogism and the contrapositive where the base is wrong, the base is right on De Morgan where TwIL-LM3 is wrong, and both fail the vacuous-truth item. A 2–1 split among three discordant items is indistinguishable from chance, and a 12-item instrument can only detect large effects in any case. The honest summary is that this test was not sensitive enough to confirm or refute the specialisation, and the one-point total should not be quoted as evidence for it.

The harness is scripts/eval_logic.py in the converter repo, so the comparison can be repeated or extended.

Performance

litert-lm benchmark (litert-lm 0.16.0), Apple M4 Max, -p 256 -d 256 --cache no, quiet machine, serialized, measured twice (run-to-run spread under 1%):

Backend	Prefill (256)	Decode	TTFT	Init
GPU (Metal)	1341 tok/s	93.0 tok/s	0.206 s	2.50 s
CPU	141.5 tok/s	25.2 tok/s	2.23 s	1.97 s

iPhone 17 Pro, same file, own harness driving the LiteRT-LM Swift API:

Backend	8Q	Context	Init
GPU (Metal)	8/8	bundle default (4096)	6.34 s
CPU	8/8	bundle default (4096)	3.52 s

The phone harness measures correctness and load time only — it does not measure throughput, so no phone tok/s figure is quoted here rather than one being estimated from the desktop.

Usage

litert-lm run ./TwIL-LM3_int4.litertlm --prompt "If it is raining then the ground is wet. The ground is not wet. Is it raining?"

# GPU
litert-lm run ./TwIL-LM3_int4.litertlm --backend gpu --cache no --prompt "..."

The bundle carries the tokenizer and the SmolLM3 chat template with its thinking controls (/think, /no_think) — not plain ChatML. A 4096-token KV budget.

Conversion notes

Converted with litert-torch 0.9.3 / litert-converter 0.3.1 / ai-edge-quantizer 0.8.0 / litert-lm-builder 0.16.0 — a pristine released stack, no patched checkout.

• Same rail as SmolLM3-3B, and the file size proves it. TwIL-LM3 is a SmolLM3-3B finetune, so it converts with the identical one-command recipe. The result is 2,002,241,456 bytes against the shipped SmolLM3-3B's 2,002,257,840 — a 16 KB difference on the same architecture and recipe, which is the strongest single check that the rail applied correctly.
• EXTERNALIZE_EMBEDDER=1 is required, not cosmetic. The model ties its embedding and lm_head, so a recipe asking for int4 linears and an int8 embedder describes one tensor two ways; the quantizer resolves it by copying the 128256-row table once per prefill signature. Externalising the table removes the conflict. Verified duplication-free after export (0.65 bytes/param).
• No start_token in the bundle, which is correct here: upstream sets bos_token: None. A bundle that prepends a BOS the model was never trained to see is a quiet quality killer, so it is worth confirming rather than assuming.
• Quantization: int4 blockwise-32 with OCTAV clipping on the linears, int8 on the embedding.

License and changes

Distributed under the webAI Non-Commercial License v1.0, inherited unchanged from the base model; the complete license text is included as LICENSE.md as its §3.1 requires. Use is limited to non-commercial research and educational purposes (§3.3).

Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged into the .litertlm bundle. No weights were retrained or altered in value beyond quantization.

This repository is a community conversion and is not affiliated with webAI.