Qwen3-4B-openbob-i8

A local 4B assistant with byte-reproducible inference.

Bob runs Qwen3-4B from a frozen 8.01-bit-per-weight integer artifact. On x86-64, the same model bytes and the same journaled state produce the same output bytes: CUDA is qualified on this release, the CPU path on 5.0.0 with its re-qualification pending. Reset him and you get the same trajectory back exactly. Everything needed to test that claim is in this repository.

What this release demonstrates

• deterministic local text generation
• byte-exact replay from the same session state
• CPU and CUDA output identity, byte for byte, shown on 5.0.0; the CPU re-qualification on this release is pending
• digest verification of every weight region before use
• no cloud dependency of any kind

The invariant, in three lines

fresh state  -> prompt -> h1
same session -> prompt -> h2
reset        -> prompt -> h1, byte for byte

Quick start

x86-64 Linux. CUDA on a supported NVIDIA card is the primary path; the CPU path is slow and is re-qualifying on this runtime (see Performance).

tar xf bob-5.0.1-4b.tar.zst
cd bob-5.0.1-4b
sha256sum -c SHA256SUMS
./openbob serve

Wait for READY, then talk to Bob:

1. Type /new, then write a haiku. That reply is h1.
2. Send write a haiku again. The state has advanced; that reply is h2.
3. Type /new, then write a haiku again. That reply is h1, byte for byte.

/diag prints the receipt for the current turn: version, model, the digests of the model and runtime in use, and the execution path.

Results

Measured at packaging on one machine (AMD Ryzen 9 5900X, NVIDIA RTX 3090 Ti, x86-64 Linux). Every number below has its receipt in receipts/.

Check	Result
Stored goldens replayed, twice	7 of 7 pass, both runs
Six fresh-session prompts, run twice	6 of 6 answered from the model; both runs byte-equal
The same six prompts on CPU and CUDA	6 of 6 answers identical, byte for byte, on 5.0.0; this runtime's CUDA answers are those same bytes; CPU re-qualifying
Weight regions verified before use	398 of 398, every run
Denied tool: an invalid retry refused by name, the reply states the denial	5 of 5

Reproducibility

• SHA256SUMS covers every file in the kit; sha256sum -c is the first step and every line must read OK.
• The frame carries TWO digests and both are the same frame. The file digest 599a917e... is the digest of frame/frame.bin and is what sha256sum -c checks; the state digest aa718d25... is the digest of the frame as it enters the session state, and it is the constituent the startup prefix and the golden store are keyed by. receipts/BUILD.txt prints both.
• model/manifest.json carries one digest per weight region and the pinned digest of the CUDA library. The runtime checks each region against it before use, on every run. A failed check is reported by name and the run stops. There is no silent fallback to another path.
• ./openbob run --goldens goldens/ replays the stored golden outputs and compares bytes. ./clean-boot-check.sh runs the six-prompt check twice and compares the runs.
• ./denial-check.sh runs the tool-denial check. Start the assistant with ./openbob serve --trip-budget 0 for this check.
• Every turn is journaled and can be replayed.

Performance

Path	Qualified	Startup	Decode	Notes
CUDA, RTX 3090 Ti	Yes	134.4 s cold	8.6 tokens/s	primary path; about 8.5 GB of card memory in use
CPU, Ryzen 9 5900X	On 5.0.0; re-qualifying on this runtime	935 s first boot, 6 s after	1.4 tokens/s	the 5.0.0 numbers; see receipts/CPU-QUAL.txt

The cold CUDA start is the runtime verifying and staging every weight region before it prints READY. Detailed timing is in receipts/STARTUP.txt.

CUDA is qualified on this runtime. The CPU path is qualified on 5.0.0 (release 5.0.0, the same model artifact and frame) and its re-qualification on this runtime is pending; see receipts/CPU-QUAL.txt.

Limitations

• This is a 4B model. It is wrong about the things a model of this size is wrong about. Determinism is not accuracy.
• The identity claim covers this model artifact, this runtime build and the journaled session state. Changes to any of the three fall outside this identity claim.
• Qualified platform: x86-64 Linux. CUDA is qualified on this runtime; the CPU path is qualified on 5.0.0 and re-qualifying. Metal is not qualified and is not claimed.
• The CPU path computes its startup prefix on first boot (935 s here). A later release ships that prefix as a verified artifact.
• Messaging integrations are not part of this kit.

Technical artifacts

bob-5.0.1-4b.tar.zst is the kit; its digest is in bob-5.0.1-4b.tar.zst.sha256. Unpack it to read any file; the 5.0.0 kit remains browsable under bob-5.0.0-4b/.

openbob                 the runtime, x86-64 Linux
libopenbob_cuda.so      the CUDA library, digest pinned in the manifest
model/                  the model artifact and its manifest
frame/                  the fixed system prompt the assistant runs under
goldens/                the stored golden outputs
clean-boot-check.sh     the six-prompt check
receipts/               every measurement taken at packaging
SHA256SUMS              a digest for every file above
KIT-LAYOUT.txt          one line per file, and the receipt that proves it

The model artifact is Qwen3-4B frozen in an integer representation at 8.01 bits per weight; the base_model_relation field reads quantized because that is the vocabulary the field allows. Package bob-5.0.1-4b, version 5.0.1.

License

Qwen3-4B is Apache-2.0. The runtime and tools in the kit are Apache-2.0. (c) 2026 i64.