Instructions to use divagr1925/Qwenputer-npu with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use divagr1925/Qwenputer-npu with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="divagr1925/Qwenputer-npu")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("divagr1925/Qwenputer-npu")
model = AutoModelForCausalLM.from_pretrained("divagr1925/Qwenputer-npu", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use divagr1925/Qwenputer-npu with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "divagr1925/Qwenputer-npu"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "divagr1925/Qwenputer-npu",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/divagr1925/Qwenputer-npu

SGLang

How to use divagr1925/Qwenputer-npu with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "divagr1925/Qwenputer-npu" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "divagr1925/Qwenputer-npu",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "divagr1925/Qwenputer-npu" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "divagr1925/Qwenputer-npu",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use divagr1925/Qwenputer-npu with Docker Model Runner:
```
docker model run hf.co/divagr1925/Qwenputer-npu
```

Configuration Parsing Warning:Config file tokenizer_config.json cannot be fetched (too big)

Code on GitHub · Qwenputer · Base model: Qwen/Qwen3.5-4B · Apache-2.0

Qwenputer-npu is Qwen3.5-4B with a 32-bit RISC-V CPU compiled into its weights, and that CPU now has a neural-network instruction set. It is fast enough to run a second language model: Karpathy's stories260K runs as a C program on the CPU inside Qwen and writes its stories.

Like Qwenputer, the CPU is 11 transformer layers in front of Qwen's own, every weight set by a compiler, with Qwen3.5's Gated DeltaNet state as its RAM. This version adds two instructions and a weight memory:

VDOT rd, rs1, rs2: the dot product of 8 int8 activations (two registers) with the next 8 int4 weights of the weight memory, added to rd. WB rs1 sets where the weights start.
SMAQA rd, rs1, rs2: four signed byte products added to rd, as in the RISC-V P extension.
A weight memory of 32,640 words (261,120 int4 weights) in DeltaNet heads, loaded from the program's .wrom section.

Each is one instruction for the CPU and one trace token, computed exactly by the compiled layers.

A language model inside the language model

stories260K is a 260K-parameter Llama trained on TinyStories. Here it is a C program (guests/stories260k/): its weights are data in the CPU's memory, and every multiply is an instruction the CPU executes. None of it is compiled into the checkpoint, so any other model that fits the memory runs the same way.

Build of stories260K	Trace tokens per story token
Plain C, int8 weights	2,430,000
Unrolled kernels, no new instructions	1,110,000
VDOT + SMAQA, int4 weights	225,000

The int4 weights come from distilling the float model on its own samples. The CPU writes the same story as the float model for all 60 tokens checked:

Once upon a time, there was a little girl named Lily. She loved to play outside in the park. One day, she saw a big, red ball. She wanted to play with it, but it was too high

A new fused executor runs the CPU at about 5,500 trace tokens a second on an L4, so a story token takes about 41 seconds.

Checks

The checkpoint's own chip layers run VDOT and SMAQA test programs and the first 30,000 steps of the story program, teacher-forced against a reference model of the CPU: zero mismatches.
The executor runs the story free, checked against the reference: exact.
An IR replay of 39 of the mul_big conversations (114,736 positions, 93,381 of them the computer's) against the reference: zero mismatches.
calc still works the same way: every answer the CPU gave in these runs matched the reference model. The published sets, 40 problems each, batched decoding:

Benchmark	Base Qwen3.5-4B	Qwenputer-npu
Synthetic integer arithmetic	75.0	90.0
DeepMind `mul_big`	55.0	97.5

The four arithmetic misses are Qwen repeating calc calls until it runs out of budget, with the CPU answering each one correctly; the published Qwenputer scores 97.5 on that set. Qwen's text behaviour matches base: KL 5.8e-4, against a noise floor of 5.1e-4.

Quickstart

git clone -b npu https://github.com/divagr18/qwenputer && cd qwenputer/experimental/riscv && uv sync --extra hf
hf download divagr1925/Qwenputer-npu --local-dir qwenputer-npu
uv run python -m chipformer.rv.chat qwenputer-npu --firmware tests/data/rv/calc-bridge.elf \
    --ask "Use calc: what is 123456789 * 987654321?"
uv run python -m chipformer.rv.npucheck qwenputer-npu guests/stories260k/stories260k.elf
uv run python guests/stories260k/storyrun.py guests/stories260k/stories260k.elf

npucheck runs the new instructions on this checkpoint's layers; storyrun writes the story on the CPU's executor.

Gated DeltaNet as RAM

Gated DeltaNet state used as RAM: a one-hot key overwrites one row, a one-hot query reads it back, and the CPU's registers, memory and program counter live in DeltaNet heads.

Model details


Base	Qwen/Qwen3.5-4B, Qwen's own weights unchanged
Architecture	stock `Qwen3_5ForCausalLM` (text only); config-only changes
Computer	11 compiled layers before Qwen's 32; RV32IM + Zba + bignum unit + VDOT, WB, SMAQA
Weight memory	32,640 words of 8 int4 weights, in DeltaNet heads
Hidden size	4096 (Qwen's 2560 plus the computer's dims)
Vocabulary	Qwen's, plus 263,092 trace tokens
Size	19.6 GB in bf16, mostly zero padding
Precision	exact in bf16

Citation

@software{agrawal2026qwenputer,
  author  = {Agrawal, Divyansh},
  title   = {{Qwenputer}: a computer compiled into the weights of a language model},
  year    = {2026},
  version = {0.3.0},
  url     = {https://github.com/divagr18/qwenputer},
  license = {MIT}
}

License

Apache-2.0. This is a modified version of Qwen/Qwen3.5-4B: the config is widened and computer layers are added, but Qwen's own weights are unchanged. stories260K is by Andrej Karpathy (MIT).

Downloads last month: 251

Safetensors

Model size

10B params

Tensor type

BF16

F32

Model tree for divagr1925/Qwenputer-npu

Qwen/Qwen3.5-4B

Merge model

this model