Instructions to use torchcast-ai/torchcast-decision-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use torchcast-ai/torchcast-decision-4b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="torchcast-ai/torchcast-decision-4b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("torchcast-ai/torchcast-decision-4b")
model = AutoModelForMultimodalLM.from_pretrained("torchcast-ai/torchcast-decision-4b", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use torchcast-ai/torchcast-decision-4b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "torchcast-ai/torchcast-decision-4b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "torchcast-ai/torchcast-decision-4b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/torchcast-ai/torchcast-decision-4b

SGLang

How to use torchcast-ai/torchcast-decision-4b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "torchcast-ai/torchcast-decision-4b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "torchcast-ai/torchcast-decision-4b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "torchcast-ai/torchcast-decision-4b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "torchcast-ai/torchcast-decision-4b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use torchcast-ai/torchcast-decision-4b with Docker Model Runner:
```
docker model run hf.co/torchcast-ai/torchcast-decision-4b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Torchcast Decision 4B (v1.3)

A text-based decision model developed by Torchcast AI, based on Qwen3.5-4B. It returns yes/no probabilities (noul), choice distributions (choice) and ordinal score distributions (score) through a TypeSafe-compatible API. Each decision reads the option letters at the answer position in one forward pass (choice lists with more than 26 options are decided in a few passes over groups of options); nothing is generated.

Use the bundled torchcast_decision serving package with the bundled decision_config.json. Generic Hub text-generation examples and causal-attention runtimes (vLLM, llama.cpp, MLX) do not reproduce the evaluated interface: in this version the full-attention layers attend to the whole prompt in both directions (see below).

Serving instructions

What changed in v1.3

Weights: a new full fine-tune of Qwen/Qwen3.5-4B (revision 851bf6e8, same recipe as v1.2, 3,505 steps) on a new training data mix of 897,139 rows: the earlier v1.3 build's 878,791 rows plus 18,348 new rows from a further non-Claude data block, with 29,385 question-bank rows reworded by MiniMax, re-smoothed targets for easy and standard items (label smoothing 0.1 to 0.03 on 267,530 rows) and 18,877 paired twin items placed in the same batches. The architecture, tokenizer, prompt format (jev_render_v3) and non-causal attention mode are the same as in v1.2.
Speed: the non-causal mode now runs with CUDA graphs (torchcast_decision/model.py). Each graph records a key-padding mask for its rows and uses the same padded lengths as the eager path, so it replays the kernels v1.2 ran eagerly: probabilities are bit-identical to the eager path, and the median latency per request is about half (67.9 ms to 31.9 ms on one A40). /health reports the number of recorded graphs (cuda_graphs).
Memory: the server keeps only the option-letter rows of the output head (1.18 GiB less GPU memory allocated after loading; probabilities unchanged).
Serving temperature: decision_config.json sets T = 0.9 for choice, yes/no and score questions (v1.2: T = 1.0).

An earlier internal v1.3 build (FULL_v5 weights) was not released; these weights supersede it.

Results

v1.3 has not been measured on the JevBench leaderboard yet.

Our own reads, paired in the same harness (the bundled server or its offline equivalent and the JevBench typesafe adapter), all at T = 0.9, against the earlier v1.3 build (FULL_v5 weights):

Set	Earlier v1.3 build (Capability)	v1.3 (Capability)	v1.3 Intelligence / Calibration
231 public JevBench items	81.55	81.4	74.9 / 87.9
Held-out proxy set (proxyA2)	74.4	75.3
Independent JevBench replica (2,178 items)	78.7	82.1 (+3.5, significant)	+3.4 / +3.5 over the earlier build (both significant)

On the public items and our Proxy C set, v1.3 is level with the earlier build; the gain is on the independent replica. On the public items, v1.2 at the same T = 0.9 scores Capability 80.3 (as shipped, at T = 1.0: 77.3), against 81.4 for v1.3. These are our measurements on public or internal items, not leaderboard scores.

v1.2 (history)

On the JevBench leaderboard (release v1.6.34, as of 2026-10-20), Torchcast Decision 4B v1.2 was #1 among open-weight models by composite score:

Model	Composite	Intelligence	Calibration	Speed	Cost
Torchcast Decision 4B v1.2	73.03	60.26	84.95	91.13	64.81
H2O-Lightning-4B v1.1	72.52	60.04	89.95	92.61	60.30

Source: benchmarkheaven.com/jev-models. v1.2 was measured on the second regular v1.6 draw (anchor runs pending), so scores may be re-placed when the board's anchor offsets are applied. These numbers are for v1.2 (tag v1.2.0 of this repository), not v1.3.

Artifact

Pinned revision: tag v1.3.0 of this repository.
Weight SHA-256: see SHA256SUMS.
Served model name: torchcast-decision-4b-v1.3.

Serving settings

decision_config.json sets a calibration temperature T = 0.9 for every question type (temperature_by_type: the option-letter logits are divided by 0.9 before normalising, which sharpens the probabilities; the answer does not change) and a yes/no commit floor of 0.801 (a yes/no probability whose max(P, 1 - P) is below 0.801 is moved to 0.801 on its own side; the answer does not change). Within a prompt, the full-attention layers use no causal mask (attention_mode: noncausal_full_attention, as in training); the bundled PyTorch backend implements this, and the MLX and GGUF backends refuse the model. Prompts longer than 16,384 tokens keep the head and the tail of the state. Triton kernel selection is pinned, so repeated runs on the same GPU type give identical probabilities from a cold or a warm cache. See NOTICE and the serving instructions. Loading the weights with plain transformers (causal attention) gives different probabilities; serve through the bundled torchcast_decision package.

Intended use and limitations

English text-based decision research, such as classification, routing and bounded rubric evaluation. The server returns decisions and does not execute actions. Probabilities can be unreliable on new domains.

Licence and attribution

Model weights: Apache License 2.0 (see LICENSE). Base model: Qwen3.5-4B by the Qwen team, Apache License 2.0, Copyright 2026 Alibaba Cloud (LICENSE-qwen-Apache-2.0, NOTICE). Serving package: Apache License 2.0, derived from the StartLux-Decision inference code (torchcast_decision/LICENSE, torchcast_decision/NOTICE). This project is independent of TypeSafe AI and the JevBench maintainers. Earlier releases keep their own licence: v1.1 (tag v1.1.0) remains under CC BY-NC 4.0.

Downloads last month: 217

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for torchcast-ai/torchcast-decision-4b

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Finetuned

(1103)

this model

Quantizations

1 model