Instructions to use KatterMobile/Tinfield-1-EXL3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KatterMobile/Tinfield-1-EXL3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="KatterMobile/Tinfield-1-EXL3")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("KatterMobile/Tinfield-1-EXL3")
model = AutoModelForMultimodalLM.from_pretrained("KatterMobile/Tinfield-1-EXL3", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KatterMobile/Tinfield-1-EXL3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KatterMobile/Tinfield-1-EXL3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KatterMobile/Tinfield-1-EXL3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/KatterMobile/Tinfield-1-EXL3

SGLang

How to use KatterMobile/Tinfield-1-EXL3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KatterMobile/Tinfield-1-EXL3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KatterMobile/Tinfield-1-EXL3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KatterMobile/Tinfield-1-EXL3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KatterMobile/Tinfield-1-EXL3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use KatterMobile/Tinfield-1-EXL3 with Docker Model Runner:
```
docker model run hf.co/KatterMobile/Tinfield-1-EXL3
```

Configuration Parsing Warning:In config.json: "quantization_config.bits" must be an integer

🤖 Tinfield-1-EXL3 (4.05 bpw)

Calibrated ExLlamaV3 (EXL3) 4.05 bpw quantization with 512k max output tokens, 1M context ceiling, native n-gram speculative decoding, and official Qwen thinking runtime for single NVIDIA DGX Spark GB10 / Grace Blackwell (and multi-GPU systems).

☕ Support on Ko-fi: https://ko-fi.com/kattermobile
📬 Support & Inquiries: support@kattermobile.com

⚡ Original Tinfield 1 Benchmarks & Capabilities

Tinfield 1 is an agentic model purpose-built for terminal operations and long-horizon software engineering.

It achieves 33.0 on Terminal-Bench 4.0 (ahead of Claude Opus 4.8 at 23.6 and Claude Sonnet 5 at 12.4) and 62.0 on DeepSWE v1.1 (ahead of Claude Opus 4.8 at 59, Qwen3.8 Max at 57, and Claude Sonnet 5 at 54). It achieves this on only 6.6B active parameters per token out of 177B total across 512 MoE routed experts.

🏆 Benchmark Comparison (Evaluated with mini-swe-agent at k=5)

Terminal-Bench 4.0

Model	Agent	Score
GLM-5.3 (max)	Claude Code	41.8
GPT-5.6 Sol (max)	Codex	37.3
Tinfield 1	mini-swe-agent	33.0
Qwen3.8-Flash-Next		29.0
Claude Opus 4.8 (max)	Claude Code	23.6
GPT-5.6 Terra (max)	Codex	21.5
Grok 4.6 (high)	Grok Build	20.3
Gemini 3.8 Flash (high)	mini-swe-agent	19.1
Claude Sonnet 5 (max)	Claude Code	12.4

DeepSWE v1.1

Model	Score
GLM-5.3 (max)	69.0
GLM-5.3 Flash (max)	63.0
DeepSeek V4 Pro (max)	63.0
Tinfield 1	62.0
Claude Opus 4.8 (max)	59.0
Qwen3.8 Max (xhigh)	57.0
Muse Spark 1.2 (xhigh)	55.0
Claude Sonnet 5 (max)	54.0
DeepSeek V4 Flash (max)	53.0

⚙️ Mandatory Thinking & Recommended Sampling Parameters

Preserve Reasoning Traces: To obtain full reasoning performance and prevent premature cutoff in multi-turn dialogues, ensure your client or UI preserves <think> ... </think> tokens and exposes them in reasoning_content.

Parameter	Recommended Value	Purpose & Notes
`temperature`	`1.0`	Mandatory baseline for Qwen thinking exploration dynamics.
`top_p`	`0.95`	Nucleus sampling threshold.
`top_k`	`20`	Stabilizes token choices during multi-step reasoning.
`min_p`	`0.0`	Disabled (avoids clipping low-probability reasoning branches).
`presence_penalty`	`0.0`	Neutral presence penalty.
`repetition_penalty`	`1.0`	Neutral (prevents damaging code indentation, formulas, and syntax structures).
`max_tokens`	`524,288` (512k)	Huge output budget for full repository solutions and exhaustive reasoning.
`context_window`	`1,048,576` (1M)	1 Million token context ceiling fully supported with native YaRN RoPE.
`preserve_reasoning`	`true`	Keep internal `<think>` traces intact across conversation turns.
`preserve_thinking`	`true`	Template alias for multi-turn thinking preservation.
`reasoning_effort`	`xhigh`	Triggers deep reasoning and validation steps.

🧠 Transparent Architecture: Quantization & Build Pipeline

This build is quantized using Turboderp's ExLlamaV3 (EXL3) with second-order Hessian calibration specifically targeted for single NVIDIA DGX Spark GB10 (128 GB unified memory):

[badtheorylabs/Tinfield-1 Base]
         │
         ▼
[In-Place LoRA Fusion] (420 weight tensors merged directly into FP16 shards)
         │
         ▼
[EXL3 4.05 bpw Quantization] (Trellis calibration across 48 layers / 512 MoE experts)
         │
         ▼
[Deployment on TabbyAPI :8899] (1M Context, 512k Output, N-Gram Drafting @ 58–70 tok/s)

Allocation Breakdown (4.05 bpw Pareto Allocation)

Attention Blocks (Q/K/V/O): 6.00 bpw (ensures rock-solid attention fidelity across 1M context)
512 Routed MoE Experts: 4.00 bpw trellis quantization across all 48 transformer layers
LM Head: 6.01 bpw (preserves token vocabulary projection accuracy for complex code syntax)
MTP Auxiliary Draft Head: 4.02 bpw across 512 draft experts, enabling native speculative decoding
Vision Tower: 6.03 bpw across 27 ViT blocks
N-Gram Lookup Table: 39 GB disk-mapped table (ngram_embedding.safetensors) for zero-VRAM speculative lookups

💻 Quick Start with Pi Code CLI (`pi`)

To use this model directly inside Pi CLI, add the following to your ~/.pi/agent/models.json:

{
  "providers": {
    "vllm-home": {
      "baseUrl": "http://<YOUR_HOST_IP>:8899/v1",
      "api": "openai-completions",
      "compat": {
        "supportsDeveloperRole": false,
        "supportsReasoningEffort": true,
        "thinkingFormat": "qwen-chat-template",
        "chatTemplateKwargs": {
          "enable_thinking": true,
          "preserve_thinking": true,
          "preserve_reasoning": true
        }
      },
      "models": [
        {
          "id": "Tinfield-1-EXL3",
          "name": "Tinfield-1 EXL3 (4.05 bpw)",
          "reasoning": true,
          "contextWindow": 1048576,
          "maxTokens": 524288,
          "samplingParams": {
            "temperature": 1.0,
            "top_p": 0.95,
            "top_k": 20,
            "repetition_penalty": 1.0
          }
        }
      ]
    }
  }
}

Then run in your terminal:

pi --provider vllm-home --model Tinfield-1-EXL3

🚀 Serving with TabbyAPI on NVIDIA DGX Spark

# Recommended environment flags for Grace Blackwell GB10
export EXL3_MOE_MIXEDK_NOSYNC=1
export EXL3_MOE_COOP_MIXEDK=1
export EXL3_MTP_SOLO=1
export PROFILE=concurrent
export MAX_SEQ_LEN=1048576
export CACHE_SIZE=1048576
export NGRAM_RAM=false
export VISION=true
export DRAFT_NUM_TOKENS=5
export HOST="0.0.0.0"
export PORT="8899"
export MODEL_DIR="/path/to/Tinfield-1-EXL3"

# Run inference server
python serve.py

Measured Local Performance (GB10 Grace Blackwell)

Generation Speed: ~58–70 tokens/sec
N-Gram Speculative Acceptance: ~69%
Resident GPU Memory: ~88.3 GB (leaving ~40 GB free for KV cache and context scaling up to 1M tokens!)

📬 Contact, Support & Feedback

☕ Ko-fi Donations: https://ko-fi.com/kattermobile
📧 Support & Bug Reports: support@kattermobile.com
💬 Community Discussions: Feel free to open an issue or discussion thread right here on Hugging Face!

📜 Credits & Licensing

Bad Theory Labs: For the outstanding badtheorylabs/Tinfield-1 base model.
Qwen Team (Alibaba Cloud): For the monumental Qwen3.8-Flash-Next architecture.
Turboderp: For the incredible ExLlamaV3 engine and EXL3 format.
Unsloth Team: For making efficient large-scale QLoRA fine-tuning possible.

This model follows the Qwen Community License 1.0.

Downloads last month: 144

Safetensors

Model size

34B params

Tensor type

BF16

F16

I16

Model tree for KatterMobile/Tinfield-1-EXL3

Base model

Qwen/Qwen3.8-Flash-Next

Finetuned

badtheorylabs/Tinfield-1

Quantized

(7)

this model