Instructions to use ai-forever/SMITH-Exp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ai-forever/SMITH-Exp with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ai-forever/SMITH-Exp")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ai-forever/SMITH-Exp")
model = AutoModelForCausalLM.from_pretrained("ai-forever/SMITH-Exp", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ai-forever/SMITH-Exp with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ai-forever/SMITH-Exp"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ai-forever/SMITH-Exp",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ai-forever/SMITH-Exp

SGLang

How to use ai-forever/SMITH-Exp with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ai-forever/SMITH-Exp" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ai-forever/SMITH-Exp",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ai-forever/SMITH-Exp" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ai-forever/SMITH-Exp",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ai-forever/SMITH-Exp with Docker Model Runner:
```
docker model run hf.co/ai-forever/SMITH-Exp
```

SMITH-Exp

SMITH-Exp is an instruction-tuned Mixture-of-Experts checkpoint based on GigaChat 3.1 Lightning (10B total parameters, 1.8B active, BF16). It is optimized for multi-hop retrieval through GigaChat 3 tool calls: the model iteratively invokes search_index, then returns an answer with supporting_corpus_ids.

For reproducible results, use the checkpoint with the SMITH retrieval harness or an equivalent implementation of the same tool-calling loop. The evaluation results below assume this execution protocol; behavior and quality are not guaranteed in general-purpose chat frameworks or integrations that use different prompts, tools, parsing, or stopping logic.

The included gigachat3_guided_decoding.py plugin integrates with vLLM's GigaChat 3 tool parser and constrains tool calls to the expected structure. This reduces malformed calls and allows the harness to parse search_index and final answer actions reliably. Use the plugin with the supplied chat template, as shown below.

Evaluation

MuSiQue answerable validation (bdsaglam/musique), 100 queries mixing the three four-hop topologies: 50 4hop1, 25 4hop2, and 25 4hop3. Ranking is against gold supporting paragraphs at k=10 on a text-deduplicated train+validation FAISS index. Agent runs use temperature 0, retriever_k=100, agent_max_k=10, and at most six searches.

Citation scores the supporting_corpus_ids in the final answer. Search scores the union of documents retrieved over the trajectory. Searches is the mean number of search_index calls per question.

The embedder-only row is a comparison baseline (single top-10 search of the original question, no agent). Agents still call search_index as a tool; that tool uses Qwen3-Embedding-4B (2,560-d).

Model	nDCG@10 (citation)	nDCG@10 (search)	Recall@10 (citation)	Recall@10 (search)	Searches
Agents
SMITH-Exp	0.664	0.766	0.629	0.680	5.73
GLM-5.3	0.611	0.709	0.515	0.610	4.64
DeepSeek-V4-Flash-0731	0.591	0.726	0.523	0.638	5.84
Qwen3.6-35B-A3B	0.546	0.712	0.445	0.620	5.57
Embedders
Qwen3-Embedding-4B	0.333	0.333	0.300	0.300	—

Usage

Serving dependencies

uv venv --python python3.11 --seed .venv
uv pip install -r requirements.txt \
  --python .venv/bin/python \
  --torch-backend=cu130 \
  --index-strategy unsafe-best-match

Package versions and license information are listed in requirements.txt. The model fits on a single 80 GB GPU.

transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig

model_name = "ai-forever/SMITH-Exp"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.generation_config = GenerationConfig.from_pretrained(model_name)
messages = [
    {"role": "user", "content": "Which corpus passages support the answer?"}
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt")
inputs = {k: v.to(model.device) for k, v in inputs.items()}
outputs = model.generate(**inputs, max_new_tokens=512)
prompt_len = inputs["input_ids"].shape[1]
print(tokenizer.decode(outputs[0][prompt_len:], skip_special_tokens=True))

vLLM

vllm serve ai-forever/SMITH-Exp \
  --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-call-parser gigachat3 \
  --tool-parser-plugin ./gigachat3_guided_decoding.py \
  --chat-template ./chat_template.jinja \
  --tensor-parallel-size 1 \
  --dtype bfloat16

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ai-forever/SMITH-Exp",
    "temperature": 0,
    "tool_choice": "required",
    "messages": [
      {"role": "user", "content": "Which corpus passages support the answer?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "search_index",
          "description": "Search the local semantic index and return relevant text snippets.",
          "parameters": {
            "type": "object",
            "properties": {
              "query": {"type": "string", "description": "The search query string."},
              "k": {"type": "integer", "description": "Optional number of top results to return."}
            },
            "required": ["query"]
          }
        }
      }
    ]
  }'