Instructions to use rish-huggingface/alfred-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rish-huggingface/alfred-4b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rish-huggingface/alfred-4b", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rish-huggingface/alfred-4b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("rish-huggingface/alfred-4b", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rish-huggingface/alfred-4b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rish-huggingface/alfred-4b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rish-huggingface/alfred-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/rish-huggingface/alfred-4b

SGLang

How to use rish-huggingface/alfred-4b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rish-huggingface/alfred-4b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rish-huggingface/alfred-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rish-huggingface/alfred-4b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rish-huggingface/alfred-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use rish-huggingface/alfred-4b with Docker Model Runner:
```
docker model run hf.co/rish-huggingface/alfred-4b
```

Access Alfred 4B for non-commercial evaluation

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Free for non-commercial evaluation under the Alfred 4B Evaluation License (LICENSE) and NVIDIA's license (LICENSE-NVIDIA).

Alfred 4B

A 4B model that outperforms NVIDIA's 120B Nemotron 3 Super at knowing when to act, when to ask, and when to say it can't: 0.70 vs 0.60 macro F1 on NVIDIA's When2Call benchmark.

A 4B-parameter assistant model trained for judgment in tool use: call a tool when everything it needs was said, ask when a required detail is missing, look things up instead of guessing, and say "I can't" when no tool fits. Fine-tuned from NVIDIA Nemotron 3 Nano 4B with teacher distillation and reinforcement learning with verifiable rewards (GRPO) in Alfred, a sandboxed environment that simulates a person's email, calendar and contacts, plus twelve procedurally generated apps.

Small enough to run on a laptop: a 4-bit GGUF is in alfred-4b-GGUF. Shared for non-commercial evaluation only (see License); access is granted automatically once you accept the terms.

What it does differently

Two real episodes from the evaluation, on apps it never trained on. Same request, same tools, same simulated user; "untrained" is NVIDIA Nemotron 3 Nano 4B, the model Alfred 4B was trained from.

"Which contractors have I saved?" The apps on offer handle bikes, tutoring and membership passes; none has contractors.

Untrained 4B, 3 of 3 tries: opens the tutoring app and answers with the user's tutors ("Based on your saved tutors, you have 3 contractors saved: Amélie D. ...").
Alfred 4B, 3 of 3 tries: "I don't have access to contractor information or saved contractor details. The available tools only relate to bike reservations, tutoring, and membership passes."

"Set up a session with my tutor Yusuf A. on 4/2 at 1:30pm." The booking tool takes the tutor's id, not a name.

Untrained 4B, 2 of 3 tries: books with tutor_id: "Yusuf A." (once also guessing the year 2024; it's January 2027), and the user rejects the draft.
Alfred 4B, 3 of 3 tries: looks up the user's tutors, finds Yusuf A. is T-475, and books T-475 on 2027-04-02 at 13:30, approved the first time.

Full transcripts are in the GitHub release (runs r4-base-newapps-* and r4-rl4-newapps-*).

Results

Our own runs, reasoning off. Everything below recomputes from the evidence in the GitHub repo (python scripts/verify_results.py), and the full write-up is RESULTS.md.

Measure	Nemotron 3 Nano 4B (base)	Alfred 4B	Nemotron 3 Super 120B
When2Call macro F1	0.61	0.70	0.60
When2Call: asked for missing information	59%	65%	57%
When2Call: said "I can't" when no tool fits	32%	50%	30%
When2Call: called a tool when it should	94%	96%	96%
BFCL multi-turn: asked for a missing detail	23%	28%
BFCL multi-turn: said a tool was missing	20%	18%
Apps never trained on, clean success (tools)	68%	99%
Made-up tool arguments on new tasks	23%	4%

On When2Call's 300 items, against Nemotron 3 Super on the same items and the same judge (paired bootstrap, 95%): macro F1 +0.10 [+0.06, +0.15], asking +8 points [+1, +15], saying "I can't" +20 points [+12, +29]. The base model and Super were run with their own settings (Super with reasoning on); Alfred 4B with reasoning off.

Use

Turn reasoning off, as in training and every evaluation: pass chat_template_kwargs: {"enable_thinking": false}. NVIDIA's recommended sampling for tool calling is temperature 0.6, top-p 0.95.

With vLLM, using NVIDIA's recommended flags for Nemotron 3 Nano (the reasoning parser plugin is nano_v3_reasoning_parser.py from the base model's repository):

vllm serve rish-huggingface/alfred-4b --trust-remote-code --mamba_ssm_cache_dtype float32 \
    --enable-auto-tool-choice --tool-call-parser qwen3_coder \
    --reasoning-parser-plugin nano_v3_reasoning_parser.py --reasoning-parser nano_v3

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")
reply = client.chat.completions.create(
    model="rish-huggingface/alfred-4b", messages=[{"role": "user", "content": "Find me a pharmacy."}], tools=[...],
    temperature=0.6, extra_body={"top_p": 0.95, "chat_template_kwargs": {"enable_thinking": False}})

With llama.cpp: llama-server -m alfred-4b-Q4_K_M.gguf --jinja, and the same chat_template_kwargs in each request. The 4-bit GGUF scores within the margin of the full model on the same tasks: 100% vs 99% clean success on trained tasks with tools, 97% vs 98% with plain replies, and 97% vs 100% on apps never trained on.

Training

Base: nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16, LoRA rank 32 on the attention and MLP projections, merged.
Distillation: 3,900 conversations across 20 task types, in a tool interface and a plain-text one: clean successes of Nemotron 3 Nano 30B as a teacher (71%), topped up with scripted correct episodes.
RL: GRPO, 8 attempts per task, rewards computed by code from what happened (no judge model): a task done earns reward; made-up values, unneeded or missing questions, unrequested actions, leaks and acting on prompt injections cost it. 60 steps; the best checkpoint by validation reward was kept.
Data: synthetic. Email text was written by Nemotron 3 Super with real emails from the Enron corpus as style references only. No data from When2Call, BFCL or AgentDojo, or their tool sets, was used in training.

Limitations

Asked for a two-part answer (a meeting's time and place), it can answer from a search result and never open the record that has the second part.
It recovers poorly after a user rejects a draft, sometimes repeating the same search.
Turning a weekday into a date ("Friday") is unreliable.
It declines and then offers alternatives, which some benchmarks count as asking.
English only; evaluated on synthetic tasks and three public benchmarks. It is not a substitute for a human approving actions: in Alfred, every action is a draft the user approves.

License

Evaluation only. The weights are licensed under the Alfred 4B Evaluation License (LICENSE in this repository): you may download and run them privately to evaluate the model and reproduce its results, non-commercially, and publish your evaluation results with credit. You may not use them commercially, make derivatives (fine-tuning, merging, distilling, training on outputs), or redistribute, publish or host them or any derivative. As a derivative of NVIDIA Nemotron 3 Nano 4B they are also governed by the NVIDIA Nemotron Open Model License (LICENSE-NVIDIA). NVIDIA and Nemotron are trademarks of NVIDIA Corporation. The training and evaluation code is separate, under AGPL-3.0, at github.com/Rishabh-Fuke/alfred.

Downloads last month: -

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for rish-huggingface/alfred-4b

Base model

nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base

Finetuned

nvidia/NVIDIA-Nemotron-Nano-12B-v2

Finetuned

nvidia/NVIDIA-Nemotron-Nano-9B-v2

Finetuned

nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16

Finetuned

(31)

this model