Instructions to use devopsfortunato/Fonzie0.2-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use devopsfortunato/Fonzie0.2-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="devopsfortunato/Fonzie0.2-4B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("devopsfortunato/Fonzie0.2-4B")
model = AutoModelForMultimodalLM.from_pretrained("devopsfortunato/Fonzie0.2-4B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use devopsfortunato/Fonzie0.2-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "devopsfortunato/Fonzie0.2-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "devopsfortunato/Fonzie0.2-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/devopsfortunato/Fonzie0.2-4B

SGLang

How to use devopsfortunato/Fonzie0.2-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "devopsfortunato/Fonzie0.2-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "devopsfortunato/Fonzie0.2-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "devopsfortunato/Fonzie0.2-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "devopsfortunato/Fonzie0.2-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use devopsfortunato/Fonzie0.2-4B with Docker Model Runner:
```
docker model run hf.co/devopsfortunato/Fonzie0.2-4B
```

Fonzie0.2-4B

A 4B model that searches when it doesn't know, answers with sources, speaks Italian and English, and looks at pictures. It is a fine-tune of Qwen3.5-4B, trained to work inside real agent apps: oh-my-pi and opencode. It was also tested in Hermes Agent, which it never saw in training.

In italiano: un 4B che cerca quando non sa, risponde con le fonti, parla italiano e inglese e guarda le foto. Rispetto a Fonzie0.1 riprova dopo una ricerca fallita, cerca prima di rispondere su "l'ultimo X", dice quando una cosa non esiste e in oh-my-pi fa le domande all'utente con lo strumento ask. È un progetto di studio personale di Alfonso Fortunato: i numeri qui sotto dicono onestamente cosa migliora e cosa peggiora.

GGUF for llama.cpp / LM Studio (Q8_0, Q4_K_M, vision projector): Fonzie0.2-4B-GGUF. Previous version: Fonzie0.1-4B.

Smaller versions: Q4_K_M and Q8_0

To run it in a few GB, Fonzie0.2-4B-GGUF has a Q4_K_M (2.8 GB) and a Q8_0 (4.6 GB), plus the 0.7 GB vision projector. GPU memory used by llama-server with the projector loaded, MTP on and the default f16 KV cache:

Context (tokens)	Q4_K_M	Q8_0
8,192	4.2 GiB	5.9 GiB
32,768	5.1 GiB	6.8 GiB
65,536	6.3 GiB	8.0 GiB
131,072	8.7 GiB	10.4 GiB
163,840	10.1 GiB	11.8 GiB
262,144 (the model's limit)	≥ 13.4 GiB	≥ 15.1 GiB

The Q4_K_M was measured on an RTX 4070 Ti (12 GB) at 64k, 128k and 160k; the other rows are computed from those measurements, and the Q8_0 was not run. With a q8_0 KV cache the Q4_K_M fits 229,376 tokens on 12 GB.

What the Q4_K_M loses against this bf16 model (bold: paired 95% interval excludes zero):

	bf16	Q4_K_M
frozen test: accuracy	66.5%	64.8%
frozen test: confident wrong answers	24.6%	27.4%
frozen test: invented links	2.6%	7.2%
everyday questions, accuracy (oh-my-pi / opencode / Hermes)	66.9 / 66.7 / 76.4%	54.2 / 66.7 / 73.2%
new test set, accuracy (oh-my-pi / opencode / Hermes)	55.4 / 61.1 / 67.8%	52.1 / 67.2 / 60.6%

In opencode and Hermes it is as good as the bf16, within the noise; in oh-my-pi it loses about 13 points on everyday questions and writes more links that no tool returned. The Q8_0 has not been evaluated. Details and the fastest 12 GB setup are in the GGUF model card.

What is new compared to Fonzie0.1

Fonzie0.2 has the same recipe as Fonzie0.1; only the data changed. It adds conversations for the failures seen in Fonzie0.1. Fonzie0.1 was measured again in exactly the same setup (merged bf16 on vLLM 0.32, same sampling), so its numbers here differ slightly from its own card. Differences in bold are outside the paired 95% bootstrap interval.

New tests, written for these failures and never used in training: 20 questions per category, half Italian and half English, 3 runs. Search goes only to a local Wikipedia index, so the results don't depend on the web of the day.

"the latest X" with no date: it must search first, then answer correctly;
false premises (versions, events or URLs that don't exist): it must search and say the thing doesn't exist, without adding facts from memory;
every search fails: it must say what failed, not answer from memory and not go round in circles;
ambiguous requests in oh-my-pi: it must ask the user with the ask tool.

	oh-my-pi			opencode			Hermes (unseen)
	base	0.1	0.2	base	0.1	0.2	base	0.1	0.2
Accuracy	29.2%	36.7%	55.4%	36.1%	47.8%	61.1%	45.0%	51.1%	67.8%
Confident wrong answers	69.2%	57.5%	38.8%	62.8%	48.3%	31.7%	52.2%	46.1%	25.6%
Follows the category's rule	25.0%	33.8%	53.3%	32.8%	45.0%	58.9%	41.1%	46.1%	63.3%
- "the latest X"	35.0%	30.0%	43.3%	38.3%	28.3%	46.7%	38.3%	36.7%	50.0%
- false premises	50.0%	58.3%	70.0%	48.3%	56.7%	60.0%	65.0%	65.0%	76.7%
- every search fails	15.0%	45.0%	68.3%	11.7%	50.0%	70.0%	20.0%	36.7%	63.3%
- ambiguous requests (`ask`)	0.0%	1.7%	31.7%
Asks the user with `ask`	1.7%	3.3%	45.0%
Loops / timeouts	22.9%	9.2%	12.5%	32.2%	10.0%	10.6%	23.9%	7.8%	14.4% (worse)

Recovery after a failed search (rate limit or timeout, failure types never seen in training; 60 questions, 48 in Hermes, 1 run), Fonzie0.1 → Fonzie0.2:

	oh-my-pi	opencode	Hermes
Tries again after the failure	24% → 60%	74% → 95%	72% → 91%
Then finds results	16% → 38%	41% → 68%	69% → 85%
Confident wrong answers	55.0% → 38.3%	38.3% → 30.0%	39.6% → 22.9%

The 130 test questions of Fonzie0.1 (94 for Hermes, text only), 3 runs, Fonzie0.1 → Fonzie0.2:

	oh-my-pi	opencode	Hermes
Searches when it doesn't know	83.9% → 92.2%	83.9% → 93.3%	85.4% → 95.1%
Searches when it doesn't need to	27.6% → 31.0%	32.4% → 40.5%	26.1% → 29.7%
Accuracy	63.3% → 65.5%	65.0% → 66.7%	73.2% → 76.4%
Confident wrong answers	27.4% → 24.0%	27.1% → 24.3%	22.8% → 18.7%
Loops / timeouts	7.9% → 9.5%	4.9% → 5.9%	5.3% → 5.7%
Links not returned by a tool	4.1% → 5.9%	1.8% → 4.6% (worse)	0.7% → 2.8%
Answers in the user's language	95.3% → 97.7%	95.1% → 96.9%	98.9% → 99.6%

On our own agent format (250 frozen questions, 2 runs) it is slightly better than Fonzie0.1 on every metric, none significant: accuracy 66.5% vs 64.8%, confident wrong answers 24.6% vs 25.9%, loops 4.0% vs 5.6%, right language 99.4% vs 98.7%.

What it does not do (yet)

It writes more links that no tool returned: 4.6% of answers in opencode, against 1.8% for Fonzie0.1. In Hermes, after a failed search, 12.5% against 2.1% (48 questions, not significant).
More loops in Hermes on the new tests: 14.4% against 7.8%.
It searches a bit more when it doesn't need to (+3 to +8 points, not significant).
"The latest X" still fails about half the time, even if it now searches first more often.
Accuracy on the 130 questions is about the same as Fonzie0.1, still 1-2 points below the base model's numbers (measured on an older vLLM, 0.24).
Photos were not measured separately: identifying specific landmarks and artworks was unreliable in Fonzie0.1, and this version has only 35 new "recognize, then verify" conversations.
Outside an agent context (plain chat, no system prompt, no tools) it falls back to the base model's long thinking, like Fonzie0.1.

How to use it

It expects an agent harness that gives it a web search tool. To reproduce the training setup in opencode, serve it with vLLM and add a search MCP server.

vllm serve devopsfortunato/Fonzie0.2-4B --served-model-name fonzie --max-model-len 65536 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3

Sampling used in all evaluations: temperature 1.0, top_p 0.95, top_k 20 (also in generation_config.json).

The LoRA adapter alone (rank 16, on top of unsloth/Qwen3.5-4B) is in lora/.

How it was made

Data: 1,788 conversations, 68% in Italian, 344 with a picture:

the 1,390 of Fonzie0.1 without two sources removed for the license (no_robots and evol-instruct-italian, 89 conversations): 1,301;
487 new conversations, written for the failures of Fonzie0.1:
- 170 recoveries: the first search fails with the apps' real error messages, and the next turn says what failed and searches again;
- 141 "the latest X" with no date, where searching is the first step;
- 36 false premises;
- 52 oh-my-pi conversations that use the ask tool;
- 35 photos, "recognize, then verify with a search";
- 53 general requests from Aya and oasst2, replacing the removed sources.
Formats: oh-my-pi 678, opencode 537, plain agent format 573.
Decisions: 38% answer directly, 52% search, 10% search and fail.
Answers are capped at 100 words, except tasks and math.

Teachers, all run locally on a DGX Spark:

GLM-5.3-Flash (text, the Fonzie0.1 data);
Qwen3.8 Flash Next (the new data, text and pictures).

Filters: every conversation passed code checks before an LLM judge (Grok 4.7) looked at it:

the answer matches the reference;
no answer or guessed names in the search query;
every link comes from a tool result;
right language;
short reasoning;
no repeated identical calls.

The judge was checked by hand on 15 conversations (15 agreements out of 15). Answers rejected for details without a source were rewritten without those details, then judged again. A report checked that no sentence appears in more than 5 answers and that no test question is in the training data.

Training: the same recipe as Fonzie0.1. LoRA rank 16 on the language layers (vision encoder frozen), bf16, learning rate 1e-4, 1 epoch, up to 24,576 tokens, with Unsloth. Training took about 2.5 hours on an NVIDIA DGX Spark. No RL.

Credits

Javi0.2-4B and Javi0.3 by Javier (@javierbits): this project follows their approach (test in the real harness, check with code, fix failures with data) and many of their lessons. The new tests of this version come from ideas in Javi0.3.
Qwen for the base model and the Qwen3.8 teacher, Z.ai for the GLM-5.3-Flash teacher, Unsloth for training, and llama.cpp for the GGUF files.
Questions from:
- MKQA (Apple, CC BY 3.0);
- TriviaQA;
- HotpotQA (CC BY-SA 4.0);
- Wikipedia and Wikidata (CC BY-SA, CC0);
- Wikimedia Commons images;
- ChartQA (GPL-3.0);
- DocVQA (MIT);
- gsm8k_italian (Sapienza NLP, MIT);
- Aya (Cohere, Apache 2.0);
- oasst2 (OpenAssistant, Apache 2.0).

License

Apache 2.0, like the base model Qwen3.5-4B. The training data is not redistributed. The teachers' licenses (GLM-5.3-Flash: MIT; Qwen3.8 Flash Next: Qwen license) put no limits on their outputs.

Author: Alfonso Fortunato.

Downloads last month: -

Safetensors

Model size

5B params

Tensor type

BF16

F32

Model tree for devopsfortunato/Fonzie0.2-4B

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Adapter

(977)

this model