Instructions to use F-A-I-L/mini-agent-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use F-A-I-L/mini-agent-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="F-A-I-L/mini-agent-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("F-A-I-L/mini-agent-v2")
model = AutoModelForCausalLM.from_pretrained("F-A-I-L/mini-agent-v2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use F-A-I-L/mini-agent-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "F-A-I-L/mini-agent-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "F-A-I-L/mini-agent-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/F-A-I-L/mini-agent-v2

SGLang

How to use F-A-I-L/mini-agent-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "F-A-I-L/mini-agent-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "F-A-I-L/mini-agent-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "F-A-I-L/mini-agent-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "F-A-I-L/mini-agent-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use F-A-I-L/mini-agent-v2 with Docker Model Runner:
```
docker model run hf.co/F-A-I-L/mini-agent-v2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

mini-agent-v2

A 1.5B native tool-calling, Bash and Python fine-tune of Qwen2.5-Coder-1.5B-Instruct, shipped as merged bf16 weights and six GGUF quants.

bf16 weights · GGUF quants

checkpoint b15 (d1497d77e5) · evaluated 2026-10-04

Summary

mini-agent-v2 is a QLoRA fine-tune of Qwen2.5-Coder-1.5B-Instruct, trained jointly on function calling, Bash and Python on a single 8 GB consumer GPU. It is small enough to run on a laptop. It ships as merged bf16 weights and as GGUF builds from Q3_K_M to bf16.

It speaks the standard tool-call protocol: Qwen's own chat template with <tool_call>{"name": ..., "arguments": {...}}</tool_call>. That is the format llama.cpp (--jinja), vLLM, Ollama and OpenAI-style clients already parse. You don't need a custom system prompt or parser: pass your tools in the normal tools field. On the Berkeley Function Calling Leaderboard (BFCL) questions it answers 86.2% correctly, up from 52.7% for the base model.

At a glance


✅ Use it for	native tool calls through any OpenAI-compatible server · Bash one-liners · small Python functions · local/offline agents
⚠️ Don't use it for	long autonomous multi-step agent loops · shell commands you won't read · anything safety-critical
📈 Best result	86.2% on BFCL non-live in FC mode (base: 52.7%), level with Qwen3-1.7B and xLAM-2-3B
📉 Biggest weakness	real-user BFCL live questions: 65.7%, so about 1 in 3 is still wrong
🧰 Works with	`llama-server --jinja`, vLLM, transformers; tool calls checked on every GGUF
💾 Smallest build	824 MB (Q3_K_M GGUF)

Specifications


Base model	`Qwen/Qwen2.5-Coder-1.5B-Instruct` (1.54 B params, 28 layers, tied embeddings)
Method	QLoRA over `unsloth/qwen2.5-coder-1.5b-instruct-bnb-4bit`: LoRA rank 16, alpha 32, dropout 0, bias none
Trainable params	18,467,840: 392 LoRA A/B tensors (28 layers × 7 projections) + 2 re-initialised tool-tag embedding rows
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Training context	2048 tokens, no packing
Inherited context	32,768 tokens
Release precision	adapter merged into the bf16 base
License	Apache-2.0

Results

Grouped bar chart comparing Qwen2.5-Coder-1.5B-Instruct against mini-agent-v2. BFCL standard 52.7% base vs 86.2%; BFCL live 56.9% vs 65.7%; NL2SH-ALFA bash 41.7% vs 50.3%; InterCode bash 30.5% vs 36.0%; MultiPL-E sh 32.8% vs 32.6%; HumanEval+/MBPP+ 59.2% vs 66.2%.

These are the three tasks the model was trained for, compared with the base model under an identical harness. Every answer is executed, not string-matched: Bash runs in Docker, Python runs against unit tests, and tool calls go through BFCL's AST checker.

Benchmark	n	Base `Qwen2.5-Coder-1.5B-Instruct`	mini-agent-v2	Δ	Paired test
Tool calls: BFCL non-live, FC mode	1000	52.7%	86.2%	+33.5 pp	McNemar p < 0.001
Tool calls: BFCL live, FC mode	1335	56.9%	65.7%	+8.8 pp	McNemar p < 0.001
Bash: NL2SH-ALFA, executed	300	41.7%	50.3%	+8.6 pp	McNemar p < 0.001
Bash: InterCode-Bash, executed	200	30.5%	36.0%	+5.5 pp	McNemar p = 0.080, n.s.
Bash: MultiPL-E sh, unit tests	540	32.8%	32.6%	−0.2 pp	McNemar p = 1.000, n.s.
Python: HumanEval+ / MBPP+, executed	542	59.2%	66.2%	+7.0 pp	McNemar p < 0.001

Tool calling is the big gain. The base model's tool-tag embeddings are untrained, so it barely emits its own <tool_call> format, and when asked for tools it mostly answers in Markdown JSON. Fixing that, plus the tool data, takes it from 52.7% to 86.2%.
Bash and Python improve without being traded away. NL2SH-ALFA gains 26 commands and Python gains 38 problems, both significant. MultiPL-E sh (writing whole shell programs) is unchanged.
InterCode needs a caveat. 127 of its 200 tasks overlap NL2SH-ALFA's test split, and the whole gain sits there. On the 73 independent tasks it is flat.

Compared with other small tool-calling models

Bar charts of BFCL FC-mode accuracy. Standard: xLAM-2-3b 87.7%, Qwen3-1.7B 86.5%, mini-agent-v2 86.2%, Arch-Function-1.5B 82.1%, xLAM-2-1b 75.7%, Hammer2.1-1.5b 71.6%, base 52.7%. Live: mini-agent-v2 65.7%, Hammer2.1-1.5b 57.2%, base 56.9%, Arch-Function-1.5B 55.6%, xLAM-2-1b 52.2%.

Every model was run through the same harness in bf16, using its own chat template and native tool-call format.

Model	BFCL non-live	BFCL live	NL2SH-ALFA	InterCode	MultiPL-E sh	HE+/MBPP+
mini-agent-v2 (1.5B)	86.2%	65.7%	50.3%	36.0%	32.6%	66.2%
Qwen2.5-Coder-1.5B-Instruct (base)	52.7%*	56.9%*	41.7%*	30.5%	32.8%	59.2%*
Hammer2.1-1.5b	71.6%*	57.2%*	41.3%*	28.5%*	34.3%	62.0%*
xLAM-2-1b-fc-r	75.7%*	52.2%*	26.0%*	21.0%*	22.4%*	54.6%*
Arch-Function-1.5B	82.1%*	55.6%*	34.3%*	23.0%*	24.1%*	55.4%*
xLAM-2-3b-fc-r (3B)	87.7%	**	**	**	**	**
Qwen3-1.7B, no-think	86.5%	**	**	**	**	**

Notes: * = the gap to mini-agent-v2 is statistically significant (both models answered the same questions; paired exact McNemar test, p < 0.05), so it is unlikely to be luck. No * means the two are effectively tied. ** = that model was not tested on this benchmark (not a score of zero).

Files

The weights are split across two repos: F-A-I-L/mini-agent-v2 holds the bf16 safetensors for transformers / vLLM, and F-A-I-L/mini-agent-v2-GGUF holds the GGUF builds for llama.cpp and GGUF apps.

File	Repo	Size	Bits per weight
`model.safetensors` + `config.json` + tokenizer files	`mini-agent-v2`	3.09 GB	16 (bf16)
`mini-agent-v2-Q3_K_M.gguf`	`mini-agent-v2-GGUF`	824 MB	4.24
`mini-agent-v2-Q4_K_M.gguf`	`mini-agent-v2-GGUF`	986 MB	5.08
`mini-agent-v2-Q5_K_M.gguf`	`mini-agent-v2-GGUF`	1.13 GB	5.80
`mini-agent-v2-Q6_K.gguf`	`mini-agent-v2-GGUF`	1.27 GB	6.56
`mini-agent-v2-Q8_0.gguf`	`mini-agent-v2-GGUF`	1.65 GB	8.50
`mini-agent-v2-bf16.gguf`	`mini-agent-v2-GGUF`	3.09 GB	16 (unquantized)

GGUF metadata: architecture qwen2, 1,543,714,304 params, context 32768, EOS <|im_end|>, with the stock Qwen2.5 chat template (tool support intact). All quants were made from the bf16 GGUF with llama-quantize. Plan on the file size plus about 1 GB of memory.

Scores above are for the bf16 weights. Fewer bits per weight means a smaller file and lower quality. Every GGUF was checked to produce a correct tool call, but the quants were not re-benchmarked, so expect somewhat lower scores than these, more so the smaller the file.

How to prompt it

Tool calling: pass tools through the chat template's tools= field (or the OpenAI tools field on a server). No system prompt is needed; the template inserts the tool list. It replies with one or more <tool_call>{"name": ..., "arguments": {...}}</tool_call> blocks, or with plain text when no tool fits.

Bash

You are a Bash assistant. Translate the request into a single Bash command.

For a full script: You are a Bash assistant. Write a complete Bash script for the request.

Python

You are a Python coding assistant. Write correct code. Respond with a single code block.

The benchmark numbers on this card were measured with greedy decoding (temperature 0).

Usage

llama.cpp

# downloads the chosen quant from the GGUF repo on first run
# (QUANT = Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0 or bf16)
llama-server -hf F-A-I-L/mini-agent-v2-GGUF:QUANT --jinja -c 4096 -ngl 99
# or, with a file you already downloaded
llama-server -m mini-agent-v2-QUANT.gguf --jinja -c 4096 -ngl 99

--jinja is required for tool calling. Without it, llama.cpp ignores the template's tool support.

curl localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Berlin in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather",
    "parameters": {"type": "object", "properties": {
      "city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}},
    "required": ["city"]}}}]}'
# -> tool_calls: get_weather({"city": "Berlin", "unit": "celsius"})

Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "F-A-I-L/mini-agent-v2"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")

tools = [{"type": "function", "function": {"name": "get_weather",
          "parameters": {"type": "object", "properties": {"city": {"type": "string"}},
                         "required": ["city"]}}}]
msgs = [{"role": "user", "content": "Weather in Paris?"}]
enc = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True,
                              return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**enc, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=False))
# <tool_call>
# {"name": "get_weather", "arguments": {"city": "Paris"}}
# </tool_call><|im_end|>

Any GGUF app

LM Studio, Jan and KoboldCpp load the .gguf directly. To use tools, turn on the app's tool/function-calling support.

🖥️ Built on one 8 GB laptop GPU, looking for a bigger one

Every run behind this model was trained and evaluated on a single RTX 4060 (8 GB). That is why it's 1.5B, trained on a 4-bit base with 2k-token examples. The same weights already score significantly higher on Bash and Python when run in full bf16 instead of 4-bit.

If your team has an NVIDIA DGX Spark or an AMD Ryzen AI Max+ ("Strix Halo") AI dev kit to spare, it would go straight into full-precision training, longer context, 3B–7B bases and verified-reward RL, all released openly.

Get in touch via Discussions. No obligation; the model stays free either way. Thank you!

Evaluation

How this was measured

All results come from a run on 2026-10-04 (external models 2026-10-05), scored with each benchmark's own scorer. Where a task can be executed, it was executed.

Decoding: greedy, batch size 8, 4096-token eval context, bf16 weights, the same harness for every model.
Tool calling: BFCL non-live (simple, multiple, parallel, parallel-multiple) and live, in FC mode. Tools go through each model's tools= chat template, the answer is decoded from its native tool-call output, and it is checked with a reimplementation of BFCL's official AST checker. This is this project's harness, not the public leaderboard's prompt mode, so the numbers are not directly comparable with the leaderboard.
Bash: NL2SH-ALFA test split, executed through InterCode-ALFA in Docker and scored on filesystem/output equivalence. InterCode-Bash runs in its own images. MultiPL-E sh runs unit tests in a network-less container.
Python: EvalPlus HumanEval+ and MBPP+ (base + hardened "plus" tests), executed.
Significance: paired exact McNemar test on per-question outcomes. Same problems, same prompts, same decoding.

Tool calling, in detail

Split	mini-agent-v2	Base
non-live micro / macro	862/1000 (86.2%) / 85.1%	527/1000 (52.7%) / 43.7%
— simple / multiple	362/400 · 181/200
— parallel / parallel-multiple	162/200 · 157/200
live micro / macro	877/1335 (65.7%) / 56.3%	759/1335 (56.9%) / 29.0%
— live simple / live multiple	160/258 · 697/1037
— live parallel / live parallel-multiple	6/16 · 14/24

The remaining errors are mostly argument values, not tool choice:

Optional parameters left out or added when the user didn't ask for them
User wording not mapped onto a schema enum value
Date arithmetic ("next Tuesday")
Very long tool lists

Development history

Line chart of BFCL FC-mode accuracy across runs b10-10pct to b16. Standard rises from 80.3% to 85.7% at b15; live from 59.7% to 65.8% at b15; b16 is slightly lower on both.

mini-agent-v2 is run b15 of a series:

b1–b9 trained BFCL's prompt format ([f(a=1)]). It scored well on the benchmark, but no inference server parses it.
From b10, every run is native-FC only. b10 also brought the tool-tag fix: in the base model, the <tool_call> / </tool_call> embedding rows are byte-identical untrained vectors with tied embeddings. LoRA doesn't touch embedding rows, so both were re-initialised and trained as extra tokens.
b11–b15: each run fixed one failure class found by row-level error analysis of the previous run.
- b12 reverted a rule that taught dropping optional parameters and removed rows with invented years.
- b14 added targeted slices.
- b15 removed the b14 slice that caused its non-live regression.
b16 added longer context and a "fill discipline" filter. It came out null against b15, so b15 is the release.

The chart shows each run's 4-bit evaluation, which is why b15 reads 85.7% / 65.8% there and 86.2% / 65.7% in bf16 above. Neighbouring runs are often within noise: b15 vs b14 was p < 0.001 on non-live and p = 0.128 on live.

Training recipe


Epochs	1
LR / schedule	1e-4, cosine
Batch	4 × grad-accum 2 (effective 8)
Sequence	max 2048, `packing=False`
Regularization	none (NEFTune off)
Loss masking	`train_on_responses_only`, anchored on the user header so tool results are never trained as model output
Seed	3407
Framework	Unsloth + TRL `SFTTrainer`, 1 × RTX 4060 8 GB

Data: 24,745 rows after the length filter: tool 49%, Python 28%, Bash 23%.

Tool (12,275, all native FC, rendered through the real chat template):
- xLAM calls, weighted toward parallel and multi-call shapes
- Toucan MCP calls from real servers
- 1,000 ToolACE rows
- 1,000 full glaive loops (call → tool result → answer)
- 500 "irrelevant" rows, where the tools are listed but the right answer is plain text
- Slices for array arguments, enum mapping, date phrasing and a generic shell tool
- No trained call contains a year the user didn't state.
Python (7,000): LeetCode problems and stub completions, plus KodCode rows (CC-BY-NC-4.0, see License) whose answers were re-executed against their unit tests.
Bash (5,670): NL2SH-ALFA train, bash-instruct-III one-liners and scripts, and linux-command-dataset.
Leak protection: training prompts that match BFCL, NL2SH-ALFA test, HumanEval or MBPP+ were removed.

Known limitations

About 1 in 3 real-user tool questions is wrong. See the error classes above.
Not a multi-step agent. Training is mostly single-call, with 1,000 two-hop loops. On a Docker agent probe it completes 7/54 tasks, and 8/45 on held-out task families.
Bash output is a single command or a short script, unverified. Sandbox it.
The quants were not re-benchmarked. Expect lower scores than the bf16 numbers, more so the smaller the file.
English only. No safety tuning beyond the base model.

Intended use and limits

Good for: local and offline tool-calling back-ends behind any OpenAI-compatible server, shell one-liner translation, small Python functions, and edge deployment.

Not for: production automation without validation, long autonomous agent loops, or anything safety-critical. Never execute generated shell commands unreviewed.

Citation

@misc{mini_agent_v2,
  title  = {mini-agent-v2: a 1.5B native tool-calling, Bash and Python fine-tune of Qwen2.5-Coder-1.5B-Instruct},
  author = {Frost2o24},
  year   = {2026},
  url    = {https://huggingface.co/F-A-I-L/mini-agent-v2},
  note   = {Merged bf16 weights and GGUF builds; evaluated on BFCL (FC mode), NL2SH-ALFA, InterCode-Bash, MultiPL-E sh and EvalPlus}
}

License

The model weights are released under Apache-2.0, inherited from Qwen2.5-Coder-1.5B-Instruct.

⚠️ Training-data notice: about 5,100 of the 24,745 training rows (the KodCode Python slices: 4,300 general + 800 odd-function-name rows, via F-A-I-L/kodcode-verified-python-235k) come from KodCode-V1, which is licensed CC-BY-NC-4.0 (non-commercial). All other training sources are Apache-2.0, MIT or CC-BY-4.0. Whether a dataset's license carries over to a model trained on it is legally unsettled. If you plan to use this model commercially, review that dataset's terms and decide for yourself; this notice is not legal advice.

Training data licenses are those of the upstream sources listed in the metadata. xLAM (CC-BY-4.0) is credited to Salesforce Research.