TinyToolRouter-0.6B

A lightweight, CPU-first tool router fine-tuned from Qwen3-0.6B.

GitHub: @roboroyce

TinyToolRouter-0.6B is a 0.6-billion-parameter language model designed to select tools from a supplied list of tool schemas. Given a natural-language request and the available tools, it produces exactly one structured JSON tool call or explicitly abstains when no tool applies.

Example

Input request: What's the weather in Oslo?

Output:

{
  "name": "get_weather",
  "arguments": {
    "city": "Oslo"
  }
}

When no listed tool applies, the model abstains:

{
  "name": "no_tool",
  "arguments": {}
}

Key features

• Small footprint: 0.6B parameters, suitable for CPU-first inference.
• Schema-aware routing: Selects from tool definitions supplied at inference time.
• Structured output: Produces a single JSON object representing a proposed tool call.
• Explicit abstention: Supports a dedicated no_tool response.
• Paraphrase robustness: Evaluated on requests phrased differently from the training examples.
• Unseen-schema evaluation: Tested on tools absent from the training set.
• Multiple formats: Safetensors and GGUF builds are available in this repository.

Model details

Property	Value
Model	TinyToolRouter-0.6B
Base model	Qwen/Qwen3-0.6B
Parameters	0.6B
Fine-tuning	Supervised fine-tuning with LoRA (PEFT)
Deployment	Merged fine-tuned weights
Primary language	English
License	Apache-2.0
Intended hardware	CPU-only and low-VRAM systems
Safetensors size	Approximately 1.2 GB
GGUF Q4_K_M size	Approximately 397 MB
GGUF Q8_0 size	Approximately 639 MB
GGUF F16 size	Approximately 1.2 GB

File sizes are approximate and may vary slightly by build.

Benchmark results

TinyToolRouter was evaluated on a held-out set of 231 cases covering three distinct evaluation families.

Overall performance

Metric	Qwen3-0.6B (base)	TinyToolRouter-0.6B
Exact match	51.95%	92.21%
Valid structured output	99.13%	99.13%
Tool selection	93.51%	98.70%
Argument accuracy	51.95%	92.21%
no_tool precision	76.47%	97.56%
no_tool recall	97.50%	100.00%
no_tool F1	85.71%	98.77%

Exact match improved by 40.26 percentage points over the base model.

Performance by evaluation family

Evaluation family	Cases	Base model	Fine-tuned
seen_paraphrase	131	22.90%	92.37%
unseen_schema	60	85.00%	86.67%
no_tool	40	97.50%	100.00%

What these results mean

• Paraphrase robustness: Exact match increased from 22.90% to 92.37% on requests that use unseen phrasing for tools encountered during training.
• Unseen tool schemas: Exact match increased from 85.00% to 86.67% on tools absent from the training set. This is a modest improvement, not evidence of fully general tool-schema understanding.
• Abstention: The fine-tuned model correctly abstained on all 40 held-out no_tool cases, with 97.56% measured precision.

Evaluation methodology

Both models were evaluated using identical prompts and generation settings:

• do_sample=False
• max_new_tokens=128
• Held-out evaluation set: 231 cases
• Evaluation metric: exact match of the expected structured output

The benchmark separates familiar tool schemas with paraphrased requests, genuinely unseen tool schemas, and requests that should not trigger a tool call.

The reported metrics are from the safetensors model. The GGUF conversions have not been independently benchmarked, so equivalent performance across formats should not be assumed.

Quickstart

Install dependencies

pip install torch transformers

Run with Transformers

import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Robokid1134/TinyToolRouter-0.6B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
model.eval()

tools = [
    {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"}
            },
            "required": ["city"],
        },
    }
]

system_prompt = (
    "You are a tool router. You are given a list of available tools "
    "and a user request. Reply with exactly one JSON object of the form "
    '{"name": "<tool_name>", "arguments": {"<argument>": "<value>"}}. '
    "Use only the listed tools and their declared arguments. "
    'If no listed tool applies, reply {"name": "no_tool", "arguments": {}}. '
    "Output JSON only, with no additional prose."
)

messages = [
    {"role": "system", "content": system_prompt},
    {
        "role": "user",
        "content": json.dumps({
            "tools": tools,
            "request": "What's the weather in Oslo?"
        }),
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

inputs = tokenizer(prompt, return_tensors="pt")

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
    )

new_tokens = output[0][inputs["input_ids"].shape[1]:]
response = tokenizer.decode(
    new_tokens,
    skip_special_tokens=True,
)

print(response)

Expected output:

{"name": "get_weather", "arguments": {"city": "Oslo"}}

Note: enable_thinking=False is recommended for direct routing output. The example uses the Qwen3 chat template and assumes a Transformers version that supports this argument.

GGUF: CPU inference

GGUF builds are available in the repository's gguf/ directory.

File	Approximate size	Intended use
gguf/model-Q4_K_M.gguf	397 MB	Compact CPU inference
gguf/model-Q8_0.gguf	639 MB	Higher quantization fidelity
gguf/model-f16.gguf	1.2 GB	Full-precision weight storage

Run a GGUF model using llama.cpp:

llama-cli \
  -m gguf/model-Q4_K_M.gguf \
  -p "<your prompt>"

The prompt should follow the model's expected chat template and include the available tool schemas and user request.

For repeated use, a dedicated prompt template or a compatible Ollama model configuration can simplify integration.

Benchmark caveat: The GGUF files were converted from the merged fine-tuned model but have not been independently re-evaluated. The benchmark figures above apply to the safetensors model.

Tool schema format

The model expects a JSON object containing a list of available tools and a natural-language request.

{
  "tools": [
    {
      "name": "get_weather",
      "description": "Get the current weather for a city.",
      "parameters": {
        "type": "object",
        "properties": {
          "city": {
            "type": "string"
          }
        },
        "required": ["city"]
      }
    }
  ],
  "request": "What's the weather in Oslo?"
}

The schema format is JSON-Schema-style, with a name, description, and parameters object for each tool.

Training covered the following argument types:

• string
• integer
• number
• boolean
• array of strings
• array of numbers

Examples included required and optional arguments, along with distractor tools that should not be selected.

The model returns a proposed tool name and its arguments. It does not execute the tool or verify that the underlying action succeeds.

Training

Setting	Value
Base checkpoint	Qwen/Qwen3-0.6B
Base revision	c1899de289a04d12100db370d81485cdf75e47ca
Fine-tuning method	LoRA using PEFT
Adapter rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	Attention and MLP projections
Epochs	3
Training steps	57
Learning rate	1e-4
Scheduler	Cosine
Warmup ratio	3%
Batch size	8
Gradient accumulation	4
Maximum sequence length	512 tokens
Training precision	BF16
Random seed	7
Training examples	599
Held-out examples	231
Hardware	NVIDIA A10G, 24 GB VRAM
Approximate training time	150 seconds
PyTorch	2.5.1
Transformers	4.57.6
PEFT	0.17.1
Python	3.9

The LoRA adapter was merged into the base model for deployment.

Training loss was computed on the assistant's JSON output, with prompt tokens masked from the loss.

Dataset

The dataset consists of original synthetic examples authored for this project and released under Apache-2.0.

No scraped text, chat logs, or personal data were used. Slot values are synthetic, including example email addresses, dates, and city names.

Tool coverage

Tools represented in training (9):

• get_weather
• send_email
• create_calendar_event
• search_web
• convert_currency
• set_timer
• translate_text
• filter_items
• tag_note

Unseen tools reserved for evaluation (5):

• play_music
• get_stock_price
• book_restaurant
• draw_chart
• create_reminder

Data separation

The training and held-out sets use disjoint templates, value pools, and example IDs.

The unseen_schema evaluation family contains tools absent from training entirely. The training set also includes abstention examples, while the held-out no_tool requests are disjoint from those training examples.

These design choices help test whether the model learned useful routing patterns rather than simply memorizing identical examples. They do not eliminate every possible source of dataset bias or leakage.

Limitations

• English only: The model was trained and evaluated on English-language examples.
• Limited tool coverage: Training covers nine tool schemas, with five additional schemas used for evaluation.
• Synthetic data: Performance on real-world agent requests may differ from the reported benchmark.
• Modest unseen-schema gains: The improvement on unseen schemas is only 1.67 percentage points.
• Limited argument extraction: Arguments often appear explicitly in the request. The evaluation primarily tests tool selection, argument correctness, and structured output rather than unrestricted information extraction.
• Small evaluation set: Results are based on 231 held-out examples. Small differences may reflect only a few cases.
• Single random seed: Multi-seed variance has not been measured.
• Quantized models not re-benchmarked: The reported metrics do not establish equivalent performance for the GGUF builds.
• Context length: Training used a maximum sequence length of 512 tokens. Long tool lists may exceed the model's familiar input distribution or be truncated by an inference pipeline.

Intended use

TinyToolRouter-0.6B is intended for:

• Local AI agents running on CPU-only or low-VRAM machines.
• Lightweight tool-selection front ends for larger language models.
• Routing between a predefined set of application tools.
• Explicitly declining requests that do not match any available tool.
• Research into compact language models, tool calling, and structured generation.
• Reproducible experiments comparing a small base model against a narrowly fine-tuned router.

Out of scope

This model is not a general-purpose assistant. It is not intended to independently answer general questions, execute tools, or make safety-critical decisions.

Do not rely on its output alone to authorize sensitive actions or execute shell commands. The calling application should validate the selected tool, enforce an allowlist, validate arguments against the actual schema, and require appropriate authorization before executing an action.

Citation

If you use this model in research or another project, you can cite it as follows:

@misc{tinytoolrouter06b,
  title  = {TinyToolRouter-0.6B: A CPU-First, Schema-Strict Tool Router},
  author = {Robokid1134},
  year   = {2026},
  note   = {LoRA fine-tune of Qwen/Qwen3-0.6B},
  url    = {https://huggingface.co/Robokid1134/TinyToolRouter-0.6B}
}

Attribution and license

TinyToolRouter-0.6B is fine-tuned from Qwen/Qwen3-0.6B, which is released under the Apache-2.0 license.

This derivative model is distributed under Apache-2.0, with the applicable upstream license and copyright notices preserved. The project's original code and generated dataset are also released under Apache-2.0.