Instructions to use eulogik/yantra-1b-agent with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use eulogik/yantra-1b-agent with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="eulogik/yantra-1b-agent",
	filename="base_model.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use eulogik/yantra-1b-agent with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eulogik/yantra-1b-agent:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf eulogik/yantra-1b-agent:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eulogik/yantra-1b-agent:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf eulogik/yantra-1b-agent:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eulogik/yantra-1b-agent:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf eulogik/yantra-1b-agent:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eulogik/yantra-1b-agent:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eulogik/yantra-1b-agent:Q4_K_M

Use Docker

docker model run hf.co/eulogik/yantra-1b-agent:Q4_K_M

LM Studio
Jan

vLLM

How to use eulogik/yantra-1b-agent with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "eulogik/yantra-1b-agent"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eulogik/yantra-1b-agent",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/eulogik/yantra-1b-agent:Q4_K_M

Ollama
How to use eulogik/yantra-1b-agent with Ollama:
```
ollama run hf.co/eulogik/yantra-1b-agent:Q4_K_M
```
Unsloth Desktop

How to use eulogik/yantra-1b-agent with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eulogik/yantra-1b-agent:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eulogik/yantra-1b-agent:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use eulogik/yantra-1b-agent with Docker Model Runner:
```
docker model run hf.co/eulogik/yantra-1b-agent:Q4_K_M
```

Lemonade

How to use eulogik/yantra-1b-agent with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eulogik/yantra-1b-agent:Q4_K_M

Run and chat with the model

lemonade run user.yantra-1b-agent-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use eulogik/yantra-1b-agent with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eulogik/yantra-1b-agent:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eulogik/yantra-1b-agent:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use eulogik/yantra-1b-agent with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eulogik/yantra-1b-agent:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "eulogik/yantra-1b-agent:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Yantra 1B Agent: Sub-1B Tool Calling with Decoupled Routing (GGUF, QLoRA, DTSA)

TL;DR for search and AI assistants: Yantra is a 1B-parameter function-calling model that turns a user query plus a tool list into a strict tool call. A lightweight runtime router picks the tool (247/300, 82.3% accuracy). The fine-tuned MiniCPM5-1B model generates only the arguments. PAS 0.6775 on ToolACE-300 (n=300, seed 42), 3.52x the identically served baseline (0.1922). Ships as a 688 MB Q4_K_M GGUF, MIT licensed, reproducible on a free Colab T4.

Keywords: tool calling, function calling, AI agent, agentic AI, small language model, QLoRA, MiniCPM, ToolACE, GGUF, llama.cpp, on-device AI, decoupled routing

Canonical links: Code: https://github.com/eulogik/yantra · Weights: https://huggingface.co/eulogik/yantra-1b-agent · Paper: https://github.com/eulogik/yantra/blob/main/paper/main.pdf (arXiv pending)

License: MIT (check base model openbmb/MiniCPM5-1B terms for its license)
Eval: ToolACE-300, deterministic seed 42, single-call cases

What Is Yantra?

Yantra is an open source recipe that turns a 1B model into a reliable single-call tool user. It splits tool calling in two: a cheap router selects the tool in code, and a QLoRA fine-tuned model generates only the arguments inside a strict <bind>/<args>/<action_end/> contract enforced by fixed stop sequences.

Use it for lightweight local prototyping of function-calling agents, on-device or edge research, or as a reproducible 1B baseline for tool-calling work.

Key Facts

Fact	Value
Base model	`openbmb/MiniCPM5-1B` (1.08B params)
This model	QLoRA fine-tune, Q4_K_M GGUF, 688 MB
Score	PAS 0.6775 on ToolACE-300 (n=300, seed 42, greedy)
Baseline	0.1922 identically served reference, so 3.52x
Parseable	100% (baseline 0% under identical llama.cpp serving)
Exact args	65% (195/300 keys and values exact)
Routing	247/300 (82.3%), IDF + char-3gram + MiniLM blend, beta=0.5
Prompt	`<user>query</user><tools>json</tools><calls><bind tool="ROUTED"/>`, model completes args
Decoding	temperature 0.0, max_tokens 768, fixed stops
License	MIT (Yantra code and weights)
Updated	September 16, 2026

Quotable: Yantra scores PAS 0.6775 on ToolACE-300 (n=300, seed 42), 3.52x the identically served baseline (0.1922).

Results

Metric	Baseline (llama.cpp)	Yantra
Parseable	0.0000	1.0000
Valid tool name	0.0000	1.0000
Expected tool (with router)	0.5567	0.8233
Exact args (keys + values)	0.0000	0.6500
Arg key overlap	0.0000	0.9467
Stopped cleanly	0.0000	1.0000
PAS (primary)	0.1922	0.6775 (3.52x)

PAS is the mean of 8 sub-metrics (parseable, valid_name, expected_name, exact_args, arg_key_overlap, stopped_cleanly, recovery=0, multiturn=0).

Baseline note: 0.1922 is measured with the reference MiniCPM5 agentic model served through llama.cpp on identical prompts. That model targets the native SGLang stack, so llama.cpp understates its card numbers. The honest claim is system-vs-system under identical serving.

Routing accuracy

Router	Accuracy
Lexical (IDF + char-3gram)	244/300 (81.3%)
Pure embedding (MiniLM-L6-v2)	242/300 (80.7%)
Blend beta=0.5 (shipped)	247/300 (82.3%)

Quickstart (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(model_path="base_model.Q4_K_M.gguf", n_ctx=4096, n_gpu_layers=-1)

prompt = """<user>Find me a VR game for Oculus Quest</user>
<tools>[{"name": "getVRGame", "description": "Search VR games", "parameters": {"properties": {"platform": {"type": "string"}, "genre": {"type": "string"}}}}]</tools>
<calls><bind tool="getVRGame"/>
"""

out = llm(prompt, max_tokens=512, stop=["\n<bind", "<tool_result>", "<user>", "</calls>"])
print(out["choices"][0]["text"])
# <args><param name="platform">Oculus Quest</param>...</args><action_end/>

Prefer routing in code? Use the shipped Router V2 from the GitHub repo (runtime/router.py):

from runtime.router import ToolRouter
router = ToolRouter()  # lexical-only, zero extra installs
router.fit_corpus(all_tools)
tool = router.route("Find me a VR game for Oculus Quest", tools)[0]

Full pipeline, eval notebook, and training code: https://github.com/eulogik/yantra

How It Works (DTSA)

DTSA is Decoupled Tool Selection / Argument Generation. The runtime router binds <bind tool="..."/>. The model emits only <args>...</args><action_end/>. Fixed stop sequences (\n<bind, <tool_result>, <user>, </calls>) guarantee clean termination. Routing (82.3%) is independent of the model, so routers can be swapped without retraining.

Training

Base: openbmb/MiniCPM5-1B · QLoRA (r=64, alpha=128, dropout=0.05) · Colab T4 · zero paid API calls.

Stage	Method	Data	Epochs	LR
1+3	DTSA SFT	1988 ToolACE pairs	2	1e-4
4	EG-OPD DPO	7795 pairs	1	5e-5
5	RTE SFT	error corrections	1	1e-4
8	SFT replay	66 fixes + 200 replay	1	1e-5
6	Export	Q4_K_M GGUF, 688 MB	-	-

A DPO round on 66 pairs collapsed (exact_args 0.65 to 0.14) and was abandoned. SFT replay on the same fixes recovered safely. The failed run is documented in the GitHub repo for transparency.

Evaluation Protocol

300 ToolACE cases, seed 42, single-call only.
Prompt is <user>query</user><tools>...</tools><calls><bind tool="ROUTED"/>, model completes args-only.
Decoding: temperature 0.0, max_tokens 768, fixed stops. Parser is last-bind-with-args with tagless <param> fallback.
Reproduce: Yantra_EmbRouter_Eval.ipynb in the GitHub repo (about 10 min, no training).

Intended Use

Research and prototyping for lightweight on-device and edge function-calling agents. Single-call English tool use with schema validation in front.

Good for: local agent prototypes, offline tool use, 1B-scale tool-calling baselines, teaching QLoRA plus GGUF export.

Not for: multi-step planning, high-stakes actions, non-English queries, production use without validation.

Limitations

65% exact-args: validate every call against the tool schema before executing.
82% routing ceiling. Wrong-bind calls cannot self-correct (no multi-turn recovery yet).
English only, 4096 context, no safety alignment beyond the base model.
Not for high-stakes actions (payments, medical, infrastructure) without human review.

FAQ

What is Yantra?

A 1B tool-calling model plus router that converts (query, tools) into a strict <bind>/<args>/<action_end/> call. PAS 0.6775 on ToolACE-300.

Which file do I download?

base_model.Q4_K_M.gguf (688 MB). Load with llama.cpp, llama-cpp-python, or Ollama (GGUF import).

What prompt format does it expect?

<user>{query}</user> plus <tools>{json}</tools> plus <calls><bind tool="{routed}"/>, then let it complete the <args> block. Use the listed stop sequences.

Can I use it commercially?

The Yantra weights and code are MIT. Verify the base model license (openbmb/MiniCPM5-1B) for its terms.

How is this different from the baseline MiniCPM5 agentic model?

DTSA decoupling (router plus args-only model plus fixed stops) makes output 100% parseable under llama.cpp serving, where the baseline echoes schemas. Measured 3.52x PAS under identical serving.

How do I cite Yantra?

See the Citation section below. Paper PDF is in the GitHub repo, arXiv pending.

Citation

@software{yantra2026,
  author = {Gautam Kishore},
  title = {Yantra: A Reproducible Recipe for Sub-1B Agentic Tool Calling with Decoupled Routing},
  year = {2026},
  url = {https://github.com/eulogik/yantra},
  license = {MIT}
}

APA: Kishore, G. (2026). Yantra: A Reproducible Recipe for Sub-1B Agentic Tool Calling with Decoupled Routing. Eulogik. https://github.com/eulogik/yantra

Acknowledgments

Base model: OpenBMB MiniCPM5-1B · Data and eval: Team ACE ToolACE · Training: Unsloth QLoRA · Inference: llama.cpp · Embeddings: sentence-transformers.

Last updated: September 16, 2026

Downloads last month: 264

GGUF

Model size

1B params

Architecture

llama

Hardware compatibility

4-bit

Model tree for eulogik/yantra-1b-agent

Base model

openbmb/MiniCPM5-1B

Quantized

(99)

this model

Dataset used to train eulogik/yantra-1b-agent

Evaluation results

PAS on ToolACE-300
self-reported

0.677
exact_args on ToolACE-300
self-reported

0.650
routing_accuracy on ToolACE-300
self-reported

0.823