animicaorg/Animica-Agent-2B

Animica Agent 2B

A small model fine-tuned to call APIs it has never seen — trained on schema-derived tool-calling data from 17,987 real machine-payable services.

Read the results section before you use this. It is good at one thing and measurably not better than its base at another, and we would rather you know which is which.

What it is

Qwen3.5-2B (Apache-2.0) + a LoRA adapter trained on tool-calling data generated from real API surfaces: schema-validated arguments, hard negatives drawn from each service's own sibling endpoints, and explicit abstention examples for when no tool fits.

Ships as GGUF (Q4_K_M, 1.19 GB) so it runs on a phone, and as a 257 MB adapter that can be applied to the base at runtime (llama-server --lora), which is how it is served across a swarm of devices.

Results

On API surfaces it had never seen — this is what it is for

Held out by host, so no service in the test set appears in training:

metric	base Qwen3.5-2B	this model
tool selection accuracy	0.611	1.000
argument accuracy (extractable-exact)	0.126	0.798
argument F1	0.256	0.837
JSON validity	0.903	1.000
correct abstention	1.000	1.000

Argument accuracy is measured only on values the user's request actually contained. Values that our data generator synthesised are scored on type and presence, never on exact match — grading a model for failing to guess an unstated value teaches it to invent confident ones, which is the failure this model exists to remove.

On a public general benchmark — no improvement

	base	this model
BFCL v4 (overall)	20.37%	20.35%
IFEval	66.15%	66.15%

The fine-tune did not move general tool-calling ability. Both figures exclude the web_search category (no API key) and were measured through the same harness, so the comparison is fair even though the absolute numbers are a floor. The published ≤2 GB leader scores 56.88 on BFCL v4; we are not near it and we are not claiming to be.

Those two tables together are the honest description of this model: specialisation to a known API surface works; transfer to unseen general benchmarks did not. If you want a strong general agent model, use a larger one. If you want a small model that reliably calls a specific set of APIs, this is the recipe — and the same pipeline can be pointed at yours.

What it is bad at

• General agentic reasoning beyond tool selection (see BFCL above).
• Instruction-following is unchanged from the base, which is mediocre (66% IFEval).
• Filling arguments the user never stated — by design; it abstains or leaves them out rather than inventing.
• Long multi-turn tool sequences (our weakest held-out task at 0.28).

Get it

Weights are mirrored on our own infrastructure as well as here:

file	size	sha256
animica-agent-2b-q4_k_m.gguf	1.19 GB	793ba92b1dd5dc2f00d935cd8001bc5ef911857b9282bfa018cd03b08ec44e9c
animica-agent-2b-lora.gguf	257 MB	17b02e6e28f6d37f17487de6f46b3491cfaecbfd6352dcd18c21061a3722deb4

Mirror: https://pool.animica.org/downloads/models/ · Checksums: https://animica.dev/models/SHA256SUMS

Verify the checksum before serving it. An earlier export of this model shipped under this exact filename with no chat template embedded and rambled instead of calling tools; the hash above is the artifact that passes the tool-calling smoke test.

Call it without downloading anything

It is served free on the Animica network, no key, OpenAI-compatible:

curl https://animica.dev/v1/chat/completions -H 'content-type: application/json' -d '{
  "model": "animica-agent-2b",
  "messages": [{"role":"user","content":"What is the weather in Tokyo in celsius?"}],
  "tools": [{"type":"function","function":{"name":"get_weather","description":"Get current weather.",
    "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]
}'

Answers are produced by volunteer devices racing each other, so latency is ~20-40s, not datacenter-fast. Serve it yourself and earn per job: pip install animica-serve && animica-serve --model animica-agent-2b --address anim1…

Usage

# GGUF, llama.cpp
llama-server -m animica-agent-2b-q4_k_m.gguf --jinja

# or the adapter on the base, swappable at runtime
llama-server -m qwen3.5-2b-q4_k_m.gguf --lora animica-agent-2b-lora.gguf

Pass tools through the API's tools field. Hand-rolling a tool list into the system prompt produces correct arguments wrapped in unparseable markup with tool_calls: null — a failure that looks like a bad model and is not.

Training

LoRA (rank 128, all linear layers including MLPs), bf16, lr 1e-4, on a Llama-free corpus so the model carries no naming clause. Data: our own schema-derived rows plus public function-calling sets (APIGen/xLAM lineage, Qwen-generated general chat), deduplicated and decontaminated against BFCL and API-Bank evaluation questions (23 overlapping rows removed).

Full pipeline, corpus builder and evaluation harness: https://gitlab.com/Animica/animica-model

Attribution

Built on Qwen3.5-2B (Apache-2.0). The NOTICE and licence are included; changes are described above.

Want this for your own APIs

The pipeline that produced this is available as a service: submit an OpenAPI spec and get a free report showing coverage, the rows we would generate, real sample rows from your spec, and where it would still be weak — https://animica.dev/factory/