🤖 botte-nano-nn — Micro Neural Networks for Agent Pipelines

Replace trivial LLM calls with tiny (<3 KB) neural networks. Deterministic, 5 µs inference, 0 tokens, 0 API calls.

These models are designed for skill/tool pipelines and deterministic systems that need just a little bit of intelligence — not a full LLM call.

Why micro NNs in agent pipelines?

Modern AI agents call LLMs for everything — even decisions that are trivially classifiable. A pipeline step like "should I retry this operation?" or "is this task simple enough for a local model?" does not need GPT-5. It needs a lightweight classifier that runs locally, deterministically, at 5 µs.

Auto-recovery & degraded mode

Scenario	What the micro-NN decides	Without it	With it
API call failed	Should we retry or fall back?	LLM call (~500 tokens)	5 µs, 0 tokens
Resource low	Which features to disable?	LLM call (~800 tokens)	5 µs, 0 tokens
Queue full	Process now or delay?	LLM call (~400 tokens)	5 µs, 0 tokens
Error spike	Alert or silence?	LLM call (~600 tokens)	5 µs, 0 tokens

Tool & skill routing

Scenario	What the micro-NN decides	Without it	With it
New task	Run locally or send to cloud?	LLM call (~300 tokens)	5 µs, 0 tokens
Code file	Audit or skip?	LLM call (~400 tokens)	5 µs, 0 tokens
Log line	Anomaly or normal?	LLM call (~500 tokens)	5 µs, 0 tokens

The pattern: "just enough intelligence"

Input → [micro-NN] → decision
              ↓
        (if uncertain) → [small LLM] → decision
                               ↓
                    (if complex) → [full LLM] → decision

This tiered escalation saves 60-90% of tokens by keeping trivial decisions at the micro-NN level. Only the hard cases reach the LLM.

Models

Model	Inputs	Outputs	Size	What it decides
effort_classifier	4	3	3.1 KB	Easy/medium/hard task effort → route to local/cloud
binary_router	3	2	1.7 KB	Local vs cloud: should this use a full LLM?
anomaly_detector	5	2	2.6 KB	Log is normal or anomalous → alert or ignore

Features

• 0 tokens, 0 API calls — pure CPU math, ~5 µs inference
• Rust inference — feedforward, f64, no external ML deps
• Python training — pure numpy, backprop from scratch, ~50 lines
• Deterministic — same input = same output, every time
• Portable — 50 KB Rust binary, no GPU, no cloud

Quick start

Python inference (no build needed)

pip install numpy

# Classify task effort (4 features)
python cli.py predict models/effort_classifier.json \
    --input 0.1 0.2 0.8 0.0
# → Prediction: easy (local) (confidence: 0.9797)

# Route to local or cloud (3 features)
python cli.py predict models/binary_router.json \
    --input 0.7 0.2 0.5 --probabilities
# → 0: local    [░░░░░░░░░░░░░░░░░░░░] 0.1234
# → 1: cloud    [████████████████████] 0.8766

# Auto-detect anomaly (5 features)
python cli.py predict models/anomaly_detector.json \
    --input 0.1 0.3 0.9 0.2 0.5
# → Prediction: normal (confidence: 0.9234)

# Or auto-select the right model by input size:
python cli.py which --input 0.1 0.3 0.9 0.2 0.5
# → 🧠 normal (confidence: 92.34%)

Rust inference (faster, 50 KB binary)

cargo build --release
./target/release/botte_nn_cli predict models/effort_classifier.json \
    0.1 0.2 0.8 0.0
# → [0.9797, 0.0123, 0.0080]

As an MCP tool (for any AI agent)

These models are exposed as MCP tools through Botte Secrète's MCP Gateway. Any MCP-compatible agent (Claude Code, Codex, Cursor) can call them directly:

{
  "mcpServers": {
    "botte-gateway": {
      "command": "python",
      "args": ["-m", "skills.mcp_gateway.server"],
      "cwd": "/path/to/botte-secrete"
    }
  }
}

Then: Call tool: botte_nn (model: "effort_classifier", input: [0.1, 0.2, 0.8, 0.0])

Integration in agent pipelines

┌──────────────────────────────────────────────────┐
│              Agent Pipeline                       │
│                                                    │
│  Input → [micro-NN classifies effort]             │
│              │                                     │
│              ├─ easy ───→ local model (0 tokens)   │
│              ├─ medium ─→ hybrid (small LLM)       │
│              └─ hard ───→ cloud LLM                │
│                                                    │
│  Error → [micro-NN detects anomaly]               │
│              │                                     │
│              ├─ normal ──→ log + continue          │
│              └─ anomaly ─→ alert + escalate        │
└──────────────────────────────────────────────────┘

Training your own for pipeline decisions

python training/train.py \
    --layers 4 16 3 \
    --activations relu softmax \
    --epochs 200 \
    --samples 1000 \
    --save my_router.json

Train on your own data: logs, routing decisions, error patterns. The network learns in seconds, then runs at 5 µs indefinitely — 0 recurring cost.

Architecture

botte_nn/
├── src/
│   ├── lib.rs          — Public API
│   ├── matrix.rs       — Dot product, mat-vec mul
│   ├── activation.rs   — ReLU, Sigmoid, Softmax, Linear
│   ├── layer.rs        — Layer: weights + bias + activation
│   ├── model.rs        — Model: load JSON → predict
│   └── inference.rs    — predict_json helper
├── training/
│   └── train.py        — Training loop (numpy, backprop)
├── cli.py              — CLI: predict, list, which
└── Cargo.toml          — 2 deps (serde, serde_json)

License

MIT — part of Botte Secrète, a multi-agent token optimization platform.