Artoru # EDGE 65M

First preview. The local agentic brand starts here.

Not another cloud chat clone. Artoru is the local-first, multi-agent line you run on your own machine — and EDGE 65M is the opening chapter: the tiny routing engine inside Artoru Desk, shipped as a public first preview with LoRAs and more already on the road.

The arena already has its myths. Artoru is our line in that sky — built to live on your hardware.


Brand	Artoru
Model	Artoru EDGE 65M (first EDGE preview)
Product	Artoru Desk — local-first multi-agent workspace
Routing win	~98.66% product-scored · ~99.16% handled (EDGE + Chief)
Speed	ORT CUDA FP32 ~134,462 tok/s (batch 32, RTX 5090)
Params	~65.39M

Why this card exists

Most model cards apologize for being small. This one doesn’t.

EDGE 65M is not trying to write your novel, debate philosophy, or impersonate a search engine. It has one job and it does that job like a blade: read the turn, name the intent, route the desk — on-device, at reflex speed, without asking the cloud for permission to think.

If Desk is the stage, EDGE is the stage manager with a stopwatch.

The win

Cloud models talk. EDGE routes.

EDGE + Chief on the product eval (n=372):

~98.66% product-scored accuracy
~99.16% handled-rate

That’s the number. Own it.

Throughput: ORT CUDA FP32 batch 32 ≈ 134k tok/s (~0.11 ms/sample, ~8.9k samples/s) on short routing utterances. Batched PyTorch FP16 ≈ 84k tok/s.

Tiny model. Massive local routing power.

No asterisk that says “but only if the API is up.” No waiting room. No meter ticking while you decide whether to draft an email or ask the Chief.

The story: from Micro to EDGE

Before Artoru had a name people could shout, there was a quieter public router — AMS Micro on vvarghese/ams-micro-70m. Honest work. On-device. Already thinking in intents.

Artoru EDGE 65M is that lineage stepping into the light with a brand, a desk, and a scoreboard:

	Prior public Micro	Artoru EDGE 65M
Identity	Earlier published on-device router (`vvarghese/ams-micro-70m`)	Today’s Artoru preview — Desk’s local routing engine
Product routing	Earlier public baseline	~98.66% / ~99.16% handled with EDGE + Chief
Speed	Prior CPU ORT full-test ~9.2 s (older publish)	~134k tok/s ORT CUDA FP32 b32
This pack	—	FP32 + int8 + Q4

Same obsession — understand first, spend heavy compute second. New costume. Sharper teeth. Product attached.

Why local still wins

Because your desk is not a demo.

Because “upload everything to someone else’s GPU” is a lifestyle choice, not a law of physics.

Because routing is a reflex, and reflexes that cross the ocean feel like lag even when the chat model is brilliant.

Local means:

Privacy by architecture — intent classification happens where the keystrokes already are
Latency you can feel — ~134k tok/s class throughput on the FP32 ORT path (batch 32, RTX 5090)
A desk that still works when the wifi sulks
Agents that ask permission instead of shipping your draft into the void

Artoru is not anti-cloud. Artoru is pro-sovereignty. Cloud models are invited guests. EDGE is the bouncer who knows your name.

Artoru Desk — the product frame

Artoru Desk is the product: a local-first multi-agent workspace — agents, tools, approvals, panic, memory, and a full surface that lives on your hardware.

EDGE 65M is the fast intent/routing model in that stack.
Chief is the handoff layer that rides with EDGE on hard turns — one piece of Desk, not the whole story.

[ You type ] → [ EDGE 65M names the intent ] → [ Desk routes ]
                    └─ low confidence / hard intent → [ Chief ]

Desk > any single model.

EDGE is how Desk decides what to do next without round-tripping the cloud for every reflex. Chief is how Desk stays honest when the turn is sharper than a tiny router should pretend.

Chat is never empty theater. Someone is always addressed — usually the Chief — and EDGE is the quiet nerve that decides whether the turn is a greeting, a task assign, an escalate, a skill suggest, or “ask the human.”

How EDGE + Chief feels

Imagine opening Desk.

You say something ordinary. EDGE fires. The room already knows whether this is smalltalk, a meeting, a fetch, or a handoff.

You say something that matters — escalate, assign, a Chief question. Confidence dips or the intent lands in the handoff set. Chief steps in. Visible. Ownable. Not a silent model swap behind a spinner.

That duet is the product eval story:

Metric	Value
Product-scored accuracy	~98.66%
Handled-rate (EDGE + Chief)	~99.16%
Eval size	n=372

EDGE alone is the sprinter.
EDGE + Chief is the relay team that finishes the race.

If routing were only accuracy cosplay, we’d ship a bigger model and call it a day. This pack is about feel: instant local triage, then a named handoff when the desk needs a brain with more room.

What’s in the pack (and why the drama)

Four files. Four moods. Same EDGE soul.

File	Precision	Use when
`artoru-edge-65m.safetensors`	FP32	Flagship PyTorch load + vocab metadata
`artoru-edge-65m.onnx`	FP32	ORT serve / GPU peak throughput (self-contained)
`artoru-edge-65m-int8.onnx`	INT8	Smaller CPU-oriented ORT (~63 MB)
`artoru-edge-65m-q4.safetensors`	INT4	Smallest download (~33 MB); dequant then load

FP32 is the headline — quality plus the ~134k tok/s ORT CUDA story.
Int8 is the everyday carry for lean CPU desks.
Q4 is the suitcase that still somehow holds the whole outfit: a custom RTN int4 pack — dequant before load_state_dict. After dequant, compute is FP32-class.

Download the flagship if you want the poster on the wall.
Download int8 if your fans already hate you.
Download Q4 if you believe small can still be serious.

Architecture

~65.39M params · vocab 9225 · up to 2048 positions · 768-d · 8 heads · 8 layers · intents + skills + slots + risk.

Built for agentic routing inside Desk: register, assign/resume, browse/search/fetch, skills, memory/meeting, calculate/time/weather.

Not a toy transformer cosplaying as a chat model. A routing encoder with a job description carved into the heads.

Load — FP32 Safetensors

import json
from pathlib import Path
import torch
from safetensors import safe_open
from safetensors.torch import load_file

from scripts.train_canonical import build_canonical_encoder
from ams.tokenize import tokenize_legacy

path = Path("Model/artoru-edge-65m-hf/artoru-edge-65m.safetensors")
tensors = load_file(str(path))
with safe_open(str(path), framework="pt") as f:
    meta = f.metadata() or {}

vocab = json.loads(meta["vocab_json"])
canonical = json.loads(meta["canonical_json"])
skills = json.loads(meta["skills_json"])

model = build_canonical_encoder(len(vocab), len(canonical), len(skills), dropout=0.0)
model.load_state_dict(tensors, strict=False)
model.eval()

def encode(text: str, pack: int = 512):
    content = tokenize_legacy(text)
    room = pack - 1
    if len(content) > room:
        content = content[-room:]
    ids = [vocab["<cls>"]] + [vocab.get(tok, 1) for tok in content]
    return torch.tensor([ids], dtype=torch.long)

Load — FP32 ONNX

import onnxruntime as ort

sess = ort.InferenceSession(
    "Model/artoru-edge-65m-hf/artoru-edge-65m.onnx",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)

Load — INT8 ONNX

import onnxruntime as ort

sess = ort.InferenceSession(
    "Model/artoru-edge-65m-hf/artoru-edge-65m-int8.onnx",
    providers=["CPUExecutionProvider"],
)

Pad/truncate to the graph width when needed (bench used seq≈42).

Load — Q4

Unpack *.q4 + *.scale from safetensors metadata (layouts_json), rebuild FP32 tensors, then load_state_dict into the EDGE encoder. After dequant, compute is FP32-class.

Roadmap swagger

This is the 1st public preview of Artoru EDGE on the path to Desk.

Coming next — proudly:

LoRAs that specialize the router without replacing the soul
More EDGE drops as the desk surface deepens
Richer agent lanes, sharper handoffs, louder product feel
The boring-but-holy work: eval harnesses, calibration, thresholds you can actually trust

The brand ladder is just starting. Watch Artoru climb.

We are not shipping “a 65M curiosity.” We are shipping the first named nerve of a local multi-agent brand — and inviting you to load it, break it, score it, and tell us where it flinches.

Naming

Name	What it is
Artoru	The brand line — local / agentic
Artoru EDGE 65M	This model (first EDGE preview weights)
Chief	Handoff layer with EDGE
Artoru Desk	Local-first multi-agent product

Intended use (short and honest)

Yes: on-device intent / routing for agentic desks; triage before expensive generation; local-first pipelines that want a fast first decision.

Use EDGE to decide. Use your generative stack to speak. Use Chief when the turn deserves a longer brain.

Artoru EDGE 65M — first preview. The ladder begins.

Downloads last month: -; Downloads are not tracked for this model. How to track