Instructions to use NamanAgnih0tri/AlphaRoute-0.8B-v1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NamanAgnih0tri/AlphaRoute-0.8B-v1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NamanAgnih0tri/AlphaRoute-0.8B-v1.0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NamanAgnih0tri/AlphaRoute-0.8B-v1.0")
model = AutoModelForCausalLM.from_pretrained("NamanAgnih0tri/AlphaRoute-0.8B-v1.0", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NamanAgnih0tri/AlphaRoute-0.8B-v1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NamanAgnih0tri/AlphaRoute-0.8B-v1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NamanAgnih0tri/AlphaRoute-0.8B-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NamanAgnih0tri/AlphaRoute-0.8B-v1.0

SGLang

How to use NamanAgnih0tri/AlphaRoute-0.8B-v1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NamanAgnih0tri/AlphaRoute-0.8B-v1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NamanAgnih0tri/AlphaRoute-0.8B-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NamanAgnih0tri/AlphaRoute-0.8B-v1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NamanAgnih0tri/AlphaRoute-0.8B-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use NamanAgnih0tri/AlphaRoute-0.8B-v1.0 with Docker Model Runner:
```
docker model run hf.co/NamanAgnih0tri/AlphaRoute-0.8B-v1.0
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

AlphaRoute-0.8B-v1.0 🔥

AlphaRoute-0.8B-v1.0 is a compact language model specialized for instruction-conditioned semantic intent routing and structured information extraction. Given a user query and a dynamically defined set of categories with semantic descriptions, it predicts the most appropriate intent, extracts parameter slots, and produces a constrained structured JSON decision.

Unlike traditional classifiers bound to fixed output heads, AlphaRoute operates as a zero-shot meta-router: you can define arbitrary runtime categories, out-of-scope policies, parameter extraction slots, and target nested JSON schemas on the fly without retraining or fine-tuning the model.

This repository provides the standalone unquantized FP16 weights (Hugging Face Safetensors) suitable for PyTorch, Transformers, vLLM, SGLang, and cloud GPU inference pipelines.

🏆 Benchmark Highlights (Model Family Reference)

Evaluated across standardized held-out public benchmark test splits:

Benchmark Dataset	Domain Scope	8-bit Accuracy	4-bit Accuracy	JSON Valid %
Banking77 (`official test`)	77 Fine-Grained Banking Intents	93.00%	89.60%	100.0%
CLINC150 (`official test + OOS`)	150 Intents + Out-of-Scope	95.00%	94.20%	100.0%
HWU64 (`official test, 1,076 queries`)	64 Voice Assistant Intents	85.04%	80.20%	100.0%

Note on Benchmarks: Benchmarks were measured using the model family's reference engine. Empirical accuracy across the corresponding standalone and quantized model formats is virtually identical as per our usage and evaluations.

🧠 Advanced Capabilities Beyond Simple Routing

While traditional classifiers only output a flat integer class ID, AlphaRoute-0.8B is trained as a structured semantic engine capable of rich schema synthesis and contextual extraction:

1. 🏗️ Arbitrary Deeply Nested JSON Schemas

AlphaRoute natively follows complex multi-level JSON schemas, including nested metadata blocks, arrays of action items, and multi-tier operational telemetry:

{
  "routing": {
    "target_category": "iam_privilege_escalation",
    "severity": "CRITICAL",
    "out_of_scope": false
  },
  "extracted_entities": {
    "compromised_identity": "admin-service-account",
    "affected_resource": "arn:aws:iam::123456789:role/ProdAdmin",
    "action_taken": "policy_attachment"
  },
  "recommended_actions": [
    "revoke_active_session",
    "quarantine_role"
  ]
}

2. 🔍 Dynamic Slot Extraction & Parameter Parsing

Extracts contextual parameters (amounts, dates, currencies, affected servers, user accounts, error codes) directly into arbitrary developer-defined keys in a single forward pass without needing a secondary NER model.

3. 🛡️ Out-of-Scope (OOS) Rejection Guardrail

When user input does not match any candidate categories in the active ontology, AlphaRoute reliably flags "out_of_scope": true and sets "intent": null, preventing hallucinated routing.

4. 🧩 Zero-Shot Runtime Schema Adaptability

You can change the schema layout, field names, or category definitions on every single API request. The model condition itself dynamically to mirror the requested developer schema.

💡 Best Practices & Prompt Engineering Tips

To achieve optimal routing accuracy and reliable JSON generation from AlphaRoute-0.8B, follow these established best practices:

1. 🏷️ Always Provide Actionable Category Descriptions

Rather than providing bare category names, supply 1–2 sentences defining the semantic boundaries and trigger conditions:

Avoid: - billing
Recommended: - billing: Inquiries regarding invoices, credit card charges, overbilling, refund requests, or payment disputes.

2. 📋 Specify an Explicit JSON Output Schema

Explicitly list allowed values inside the schema definition to guide token generation:

{
  "intent": "billing | technical_support | account_security | null",
  "out_of_scope": "boolean"
}

3. 🎯 Use 1–2 In-Context Few-Shot Exemplars for Complex Logic

For nuanced domains, complex entity extractions, or custom severity scoring, providing 1–2 dynamic few-shot examples dramatically steers the output:

EXAMPLES:
Input: "Someone from Russia logged into our admin console"
Output: {"intent": "account_security", "severity": "CRITICAL", "out_of_scope": false}

Input: "Can you change my invoice address?"
Output: {"intent": "billing", "severity": "LOW", "out_of_scope": false}

📐 Canonical Prompt Structure

For highest fidelity, format your prompts following this standard block structure:

You are a precise semantic routing engine. Output only valid JSON.
TASK:
<High-level routing objective>

CATEGORIES:
- <category_name_1>: <semantic definition and trigger rules>
- <category_name_2>: <semantic definition and trigger rules>

OUTPUT SCHEMA:
<JSON Schema with allowed literals>

[OPTIONAL] EXAMPLES:
Input: "<exemplar_query>"
Output: <exemplar_json>

INPUT:
"<user_query>"

JSON:

⚡ Quickstart (Transformers / PyTorch)

import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NamanAgnih0tri/AlphaRoute-0.8B-v1.0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

prompt = '''You are a precise semantic routing engine. Output only valid JSON.
TASK:
Route cloud infrastructure event and extract parameters.
CATEGORIES:
- finops_budget_anomaly: Unpartitioned scans, large spend surge.
- iam_privilege_escalation: Unauthorized root policy attachment.
OUTPUT SCHEMA:
{"routing": {"target_category": "finops_budget_anomaly | iam_privilege_escalation | null", "severity": "CRITICAL | HIGH | LOW", "out_of_scope": "boolean"}, "metadata": {"affected_resource": "string or null", "cost_or_metric": "string or null"}}
INPUT:
"BigQuery project 'analytics-warehouse' incurred $14,250 in 4 hours due to cross-join scans."
JSON:'''

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=100, do_sample=False)

generated = tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True).strip()
print(generated)

📦 Model Family Releases

⚡ Apple MLX 4-bit (404 MB): NamanAgnih0tri/AlphaRoute-0.8B-v1.0-MLX-4bit
💎 Apple MLX 8-bit (763 MB): NamanAgnih0tri/AlphaRoute-0.8B-v1.0-MLX-8bit
🦙 llama.cpp / Ollama GGUF (503 MB): NamanAgnih0tri/AlphaRoute-0.8B-v1.0-GGUF
🔥 Standalone FP16 Safetensors (1.4 GB): NamanAgnih0tri/AlphaRoute-0.8B-v1.0

Downloads last month: 209

Safetensors

Model size

0.8B params

Tensor type

F16

Model tree for NamanAgnih0tri/AlphaRoute-0.8B-v1.0

Base model

Qwen/Qwen3.5-0.8B-Base

Finetuned

(101)

this model

Quantizations

1 model