chris0809/scope-intent-distilmbert

Multilingual DistilBERT Scope Intent Router

This full fine-tune of distilbert/distilbert-base-multilingual-cased routes requests into:

• DIRECT_RESPONSE: no external state or structured scope resolution is needed.
• REQUIRES_SCOPE_CONTRACT: tools or external state are needed, so a scope contract should be resolved first.

Use the conservative threshold stored in router_metadata.json.

Held-out synthetic-family evaluation

Accuracy	ROC-AUC	Scope precision	Scope recall
84.80%	97.42%	77.10%	99.00%

The split contains 3,000 examples from seed families excluded from training. These are synthetic results and do not establish production accuracy.

import json
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "chris0809/scope-intent-distilmbert"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()
metadata = json.loads(open(hf_hub_download(model_id, "router_metadata.json"), encoding="utf-8").read())
inputs = tokenizer("把我收藏夹里的文件移到归档区", return_tensors="pt", truncation=True)
with torch.inference_mode():
    probability = torch.softmax(model(**inputs).logits, dim=-1)[0, 1].item()
decision = "REQUIRES_SCOPE_CONTRACT" if probability >= metadata["threshold"] else "DIRECT_RESPONSE"