agent-task-classifier

(laya loop-taxonomy v2 — nine labels)

A fine-tuned laya checkpoint that classifies a user request to a personal-assistant agent by the agent-loop shape it demands — the skeleton of the high-level process required to fulfil it — rather than by topic.

Same interface as every laya checkpoint: text in, one typed choice decision out, one forward pass, no generation. English, ~0.3 s/query on CPU.

v2 changes the label set (v1 had eleven labels; git history keeps it). v1 → v2: make→produce; react→stabilize; arrange, acquire, execute→transact; tend→conversation (reviewing a standing concern is a lookup; the wake that raises it is a trigger, not a request); plan is new — setting up or changing a person's own goals, routines, budgets and trackers.

The nine labels

label	the loop it demands
conversation	questions, quick asks, lookups, a reported measurement, a single reminder, chatter — the default, safe fallback
companion	inner-life support with no outcome in the world (reflection, therapy)
learn	build the user's own ability over sessions (practice, tutoring)
diagnose	find the cause of a specific malfunction or anomaly, including in the user's own data
research	assemble understanding from investigation (multi-source, comparison, synthesis)
produce	make an artifact judged against a standard (document, spreadsheet, script, letter)
transact	a commitment or possession in the world: book, buy, pay, transfer, or complete a procedure
stabilize	urgent or deteriorating situation — act before full understanding
plan	set up or change the person's own standing structure: a goal, routine, budget/limit, or tracker

Boundary rule: two requests share a label iff they demand the same loop skeleton — same phases, same iteration driver, same exit criterion. Topic is irrelevant. Labels are checked in the order stabilize, transact, plan, produce, diagnose, research, learn, companion, conversation; the first match wins. plan is not a single reminder ("remind me to call Sam"), a lookup ("how much did I spend on dining"), a reported measurement ("I weighed 82 kg"), a document to write ("build me a budget spreadsheet") or a payment ("transfer $500 to savings").

Usage

import laya

agent = laya.load("kumarabd1996/agent-task-classifier", device="cpu")
questions = json.load(open(hf_hub_download("kumarabd1996/agent-task-classifier", "questions.json")))
out = agent.predict("I want to keep my dining spending under $200 a month", questions)
out["answers"]["loop"]["choice"]         # "plan"
out["answers"]["loop"]["probabilities"]  # calibrated (temperatures fitted at train time)

Continuations: pass the request being continued in the state, the format the model was trained on:

Earlier request: I want to keep my dining spending under $200 a month
New request: All

Evaluation (held out, never trained on)

54 items: the 24 taxonomy boundary drills (relabelled to v2) plus 30 hand-written personal-records and finance cases.

	accuracy	mean confidence	ECE
untuned laya	24.1% (13/54)	0.31	0.19
this model	87.0% (47/54)	0.96	0.09

plan: 11/13. Nothing outside plan was predicted as plan. The two misses were implicit phrasings ("Don't let me overspend on clothes this quarter", "I keep meaning to exercise more, can you help me actually do it?"). Remaining misses are older boundary drills (an implied purchase, "is X worth it?" as research, a money transfer) and the model is confident when it is wrong (~0.95), so a confidence floor does not catch them. 54 items is a small sample: treat the numbers as a smoke test, not a benchmark.

Training

Full fine-tune of laya on 4,754 examples (400 held back to fit calibration temperatures), 4 epochs, effective batch 64. Data: kumarabd1996/agent-task-classifier-dataset.

Limitations

• English only; nine fixed labels; out-of-taxonomy requests resolve to conversation.
• Much of the training data is synthetic, so template phrasing biases it; implicit or indirect requests ("I keep meaning to…") are the weak spot.
• Calibrated on a small held-out slice; probabilities are not reliable above ~0.9.