ludolua/nextaction-agent

NextAction: agent

A small, auditable classifier that predicts what an assistant should do next in a task-oriented conversation. Given the current state, it returns one action from a fixed vocabulary with a probability for each, so an agent can act alone when it is confident and hand the decision to a person when it is not. It ships with a decision policy (config.toml) that blocks impossible actions and sets a minimum confidence per action.

Model details

Developed by	Luiz Araujo
Model type	TF-IDF features + multinomial logistic regression (scikit-learn)
Task	next-action prediction for task-oriented agents (text classification)
Language	English
License	cc-by-sa-4.0
Size	4.7 MB
Code	github.com/leduardoaraujo/NextAction

Uses

Direct use. Decide the next step of a conversational agent (ask, search, present options, confirm, execute, end the conversation), route uncertain decisions to human review, and log decisions with feedback to collect training data.

Out of scope. Executing irreversible actions (payments, bookings) from a prediction alone, domains far from the training data without re-evaluating, and any use as a source of facts: the model chooses an action, it does not generate or verify content.

How to get started

from huggingface_hub import hf_hub_download
import skops.io as sio

model = sio.load(hf_hub_download("ludolua/nextaction-agent", "model.skops"))  # no types outside skops' trusted list

state = "[intent SearchOnewayFlight] [filled destination_airport origin_airport] [missing departure_date] [requested none] || user: I need a one way flight from New York to Seattle."
probabilities = dict(zip(model.classes_, model.predict_proba([state])[0]))
print(max(probabilities, key=probabilities.get), probabilities)

Dependencies: scikit-learn, skops and huggingface_hub. Use a scikit-learn version compatible with the training one. The full toolkit (CLI, REST API, decision log, evaluation, policy) is in the NextAction repository.

Input format

The state is the dialogue state followed by the last turns, separated by ||:

[intent <ActiveIntent>] [filled <slots>] [missing <slots>] [requested <slots>] || user: <text> || assistant: <text> || user: <text>

Tag	Content
intent	the user's active intent, e.g. SearchOnewayFlight
filled	slots the user already provided (names only, no values)
missing	required slots of the intent that are still empty
requested	slots the user asked about

Use none for empty tags. Knowing which required slots are missing is the strongest signal: without the tags, accuracy drops from 0.90 to 0.77. Portuguese tags ([intencao] [preenchidos] [faltando] [pedidos], usuario:) are accepted by nextaction.vocabulary.normalize_state, which the NextAction package applies before predicting.

Actions

Action	Meaning
answer	answer a question about an item or the task
ask	ask the user for missing information
confirm	confirm the details with the user before a transaction
end_conversation	close the conversation
execute	run the transaction (booking, payment, transfer) and report the outcome
offer_more_help	ask whether the user needs anything else
present_options	present options from results already fetched
search	query a tool, API or database before replying
suggest_next_step	propose the next step, such as booking what was found

Training details

Data. 164982 assistant turns from the Schema-Guided Dialogue (Rastogi et al., 2020, CC BY-SA 4.0): conversations between people and a virtual assistant across 20 domains, written by crowd workers from simulated outlines. Build the CSV with nextaction import-sgd. The training data is not redistributed here; download it from the original source.

Preprocessing. States are normalized to the English tag format, then vectorized with TF-IDF over word unigrams and bigrams with accents stripped.

Hyperparameter	Value
ngram_range	(1, 2)
min_df	5
C	1.0
class_weight	None
max_iter	1000
calibration	none

The solver is L-BFGS, run in warm-started chunks so training progress can be reported. Hyperparameters live in config.toml and can be searched with nextaction tune.

Evaluation

Method. separate test split (sgd_generic_test.csv), with 42297 examples evaluated.

Results. Accuracy 0.901 and macro F1 0.865; always choosing the most frequent action would give 0.213. Expected calibration error: 0.084.

Action	Precision	Recall	F1	Support
answer	0.983	0.962	0.972	4170
ask	0.997	0.941	0.968	9016
confirm	0.901	0.847	0.873	5697
end_conversation	0.926	0.892	0.908	4201
execute	0.939	0.921	0.93	5006
offer_more_help	0.73	0.767	0.748	2684
present_options	0.686	0.714	0.7	1030
search	0.849	0.958	0.9	8233
suggest_next_step	0.795	0.779	0.787	2260

Review thresholds. Decisions with confidence below the threshold go to a person:

Threshold	Decisions automated	Accuracy on automated
0.3	98.7%	90.9%
0.4	95.4%	92.3%
0.5	90.0%	94.3%
0.6	82.9%	96.3%
0.7	74.7%	97.8%
0.8	64.9%	98.8%
0.9	49.8%	99.6%

Decision policy

config.toml holds the decision policy used with this model: rules that block impossible actions, a minimum confidence per action, risk levels and error costs, in units of one human review (review_cost = 1). nextaction calibrate-policy re-fits the thresholds from the costs.

Action	Risk	Min. confidence	Unreviewed error cost	Confirm with user
answer	low	0.3	2	no
ask	low	0.33	2	no
confirm	medium	0.71	10	no
end_conversation	medium	0.54	8	no
execute	irreversible	0.86	50	yes
offer_more_help	low	0.31	2	no
present_options	low	0.48	3	no
search	low	0.64	3	no
suggest_next_step	low	0.42	3	no

Rules:

• no_search_or_execute_with_missing_fields: deny search, execute when any required slot is missing

Simulated on the evaluation data above, the policy automates 85.1% of decisions at 95.2% accuracy, with an expected cost of 0.309 reviews per decision.

Apply it with the NextAction package (pip install git+https://github.com/leduardoaraujo/NextAction):

import tomllib
from nextaction.policy import apply, from_dict

with open(hf_hub_download("ludolua/nextaction-agent", "config.toml"), "rb") as file:
    policy = from_dict(tomllib.load(file)["policy"])
decision = apply(probabilities, state, policy)  # probabilities and state from the snippet above
print(decision["action"], decision["needs_review"], decision["reasons"])

Bias, risks and limitations

• The model matches word patterns. It does not understand business rules or verify facts, so the calling application must check preconditions before acting.
• Logistic regression probabilities are not calibrated; pick thresholds from the tables above, and re-evaluate on your own traffic before automating.
• The training conversations were written by crowd workers from simulated outlines in 20 service domains, so real users will be messier and other domains less accurate.
• The action vocabulary is fixed; new actions require new labeled data.

Citation

@software{araujo2026nextaction,
  author = {Araujo, Luiz},
  title  = {NextAction: a small, auditable next-action model for AI agents},
  year   = {2026},
  url    = {https://github.com/leduardoaraujo/NextAction}
}

Training data:

@inproceedings{rastogi2020schema,
  title     = {Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset},
  author    = {Rastogi, Abhinav and Zang, Xiaoxue and Sunkara, Srinivas and Gupta, Raghav and Khaitan, Pranav},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
  volume    = {34},
  number    = {05},
  pages     = {8689--8696},
  year      = {2020}
}

Model card contact

Open an issue in the NextAction repository.