P y TOD : Programmable Task-Oriented Dialogue with Execution Feedback (2025)

Abstract: Programmable task-oriented dialogue (TOD) agents enable language models to follow structured dialogue policies, but their effectiveness hinges on accurate dialogue state tracking (DST). We present PyTOD, an agent that generates executable code to track dialogue state and uses policy and execution feedback for efficient error correction. To achieve this, PyTOD employs a simple constrained decoding approach, using a language model instead of grammar rules to follow API schemata. This leads to state-of-the-art DST performance on the challenging SGD benchmark. Our experiments show that PyTOD surpasses strong baselines in both accuracy and cross-turn consistency, demonstrating the effectiveness of execution-aware state tracking.

## introduction tod agents provide natural language interfaces which enable users to control their digital environment to complete daily tasks. such agents typically include a dialogue state tracking (dst) component, which maps the conversation history to a symbolic representation of the task-relevant information communicated during the exchange. at each turn, a dialogue manager (dm) uses this information to take system actions 1 necessary to help the user complete the task. the agent behaviour is controlled by a dialogue policy defined by application developers. domain adaptation has long been a challenge for state tracking @xcite , dialogue management @xcite and end-to-end (e2e) @xcite agents, as it often requires developers to collect and annotate new datasets for retraining. to address this challenge, fine-tuning pretrained language models (plms) @xcite within the schema-guided a schema supervisor is invoked by the dm to correct errors if predicted statements contain slot names that are not part of the api schema (e.g., the slot to_city is mapped to destination_city, a member of the schema). given knowledge of previous system actions (x2, x3) the dm detects slot omissions and invokes a parser supervisor to correct them (e.g., flight date is recovered). despite its strengths, anytod has some limitations. first, it re-estimates the state and action history at every dialogue turn solely based on the dialogue history and schema, which increases generation length and amplifies the risk of state-tracking errors. second, it fails to exploit previous system actions to verify the correctness of the state sequence. finally, its reliance on symbolic representations of state and action sequences requires additional system components to translate them into executable code, adding deployment complexity. we present pytod, a programmable dialogue system that addresses these challenges by directly communicating with its execution environment and policy programs to perform accurate state tracking for unseen apis and domains, as discussed next. pytod overview pytod incrementally generates dialogue states as code, using policy-and execution feedback for accurate state estimation (figure 1 ). it operates as follows: (1) an action parser (ap) ( §3.1) processes the user query to produce one or more python statements; (2) the dialogue manager (dm) ( §3.2) executes these statements, ensuring they valid according to the schema; (3) if invalid slots are found, the dm calls the schema supervisor (ss) ( §3.2.2), a language model that constrains code generation according to the decoding schema; and (4) the dm evaluates the constrained output by comparing expected and current environment states, invoking a parser supervisor (ps) ( §3.2.3) to recover from possible omissions or semantic errors. the dialogue state is derived by executing the program generated by pytod at each user turn. contributions unlike pytod, tod agents of-ten optimise dst and dm independently, overlooking how policy information can enhance state generation accuracy. we further demonstrate that feedback from the execution environment enables language models to constrain decoding without requiring additional training data, with minimal developer effort and only a slight increase in system latency. pytod achieves sota performance on the challenging schema-guided dialog (sgd) dataset @xcite . as an additional contribution, we release pytodlib foot_1 , a python library that simulates sgd apis, including database responses and api behavior. this toolkit addresses the scarcity of conversational tool-use corpora, providing a valuable resource for benchmarking large language models' ability to handle complex, multi-task, goal-oriented conversations while following predefined policies. ## simulated environment as a programmable dialogue system, pytod relies on a simulation of its operating environment and an execution engine, described in this section. our simulation is based on sgd due to its large ontology, complex policy and high-quality annotations. ## selection [select] select [3] generated when: user selects an entity suggested by the agent (e.g., a restaurant). syntax: select(evar, @xmath0. side effect: api state updated with selected entity properties (e.g., @xmath1. 4 suspend task [suspend] negate_intent [3], [4] generated when: a user switches context, suspending a follow-up task suggested by the agent. syntax: svar suspend(sugvar) where sugvar is a variable bound to an agent suggested task (e.g., x7 in figure 2 ). side effect: svar stores state relevant to the follow-up task (e.g., @xmath2. 5 resume task [resume] inform_intent [5] generated when: a user continues a previously suspended task. syntax: resume(svar) where svar is a variable referencing a suspended task. side effect: ensures resumed tasks (e.g., x12 in figure 2 ) are updated with state stored in svar. 6 transaction api call [e.g., book_restaurant] inform_intent [5] generated when: a user initiates a task that changes their device state or that of an external service (e.g., bookings). syntax: see #1. side effect: see #1. 7 provide task constraints [-] inform [6] generated when: a user provides task constraints or corrects slot values. syntax: dot assignment: variable.@xmath3, where variable references a prior call and field is an api slot. side effect: updates referenced api state with slot values provided by the user. 8 pause [conversation_pause] -[7] generated when: detecting an out-of-scope utterance. syntax: zero-arguments function call to conversation_pause(). side effect: carries over the current api state to the next turn. 9 confirm [confirm] affirm [8] generated when: a user confirms a transaction for execution. syntax: confirm(apivar) where apivar references a transaction api call. side effect: updates api state agent values (e.g., @xmath4, @xmath5, @xmath6. 10 decline suggestions [decline_alternative] negate [9] generated when: a user declines agent's suggestion to update task constraints following a transaction failure. syntax: zero-arguments function call to decline_alternative(). side effect: carries over the current api state to the next turn. ## experimental setup we evaluate pytod on schema-guided dst. while zhao et al. (2023) also evaluate their system on next-action prediction (nap), pytod is designed such that, given the correct dialogue state, it is guaranteed to take a correct action according to the dialogue policy. hence, nap is highly correlated with dst performance and does not provide additional insight into system behavior. such turnbased evaluations often fail to reflect a system's ability to satisfy complex user goals in real interactions @xcite . ## main result tod agents are deployed in resource-constrained settings, so we implement the ap and ps components using a small plm and compare pytod against sota models of similar size and llms. pytod retrieves apis from the prompt while the jga of all other models is invariant to intent parsing errors, making the figures less sensitive to anno- however, sdt-seq requires developers to manually craft example demonstrations for each intent in addition to service schemas, increasing deployment overhead. furthermore, sdt-seq does not perform api retrieval, requiring an external intent detection model. finally, pytod demonstrates substantially higher consistency, outperforming sdt-seq by 5.0% (#5&10) and 4.2% (#7&11) in c-jga, reinforcing its stability in multi-turn interactions. llm comparison pytod compares favourably with llms. compared to llama, pytod (b) improves jga by 1.5% (#8&10) while being 32 times smaller. pytod (l) improves it by 6.9% while being 10 times smaller (#8&11). these results show that automatic prompt generation given execution errors and policy feedback enables small language models to perform competitively with llms while operating at a fraction of a cost. ldst is trained with data generated with multiple prompt templates, outperforming our best model by 2.3% (#9&11). we do not perform augmentation. ## conclusion we introduced pytod, a model that generates code incrementally, tracks state through execution, and uses policy and execution feedback to automatically generate error-correcting prompts. by coupling state tracking with execution, pytod shows enhanced cross-turn consistency and thus improved real-world reliability. we release pytodlib, a simulation grounding sgd, to advance research on zero-shot tod agents and conversational tool use. future work will focus on enhancing pytod's robustness to copy errors and applying it to llmbased zero-and few-shot e2e dialogue modeling.