Exploring R e A ct Prompting for Task-Oriented Dialogue: Insights and Shortcomings (2025)

Abstract: Large language models (LLMs) gained immense popularity due to their impressive capabilities in unstructured conversations. Empowering LLMs with advanced prompting strategies such as reasoning and acting (ReAct) (Yao et al., 2022) has shown promise in solving complex tasks traditionally requiring reinforcement learning. In this work, we apply the ReAct strategy to guide LLMs performing task-oriented dialogue (TOD). We evaluate ReAct-based LLMs (ReAct-LLMs) both in simulation and with real users. While ReAct-LLMs severely underperform state-of-the-art approaches on success rate in simulation, this difference becomes less pronounced in human evaluation. Moreover, compared to the baseline, humans report higher subjective satisfaction with ReAct-LLM despite its lower success rate, most likely thanks to its natural and confidently phrased responses.

## introduction task-oriented dialogue (tod) systems solve tasks, such as accessing information or booking places and tickets, by interacting with humans in natural language @xcite . traditional tod pipelines integrate specialized components for natural language understanding (nlu), dialogue state tracking (dst), dialogue management and natural language generation (nlg), and optionally speech recognition and synthesis @xcite . on the other hand, end-to-end architectures model all components in a single neural networks @xcite @xcite . both approaches are costly to develop, requiring manual engineering and/or collecting large dialogue corpora. large language models (llms) @xcite offer an alternative to this by generalizing from instructions or a small number of examples and promise fluent and natural replies. however, unlike standard llm use scenarios, task-oriented dialogues typically have a rigid structure and require access to an external database to retrieve necessary information, such as venues or objects to search for and their properties. recently, synergizing reasoning and acting in llms (react) @xcite has shown promising results in controlled tasks that need external information access. react employs few-shot llm prompting with a sequence of thoughts, actions, and observations. thoughts refer to internal reasoning that decomposes a problem into sub-problems. actions execute external api calls or programs, and observations analyze the results of actions. in this work, we investigate the ability of llms guided by react to solve task-oriented dialogue. our contributions are as follows: (i) we implement a react-based system for tod, using gpt-3.5 and gpt-4 llms respectively @xcite , for the multiwoz tourist information domains @xcite . (ii) we evaluate our react-llm system both with a simulated user and with humans, and we include a manual qualitative analysis of human dialogues and a api cost versus performance trade-off assessment. (iii) our results show that react-llms clearly underperform state-of-the-art baselines in terms of success rate in simulation. however, this difference is less pronounced in human evaluation, and humans even rate their satisfaction with the react-llm system higher than the baseline. ## related work classical approaches for dialogue management @xcite , varied from handcrafted (hdc) to reinforcement learning (rl). the combination of deep rl with imitation learning @xcite as well as structural rl have also been applied to multi-domain, multitask dialogue @xcite . however, these approaches require separate specialized components, involving extensive engi-neering, the need for semantically annotated data as well as user simulators operating at the semantic level. latest end-to-end neural architectures are built on top of pretrained language models @xcite and involve two-step generation: the model first generates the dialogue state or database query based on user input; then, it generates the reply based on external database search. these approaches provide more flexibility and potentially better fluency, but require even larger training corpora. recent approaches explore simple zero-shot or few-shot llm prompting for tod, using examples of relevant dialogue turns @xcite , database or api definitions @xcite , templated conversation snippets @xcite , or detailed behavior and api calling instructions @xcite . in contrast, in this work we propose to use the react strategy @xcite to guide llms towards task oriented dialogue through domain selection, slot detection and belief state tracking. since dialogue is dynamic and evaluation on static data in single-turn replies may not be consistent with full dialogue performance @xcite , we evaluate the system on full dialogues. first, we evaluate performance in a simulated environment, which, unlike @xcite , utilizes the well established convlab3 benchmark @xcite to guarantee reproducibility and fair comparison with previous approaches. we also assess the conversations with humans. unlike previous works on dialogue management @xcite @xcite , the simulator and our system do not interact at the semantic level, instead they interact in natural language. unlike previous end-to-end and llm-based approaches, which generated delexicalized responses with placeholders and replaced these with valid entity names from the database in a rule-based fashion, we generate full responses including entity names. this avoids constraining the inherent capabilities of llms in generating natural language. instead, we guide the llm towards the database constraints through react prompting. step 1: the sa should try to understand the user input. its first thought should be explicitly planning out the next course of action, to be executed in steps 2-5. the sa should plan for identifying the domain and the corresponding slots and values from the user input, forming the belief state and using it to query the database, retrieving the results and forming the final answer based on the results. step 2: the sa should call list_domains, observe its output and decide which domain the user request belongs to. step 3: now it needs the list of slots available in the selected domain to identify the slot values from the user request. the system should use the list_slots tool with the domain as input, observe the list of slots available and decide which slots have been mentioned in the user input and form/update the belief state. step 4: it should call db_query using the state as input to retrieve records from the database that match the user request. ## experimental setup we use the langchain foot_0 library for implementing react-llm. the prompt details are given in appendix a, figure 2 . we use langfuse foot_1 for debugging the reasoning traces and to keep track of the computational costs of our experiments. we experiment with openai gpt-3.5 (gpt-3.5-turbo-0301) and gpt-4 (gpt-4-32k) models. we first couple our sa with a simulated user, then proceed to evaluate it with humans. ## results we now assess both simulated and real users. ## conclusion the performance of react-llm falls short compared to hdc and rl baselines, which perform better mainly due to their fine-grained control at each step in the pipeline. by relying completely on the reasoning abilities of react-llm, we lose the ability to control its reasoning traces and response generation. additionally, difficulty in understanding the system requests by the user simulator, due to bert-nlu errors, leads to repeated utterances and thus a higher number of turns on average. the human evaluation shows that react-llm is preferred by users over the hdc baseline, despite its lower success rate. heng-da xu, xian-ling mao, puhai yang, fanshu sun, and heyan huang. 2024. rethinking task-oriented dialogue systems: from complex modularity to zeroshot autonomous agent. in proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: long papers), pages 2748-2763, bangkok, thailand. association for computational linguistics. shunyu yao, jeffrey zhao, dian yu, nan du, izhak shafran, karthik narasimhan, and yuan cao. 2022. react: synergizing reasoning and acting in language models. arxiv preprint arxiv:2210.03629. xiaoying zhang, baolin peng, kun li, jingyan zhou, and helen meng. 2023. sgp-tod: building task bots effortlessly via schema-guided llm prompting. in findings of the association for computational linguistics: emnlp 2023, pages 13348-13369, singapore. association for computational linguistics. qi zhu, christian geishauser, hsien chin lin, carel van niekerk, baolin peng, zheng zhang, michael heck, nurul lubis, dazhen wan, xiaochen zhu, jianfeng gao, milica gašić, and minlie huang. 2022. convlab-3: a flexible dialogue system toolkit based on a unified data format. arxiv preprint arxiv:2211.17148. qi zhu, zheng zhang, yan fang, xiang li, ryuichi takanobu, jinchao li, baolin peng, jianfeng gao, xiaoyan zhu, and minlie huang. 2020. convlab-2: an open-source toolkit for building, evaluating, and diagnosing dialogue systems. in proceedings of the 58th annual meeting of the association for computational linguistics: system demonstrations, pages 142-149.