An Adaptive Prompt Generation Framework for Task-oriented Dialogue System (2023)

Abstract: The de facto way of utilizing black-box large language models (LLMs) to perform various downstream tasks is prompting. However, obtaining suitable prompts for specific tasks is still a challenging problem. While existing LLM-based methods demonstrate promising performance in task-oriented dialogue (TOD) task, they often require manual adjustment in prompt selection, or focus solely on dialogue understanding or generation. To address these issues, we propose an adaptive prompt generation framework to fully unleash the potential of LLMs for the comprehensive TOD system. Firstly, we design a trainable slot generator (TSG) that can generate domain and slot information in the belief state, which serves as prior knowledge for subsequent prompt generation. Next, we propose an adaptive prompt generator (APG) that utilizes the prior knowledge to generate prompts for the LLM, deriving the belief state and system response of the dialogue for evaluation. Finally, we evaluate our framework on the MultiWOZ 2.0 dataset. Extensive experiments demonstrate that our method outperforms existing methods. Our code and data will be released.

## introduction in recent years, significant progress has been made in llms, such as instruct-gpt @xcite and gpt4, and remarkable results have been achieved in their application to various downstream tasks such as task-oriented dialogue (tod) and text summarization @xcite . prompting has become a de facto method for utilizing black-box llms, as appropriate prompts can significantly enhance the capabilities of these models. however, different tasks require different prompts, and obtaining these prompts often requires manual adjustment. this is particularly challenging in dynamic figure 1 : comparison of our proposed adaptive prompt framework with the standard prompting method for black-box llm-based tod system. our method uses a trainable slot generator to obtain domain and slot, which serve as prior knowledge for prompt generation. this allows for better adaption to diverse dialogues involving various domains and slots. scenarios like the tod task, where prompt selection should adapt, usually indicating a large amount of manual labor is required. in the context of the tod system, two crucial components for measuring the success of a dialogue are belief state and system response. when using prompts to generate the system response from black-box llms, a problem arises: belief states and system responses vary with the domains and slots involved in dialogues. if the same prompt is used for all dialogues, the prompt becomes overly long and complex, resulting in hallucinations in llms. on the other hand, manually designing different prompts for different dialogues would incur a significantly higher labor cost and would be in- dialog history ... user : i would like to find an expensive restaurant that severs chinese food . system : sure, which area do you prefer ? user : how about in the north part of town . domain and slot restaurant{pricer ange, food, area} system response the peony is a great chinese food restaurant. would you like to book a table there ? adaptive prompt framework prompt for system response prompt for belief state belief state restaurant { @xmath0, @xmath1, @xmath2. to address these issues, an automated method is needed to generate appropriate prompts for tod tasks. one approach is the continuous prompt learning method, where prompts are represented as trainable vectors that can be concatenated with inputs @xcite @xcite , enabling the acquisition of prompts suitable for the current input through gradient backpropagation. however, this approach lacks interpretability and requires model fine-tuning, making it unsuitable for black-box models @xcite . another approach is to represent prompts using discrete tokens, which overcomes the limitations of vector-based prompts @xcite . however, this approach requires designing suitable prompts for different scenarios and these prompts often cannot be learned through training. as a result, it may lead to the suboptimal performance of llms and hallucinatory outputs @xcite . traditional methods rely on static or heuristic rules to construct prompts, but such methods are only suitable for simple scenarios. recent approaches have achieved promising results in prompt generation. sun et al. (2022) propose a black-box tuning framework but only applicable to continuous prompts. diao et al. (2023) use policy gradient algorithm to find the optimal prompts, but only focus on classification tasks li et al. (2023) propose a framework to provide prompt guidance for black-box frozen llms, but only focus on dialog response generation. zhang et al. (2023b) propose a schema-guided prompting method for the tod system, which still requires manual prompt design. hence, there is currently no cost-effective method available to address the problem of adaptive prompt generation for black-box llm-based tod systems. in this paper, we propose an adaptive prompt generation framework for the comprehensive blackbox llm-based tod system to address the aforementioned challenges. to obtain appropriate prompts with minimal data, we first extract domain and slot information from the belief state as training data and introduce a trainable slot generator (tsg) that can generate domains and slots involved in each dialogue turn. this approach reduces the annotation cost when expanding to other domains. we then design a generator (apg) using this information to generate domains and slots as prior knowledge for subsequent prompt generation. to automatically generate prompts suitable for the current dialogue state, we maintain a query table of prior knowledge and candidate values. we utilize the previously generated prior knowledge to select suitable entries from the list and compose prompts that capture the current dialogue belief state and system response. this way, the content generated by the llm is precisely constrained by tailored prompts, which include the desired candidates without introducing redundant content. the main contributions of our work can be summarized as follows: ## method due to the phenomenon of hallucination that occurs during the application of black-box llms, it is hard to generate belief states accurately required for tod tasks. moreover, as the prompt becomes more complex, the content generated by black-box llms, like chatgpt, becomes even more uncontrollable. therefore, a method is needed to constrain the content generated by chatgpt. ## analysis high-frequency errors. for the unsuccess dialogues, we analyze high-frequency errors comparing the chatgpt method in the restaurant domain with our proposed method, as shown in figure 3 . specifically, "format error" is the error that prevents the belief state from being parsed, "dialogue generation error" refers to the result of a system response that does not align with logic, and "belief state error" refers to unmatched content. our method significantly reduces the number of evaluation failures caused by format errors owing to the inclusion of precise constraints in the prompt. furthermore, the proportion of dialogue generation errors is also reduced. lastly, the occurrence of belief states' errors has been mitigated due to the trainable slot generator. ## conclusion to address the issue of prompt generation of llms in the tod task, we propose an adaptive prompt generation framework for the comprehensive tod system which consists of two parts: trainable slot generator (tsg) and adaptive prompt generator (apg). the framework tackles the limitation of fixed prompts in tod and focuses on both dialogue understanding and dialogue generation. experimental results demonstrate that our proposed method significantly improves the performance compared to existing approaches.