An Adaptive Prompt Generation Framework for Task-oriented Dialogue System
Authors: Jun Gao
Liuyu Xiang
Huijia Wu
Han Zhao
Yiqi Tong
Zhaofeng He
Venue: EMNLP 2023 Findings (2023)
Keywords: adaptive prompt
LLM
task-oriented dialogue
black-box
prompt learning
TL;DR: We propose an adaptive prompt generation framework to fully unleash the potential of LLMs for the comprehensive TOD system.

Abstract: The de facto way of utilizing black-box large language models (LLMs) to perform various downstream tasks is prompting. However, obtaining suitable prompts for specific tasks is still a challenging problem. While existing LLM-based methods demonstrate promising performance in task-oriented dialogue (TOD) task, they often require manual adjustment in prompt selection, or focus solely on dialogue understanding or generation. To address these issues, we propose an adaptive prompt generation framework to fully unleash the potential of LLMs for the comprehensive TOD system. Firstly, we design a trainable slot generator (TSG) that can generate domain and slot information in the belief state, which serves as prior knowledge for subsequent prompt generation. Next, we propose an adaptive prompt generator (APG) that utilizes the prior knowledge to generate prompts for the LLM, deriving the belief state and system response of the dialogue for evaluation. Finally, we evaluate our framework on the MultiWOZ 2.0 dataset.  Extensive experiments demonstrate that our method outperforms existing methods. Our code and data will be released.

An Adaptive Prompt Generation Framework for Task-oriented Dialogue System
Jun Gao Liuyu Xiang', Huijia Wu', Han Zhao?, Yiqi Tong', Zhaofeng He
"Scooo of Artificial Intelligence, Beijing University of Posts and Telecommunications
Didi chuxing
3 School of Computer Science and Engineering, Beihang University {jungao, xiangly, huijiawu, zhaofenghe) edu. cn
zhaohan@didig com yqtongebuaa.co edu. cn
Abstract
Domain: hotel Slot: name
Domain: Diot: type entertainment
you
The de facto way of utilizing black-box large language models (LLMs) to perform various downstream tasks prompting. However, obtaining suitable prompts for specific tasks is still a challenging problem. While exist- ing L.MM-based methods demonstrate promis- ing performance in the task-oriented dialogue (TOD) task, they often require manual adjust- ment in prompt selection focus solely on dia- logue understanding or generation. To address these issues, we propose an adaptive prompt generation framework fully unleash the po- tential of LLMs for the comprehensive TOD system. Firstly, we design a trainable slot gen- erator (TSG) that generate domain and slot information in the belief state, which serves as prior knowledge for subsequent prompt genera- tion. Next, propose adaptive prompt gen- erator (APG) that utilizes the prior knowledge generate prompts for the LLM, deriving the belief state and system response of the dialogue for evaluation. Finally, we evaluate our frame- work
veriees might on the west town?
Trainable Slot (TSG)
Adaptive Prompt Gemerator (APG)
Dynamic prompt
Static prompt
Static prompt
Black-box LLM
Black-bol -bx LLM
state:
hotal name - System resporsee:
attraction type entertsinment
System response:
standard method
our method
Figure Comparison of our proposed Adaptive Prompt Framework with the standard prompting method for black-box LLM-hased TOD system. Our method uses a trainable slot generator obtain domain and slot, which serve as prior knowledge for prompt generation. This allows for better adaption to diverse dialogues involving various domains and slots.
the Multi WOZ 2.0 dataset. Extensive experiments demonstrate that our method out- performs existing methods. Our code and data will be released.
1 Introduction
scenarios like the TOD task, where prompt selec- tion should adapt, usually indicating large amount of manual labor required.
In recent years, significant progress has been made in LLMs, such as Instruct-GPT (Ouyang et al., 2022) and GPT4, and remarkable results have been achieved in their application various downstream tasks such task-oriented dialogue (TOD) and text summarization (Bang et al., 2023). Prompting has become a de facto method for utilizing black-box LLMs, as appropriate prompts can significantly en- hance the capabilities of these models. However, different tasks require different prompts, and ob- taining these prompts often requires manual adjust- ment. This is particularly challenging in dynamic
In the context of the TOD system, two crucial components for measuring the success of a dia- logue are belief state and system response. When using prompts to generate the system response from black-box LLMs, problem arises: belief states and system responses vary with the domains and slots involved in dialogues. If the same prompt is used for all dialogues, the prompt becomes overly long and complex, resulting in hallucinations in L.Mss. On the other hand, manually designing dif- ferent prompts for different dialogues would incur a significantly higher labor cost and would be in-
Corresponding author.

---

Dialog History
Belief State
User: :1 would lke
Trainable Slot
Adaptive Ganerator JAPG)
Prompt for System Plesponse
Restaurant pricerange expirslve, food = Chinese. north
to hnd
Black-box LLM
severs Chinese
food
Domain and Siot
Prompt for Beliet State
System: : surs, which area
Restaurantijrieer ange, area)
System Response
Peony grest Chinase restaurant: Would you like to book a table there? ?
north part
af town.
Adaptive Prompt Framework
Figure 2: Overview of our proposed Adaptive Prompt Generation Framework.
convenient when extending the approach domains.
TOD systems.
In this paper, we propose an adaptive prompt generation framework for the comprehensive black- box LLM-based TOD system to address the afore- mentioned challenges. To obtain appropriate prompts with minimal data, we first extract do- main and slot information from the belief state as training data and introduce trainable slot gener- ator (TSG) that generate domains and slots involved in each dialogue turn. This approach duces the annotation cost when expanding to other domains. We then design generator (APG) using this information to generate domains and slots as prior knowledge for subsequent prompt generation. To automatically generate prompts suitable for the current dialogue state, we maintain a query table of prior knowledge and candidate values. We uti- lize the previously generated prior knowledge select suitable entries from the list and compose prompts that capture the current dialogue belief state and system response. This way, the content generated by the LLM is precisely constrained by tailored prompts, which include the desired candi- dates without introducing redundant content.
To address these issues, an automated method is needed to generate appropriate prompts for TOD tasks. One approach is the continuous prompt learning method, where prompts are represented as trainable vectors that can be concatenated with inputs (Shin et al., 2020; Zhang et al., 2022, 2023a; Swamy et al., 2023), enabling the acquisition of prompts suitable for the current input through gradi- ent backpropagation. However, this approach lacks interpretabilityy and requires model fine-tuning, making it unsuitable for black-box models (Diao al., 2023). Another approach is to represent prompts using discrete tokens, which overcomes the limitations of vector-based prompts (Shin et al., 2020). However, this approach requires designing suitable prompts for different scenarios and these prompts often cannot be learned through training. As result, it may lead to the suboptimal perfor- mance of LLMs and hallucinatory outputs (Bang et al., 2023).
Traditional methods rely on static or heuris- tic rules to construct prompts, but such methods are only suitable for simple scenarios. Recent approaches have achieved promising results in prompt generation. Sun et al. (2022) propose black-box tuning framework but only applicable to continuous prompts. Diao et al. (2023) use policy gradient algorithm to find the optimal prompts, but only focus on classification tasks Li et al. (2023) propose framework to provide prompt guidance for black-box frozen L.MSs, but only focus on dia- log response generation. Zhang et al. (2023b) pro- pose a schema-guided prompting method for the TOD system, which still requires manual prompt design. Hence, there currently no cost-effective method available to address the problem of adap- tive prompt generation for black- box LLM-based
The main contributions of our work can be sum- marized follows:
1. We design prompt construction method based on domain and slot information.
2. We proposed an adaptive prompt generation framework for the comprehensive black-box LLM- based TOD system.
3. Experimental results demonstrate the effec- tiveness of approach in enhancing the capabili- ties of LLMs.
Related work
2.1 Black-Box LLMs in Downstream Tasks
In recent years, there has been a proliferation large language models such as Codex (Chen et al.,

---

2021). LaMDA (Thoppilan et al., 2022), PaLM (Chowdhery et al., 2022). chatGPT, GPT4, etc.. which have greatly enhanced various downstream tasks in NLP. However, most these LLMs are not open source and can only be accessed through query and prediction interface (Diao et al., 2023). As result, multitude of studies have emerged that focus on prompting methods tailored to spe- cific tasks for black-box LLMs. (Sun et al., 2022) propose a black-box tuning framework but only ap- plicable to common language understanding tasks. (Diao et al., 2023) use policy gradient algorithm to find the optimal prompts, but only focus on classifi- cation tasks. (Li et al., 2023) propose framework to provide prompt guidance for black-box frozen LLMs, but only focus on dialog response gener- ation. (Pan al., 2023) specifically focuses optimizing prompts for the dialogue understanding task. In summary, they are not applicable our task. In our work, we propose an adaptive prompt generation framework for the comprehensive TOD system.
Domain
Attraction
Train
Hotel
Restaurant
Train
50
50
50
50
Valid
50
50
50
50
Test
100
200
200
200
Table 1: Data statistics of four single-domain dialog
datasets (Budzianowski et al., 2020; Peng et al., 2021)
3.1 Data Preparation
In this section, we describe the steps involved in preparing the data for experiments. Our ap- proach involves selecting small subset of belief states from different domains in the MultiWOZ 2.0 dataset, based on the settings of SOLOIST. Subse- quently, the selected data is processed to remove the value component.
Selection of Belief' State Data Subset: We fol- low the setting of SOLOIST (Peng et al., 2021) and choose limited number of dialogues that contain only one domain from the MultiWOZ. 2.0 dataset. The data statistics of each domain are shown in Table
2.2 Prompt Learning
Processing Selected Data: To train a gener- ator that only generates domains and slots the belief state, we perform further processing to elim- inate the value component from the belief state. This step involves removing the actual values as- sociated with each slot in the belief state while retaining the slots and domains. By removing the value component, we focus solely on the task of predicting the belief state without considering spe- cific slot values.
Another line of our work involves prompt learning which finds optimal prompts suitable for specific tasks. One common approach is train continu- ous prompts (Liu al., 2021; Lester et al., 2021; Gu et al., 2022). However, these prompt types are not easily interpretable and require LLMs for training or fine-tuning, Consequently, some studies have proposed constructing prompts in discrete manner (Shin et al., 2020; Gao et al., 2021; Sun et al., 2022), using generated manually crafted prompts. (Schick and Schüitze, 2021) propose the pattern-verbalizer pair (PVP) method, which con- structs prompts by selecting appropriate patterns and verbalizers. Inspired by the aforementioned work, we propose a novel approach to generate appropriate prompts for the comprehensive TOD system.
By following these steps, we prepare the dataset for our experiments, which enables us to train and evaluate TSG.
3.2 Trainable Slot Generator
In Dialogue State Tracking (DST) task, We SOLOIST with pre-trained weights and then fine- tuned on a small amount of data obtained in Section 3.1. Nevertheless, unlike SOLOIST, our model is fine-tuned to generate domains and slots of belief states only. In this process, the TSG will generate prior knowledge which can be used to generate prompts in subsequent adaptive prompt generation framework.
3 Method
Due to the phenomenon of hallucination that oc- curs during the application of black-box LLMs, it is hard to generate belief states accurately required for TOD tasks. Moreover, as the prompt becomes more complex, the content generated by black-box LLMs, like ChatGPT, becomes even more uncon- trollable. Therefore, a method is needed to con- strain the content generated by ChatGPT.
Specifically, we can represent each dialog turn in the training dataset as
T (s, b)

---

where 8 is the dialog history up the current dia- log turn, is the annotated belief state only with domain and slots.
3.3 Adaptive Prompt Generation Framework for TOD System
In the TOD task, the belief state serves as an indi- cation of the user's intent and also acts inter- mediate state for extracting external information. Existing methods have demonstrated that LLMs perform well in acquiring the belief state. However, they also face certain challenges. One important factor is that large-scale models are sensitive to prompts. Clear and concise prompts tend to yield better results, while vague and lengthy prompts may lead to unexpected outcomes.
In the TOD system, the joint probability p(x)) can be expressed in the form of an autogressive model:
p(x) pp(b 8)
= p(b s)p(s)
where p(b] 8) refers to predicting the slots in the belief state. Note that we solely focus on predicting the domains and slots of belief states, excluding the values of belief state and system response, as they are generated in the subsequent part.
The TOD task differs from traditional tasks in that the domain and intent of user utterances change in each dialogue turn. When extracting informa- tion from such dialogues using LLMs, using the same prompt may not effectively capture the re- quired belief state information. On the other hand, using different prompts for each dialogue can sig- nificantly increase manual effort.
If we define the length of the belief state, which consists solely of domains and slots, as T, the train- ing objective for this process can be denoted as:
T
L1 = log 8) = Σ
log Po (b: bet, 8)
1=1
where represents the model parameters to be learned.
To address this, we propose a model-agnostic adaptive prompt generation framework in this sec- tion. This framework assists LLMs in generating the belief state and system response of dialogues. The overall structure of the framework is illustrated in Figure 2.
In line with the approach of SOLOIST, we have also incorporated contrastive learning training objective improve the efficiency of our model's learning process. However, unlike SOLOIST, when constructing negative examples (represented as æ'), we do not perform a complete replacement of the entire belief state. Instead, we perform separate replacements on the domain and slots components within the altered belief state b. We can define the label for a belief match y 1 and the label for non- non-match y O. In this case, the training objective of contrastive learning can be formulated as:
Adaptive Prompt Generator (APG): The main function of this generator is to generate prompts that are required obtain the final belief state and system response. Since the prompts dynamically change based on the dialogue process, we utilize the partial belief state generated in Section 3.2. In addition to this, the candidate lists of domains and slots Lats are required to obtain the complete belief state, well as the special token lists L are required generate the system response. Next, it is necessary to parse the generated partial belief state and generate domain sæ and slot information s.. We can consider the prompts input to the LLM as a function related to the partial belief state b. The prompts for generating the complete belief state and system response can be represented as fi and fz, respectively.
C2 = y log (2)())) + (1 - y) log (1 po (x'))
The three types of negative samples generated by approach are follows:
Only replacing the domain component of the modified belief state.
Only replacing the slots component of the modified belief state.
Simultaneously replacing both the domain and slots components of the modified belief state.
Prompt for Belief State: The prompt for gener- ating the complete belief state consists of two parts: static and dynamic. The static part includes stan- dard prompt and data examples, partly referenced from (Bang al., 2023), and can be represented as Prompts1- The dynamic part involves domain slots and candidates matched by s and can be de-
Finally, the proposed method in this section is model-agnostic and can be applied with other mod- els interchangeably. However, due to code and data availability, we have only validated the approach using the SOLOIST model in this paper.

---

fined as Promplai. The candidates of slots can be divided into two categories: one calegory includes slots with limited candidates that can be obtained from the dataset, such as the "area' slot with can- didates /'centre', 'east', 'north', 'south', 'west'), J. The other category includes candidates that cannot be exhaustively listed, such as the name slot rep- resenting various entity names related to different domains in user intents. The prompt function for generating the complete belief state utilizes the pre- viously partial belief state containing only the do- main and slot information, and can be represented as follows:
prompt function, the final result can be represented as:
r r= PLPLM (2(()) 8)
where r represents the delexicalized system re- sponse.
Therefore, the combined output of the belief state and system response generated by the LLM for evaluation is:
y = (=P)) (LLar(22(S),
Experiment
4.1
Dataset and Evaluation
fi(b) Prompta1 + Promptai (b, Lax)
We evaluated the effectivvneess of the proposed method on the MultiWOZ single-domain dialog datasets (Budzianowski et al., 2020), reorganized by (Peng et al., 2021). This dataset consists of four domains: Attraction, Hotel, Restaurant, and Train. In the TSG component, used the standard metric in DST: joint goal accuracy (JGA). This metric compares the predicted belief state with the ground truth belief state to determine if they are completely identical. successful prediction is achieved when the predicted belief state matches the ground truth belief state entirely. Note that the JGA metric requires both value and slot infor- mation to match for successful evaluation. In our work, the trainable slot generator only gener- ates domain and slot information of the belief state and does not include value information. Therefore, the evaluation is performed by complementing the value information using the LLM the second part before conducting the evaluation.
where specific b has fixed mapping relationship with the elements in I.ss-
Once we have the partial belief state b containing only the domain and slot information, we can use its prompt function fi to generate the prompt. This prompt is then inputted into the LLM, PLLM. to obtain the final result:
= PLLM
where s' represents the complete belief state in the current dialogue state.
Prompt for System Response: The prompt for generating the system response also consists of static and dynamic components. The static part serves as guidance for the LLM to simulate the generation of system response and can be represented as Prompt.2. The dynamic part involves matching special tokens based on 8a. which are placeholders used to represent system- recommended entities. The correct selection of these special tokens is crucial for evaluating the success of the dialogue and can be represented as Promplaz. The prompt function for generating the system response requires the dialogue history and can be represented as:
For the evaluation of the entire dialogue, we con- catenate the belief state and system response gener- ated by the LLM in the order of the dialogue and then evaluate them. We employ the following evalu- ation metries: Inform, which measures whether the provided entities satisfy the user's needs correctly; Success, which measures whether all requested attributes are addressed; BLEU is not adopted be- cause it measures the similarity between the gen- erated response and the reference response, and there may be significant differences between the responses generated by the LLM and the reference responses.
f2() Prompt,2 + Promptaz (Sa, List)
where specific sa has fixed mapping relationship with the elements in Let, and both f1, Promplet Promptan. f2. Prompt.2, Promptaz are all in string format, and the "+" operator denotes direct string concatenation.
4.2 Implementation Details
The slot generator and adaptive prompt framework proposed in our approach are not restricted to spe- cific models. In this paper, the TSG is based on
Next, we need to obtain the system response using the LLM. Based on the previously obtained

---

GPT-2 with 117M parameters and initialized with pretraining weights from SOLOIST. SOLOIST is pre-trained on corpora from multiple task-oriented dialogue datasets (Schema and Taskmaster) and performs tasks such as belief state prediction and response generation. Specifically, the input for the belief state prediction task is the current dialogue and all dialogue history, and the output is text sequence corresponding to the current dialogue state. In our task, we only need the domain and slot information from the belief state. An intuitive approach would be to directly extract domain and slot information from the belief state generated by the original model. However, this method would result in more complex and longer text sequences, requiring more time but not improving accuracy. Therefore, only trained the model using the domain and slot parts of the belief state.
main and slot information from the belief state in- stead of the complete belief state for training. This approach has two advantages as follows:
case manual annotation of data is required for future applications, annotators only need to label the domain and slot information dialogues. Unlike value information, which is diverse and difficult to standardize, these two types of information are usually easier to determine and have certain range. This significantly reduces the annotation cost.
Training and inferring with belief states that contain only domain and slot information make the input and output sequences of the model shorter. Compared to complete belief states, which contain more characters repre- senting the format and content, this method achieves better stability in generating results.
In the experimental part of the adaptive prompt framework, the LLM we use is chatgpt-3.5-turbo We generated belief states and system responses of dialogues by calling OpenAl's API. We find the format of the generated content affect its ac- curacy, if we use curly braces restrict the belief state, such as "(helief state: attraction type enter- tainment)", the result is better than when not using curly braces, like "belief state: attraction type entertainment". Based on our experience, we con- trol the format of the generated content to achieve optimal results. Therefore, belief states and sys- tem responses generated during the intermediate process may differ slightly in format from those in SOLOIST.
Attraction
Hotel
Restaurant
Train
ChatGPT
58.80
30.76
45.31
59.76
PPTOD
57.71
31.89
54.12
61.78
PPTOD* SOLOIST
58.11
32.76
54.34
62.90
60.10
27.83
$4.94
63.30
TSG w/o CL
58.11
32.06
55.21
63.25
TSG
58.27
33,02
55.81
63.78
Table 2: JGA results across different methods and do- mains. a fair comparison, JGA is evaluated only based on the domain and slot, excluding the consider- ation of values. "e" indicates the replacement of corre- sponding sections with the TSG method.
Inform
Success
BL.EU
Combined seure
4.3
Trainable Slot Generator (TSG) Setup
SOLOIST SOLOIST-ISG
73.88
72.22
13.11
16
76.23
74.10
13.81
88.98
4.3.1
Table Result Camrest676 dataset.
The goal of the TSG is to generate belief states that only contain slot information. We use SOLOIST baseline compare the improvement of our method on the final results. First, we train the TSG following SOLOIST, and the data statistics of dataset are shown Table The training epoch is 30, the learning rate 5e-5, and the batch size is 1. During testing, Nucleus filtering is used for decoding with a top-p value of 0.4 and a temper- ature of 0.3 to obtain the belief state. Since we only need the slot part of the belief state, we parsed the belief state and generated new belief state that only includes the domain and slot information using rules-based methods.
4.3.2 Results
The results obtained by different methods are shown in Table 2. We observe that directly uti- lizing ChatGPT for extracting slot information belief state state is not particularly effective. The method trained with shorter belief states surpasses the re- sults obtained by extracting slot information from the original SOLOIST method, and is also better than PPTOD and ChatGPT. We can observe that in more complex domains, such as hotels, the im- provement becomes more pronounced the belief state becomes more complex. This indicates that using less information for training can lead to more
For our proposed method, we only used the do-

---

accurate results, validating the potential of our pro- posed method. It is worth noting that the JGA metric of belief states containing only domain and slot information reflects the algorithm's 's advantage from one aspect. Further application in subsequent methods is needed to verify its actual improvement in the final dialogue evaluation.
use the subsequent dynamic section and include an example provide constraints on the generated format.
Then, we parse the results generated by the TSG, extract the domain and slot information, and lect the relevant entries from the existing domain- slot-value candidate list to form a list of candidate entries.
To assess the generalizability of our proposed method, we applied the TSG to the Camrest676 dataset. The results as shown in Table 3 indicate that, even when tested on novel dataset, our pro- posed method consistently enhances the results. This observation underscores the method's capac- ity to generalize to datasets beyond its training domain.
Finally, we process the input dialogue informa- tion to generate the second dynamic part of the prompt. This part consists of current user utter- ances, dialogue history, and partial belief state, combined into a string list format. The partial belief state is represented as "(slol =?}". The example prompt is presented in Appendix A.
Part 2 involves generating the system response based on the user utterance and dialogue history in the current conversation. The purpose of this part is to have the LLM act as an agent in the TOD system and respond to user requests by generating delexicalized system responses. It is important to note that, for the convenience of evaluating the success rate of subsequent dialogue, recommended or queried entities in the generated response need to be replaced with special tokens. The complete response is then generated through post-processing.
4.4
Adaptive Prompt Framework Setup
4.4.1
The purpose of the Adaptive Prompt Framework is to generate suitable prompts for inputting the current dialogue into the LLM to obtain the cor- responding belief state and system response. The underlying assumption is that the LLM possesses strong comprehension and generalization capabili- ties and can adapt to various downstream tasks by designing appropriate prompts. Specifically, the Adaptive Prompt Framework consists of two parts: Part involves refining the results from the TSG. Firstly, a candidate list of domains, slots, and their corresponding values is compiled based on the dataset. This list serves the purpose of retrieving the candidate values for given domain and slot when we obtain the domain and slot information. This candidate list includes two types of entries. The first type comprises slots with limited values, where the values can be represented by short-word sequences, such as slots indicating time or direc- tion. The second type consists of slots with poten- tially infinite values, where the values may vary during the conversation, such as the "name" slot that may change with different entities discussed in the dialogue. These entries cannot be exhaus- tively enumerated, and thus we use "?" represent candidate values. Each entry in the candidate list pertains to a single domain and slot but can contain multiple candidate values.
Similar the previous part, we need compile a list of domains and corresponding special tokens based on the dataset. This list serves the purpose of retrieving candidate values for a given domain using table lookup when we obtain the domain in- formation, to provide the LLM with options for generating the final system response. This candi- date list includes two types of special tokens. The first type consists of domain-specific special tokens represented in the form of domain-slot, such as "attraction-name" The second type comprises spe- cial tokens that are universally applicable across all domains and are represented as value-slot, such as "value-time". These special tokens represent some commonly occurring entities that may be encoun- tered in all domains.
Next, we incorporate the static prompt section for generating the system response based on the belief state prompt generated in the previous part.
Then, we generate the second dynamic section of the system response prompt using the current user utterance and dialogue history.
Next, we incorporate the static prompt section, based on existing work on generating prompts for belief states. However, these prompts are not adapt- able to subsequent dynamic utterances. Therefore, we adjust them by adding instructions on how to
Finally, based on the parsing results from the slot generator, we obtain the domain information and select the relevant entries from the domain-

---

Attraction
Hotel
Restaurant
Train
dst
inform
success
dst
inform
success
dst
inform
success
dst
inform
success
SOLOIST
86.00
68.00
75.00
51.50
84,00
62.5
81.30
74.20
GALAXY
92.00
62.00
84.50
29.00
76.50
64.50
87.31
73.60
ChatGPT
59.98
95.00
86.00
28.30
89.50
43.00
53.98
95.00
61.50
59.72
83.70
77.70
w/o APG
45,14
84,00
73.00
8,50
48.50
36,50
15.09
61.50
46,50
2.06
81.22
29.95
w/o TSG
50.13
89.00
78.00
24.06
77.00
37.50
47.56
86.50
57.50
51.82
81.73
75.63
ours
62.20
98.00
87.00
28.90
88.50
62.00
54.90
96.50
71.50
61.70
82.74
80.71
Table 4: Results four tasks. SOLOIST is quoted from (Peng al., 2021) and use the 3pt-3.5-turbo-030
version of ChatGPT.
Attraction
Hotel
Restaurant
Train
BLEU
Combined Score
BL.UU
Combined Score
BLEU
Comhined Score
BLEU
Combined Score
SOLOIST
14.60
91.60
10.09
73.34
13.17
86.42
11.90
89.18
GALAXY
9.47
86.47
5.50
62.25
11.68
82.18
6.67
87.
ChatGPT
4.11
94.61
2.12
68.37
3.20
81.45
3,56
84.26
ours
4.89
97.39
2.76
78.01
3.51
87.51
4.98
86.71
Table 5: Results of combined four tasks. Calculated from BLEU, Inform, and Success metrics in Table 4 using the formula Combined Score = (Inform Success)
BLEU BLEU.
special token candidate list form list of candi- date entries. The example prompt is presented in Appendix
The domain and slot infor- mation generated using TSG, and then the static prompt is used to input into ChatGPT for generating the final belief state and system response.
ours w/o APG:
4.4.2 Compared Methods
To demonstrate the effectiveness our proposed method, we compared with several different com- binations of methods for validation. The main methods we employed are as follows:
ours w/o TSG: Using SOLOIST to generate belief states. After that, the prompt is gener- ated using APG and input into ChatGPT to generate system responses.
Using SOLOIST alone generate belief' states and system responses, followed by direct evaluation.
SOLOIST:
These methods evaluated by concatenating the corresponding components and assessing their performance.
ChatGPT: Generating belief states and system responses separately using ChatGPT. Then, the two parts are concatenated for evalua- tion. In contrast to our proposed method in this section, this approach uses static prompts. Specifically, when generating belief states, the dynamic prompt section from our proposed method, which consists of the candidate list based on the output of TSG, replaced with the complete domain-slot-valee list. Similarly, when generating system responses, the dy- namic prompt part from our proposed method, which consists of the candidate list based on the domain information, is replaced with the complete domain-special token list. In other words, this method solely relies on ChatGPT to generate the required information, aiming to verify ChatGPT capable of completing the entire task.
4.4.3 Results
The final evaluation results are presented in Table 4. We use SOLOIST as the baseline. It can be observed that the ours w/o TSG method achieves an average improvement of 5 points. This improve- ment be attributed to the powerful comprehen- sion and generation capabilities of ChatGPT, al- lowing to generate responses that align well with the conversational context. The ChatGPT method point,
alone brings an average improvement indicating its capability to generate belief states effectively. However, we noticed that ChatGPT performs better in generating simple belief states (e.g., in the attraction domain) but struggles with complex belief states (e.g., in the train domain). We speculate that this discrepancy is due com- plex belief states containing more information, and ChatGPT's instability in generating longer pieces of information. Ultimately, our proposed method

---

Attraction
Hotel
Restaurant
Train
Info,
Suce.
BLFUU
Comb.
Info.
Succ.
BLEU
Comb.
Info.
Suce.
BLEU
Comb.
Info.
Succ.
BLEU
Comb.
85,00
69.00
12.90
89.90
74.50
43.50
8.12
67.12
81.00
55.50
12.80
81,05
80.81
64.65
9.96
82.69
ourS
98.00
87.00
4.89
97.39
88.50
62.00
2.76
78.01
96.50
71.50
3,51
87.51
82.74
80.71
4,98
86.71
Table 6: Experimental results the impact of static and dynamic prompts on system response. "L" indicates the substitution of the dynamic prompt segment with a static prompt encompassing all entries.
Attraction
Hotel
Restaurant
Train
ChatciP
Bang et al. 2023
91.00
83.50
90.50
77.27
Pan et al. 2023
93.00
84.00
91.00
78.82
ours
98.00
88,50
96.50
82.74
15.0
Table 7: Comparison of results with different prompts.
12.5
parative analysis of the effects of prompts using the frameworks presented in Bang et al. (2023) and Pan al. (2023).
7.5
5,0
As depicted in Table 7, the dynamic prompt ex- hibits significantly higher performance in the in- form metric compared to existing static prompt methods. This is attributed to the inability of static prompts adapt to different domains.
2.
0.0
Format
goneration
Belief state
Figure 3: Comparison of different methods in terms of the proportion of srg-ffreyeency errors relative to the total test data.
Similarly, to investigate the impact of static and dynamic prompts on system response, we supple- mented our study with more detailed experiments to compare their effects on the final results. Specif- ically, we replaced the dynamic part of the prompt with static prompt containing all entries. As shown in Table using only static prompt has significant impact on success, resulting in de- crease in the combined score.
outperforms the ChatGPT method with an average improvement of points. This indicates our frame- work effectively mitigates the issue hallucination in ChatGPT.
5 Analysis
Necessity of Belief State in TOD. Due to the inability of LLMs to interact extensively with exter- nal knowledge, such querying restaurant avail- ability from DB, still need to retrieve key- words from the belief state to perform DB queries. While there may better approaches in the future, currently, querying external knowledge through the belief state remains a more reliable method.
High-frequency Errors. For the unsuccess dia- logues, analyze high-frreuency errors compar- ing the chatgpt method in the restaurant domain with our proposed method, as shown in Figure Specifically, "format error" is the error that pre- vents the belief state from being parsed, "dialogue generation error" refers to the result of a system response that does not align with logic, and "belief' state error" refers to unmatched content.
Conclusion
Our method significantly reduces the number of evaluation failures caused by format errors owing to the inclusion of precise constraints in the prompt. Furthermore, the proportion of dialogue generation errors is also reduced. Lastly, the occurrence of belief states' errors has been mitigated due to the trainable slot generator.
To address the issue of prompt generation of LLMs in the TOD task, we propose an adaptive prompt generation framework for the comprehensive TOD system which consists of two parts: trainable slot generator (TSG) and adaptive prompt generator (APG). The framework tackles the limitation of fixed prompts in TOD and focuses on both dia- logue understanding and dialogue generation. Ex- perimental results demonstrate that our proposed method significantly improves the performance compared to existing approaches.
Drawback of Static Prompt. While the present study has already included comparison between static and dynamic prompts, more distinctly il- lustrate the differences between existing methods and our proposed approach, we conducted com-

---

Limitations
Philippe Tillet, Felipe Petroski Such, Dave Cum mings, Matthias Plappert, Fotios Chantzis, Eliza- beth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei. Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. 2021. Evaluating large language models trained on code.
Limitations There are several aspects of this article that can be improved in the future:
Result Updates: As ChatGPT continues to evolve, we can continue using newer versions to enhance performance.
More Scenarios: Currently, we have only ex- perimented with single-domain task-oriented dia- logues. In the future, we can improve the model's performance in multi-domain scenarios or incorpo- rate additional information, such as database query information, into the prompts.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay. Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengeheng Yin, Toju Duke, Anselm Levskaya. Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, An- drew M. Dai. Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira. Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Sacta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Tek, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language mod- eling with pathways.
Generalization validation: In the future, we can expand our approach to compare it with other methods in a broader range of tasks and validate its generalization capabilities.
Acknowledgements
The authors would like to thank Xiaoying Zhang for useful discussions. This work is supported by National Key R&D Program of China (Grant No.2022YFF1202444
Major Technology In- novation Program of Hangzhou, China (Grant 2022AIZD01 54), National Natural Science Foun- dation of China (No.62176077 National Natural Science Foundation of China (No.62301066 ...22300666,, Bei- jing Nova Program 20220484161, and the Funda- mental Research Funds for the Central Universi- ties 2023RC72. This work is also sponsored by CCF-DiDi GAIA Collaborative Research Funds for Young Scholars.
Shizhe Diao, Zhichao Huang. Ruijia Xu, Xuechun Li, Yong Lin, Xiao Zhou, and Tong Zhang. 2023. Black- box prompt learning for pre-trained language models.
Tianyu Gao, Adam Fisch, and Dançi Chen. 2021. Making pre-trained language models better few-shot learners.
References
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. A multitask. multilin- gual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity.
Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2022. Ppt: Pre-trained prompt tuning for few-shot learning.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuming.
Pawel Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ra- madan, and Milica Gašić. 2020. Multiwoz large- scale multi-domain wizard-of-oz dataset for task- oriented dialogue modelling.
Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. 2023. Guiding large language models via directional stimulus prompting.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Ka- plan, Harri Edwards. Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sas- try, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter,
Xiao Liu, Yanan Zheng, Zhengxiao Du Ming Ding. Yujie Qian, Zhilin Yang, and Jie Tang. 2021. Gpt understands, too.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller,

---

Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback.
A
Appendix
A.1 Prompt for Belief' State
The various parts of the prompt for obtaining the belief state are presented in Table 8. The dynamic part automatically adapts based different inputs generate the optimal prompt.
Wenbo Pan, Qiguang Chen, Xiao Xu, Wanxiang Che, and Libo Qin. 2023. preliminary evaluation of chatgpt for zero-shot dialogue understanding.
Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayan- deh, Lars Liden, and Jianfeng Gao. 2021. Soloist: Building task bots at scale with transfer learning and machine teaching.
Part
Prompt
Input
(utterance)
user I'd like to visit college in the center of town. could you help me find something interesting? (example)
Timo Schick and Hinrich Schütze. 2021. Exploiting cloze questions for few shot text classification and natural language inference.
Static
user hello, am looking something to do in the west part of town. it should involve multiple sports.
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV au2, Eric Wallace, and Sameer Singh. 2020. Auto- prompt: Eliciting knowledge from language models with automatically generated prompts.
system unfortunately none of those places exist here , any other preferences user : hm, can you tell me about what entertainment venues might be on the west side of town instead?
Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. 2022. Black-box tuning for manguage-modellas---seee-
Sandesh Swamy, Narges Tabari, Chacha Chen, and Rashmi Gangadharaiah. 2023. Contextual dynamic prompting for response generation in task-oriented dialog systems.
belief:: attraction t
belief': attraction type entertainment
(standard prompt)
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng. Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Hongrae Lee, Huaixiu Steven Zheng. Amin Ghafouri, Marcelo Menegali, Yamping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin. Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung- Ching Chang, Igor Krivokon, Will Rusch, Mare Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny So- raker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Ale- jandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Co- hen, Rachel Bemsteen, Ray Kurzweil, Blaise Aguera- Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. 2022. Lamda: Language models for dialog appli- cations.
According to the example, fill the blank represented as "?" of the dialogue state of the last utterance the following dialogue by using the following pre-defined slots and possible values:
Dynamic
(relevant entries)
Domain: attraction, type: l'dontcare', 'park', 'mutliple sports', 'boat', 'college', 'cinema', 'nightclub', 'concerthall', 'swimminggpol"" 'museum', 'entertainment', 'theatre', 'archi- tecture']
Domain: attraction, area I'dontcare', 'centre', eeast', 'north', 'south,, 'west']
(TSG output)
(belief attraction type =? area =?
Output
(belief: attraction type college ter
Pengfei Zhang, Tingting Chai, and Yongdong Xu. 2022. Adaptive prompt learning- analysis.
Table Prompt for belief state
based based few-shot sentiment
Rui Zhang, Yajing Sun, Jingyuan Yang, and Wei Peng- 2023a. Enowledge-augmeetee frame semantic pars- ing with hybrid prompt-tuning
Xiaoying Zhang, Baolin Peng. Kun Li, Jingyan Zhou, and Helen Meng. 2023b. Sgp-TOD: Building Task Bots Effortlessly via Schema-Guided L.LM Prompt- ing. Xrxiv, abs/2305.000 09067.

---

A.2 Prompt for System Response
Similarly, the prompt for obtaining the system re- sponse is presented in Table 9. The dynamic part also selects relevant entries based on the output of TSG.
Part
Content
Input
(utterance)
user I'd like visit college the center of town. could you help me find something interesting?
Static
(example)
user hello, am looking for something do in the west part of town. it should involve multiple sports.
system unfortunately none of those places exist here any other preferences user hm, can you tell me about what entertainment
might be the west side of town instead?
system there a fun place called [attrac- tion, name] [attraction ).
(standard prompt)
According to the example, complete the system without generating unnecessary elements. Special tokens in the system can used:
Dynamic
(relevant entries)
[attraction address]', "[attraction jareu
[[attraction pacpiijo___po_______ [[ut- traction, postoode]", "[attruction pricerange]', ptttractioo_prccraanne]]
"Tattactton_ [Tatractton_refeeeneee]] reterreeffeeeee... "[value_day dayl', vvalue_placc]'_. "value_time]] system: sure, there are several colleges in the center of town. of the interesting ones is [attraction_ name] located at [attrac- tion address).
'[value_com _coun]].
Output
Table 9: Prompt for system response


Reviews:
[{"review_id": "4dxBGCgZGN", "reviewer": "Reviewer_uoyj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Reviewer#1:__\n\nThanks for your careful and valuable comments. We will explain your concerns point by point.\n***\n\n__Q1:__ Why not use LLMs for implementing the TSG module also? This reasoning is not available. It's unclear whether TSG is the best option for generating the slots, as the success of the next phase is dependent on this. So, comparison with baselines for TSG should be done more thoroughly.\n\n__A1:__ Using black-box LLMs as the TSG module to generate domain and slot information is not very effective. \nAs shown in the table below, our proposed TSG outperforms the black-box LLM-based slot generator by 10.6%. It can be seen that black box is difficult to achieve ideal results in complex domain.\n\nWe analyze that the reason behind is that the black-box LLMs can not be directly fine-tuned, making them unable to reliably generate the domain and slot information specific to our domain. Additionally, similar phenomena were also observed in other related work [1]. \n\nTherefore, they may generate irrelevant slot generation and degrade the performance.\n\nUsing white-box LLMs might potentially reduce the amount of data required for fine-tuning. However, due to time and computational limitations, we have not fully investigate this approach. We will further investigate in future work.\n\n[1]A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding - Pan, Wenbo, et al\n\n|\t| Attraction\t| Hotel\t| Restaurant\t| Train |\n| :------- | :-------: |  :-------:  |  :-------:  |  :-------:  |\n| __ours__ | __58.27__\t | __33.02__\t | __55.81__ | \t__63.78__ | \n| ChatGPT |\t54.80 | \t30.76 | \t45.31 | \t59.76 | \n| PPTOD\t | 58.11 | \t32.76\t | 54.34 | \t62.90 | \n***\n__Q2:__ Unfair comparison with baseline in response generation task: The SOLOIST baseline is quite old now. Aren't there more recent models proposed for this? Please compare with other baselines from here: https://github.com/budzianowski/multiwoz#combined-score--inform-success05--bleu-1\n\n__A2:__ We compare with more recent state-of-the-art methods as shown in the table below. We observe from the results that our proposed method significantly outperforms these SOTA methods. \n|  | DST   | Inform | Success | BLEU  | Combined score |\n|:-----------| :-------: | :-------: |  :-------:  |  :-------:  |  :-------:  |\n| _Attraction_ | | | | | |\n| __ours__       | 62.20 | 98.00  | 87.00   | 4.89  | __97.39__ |\n| SOLOIST    | /     | 86.00  | 68.00   | 14.60 | 91.60 |\n| GALAXY     | /     | 92.00  | 62.00   | 9.47  | 86.47 |\n|----------|-------|--------|---------|-------|-------|\n| _Hotel_   | | | | | |\n| __ours__     | 28.90 | 88.50  | 62.00   | 2.76  | __78.01__ |\n| SOLOIST  | /     | 75.00  | 51.50   | 10.09 | 73.34 |\n| GALAXY   | /     | 84.50  | 29.00   | 5.50  | 62.25 |\n|----------|-------|--------|---------|-------|-------|\n| _Restaurant_  | | | | | |\n| __ours__       | 54.90 | 96.50  | 71.50   | 3.51  | __87.51__ |\n| SOLOIST    | /     | 84.00  | 62.50   | 13.17 | 86.42 |\n| GALAXY     | /     | 76.50  | 64.50   | 11.68 | 82.18 |\n|----------|-------|--------|---------|-------|-------|\n| _Train_   | | | | | |\n| ours    | 61.70 | 82.74  | 80.71   | 4.98 | 86.71 |\n| SOLOIST | /    | 80.31  | 74.24   | 11.90 | __89.18__ |\n| GALAXY  | /    | 87.31  | 73.60    | 6.67 | 87.13 |\n***\n__Q3:__ Results on generation metrics are not shown. (BLEU, METEOR or newer metrics that can capture semantics e.g. DEB )\n\n__A3:__ We include BLEU metrics in the table above. The results show that our proposed method outperforms baseline methods on Combined score. However, LLMs produce a variety of open-ended responses, and metrics like BLEU and METERO cannot fully reflect the model's capabilities e.g.[2], so we excluded it from previous evaluations. We will explore more equitable metrics in our future work, like using LLMs.\n\n[2] USR: An unsupervised and reference free evaluation metric for dialog generation - Mehri S, Eskenazi M.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The method is innovative. \n- The idea of using a trainable slot generator to provide interpretable priors for guiding adaptive prompt generation is novel. \n- Experiments on the MultiWOZ dataset show the proposed framework outperforms prior SOLOIST and ChatGPT baselines on dialogue evaluation metrics.", "reasons_to_reject": "* Why not use LLMs for implementing the TSG module also? This reasoning is not available. It's unclear whether TSG is the best option for generating the slots, as the success of the next phase is dependent on this. So, comparison with baselines for TSG should be done more thoroughly.\n* Unfair comparison with baseline in response generation task: The SOLOIST baseline is quite old now. Aren't there more recent models proposed for this? Please compare with other baselines from here: https://github.com/budzianowski/multiwoz#combined-score--inform-success05--bleu-1\n* Results on generation metrics are not shown. (BLEU, METEOR or newer metrics that can capture semantics e.g. DEB [1])\n\n[1] Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining - Sai et al.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes an adaptive prompt generation framework for task-oriented dialogue systems. The key ideas are:\n\n- A trainable slot generator (TSG) is used to generate domain and slot information from the belief state. This reduces annotation cost and improves stability.\n- An adaptive prompt generator (APG) uses the output of TSG to generate prompts for the LLM to produce the full belief state and system response. It maintains query tables to select relevant prompt candidates dynamically.\n- Experiments on MultiWOZ show the framework outperforms existing methods in dialogue evaluation metrics. The TSG improves accuracy of slot prediction. The APG mitigates LLM hallucination by constraining prompts.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HTKhkRrvd5", "reviewer": "Reviewer_E3rq", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Reviewer#2:__\n\nThanks for your careful and valuable comments. We will provide an explanation for this point.\n***\n\n__Q1:__ The authors may consider running further experiments especially where slots are complex and there may be more than one belief states\n\n__A1:__ Due to time and space constraints, we will address this aspect in our future work. We plan to explore more complex scenarios, including multi-domain scenarios or scenarios that involve additional information, such as querying databases.\n\nIn addition, we have conducted supplementary experiments on more datasets. For specific details, please refer to A2 to reviewer#3.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) Novel approach consisting of static and dynamic prompt generation - prompts for belief state and for system response ", "reasons_to_reject": "1) The authors may consider running further experiments especially where slots are complex and there may be more than one belief states", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper describes a method of dynamic prompting for multi-turn task-oriented dialog generation. Two constituent methods are a Trainable Slot Generator (TSG) - which generates the domain and slot information for the belief state and an Adaptive Prompt Generator for the LLM to generate an appropriate dialogue response.  The framework was evaluated in MultiWOZ2.0 dataset. The results are compared with SOLOIST and ChatGPT methods (for slots and belief state prediction).", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DAdhGumsk9", "reviewer": "Reviewer_RXui", "summary": "", "questions": "1. Can you provide some descriptions of the belief state and what is its structure?\n2. Are there any ablation studies to investigate the effect of static prompt, dynamic prompt and the prompt for system response to the final results?\n3. How to measure the format error and dialogue generation error?\n\nAnother question about this area: This paper introduces an Adaptive Prompt Generation for solving task-oriented dialogue tasks, and it introduces many previous experiences (e.g., Belief state) in TOD tasks. But one question is: after we have LLMs (e.g., ChatGPT), do we still need to follow these traditional settings to solve TOD tasks?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Reviewer#3:__\n\nWe appreciate your thoughtful and invaluable comments. We will explain your concerns point by point.\n***\n\n__Q1:__ The proposed Trainable Slot Generator is only validated on the SOLOIST model, which is incremental and minor. Besides, why incorporating contrastive learning can improve efficiency should be explained.\n\n__A1:__ \n\n**Effectiveness of TSG**\n\nIt is worth noting that the proposed TSG can be combined with any other TOD model apart from SOLOIST. To verify its effectiveness, we replace SOLOIST with PPTOD, and the results below demonstrate the effectiveness of the proposed TSG.\n\n| |Attraction | Hotel | Restaurant | Train |\n|:------|:------:|:-------:|:------------:|:-------:|\n| PPTOD w/o TSG    | 57.71 | 31.89      | 54.12 | 61.78 |\n| __PPTOD w/ TSG__      | __58.11__ | __32.76__      | __54.34__ | __62.90__ |\n\n**Effectiveness of contrastive learning**\n\nContrastive learning enhances training effectiveness and efficiency because it constructs negative domain and slot examples, allowing the model to learn from incorrect results, thus accelerating model convergence. The table below compares the original method with the one using contrastive learning.\n\n|                        | Attraction | Hotel | Restaurant | Train |\n|:--------------------------:|:------------:|:-------:|:----------:|:------:|\n| w/o Contrastive Learning | 58.18      | 32.06 | 55.21      | 63.25 |\n| __w/ Contrastive Learning__  | __58.27__      | __33.02__ | __55.81__      | __63.78__ |\n\n***\n__Q2:__ It seems that the proposed TSG is tuned on a small amount of data, and I am worried about whether the proposed method can be generalized to other domains. The motivation of this paper is ok, but if the proposed method is trained on the MultiWOZ dataset and can only solve tasks in MultiWOZ, its value will be minor.\n\n__A2:__ In order to verify whether the proposed method can be generalized to other domains, we conduct experiments on Camrest676 dataset and the result is shown in the table below.  We choose SOLOIST as the baseline method. The result shows that the proposed TSG is not limited to MultiWOZ dataset, but can also bring consistent improvements on other datasets.\n\n| _Camrest676_ | Inform | Success | BLEU  | Combined score |\n|:----------|:------:|:-------:|:-----:|:--------------:|\n| w/o TSG    | 73.88  | 72.22   | 13.11 | 86.16          |\n| __w/ TSG__     | __76.23__  | __74.10__   | __13.81__ | __88.98__          |\n\n***\n__Q3:__ This paper argues that using the same prompt in LLM cannot effectively capture useful belief state information. Can you provide some experiments to validate this point? And whether the belief state is necessary in TOD tasks when we have LLMs?\n\n__A3:__ \n\n**Drawback of static prompt**\n\nIn the original paper, 'ours w/o APG' refers to experimental results using a static prompt, which utilizes the static part from Appendix A.1 along with complete entries. In addition to this, we conducted several sets of experiments, as shown in the table below, to compare the results when using the same prompts, as referenced in the recent paper.\n\n| _Inform_ | Attraction | Hotel | Restaurant | Train |\n|:------:|:----------:|:-----:|:----------:|:-----:|\n| __ours__   | __98.00__      | __88.50__ | __96.50__      | __82.74__ |\n| [1]    | 91.00      | 83.50 | 90.50      | 77.27 |\n| [2]    | 93.00      | 84.00 | 91.00      | 78.82 |\n\n[1]A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity - Bang, Yejin, et al. \n\n[2]A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding - Pan, Wenbo, et al\n\n**Necessity of belief state in TOD**\n\nDue to the inability of LLMs to interact extensively with external knowledge, such as querying restaurant availability from a DB, we still need to retrieve keywords from the belief state to perform DB queries. While there may be better approaches in the future, currently, querying external knowledge through the belief state remains a more reliable method.\n\n***\n__Q4:__ Can you provide some descriptions of the belief state and what is its structure?\n\n__A4:__ The standard format of a belief state is __{domain slot1:value1; slot2:value2; ...}__. However, the improved method used in this paper employs a belief state that does not include values of slots, with the format __{domain slot1; slot2; ...}__. In the actual algorithm, some formatting constraints are applied based on LLMs to reduce generation errors, such as __{domain slot1=?; slot2=?; ...}__.\n***\n__Q5:__ Are there any ablation studies to investigate the effect of static prompt, dynamic prompt and the prompt for system response to the final results?\n\n__A5:__ The results with static prompts can be referred to in the original paper's 'ours w/o APG' and A3. Below are the additional results from the ablation study of prompts for system responses, and it can be observed that the lack of prompts for system responses leads to a significant decrease in results.\n\n|                                 | Inform | Success | BLEU | Combined score |\n|:--------------------------------:|:------:|:-------:|:----:|:-----:|\n| __Attraction__                       |        |         |      |       |\n| w/o prompts for system responses | 85.00     | 69.00      | 12.90 | 89.90  |\n| __w/ prompts for system responses__  | 98.00     | 87.00      | 4.89 | __97.39__ |\n|----------------------------------|------|------|------|-------|\n| __Hotel__                      |      |      |      |       |\n| w/o prompts for system responses | 74.50 | 43.50 | 8.12 | 67.12 |\n| __w/ prompts for system responses__  | 88.50 | 62.00   | 2.76 | __78.01__ |\n|----------------------------------|------|------|------|-------|\n| __Restaurant__                       |      |      |      |       |\n| w/o prompts for system responses | 81.00   | 55.50 | 12.80 | 81.05 |\n| __w/ prompts for system responses__  | 96.50 | 71.50 | 3.51 | __87.51__ |\n|----------------------------------|-------|-------|------|--------|\n| __Train__                       |       |       |      |        |\n| w/o prompts for system responses | 80.81 | 64.65 | 9.96 | 82.69   |\n| __w/ prompts for system responses__ | 82.74 | 80.71 | 4.98 | __86.71__ |\n\n***\n__Q6:__ How to measure the format error and dialogue generation error?\n\n__A6:__ We conduct manual analysis of failed dialogues in the test set to determine the reasons behind the failures. \n* If a failure is due to a formatting error that prevents the belief state from being parsed, it is categorized as a 'format error'.\n* If the failure is a result of a system response that does not align with logic, it is categorized as a 'dialogue generation error'.\n* If a belief state is generated that does not match the content, it is categorized as a 'belief state error'.\n***\n__Q7:__ Another question about this area: This paper introduces an Adaptive Prompt Generation for solving task-oriented dialogue tasks, and it introduces many previous experiences (e.g., Belief state) in TOD tasks. But one question is: after we have LLMs (e.g., ChatGPT), do we still need to follow these traditional settings to solve TOD tasks?\n\n__A7:__ This question is similar to Q3, and you could refer to A3 for a related response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper proposes an adaptive prompt generation framework to solve task-oriented dialogue tasks.", "reasons_to_reject": "1. The proposed Trainable Slot Generator is only validated on the SOLOIST model, which is incremental and minor. Besides, why incorporating contrastive learning can improve efficiency should be explained.\n2. It seems that the proposed TSG is tuned on a small amount of data, and I am worried about whether the proposed method can be generalized to other domains. The motivation of this paper is ok, but if the proposed method is trained on the MultiWOZ dataset and can only solve tasks in MultiWOZ, its value will be minor. \n3. This paper argues that using the same prompt in LLM cannot effectively capture useful belief state information. Can you provide some experiments to validate this point? And whether the belief state is necessary in TOD tasks when we have LLMs?", "questions_to_authors": "1. Can you provide some descriptions of the belief state and what is its structure?\n2. Are there any ablation studies to investigate the effect of static prompt, dynamic prompt and the prompt for system response to the final results?\n3. How to measure the format error and dialogue generation error?\n\nAnother question about this area: This paper introduces an Adaptive Prompt Generation for solving task-oriented dialogue tasks, and it introduces many previous experiences (e.g., Belief state) in TOD tasks. But one question is: after we have LLMs (e.g., ChatGPT), do we still need to follow these traditional settings to solve TOD tasks?", "paper_topic_and_main_contributions": "Large Language Models (LLMs) have demonstrated impressive performance in various tasks. However, it usually requires different prompt designs for different specific tasks. Therefore, in this paper, the authors propose an adaptive prompt-generation framework to automatically generate prompts for task-oriented dialogue. More specifically, the authors design a trainable slot generator (TSG) to generate domain and slot information in the belief state, and then propose an adaptive prompt generation (APG) to utilize the prior knowledge to generate prompts for LLM. Experimental results also demonstrate the proposed method.", "excitement": 2, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces an adaptive prompt generation framework designed to enhance the performance of off-the-shelf Large Language Models (LLMs) in task-oriented dialogue systems. The proposed framework incorporates a trainable slot generator that produces domain-specific slots, which are then utilized for subsequent prompt generation. This adaptive prompt generator is capable of generating belief states and system responses. The framework's effectiveness was evaluated using the MultiWOZ 2.0 benchmark and tested in conjunction with SOLOIST and ChatGPT.

The soundness scores were recorded as (2, 3, 2). The reviewers expressed multiple concerns regarding the paper's experimental fairness, highlighting:

1. The use of an outdated off-the-shelf LLM.
2. The absence of generation metrics.
3. The lack of complex-slot tasks.
4. The lack of an ablation study for the proposed sub-techniques.

In response, the authors presented new experimental results during the rebuttal phase, showcasing updated LLMs, new generation metrics, an additional TOD benchmark, and ablation studies. As a result, the reviewers reached a consensus that, if these rebuttal points are incorporated into the final paper, it would substantially strengthen the work.

As for the excitement scores, they were (3, 3, 2). While all reviewers recognized the novelty of the approach, they also noted that some of the sub-techniques were inadequately supported in the initial submission. Consequently, the overall excitement level for the paper is moderate.|meta review
Decision: Accept-Findings