Multi-Source Probing for Open-Domain Conversational Understanding
Authors: Yuanxi Li
Hao Zhou
Jie Zhou
Minlie Huang
Venue: EMNLP 2023 Main (2023)
Keywords: Open-domain dialogue systems
Dialogue comprehension
Dialogue probing
Prompt Learning

Abstract: Dialogue comprehension and generation are vital to the success of open-domain dialogue systems. Although pre-trained generative conversation models have made significant progress in generating fluent responses, people have difficulty judging whether they understand and efficiently model the contextual information of the conversation. In this study, we propose a Multi-Source Probing (MSP) method to probe the dialogue comprehension abilities of open-domain dialogue models. MSP aggregates features from multiple sources to accomplish diverse task goals and conducts downstream tasks in a generative manner that is consistent with dialogue model pre-training to leverage model capabilities. We conduct probing experiments on seven tasks that require various dialogue comprehension skills, based on the internal representations encoded by dialogue models. Experimental results show that open-domain dialogue models can encode semantic information in the intermediate hidden states, which facilitates dialogue comprehension tasks. Models of different scales and structures possess different conversational understanding capabilities. Our findings encourage a comprehensive evaluation and design of open-domain dialogue models.

Multi-Source Probing for Open-Domain Conversational Understanding
Yuanxi Li', Hao Zhou', Jie Zhou', Minlie Huang
University of Illinois at Trbana-Champalle IL, USA
Patterrn Recognition Center, WeChat AI, Tencent Inc., China
'hhe CoAI group, Tsinghua University, Beijing, China DDepartment of Computer Science and Technology, Tsinghua University, Beijing, China edu, 2 (tuxzhou, witht omzhou @tencent. com "yuanxi3@i11imool Sechuunggtsiinghhee edu, on
Abstract
2020). While these generative dialogue models can produce fluent responses, they still have many limitations in conversational understanding (Salch et al., 2020; Li et al., 2016), leading to irrelevant, repetitive, and generic responses (Li et al., 2017 Welleck et al., 2019; Cho and Saito, 2021).
Dialogue comprehension and generation vi- tal to the success of open-domain dialogue sys- tems. Although pre-trained generative conver- sation models have made significant progress in generating fluent responses, people have dif- ficulty judging whether they understand and efficiently model the contextual information of the conversation. In this study, we propose Multi Multi-Soure SSurce Probing (MSP) method prohe the dialogue comprehension abilities of open- domain dialogue models. MSP aggregates fea- tures from multiple sources accomplish di- verse task goals and conducts downstream tasks in a generative manner that is consistent with dialogue model pre-training to leverage model capabilities. We conduct probing experiments on seven tasks that require various dialogue comprehension skills, based on the internal rep- resentations encoded by dialogue models. Ex- perimental results show that open-domain dia- logue models can encode semantic information the intermediate hidden states, which facili- tates dialogue comprehension tasks. Models of different scales and structures possess different conversational understanding capabilities. Our findings encourage a comprehensive evaluation and design of open-domain dialogue models.
Research on conversational understanding can provide holistic evaluation of dialogue mod- els (Parthasarathi et al., 2020), contributing to the deployment and design of models. However, the analysis of open-domain dialogue models on con- versational understanding remains a controversial topic (Dinan et al., 2020; Tao et al., 2018; Ji et al., 2022). Some work (Sankar et al., 2019; Saleh et al., 2020) demonstrates that dialogue models have dif- ficulty in capturing the conversational dynamics in the dialog history and struggle with conversational understanding tasks such question answering, contradiction inference, and topic determination. In contrast, Parthasarathi al. (2020) affirms the conversational understanding of open-domain dia- logue models and indicates recurrent dialogue mod- els perform better than transformer-based models. These studies have limitations in probing meth- ods and experimental settings (Ravichander et al., 2020), making them not applicable to present large- scale open-domain dialogue models.
1 Introduction
Conversational understanding and response genera- tion are critical for the success of open-domain dia- logue systems. Recently, pre-trained spen-domant dialogue models, including DialoGPT (Zhang et al., 2020), BlenderBot (Roller et al., 2020), and Meena (Adiwardana al., 2020), have achieved im- pressive progress in wide range of conversational tasks. The pre-trained model has become a solid foundation for the downstream fine-tuning pro- cess, such generating empathetic (Zhong al., 2020) and persona-coherent (Wolf al., 2019b) responses, delivering knowledge- -ooledge-grounded conver- sations (Zhao et al., 2020; Wu et al., 2021), and completing task goals (Wu et al., 2020; Peng et al.,
In this work, propose Multi-Souree Probing (MSP) method to examine the conversational un- derstanding ability of open- dialogue mod- els. Specifically, MSP conducts dialogue compre- hension tasks generative manner, which is co- herent with the pre-trained dialogue generation task to take full advantage of model capabilities. In addi- tion, considering that different tasks require various information of the dialogue context, MSP aggre- gates features from multiple sources to accomplish diverse tasks. We propose multi-source cross- attention mechanism to extract local features and adopt a late fusion module to incorporate global features. With the help of these components, MSP

---

has the capability to evaluate generative dialogue models more accurately and comprchensively.
tions (Zhao et al., 2020; Wu al., 2021) and com- pleting task goals (Wu al., 2020; Peng et al., 2020). These dialogue models are capable of flu- ent response generation, but they still have many conversational understanding limits (Saleh et al., 2020; Li et al., 2016) that result in irrelevant, repet- itive, and generic responses (Serban et al., 2017; Li et al., 2017a; Welleck et al., 2019; Cho and Saito, 2021). Several studies (Sankar al., 2019; Bao et al., 2020) point out that generative dialogue mod- els don't always properly exploit the existing dialog history and they are yet unable to understand the context to provide coherent and engaging conversa- tions. Some work (Komeili et al., 2022: Zhou et al.. 2018) 1) introduce external knowledge to enhance conversational understanding, which facilitates the generation of relevant and coherent responses.
To expose the validity and reliability of our method, we conduct comprehensive experiments to compare the conventional MLP-based probing approach with MSP. Furthermore, we also set up series of ablation experiments to verify the cessity of the multiple-source attention mechanism and the late-fusion module.
In order to verify that our method also applies to even more massive scale models, we extend our MSP method to provide insight into larger pre- trained dialogue models. We found that larger models have stronger extraction capability for the information inferred via the pre- mr--trineed encoder. Our study reveals three critical findings:
Different from the conclusion reached by the vanilla probing method, we find through MSP that encoder hidden states contain more infor- mation than original embeddings in pretrained dialogue models, as reflected by the higher ac- curacy obtained on our probing tasks.
2.2 Probing Method
With the growing demand for natural language un- derstanding, the probing method has been widely employed in machine translation (Belinkov et al., 2017, 2018; Dalvi et al., 2017; Yawei and Fan, 2021) and knowledge attribution (Alishahi et al., 2017; Beloucif and Biemann, 2021) to assess the linguistic properties of sentence representations learned by models.
Generative dialogue models with single de- coder have worse understanding of the con- versation than deeoder-decoder. based mod- els, as the uni-directional attention mechanism only encodes partial context (content before each token) information for tokens, leading to asymmetric representations of dialogue his- tory and current utterance.
Although several studies have been proposed to probe the conversational understanding capabil- ity of open-domain dialogue models (Dinan et al., 2020; Tao al., 2018; Ji al., 2022), this research area is still controversial. According to certain re- search (Sankar et al., 2019; Salch et al., 2020; Das et al., 2020), conversational comprehension tasks including question answering, contradiction infer- ence, and subject determination pose challenges for dialogue models in terms of capturing the conver- sational dynamics in the dialog history. In contrast, Parthasarathi et al. (2020) validates conversational comprehension of open-domant dialogue models and demonstrates that recurrent dialogue models outperform transformer-based models. addition, previous work (Saleh et al., 2020; Alt et al., 2020; Ravichander et al., 2020; Parthasarathi et al., 2020; Richardson et al., 2020) usually adopted a certain probing method to perform model-level analysis of dialogue systems, thus they lacked an exhaustive comparison of different probing methods. These studies have reached opposite conclusions for two major reasons. First, these methods usually adopt a shallow Multi-Layer Perceptron (MLP) as the clas-
Dialogue models can capture the dialogue structure in conversational understanding. Larger dialogue models have a better under- standing of conversational information and achieve higher accuracy on probing tasks.
2 Related Work
2.1 Open-domain Conversational Models
Recently, open-domain conversation systems have been largely advanced due to the increase of dialogue corpus and the development of large- scale pre-training (Devlin et al., 2018; Rad- ford et al., 2019; Brown et al., 2020). Pre- trained open-domain dialogue models, such as Di- aloGPT (Zhang al., 2020), BlenderBot (Roller al., 2020) and Meena (Adiwardana al., 2020), demonstrate outstanding conversation skills, in- cluding empathetic (Zhong et al., 2020) and persona-coherent (Wolf et al., 2019b) response gen- eration, delivering mowleede--rounned conversa-

---

Label
aill PeRK
Locas
Late
Pro trained Macel
MLP
Pau-mirad Madell
Promps-Patiing
Pro traned Model
Multi Source Proping
(
(a) Vanilla Probing
(b) Prompt-based Probing
(c) Multi-Source Prohing
Figure 1: Overview of three probing methods. The MSP method introduces the multi-soure source attention mechanism integrate local features from multiple sources and the late fusion module capture global features, where probing tasks are conducted a generative manner, which consistent with the objective of the pre-trane pre-training task.
sifier, which cannot fully utilize the information encoded in intermediate representations to conduct probing tasks. Second, the experimental settings are also insufficient (Ravichander et al., 2020), in- cluding the probing tasks and probed model scales.
concatenate the two averaged representations to get the final feature Proobing for probing tasks, where the concatenation operation is denoted by O. The process is defined as follows:
Rhistory
3 Methodology
c
1=1
3.1 Vanilla Probing Approach
R-1
(t)
(1)
For the vanilla probing method, we first use pre- trained generative dialogue models to extract en- coder hidden states and word embeddings corre- sponding to the probing task texts and then feed the extracted representations to two-layer Multi- Layer Perceptron (MLP) classifier.
Σ
Ct
i=
1
Gp
mm)
Reurrett
(2)
Gp
R Rprobing
Rhistory
Reurrents
(3)
In this way, we calculate the accuracy of prob- ing tasks, which is empirically assumed to reflect the ability of the corresponding model to capture information that is beneficial for the goal of dia- logue understanding. Here only the parameters of the classifier are trainable during the training of probing tasks, with encoder parameters kept fixed. The vanilla probing method is shown in Fig (a).
3.2 Multi-Source Probing Approach
The prior MLP-based approach has been consid- ered fairly intuitive to detect if the dialogue model captures pertinent information in the encoder states. However, it is difficult to effectively utilize the in- formation encoded by the dialogue models using these approaches due to the divergent objectives of downstream probing tasks and dialogue generation during pre-training (Liu al., 2021; Schick and Schütze, 2021).
First, we extract inner states from the encoder to get the word-level representations on word em- beddings and encoder states in the entire prob- ing task input
To address this issue, we propose a Hulti-Souree Probing (MSP) method to probe the dialogue com- prehension abilities of open-domain dialogue mod- els. MSP conducts probing tasks in a generative manner, which is consistent with the pre-trained task to take full advantage of model capabilities. As various probing tasks require information from different aspects of the dialogue, which may differ greatly from the dialogue generation task, we pro- pose a multi-source attention mechanism to aggre- gate features from multiple sources to accomplish diverse tasks.
[u,,..., un). with
[w]") ...,, [n)] denoting the tokens in the nea ut- terance, where denotes the word numbers in the utterance u, and w." is the word embedding of the 4es word (0(") in the utterance uy-
We require utterance-level representations for probing where mixed synthesized information is present since the probing tasks rely on high-level reasoning. To get Ristory and Reurvent- we inde- pendently averaged the representations correspond- ing to historical and contemporary utterances. Then

---

Moreover, the application of potential under- standing capabilities that might be encoded in the decoder parameters also lacking MLP-based probing methods, while MSP reapplies the dis- carded information and provides better modeling of contextual information. The overview of MSP is presented in Figure 1(c).
attention by the history-source and current-source attention functions. While the integrated-sourg cross-attention mask is built to gather information from the full context of the dialogue.
Late Fusion
3.2.1 Continuous Prompt Learning
Pro tr-ttranndd Model
Multi Source Probing
Continuous (or soft) prompt learning is one of the central parts in our MSP method, which is shown in Figure I(b), as the generative approach of probing classification is considered to be more consistent with the goal in the pretraining phase, allowing the model be more adaptable.
H
(a) Dialogue History Prohing
Late Fusion
Specifically, we set the Prompt-Template as a sequence of different soft tokens for decoder in- put. Each soft token has a unique word embedding that is constantly adjusted and updated during the training stage. Consistent with the vanilla prob- ing setting, the transformer decoder is finetuned in MSP, while the encoder parameters are fixed.
Pre-trained Model
Soutre Probing
(b) Current Utterance Probing
SUL
Late Fusion
Since our probing task is based on classification, a verbalizer (Hu et al., 2022) class is constructed here to project the original probability distribution over the whole vocabulary to the set of label words given by the probing task.
Pre-trained Model
Multi-Source Probing
SHL
Integrated Context Probing
3.2.2 Multi-Source Attention
Figure 2: Overview of multi-source attention.
Different downstream probing tasks have various focuses on the location of the required informa- tion (Sankar et al., 2019), while the general atten- tion module of dialogue models tends fail in locating and extracting information from multiple aspects and sources a fine-graine grained way.
In Prompt- Template,, the three consecutive to- kens are grouped into one combined unit, which is designed to perform a round of coverage during the probing process. We assign the three soft tokens in the last combined unit as the prediction position where the averaged logit passed to the subsequent verbalizer function, and then we can calculate the loss for the corresponding classes. This step is spe- cially formulated to fuse the information captured by various soft-token representations that are fo- cused independently on the history, current, and entire section of the input text.
Therefore, to avoid overshadowing the key infor- mation contained in the dialogue context when the significance of information is unevenly distributed, we propose multi-soure, attention mechanism, by using multiple -rtss-attention masks corresponding different sources through the decoding process, to generate more reasonable attention that can ex- tract relevant local features from different sources for probing classification.
The decoder fe updates the hidden states H, conditioned on the past decoder states H with self-attention. the soft token embedding S,, well the encoded representations E, with cross- attention, where k could take values in K. [history, current, integrated), as follows:
As shown in Figure 2, the multi-source atten- tion module takes turns allocating different cross- attention to different parts of encoded representa- tions for soft prompt tokens. We adopt three types of attention masks operate the multi-source cross attentions, which are separately called history- source, current-source, and integrated-source. Only the relevant portion of the dialogue is given
fo (H S.. Emis) == 0 (mod 3)
H:
fo (H... S, Egar
(mod 3)
(4)
fo (H E
(mod 3)

---

3.2.3 Late Fusion
Schema-Guider Dialog dataset (SGD) (Rastogi et al., 2020) is an intent-tracking task that requires reasoning over multiple turns of dialogue. SNIPS (Coucke et al., 2018) is an intent classifica- tion task with crowd-sourced, single-turn queries labeled for intent.
Previous studies (Vig, 2019; Vig and Belinkov, 2019) have shown that the attention mechanism is sensitive to local features, while global features of conversations should also be considered to pro- duce a comprehensive representation.
ScenarioSA (Zhang et al., 2019) is a sentiment classification task with turn-level sentiment labels and inputs from multi-turn, open-ended dialogue.. Daily DailyDialog Topic uses dialogues from the Daily- Dialog dataset (Li et al., 2017h) to create probing task where the goal is to make inferences about the topic conversations (Salch et al., 2020).
Thus we introduce the late fusion module as a merging strategy for information integration to gen- erate a comprehensive representation from encoder hidden states, which is combined with the probing decoder states after passing through an MLP layer with dropout. Late fusion is designed to capture the global information encoded by the language model which serves the role of a complement to the probing decoder aimed at extracting precise local features, thus allowing a higher sensitivity to various features of the linguistic information.
4.2 Probing Methods
Vanilla MLP-based Probing adopts two-layer Multi-Layer Perceptron classifier (Salch et al.. 2020; Parthasarathi et al., 2020), which takes the word embedding and encoder states correspond- ing to the conversation context of the pre-trained dialogue models input.
Here we implement the late fusion by averag- ing the encoder hidden states E, that focus the desired section of the text, where k could take K. (history, current, integrated). We would receive the representation L after the late fusion, and A₃ is the final representation obtained by com- bining the decoder hidden states H₃ and the result from late fusion module.
Prompt-based Probing applies prompt learning as the principle for performing probing tasks. During training, only the embeddings of soft prompt tokens and the verbalizer parameters are finc-tuned while the pre- pre-trained encoder and decoder parameters are fixed (Liu et al., 2021).
L (EV)),
(5)
Multi-Source Probing is the proposed approach, which is characteristic of its multi-source attention and late fusion module. The parameters of the verbalizer, prompt token embeddings, and decoders are fine-tuned during training.
A₁
H: Lg.
(6)
The probability of Par(y)z) with probing label y is calculated as:
(y( = softmax(V(avg((A) (n)
K)))))
4.3 Open-domain Dialogue Models
(7)
We first train three widespread generative dialogue models for 20 epochs on DailyDialog dataset (Li et al., 2017b), using the Maximum-likelihodd objective(Sutskeee et al., 2014). Specifically, train Transformer from scratch and fine-tune BlenderBot-small and DialoGPT-small with pre- trained parameters on Daily for fair com- parison, as adopted prior work (Saleh et al.. 2020).
where V is the language model head that projects the raw logits to the label space. Here is the num- ber of combined units in the soft token sequence.
4 Experiment
4.1 Probing Tasks
TREC (Li and Roth, 2002) is question classifi- cation dataset consisting of questions labeled with relevant answer types. The task aims to determine the information category the question is requesting. NLI (Welleck et al., 2019) is natural language inference task consisting of dialog turns with entailment, contradiction and neutral labels. MultiWOZ. (Eric et al., 2020) is a multi-domain, goal- coal-directed conversational dataset for exploring natural language comprehension.
Besides, we adopt BlenderBot-meedium [400M] and DialoGPT-medium [345M] without fine-tuning for the inspection of larger pre-trained models.
Transformer (Vaswani et al., 2017) is typical language model with multiple attention layers. We implement in the form of encoder-decoder struc- ture, with a 2-layer encoder and 2-layer decoder. Ble (Roller et al., 2020) is a pre-trainne trained dialogue model based on the encoder-decoder ar-

---

Method
TREC
DNLI
MWOZ
SGD
SNIPS
SSA
Topic
Transformer
Emb.
83.1
69.9 u.79
92. fo.as]
69.91o.sa
27.9
77.1
53.31g.ol
MLP
Enc.
(9.7(o.7a)
(.
91.3|p.ao
67.0|o.oo
96.8|o.18]
77.8%o.so)
p..2
Emb.
68 68.2
52.0 06
$7.0 [o.a0]
53.7 (j..00
15.8
64.6%
30.8
PBP
Enc.
69.1p.14)
52.0 64)
56.5
55. 1 (o.5e]
94.9(u.50)
63.5
Emb.
20.
75.0g 64]
95. 95.2
78. (0.80)
97 17.9
79.4
60.7
MSP
Enc.
91.2
76.2
95.4 (0.21)
80.2 0.488
98.3 [0.21]
80.7 (0.20]
63.1
BlenderBotassau.
Emmh
85.T(0.00]
g..4
92.51g.18|
72.910.20]
97.5
(..97
(2.2.2.44
MLP
Enc.
92.1 (0.56)
84.3%
76.0%o.49
97.T(o.1s)
(3.2
65.2
Emmh.
15.5
14.5
58. 58.9ho.71
56.4p.24)
95.6j.
50.8p.12|
PBP
Enc.
(o.97)
61.0%3.00)
1..9).
80.6
97.3
(o.3
66. 66.7pp.94
Emb.
91.0 91.0pu.a4]
77.9 1o0
95.7 (o.1
82. pp.30.
18.1g..19)
80.3 f..3
60.5
MSP
Enc.
93.8 (0.70
87.4/..2
96.5 (0.
88.8 0.0ol
98.6 po.6). [0.30]
84.4 (u.30)
(1.71)
DialoGPT SMALT
Emb.
88 (8.440.43)
73.8 69)
92.9%o.29
73. 73.4f0.42
98 98.8jo.14
80.0
52.6
MLP
Enc.
90.1,0.28)
80.9% 0.30)
90.8
72. S2.244
98. 98.TTo.09)
81.1%
65.2
Emb.
37 (13553.29]
42.3 [,27]
28. (1.2(1.oo)
10.6jp.52
71.1 [2.1
57. (o.0
26.0
PBP
Eine
52.0
50.0% [.61]
44.9% (0.60)
34.3f1.02
86.2 (0.69)
57.9 57.9%
36.7
Emb.
(4.
[3.7
96.1 (o:27)
85. (p.oo
98. 18.
82.3
58.7
MSP
Enc.
96.8 96.8 jo.ca] (..4
(6.2(o.ea)
85.7 p.43]
99.0po.o4]
82.4[o.37]
66.7
Table 1: Accuracy on probing tasks for Transformer, BlenderBotswanz. DialoGPT SHATL- Experiments were conducted on three probing methods: 1) MLP: Vanilla MLP-based Probing 2) PBP: Prompt- based Probing. 3) MSP: Aulti-Source Probing. Best results are marked in bold, and data that passed the significance test (t-test, p-value < 0.05) between word embeddings and encoder states under specific probing method super-scripted with an asterisk * The numbers in square brackets represent the standard deviation.
chitecture which is first pre-trained on 1.5B Red- dit comment threads (Baumgartner et al., 2020) and later fine-tuned on Blended SkillTalk (BST)
Comparison between methods: For all prob- ing tasks and dialogue models, MSP achieves the best performances, indicating that our method more effectively leverages the relevant information en- coded in the intermediate representations to con- duct probing tasks. Besides, the majority of en- coder state results outperform word embedding re- sults, indicating that encoder states contain more semantic features than word embeddings. Thus, some extent, dialogue models learn semantic infor- mation from conversations, which required in conversational understanding tasks. For the MLP-based probing method, we ob- serve a similar phenomenon of the Transformer model that the performances of encoder states are not superior in many tasks as reported in prior work (Salch et al., 2020). This observation demon- strates that the vanilla probing method has limita- tions in utilizing encoder states to conduct conversa- tional understanding tasks, due to the gap between downstream classification tasks and the pre- trained dialogue generation task (Liu et al., 2021; Schick
dataset (Smith et al., 2020).
Di (Zhang et al., 2020) is dialogue re- sponse generation model for multi-turn conversa- tions with single decoder, which is pre-trained on large-scale Reddit data Batt@Baaggrrteer et al., 2020).
5 Analysis
We will detail experimental results in this section, including the analysis of our main experiments on the performance of different probing methods, the ablation study of Multi-Source Probing architec- tures, and evaluations of different dialogue models and experimental settings.
5.1
Main Results
The main results are presented in Table 1, where each probing task is evaluated by calculating an average score of accuracy. We analyze the results from the following perspectives:

---

Method
TREC
DNLI
MWOZ
SGD
SNIPS
SSA
Topic
Blender
Emb.
83.9p.20)
73.3 u3.3 9.777
10.2
65.3
98.3
g9.3
15.3
MLP
Enc.
(0.1
85. [..33
90.6|p.as
66.3p.
98.1 (0.12)
83.6g.43)
56.2
Emb.
22.
36.3
82 (p.40)
(8.1
82. 82.1 (0.03)
61.5pp.om
MSP
Enc.
94.1 [0.44]
90.3 (..3)
(0.00)
(0.3/0.
98.9 [0.63]
87.3 [.000]
72.3
DialoGPT MEDTUN
MEDTUM
Emb.
89 29..210.77
74.5
92.9
74.6jo.5e
98.4ro. 11]
80.4ro.
49.612.47|
MLP
Enc.
94.8fo.
84.7%
91.5
76.0 76.0p))33
98 98.6jo.13
80. (.
67.0 (0.044
Emb.
95. 2(0.5a)
88. 2. (.677
96.5 jo.44
86. 86.1gp [o.44)
99.0 99.0(o.aa)
82.8 g.na]
59.7 (p.aa]
MSP
Enc.
97.4 (0.34]
89.5 10. [0.52]
96. 96.4|o.63|
89.6 (0.38]
99.1 (o.4g)
85.7 [0.80]
70.2 (0.92)
Table The performance of MLP and MSP on two large-scale pre-traine pre-trained dialogue models BlenderBot and DialoGPTT PenTUI Best results are marked in bold, and data that passed the significance test (t-test, p-value < 0.05) are super-scripted with an asterisk * The numbers in square brackets represent the standard deviation.
and Schütze, 2021). The prompt-based probing approach obtains the worst performance among the three probing methods. Although it performs the probing tasks in generative manner, this approach cannot effectively extract relevant features required in conversational understanding tasks, leading to undesirable results.
attention mechanism and the late fusion module. The multi-source attention mechanism is designed to pay fair attention to both the historical and cur- rent turns of the conversational context, while the design of the late fusion module is motivated by the fact that many probing tasks need capture global features while maintaining high sensitivity lo- cal features, so we add a layer of late fusion after the decoder for fusing global and local features the final representation. Due to the length limit, the results are presented in Table 4 in the Appendix.
Comparison between models: As the parame- ters of dialogue models increase, the performance on dialogue understanding tasks also improves based on the MSP method. However, DialoGPT doesn't I outperform BlenderBoo on some tasks and the performances of encoder states not always significantly better than those of word embeddings. This is probably because DialoGPT adopts sin- gle decoder structure with the uni-directional at- tention mechanism, which encodes partial con- text (content before each token) information for tokens, leading to asymmetric representations of di- alogue history and current utterances. By contrast, the mucoder-decoddr-busee BlenderBot applies the bi-directional attention mechanism to encode bi- directional information (content before and after each token) for tokens, achieving more consistent and superior performances on conversational un- derstanding tasks.
5.2.1 Ablation Setting
Several sets of ablation experiments are designed to verify the necessity and effectiveness of different modules:
MSP w/o LF is ablation setting where the late fusion module is removed compared to the complete MSP method.
MSP w/o MS is an ablation setting where the multi souri- attention and late fusion module are removed. This approach has the same architecture as the prompt-based probing method except that the decoder is fine-tuned during training.
5.2.2 Effect of Late Fusion Module
We also evaluated the comprehension ability of pre-trained language models such as BERT, BART, and T5 through MSP, and the details are attached in Appendix
In our experiment settings, we set ablation experi- ments on the effectiveness of late-fusion. Through the analysis, we discover that the late-fusion archi- tecture enables a significant improvement in the ability of the probing model to extract the repre- sentational information encoded by the pre-trained language model. Furthermore, it only incorpo- rates the information provided by the encoder very well but also provides a nuanced insight for our probing architecture of representational informa-
5.2 Ablation Study
Here set up series of ablation experiments to investigate the validity and necessity of the compo- nents of our Multi-Source Probing approach, which is composed of two main parts: the multi-source

---

Method
TREC
DNLI
MWOZ
SGD
SNIPS
SSA
Topic
Transformer
Rmnd.Emb.
7..8
60.4rg
87.2 (.ouu
96. (6.6)..30)
70.6 29]
39.8 49]
Orig.Emb.
83.1 (1.0)
69.9
92.1 [.as]]
69.9/o.ss
97.99..24)
77.1 (7.17)
53.3"
MLP
Shuf.Emb.
83.0%.06)
69. 69.8%o.3v
92.0% %o
69.8 69.8fo.3
97 (0.35)
77.3(0.43]
53.5
Shuf. Enc.
77 (7.26)))))
68.8(o.28)
90.1 jo.
67. (j.35
96 6jo.3))
77-1(o.52)
55.9%
Fand.Emb.
80.4(..72
65.5
94.2 p..4]]
75.9
97-5jo.21]
75.61g.0a)
17.2
Orig.Emb.
90.3
65.0))
95.2
78.9
97.9(o.11)
79.4 79.4f0.28)
60.7".19)
MSP
Shuf Emb. Shuf.Enc.
Shuf.Emb.
64.2
68.9/o.an 4s]
94.9 197
76.9r..m
98. [0.255
77.7p
52.3
(1.90)
70.0g.244
94.7 (..7
78. 10.23
98. (8.3
78.8
57.5% vol
Bolader
Rand.Emb
76. 16)
(3.3
90.6 [.48
65.8
97 97.2
73.1
40.1 20.1
Orig.Emb.
85.7 (0)
72.4/o.24
92.5 1.18)
72.9
97. 97.551.277
78. 78.7 (0.33)
52.2% 22.2%%..
MLP
Shuf.
85.3%.oo0 (o))
[2.7
92.5 [.25]
72.9 Jo.as
97. 97.4ro.20)
78.8(a.
51.7
Shuf.Fnd
83.1 [..44)
29.)(a.22)
89.7 7 [a.2a]
68.0jp.44)
97-5(0.22)
78.5 20)
59.9
Rand.Emb.
84.0
74.33u.200
94.6 j..o0l
82.2
97 97.61u.10)
76.4g r8]
47.7
Orig. Emb.
91.0 91.0%o.o (o.644)
(1.9
95.7 75.7 [p.1s]
p..04.
98. (0.19)
80.3(u.2o]
60.5
MSP
Smb.
88. 7% 68)
75.8
95.8 p.200
83.9
98. (0.00)
79.8(a.48)
1.53
Shuf.Ene
85 (5.2().45)
84.6(o.28) 28]
95.4 (..46
84.7 (0.12
98.3
81.3(o.24)
65.9
DialoGPT SECALL
Emb.
75.3
[4.1
91.6 92.9
(8.2
96.6pu 96.6)o.19)
75.3 44]
12.222.18)
Frig.Emb.
88 38.4fo.aa)
73.8(a.e9]
73.4jo.42
98 (8..%..14)
80.0fa.aa)
52.6 (2.6(].22)
MLP
Shuf.Emb.
89.3%..
73.9 66]
93.0 0..01
73.3%..ao
98 18.9%..17)
80.2% 22
50.8 (.62
Shuf.Enc.
80. 80.11o.59]
76.7(o.20
86.9 p..s|
66.8jo.1s"
96. (6.7(0.24)
76.0jo.
65. 65.4%..32)
Rand.Emb,
87 37.88).32
78 Gg..)
92.3
70.4
97. (0.12)
77.4
52.2 (..7]
Orig.Emb.
94 2/036)
83. (o.39)
96.1
85.4
98 98.9)..15)
82.3
58.7%..e4
MSP
Shuf.
85.4p.25)
81.2(u.24)
94.5
81.5p.oz
98. 98.5
80.3 a9] 81.5
53.61g.u1)
Shuf,Enc.
(2.7(0.20]
94.6
82.3/o.34
98 98.8)..os)
66.0
Table 3:
The performance with random embeddings and shuffled order of words in dialogue context. Data that passed the significance test (t-test, p-value < 0.05) are super-scripted with an asterisk
tion at the shallow level. Consequently, this makes the structure more hierarchical and efficient in inte- grating different depths of encoded information.
synthesizing local features.
5.2.3 Effect of Multi-Source Attention
We found from the results of our experiments that when the Multi-SSurce Probing Method does not have a late-fusion layer, the probing results of encoder states in several tasks are not better than those of word embeddings, which indicates that when we discard the late fusion, we also discard some classification local features needed for the task. We found that the ground-truth labels for these tasks are often determined by a combination of certain keywords in the historical utterance and some obvious prompt words in the current utter- ance of the dialog, so if we simply use the represen- tations generated by the multi-layer transformer in the decoder, this tends to draw only the global features and ignore the involvement of local fea- tures. So in this step of the experiment, we verified the significant role of the late fusion module for
We further developed ablation experiments to ex- plore the impact of discarding the multi-source attention module of our MSP method, where an exciting discovery is found that the accuracy of the probing task was reduced by more than 10% when the MSP method did not efficiently utilize the multi-source attention module, suggesting that we need to introduce specialized designs to focus on different parts of the input text when evaluating the ability of language model to understand con- versation. This demonstrates the need to introduce specialized designs to focus different parts of the input text, which is concern that generative dialogue models are constantly focusing on during the pre-training process. We experimentally found that a single cross-ates attentioo module is not effective in accomplishing our probing purpose, and thus

---

multiple-source attention mechanism is very central component of our probing approach.
the MLP method weakens the feature of the word order by the average pooling operation, while MSP offers superior modeling the word order. We observed that the performances of both word embeddings and encoder states decrease substan- tially with shuffled input. Furthermore, the perfor- mance gap is even larger for encoder states, indi- cating that dialogue models can capture the text and flow of the dialogue for conversational understanding, rather than just processing individ- ual words or utterances isolation.
5.3 Extension Experiments
Considering the undesirable performance of the prompt-based probing approach as shown in Ta- ble 1, we adopt MSP and MLP methods in the extension experiments.
5.3.1 Impact of Model Scale
Table 2 shows the performance of large-scale pre- trained conversational models on prohing tasks. As can be seen, with the increase of model parameters, the performance on dialogue comprehension tasks also increases. The pre-trained dialogue models demonstrate strong capability of conversational understanding even without fine-tuning on down- stream dialogue corpus. For the MLP-based prob- ing method, there is no obvious difference between the performance of encoder states and word em- beddings in many tasks. By contrast, our approach is applicable to models of different scales from the 2-layer Transformer trained from scratch to the large-scale pre-trained BlenderBoot and DialoGPT.
6 Conclusion
In this paper, we propose a Multi- Solti-Souure Probing (MSP) method to probe the dialogue comprehen- sion abilitics of open-domain dialogue models. It conducts probing tasks in generative manner that is consistent with the pre-training task of dialogue models. Besides, propose the multi-source at- tention mechanism aggregate features from mul- tiple sources and the late fusion module to capture global features for downstream tasks. Our experi- mental results indicate the validity and reliability of the MSP method, which could also offer insight into the impact of the model scale, embedding qual- ity, and dialogue structure on the conversational understanding capability of dialogue models when particular experimental settings are presented. This research underscores the importance of a compre- hensive probing framework for dialogue models and paves the way for future studies aimed at en- hancing their understanding capabilities.
5.3.2 Impact of Word Embedding
During the training process of dialogue generation, word embeddings can learn and encode linguistic knowledge of conversations (Ravichander al., 2020) as Word2Vec (Mikolov et al., 2013) and Glove (Pennington al., 2014). Thus, we sub- stitute the trained word embeddings with randomly initialized ones and conduct probing experiments to investigate the impact of word embeddings.
Limitations
The performance with random embeddings is shown in Table 3. As we can see, there is signifi- cant gap between the performance of random word embeddings and original ones, indicating that the trained word embeddings encode semantic infor- mation required by conversational understanding. In addition, the performance with random embed- dings achieves above 90% on some tasks, such as SNIPS and MWOZ, while only obtaining less than 70% others. It shows that different tasks require different degrees of conversational semantics.
Although the Multi-Source Probing (MSP) method can precisely detect the conversational understand- ing of open-domain dialogue models of different scales, it still faces two limitations. First, we focus on evaluating three widespread dialogue models in our experiments due to the limitation of compu- tational resources. Dialogue models of different structures and scales could be probed with MSP in future work. Second, we adopt several representa- tive classification tasks as our probing tasks, follow- ing previous work (Saleh et al., 2020). These tasks require different dialogue comprchension skills and have different degrees of difficulty, as analyzed in Section 5.3.2. In future work, wide range of tasks of different complexity in different domains could be conducted based on MSP to construct a bench- mark of conversational understanding.
5.3.3 Impact of Dialogue Structure
To examine whether dialogue models leverage di- alogue structure in conversational understanding, we shuffle the order of the input tokens within dia- logue history and the current utterance respectively.
The results are shown in Table We note that

---

References
Alexis Conneau, German Kruszewski. Guillaume Lam- ple, Loic Barrault, and Marco Baroni. 2018. What you can cram into single S&!#* vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the As- sociation for Computational Linguistics (Volume I: Long Papers), pages 21 2126-2136, Melbourne, Aus- tralia. Association for Computational Linguistics.
Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan. Zi Yang. Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020. Towards a human-like open-domain chat- bot. arXiv preprint arXiv:2001.0997
Afra Alishahi, Marie Barking, and Grzegorz Chru- pala. 2017. Encoding of phonology in recurrent neural model of grounded speech. arXiv preprint arXiv:1706.03815
Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro. Thibault Gisselbrecht, Francesco Calta- girone, Thibaut Lavril, et al. 2018. Snips voice plat- form: an embedded spoken language understanding system for privatebb-ddsiggn voice interfaces. arXiv preprint
Christoph Alt, Aleksandra Gabryszak, and Leon- hard Hennig. 2020. Probing linguistic features of sentence-level representations in neural relation ex- traction. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 1535-1545.
Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, and Stephan Vogel. 2017. Understanding and improving morphological learning in the neu- ral machine translation decoder. In Proceedings of the Fighth International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 142-151.
Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. 2020. Plato: Pre-trained dialogue generation model with discrete latent variable. Proceedings of the 58th Annual Meeting of the Association for Com- putational I Linguistics, pages 85-96.
Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the inter- national AAA/ conference on web and social media, volume 14, pages 130-839.
Abhishek Das, Federico Carnevale, Hamza Merzic, Laura Rimell, Rosalia Schneider, Josh Abramson, Alden Hung, Arun Ahuja, Stephen Clark. Gregory Wayne, et al. 2020. Probing emergent semantics in predictive agents via question answering. Proceed- ings the 37th International Conference on Machine Learning.
Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Has- san Sajjad, and James Glass. 2017. What do neural machine translation models learn about morphology? arXiv preprint 1704v.170..
Jacob Devlin, Ming-Was Wei Chang. Kenton Lee, and Kristina Toutanova. 2018 8. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.0448
Yonatan Belinkov and James Glass, 2017. Analyz- ing hidden representations in end-to-end automatic speech recognition systems. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, et al. 2020. The second conversational in- telligence challenge (convai2). In The NeurlPS' 18 Competition, pages 187- Springer.
Yonatan Belinkov, Lluis Mårquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2018. Evalu- ating layers of representation in neural machine trans- lation part-of-speech and semantic tagging tasks. arXiv preprint arXiv:180 180..0772.
Ning Ding, Shengding Hu. Weilin Zhao, Yulin Chen, Zhiyuan Liu, Hai-Tao Zheng, and Maosong Sun. 2021. Openprompt: An open-source framework for prompt- arXiv preprint arXiv:2111.01998
Meriem Beloucif and Chris Biemann. 2021. Probing pre-trained language models for semantic attributes and their values. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2554 2559.
Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, and Dilek Hakkani-Tur 2020. Mul- tiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking base- lines. In Proceedings of the Tweifth Language Re- sources and Evaluation Conference, pages 422-428 Marseille, France. European Language Resources Association.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-19011
Itsugun Cho and Hiroaki Saito. 2021. Anna: A dap- per open-domain dialogue agent based joint attention network. Journal of Natural Language Pro- cessing, 188--1194,,
Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. 2021. Warp: Word-level adversar- ial reprogramming.

---

Shengding Hu, Ning Ding, Huadong Wang. Zhiyuan Liu, Jingang Wang. Juanzi Li, Wei Wu, and Maosong Sun. 2022. Knowledgeable prompt-tuning: Incor- porating knowledge into prompt verbalizer for text classification.
Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayan- deh, Lars Liden, and Jianfeng Gao. 2020. Soloist: Few-shot task-oriented dialog with single pre- trained auto-regressive model. arXiv preprint arXiv:2005.05298.
Tianbo Ji, Yvette Graham, Gareth JF Jones, Chenyang Lyu, and Qun Liu. 2022. Achieving reliable human assessment of open-domain dialogue systems. Pro- ceedings of the 60th Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Pupers), pages 6416-6437.
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014, Glove:
Global vectors for word rep- resentation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1543-154.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 1(8)99
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.668
Mojtaba Komeili, Kurt Shuster, and Jason Weston. 2022. Internet-augmented dialogue generation. In Proceed- ings the the 60th Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 8460-8478.
Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara. Raghay Gupta, and Pranav Khaitan. 2020. Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In P Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 34, pages 1696.
Jiwei Li. Michel Galley, Chris Brockett, Jianfeng Gao, objec- tive function for neural conversation models. Pro- ceedings of the 2016 Conference of the North Amer- ican Chapter of the Association for Computaational Linguistics: Human Language Technologies, pages 110-119.
and Bill Dolan. 2016. A
Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. 2020. Probing the probing paradigm: Does probing accuracy entail task relevance? Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguisties: Main Volume.
Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 7a. Adversarial learning for neural dialogue generation. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2157-2169.
Kyle Richardson, Hai Hu, Lawrence Moss, and Ashish Sabharwal. 2020. Probing natural language inference models through semantic fragments. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8713-8721.
Xin Li and Dan Roth. 2002. Learning question clas- sifiers. In COLING 2002: The 19th International Conference on Compulational Linguistics.
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott. Kurt Shuster, Eric M Smith, et al. 2020. Recipes for building an open- chatbot. arXiv preprint arXiv:2004.1360 13637.
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017b. Dailydialog: A manually labelled multi- dialogue dataset. arXiv preprint 17X.5:1710.03957.
AbdeIrhman Saleh, Tovly Deutsch, Stephen Casper, Yonatan Belinkov, and Stuart Shieber. 2020. Probing neural dialog models for conversational understand- ing. Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pages 132-143.
Xiao Liu, Yanan Zheng, Zhengxiao Du Ming Ding. Yujie Qian, Zhilin Yang. and Jie Tang. 2021. Gpt understands, too. arXiv preprint 1rXiv:2103.10385,
Tomas Mikolov, Kai Chen, Greg Corrado, and Jef- frey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
Chinnadhurai Sankar, Sandeep Subramanian, Christo- pher Pal, Sarath Chandar, and Yoshua Bengio. 2019. Do neural dialog systems the conversation his- tory effectively? an empirical study. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 32-37.
Prasanna Parthasarathi, Joelle Pincau, and Sarath Chan- dar. 2020. How to evaluate your dialogue system: Probe tasks as an alternative for token-level evalua- tion metries. arXiv preprint arXiv:2008. 10427.
Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Asso- ciation for Computational Linguisticx: Main Volume, pages 255-269.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Fdward Yang, Zachary VeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation pytorch. In NIPS- W.

---

Iulian Serban, Alessandro Sordoni, Ryan Lowe, Lau- rent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31.
Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Joncel-Kedzione Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, et al. 2021. A controllable model of grounded re- sponse generation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 35, pages 14085-14093.
Eric Michael Smith, Mary Williamson, Kurt Shuster, Jason Weston, and Y-Lan Boureau. 2020. Can you put it all together: Evaluating conversational agents" ability to blend skills. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2021-2030, Online. Association for Computational Linguistics.
Kong Yawei and Kai Fan. 2021. Probing modal machine translation with pre-trained language model. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3689-3699.
Yazhou Zhang, Lingling Song, Dawei Song, Peng Guo, Junwei Zhang, and Peng Zhang. 2019. Scenariosa: A large scale conversational database for interactive sen- timent analysis. arXiv preprint (rXivv:907.005
Nya Sutskever, Oriol Vinyals, and Quoe V Le. 2014. Sequence to sequence learning with neural networks. Advances in neural information processing systems. 27.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. Dialogpt: Large-scale generative pre- pre-training for conversational response generation. Proceedings of the 58th Anmual Meet- ing of the Association for Computational Linguistics: System Demonstrutions, pages 270-278.
Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2018. Ruber: An unsupervised method for automatic evaluation of open-domain dialog systems. Thirty- Second AAA/ Conference on Artificial Intelligence.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
Xueliang Zhao, Wei Wu, Can Xu, Chongyang Tao, Dongyan Zhao, and Rui Yan. 2020. Knowledge- grounded dialogue generation with pre-trained lan- guage models. Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 3377-3390.
Jesse Vig. 2019. A multiscale visualization of attention in the transformer model. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics System Demonstrations, page 37.
Peixiang Zhong. Chen Zhang. Hao Wang, Yong Liu, and Chunyan Miao. 2020. Towards persona-based empathetic conversational models. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 6556 6566.
Jesse Vig and Yonatan Belinkov. 2019. Analyzing the structure of attention in transformer language model. Proceedings of the 2019 ACL Workshop BlackbuxNIP: Analyzing and Interpreting Neural Networks for NLP, pages 63-76.
Hao Zhou, Tom Young, Minlie Huang. Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018 Commonsense knowledge aware conversation generation with graph attention. In IJCAI, pages 4629.
Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics. pages 3731-3741.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019a. Huggingface's 's transformers: St the-art natural language processing, arXiv preprint arXiv:1910.03771.
Thomas Wolf. Victor Sanh, Julien Chaumond, and Clement Delangue. 2019b. Transfertransfo: A trans- fer learning approach for neural network based con- versational agents. arXiv preprint arXiv: 1001::900.0////
Chien-Sheng Wu, Steven Hoi, Richard Socher, and Caiming Xiong. 2020. Tod-bert: Pre-trained natural language understanding for task-oriented dialogue. arXiv preprini 20X.1:2004.06871.

---

Method
TREC
DNLI
MWOZ
SGD
SNIPS
SSA
Topic
Transformer
Emb.
90.0
75.0hu.e4]
96 arr
78. qo
97. a1.))
79. 79.4pp.20
60.7
MSP
Enc.
91.2
76.2
%..4%.
80.2
98.3
80.7 2s)
63.1
MSP LF
Emb.
p0.0
75. 1p.400
95. (p..))
78.7
97. (p.
50.1j.222
61.6
Fnc.
90.66p.o
75.4p..
95. 2p.2
79.7 [o.44
98. 2s]
79.4
62. 1p.00)
Emb.
87.7
68. p0.9
14.4 sq]
71. 7p.2l.m7))
97. (p.411
2p.2p
60.5
MSP w/o MS
Enc.
89. 11j..a
69. 2
83. (3.99p.cm)
97. [p.
p2.8p.
61.9
MicuderBotsrsaee
MSP
Emb.
91.0p 21.0p.sso
93.8
87.40.211
77. .0o)
96. 85.7p.
96.5 juse
88.8
82.0 34]
98.00..o
98. 1p..
50.3pp
84.40.ss4
71.3
60.5p.54
Emhh
90.4g on
74.7p so]
95.5 .oj
81.9tu.77
98.3p as)
79.8tp.4r1
60.4g .4]
MSP LF
Enc.
93.6
55.5pp.u
(6.4
88.8
98.66p.31
$4.2jp.411
71.2
MSP w/o MS
Emb
71. p..m
90 2m.s44
2p.27 87.9% soj
97. 9u.
74.0p.
58. 18.4...0
Enc.
$0.4 to 2s]
84.7 To [o.50]
95.8 a1j
98.5 Sp oo)
83. lo 19
70.
DialoGPTssai
MSP
Emb.
94.22p..s)
83.7
969 Ip.rp
83-4pu.oo) 85.7 4Mj
38.9pp.a)]
82.3
58. 68.7]p.e4]
Enc.
96.8%
63.4jo.a1
99.0go.cu
82.4
66.7
MSP w/o LF
Emb.
93.0pp.a oo)
83 (7.7))
B5.7p.
84.8
98, ap.
81.8ppay
38.3
Fnc.
96.22o..4 49]
81.7p.md
94.7pu.0
19.1
98.7p..44
81.6go.00
66.2/o.s0
Emb.
90.44j soj
62, 1p ooj
92.6% soj
83.7po my
98. 19]
77.1p.3)
56.3 79]
MSP
Eac.
91/4pp.a
83 83.0p..11
77.3p.47) 2pu2
96. 7p.
80.7pu.7
66. 66.5)2.0)
Table 4: The performance of different dialogue models on probing tasks for ablation experiments. Here we introduce three ablation settings: 1) MSP: The complete Multi-Soure Probing method, 2) MSP w/o LF: MSP without the late fusion module, 3) MSP w/o MS: MSP without the multi-source attention mechanism and the late fusion module. Best results are marked in bold, and data that passed the significance test ( (-test, p-value < 0.05) are super-scripted with asterisk +
Method
TREC
DNLI
MWOZ
SGD
SNIPS
SSA
Topic
Emb.
90.5
75.0p..8
25.2
78.9
97.9
79.4
60.7paa am
MSP
Enc.
91.2/u.50g
76.2
95.4
80.2
98.3
80.7 80.7
63.1 Jun
Embs..
83.75
70.2p
92.0fg.se
69.2 69.2%
77 77.3
56.4p.ao 56.8/2.7a1 ra1
MLP-Deep
Eic.
80.7
69.3 62)
89.5)p.74)
67.5
96.9/o.3
f..44
Table 5: The performance of the MSP and MLPpDeep methods on probing tasks. Best results are marked in bold, and data that passed the significance test (t-test, p-value < 0.05) are super-scripted with an asterisk '
A Discussion
MSP structure.
A.1 Eliminate the Effect of Parameter Scales
A.2 Generalization on Pre-trained Language Models
We have already found positive correlation be- tween parametric size and probe model perfor- mance earlier, here we further explore the gap between MLP and MSP performance the same parameter size address the concern about the effect of the number of parameters on the probing results.
We also evaluated the results of MSP over the state- of-the-art pre-trained language models. According to the results over BERT, BART, and T5 in Table 6, we could conclude that MSP still outperforms MLP even on the pre-trained language model.
We extended the original two-layer MLP to a parameter scale consistent with the MSP, and the results are attached in Table 5. Although the pa- rameter size of the MLP is increased to a level comparable to that of the MSP, the results are still sub-optimal because it does not operate manner consistent with the goals in the pretraining phase of the dialogue model. The supplementary exper- iment results demonstrate the effectiveness of the
Among the three models of comparable parame- ter size, Bart has the most outstanding ability for dialogue understanding. Bert performs optimally on the topic classification task, and T5 has very good performance on the task of intent detection, which is consistent with the characteristics and pre- training goals of the individual models themselves.
We found that Blenderbot-Smin outperformed all of the three general pre-trained models in terms

---

Method
TREC
DNLI
MWOZ
SGD
SNIPS
SSA
Topic
Embs.
87.8
92.9
73.0g..oo
98.45
78. 78.2
46.9
Bert MLPP
Enc.
85.7
81.8 81.8%a.45)
90.2
2p.20.
95.8g.18
81. (0.
59.9
Embs.
92.7 92.7
p..2
95.4
81.9)0.37)
98.5
81 61.9/6.47)
51.4
Bert MSP
Enc.
94.9% 23
86.7/0.59)
95.8 0.42)
83.2 44)
98.6jo.cs
(..3
69.7 (1.18
Embs,
51o 57
75.1 g.
93.2 93.2 (00
73. 7/0
79.9 12)
p8.2
Bart MLP
Enc.
89.8 29)
85.9 (9.9%
0.24).
72.0
98.8 o.oal
83.7/o.14
56.0fu.7o
Embs.
91.3 ssg
96. (0.25)
84. mo..
98.4
81. 81.7 7
60.5
Bart MSP
Enc.
9.26
87.6
96.3 p.44]
(7.2
99.0
$4.90.22
68.1
Embs.
86.8
74.0
(0.4g
73.6
97.2
78.5
44.5/p.38
T5 MIP
Enc.
89.5%o.o as
83.3%
(..1
(1.4
98.1 (.11
82. 1/o
57.9/p.s1
Embs. 91.7 91.7p
s..8
95.6 95.6p.
98.6
81 4p.21
59. pp.
T3 MSP
Enc.
93.2 93.2
85.1
(..2g).
85.7 (0.02)
98.9
83.3 (o25)
67.6
Table 6:
The performance of the MSP and MLP methods with the state-of-the-art pre-trained language models BERT, BART, and T5 on several probing tasks. Best results are marked in bold, and data that passed the significance test (-test, p- p-value 0.05) are super-scripted with an asterisk".
Method
TREC
DNLI
MWOZ
SGD
SNIPS
SSA
Topic
Rand.Enn Emb.
71.8p.75)
60.4g..e
87.2
60.5m..e
96.6g.50]
70.6
39.8/2.48]
MLP
Orig. Finh.
83. 83.1 [1.07]
69.9p..7
92. [0.s8]
69.9/a.ss
97.9
77.1 [0.17]
53..)/..00
Rand. Rand.Emb,
65.71.77)
53.4%.
84.9/o
55.4 20)
95.5
69.2%
35.8%
Linear
Orig. Emb.
66.0po.74
51.2gass
84.2(p.o1]
63.8fusse)
95.5
02.3
31.2
Table 7: The performance MI.P and Linear-Probing methods with original and random embeddings on several different probing tasks. Data that passed the significance test (t-test, p-value 0.05) super-scripted with an *
Model
Parameter
Perplexity
that the linear probing method can also achieve more than 95% accuracy on random embeddings in the SNIPS task. It also proves the previous conclu- sion that linear probing is not an exclusive probing skill and non-linear probing has stronger probing performance in many aspects. We also add the ex- perimental results of MSP on Bert (See Table 6), where we could see that Bert does not outperform pretrained ODD models of comparable size the task of probing for language understanding.
Transformer
37M
30.3
Boo
90M
10.4
DialoGPT SMALE
117M
8.6
Table 8: Training results of dialogue models.
accuracy on the MWOZ SGD and Topic tasks. While DialoGPT-Small performs best on the TREC and SNIPS tasks. Another point to note is that all three models included in the supplemental exper- iments have over 25% more parameters than the corresponding BlenderBot-Smie DialoGPT.
B Dataset Examples
Examples of probing dataset are shown in Table 9.
C Implementation Details
A.3 Non-linear Probing Finetuning
Nonlinear probing is widely adopted previous works (Belinkov et al., 2017; Belinkov and Glass, 2017: Conneau et al., 2018). In fact, non- non-linear probing and linear probing are essentially similar in probing. Classifiers with even shallow lincar structure still fit well on these probing tasks. To this end, we add an experiment on linear probing, in which the original/random embeddings of the dia- logue model were connected to a single linear layer for linear regression. The results in Table 7 show
We implemented the above models with Py- Torch (Paszke et al., 2017), OpenPrompt (Ding et al., 2021) and Huggingface Library (Wolf et al., 2019a). When implementing the Multi-Source Probing method upon DialoGPT, introduced randomly initialized cross-attention parameters gether with the decoder for fine-tuning since Di- aloGPT not an senedderdecoder-baeeeee model. We utilized Soft Verbalizer (Hambardzumyan et al., 2021: Hu et al., 2022), where continuous vector

---

Dataset
Example
Class
Label
number, eailly. location.
TREC
Question Classification
[User]]: What sprawling U.S, state boases most nirpors?
Incation
[Uee[]] My chesinut.
entail,
DialogueNII
Natural Languuge Inference
[Usert]: You are blonde.
contradict, neutral
contradict
[Uert]] Can you find an expensive restaurant that serves
Venetian food?
restaurant-inform, taxi request. general ceneral-ltunk.
[User2]: Sorry, looks like there aren't Venetian restaurants
MuleWOZ
Intent Classification
are expeasive. Would you something
restaurant inform
[User]] Yes, would he intorested in one that serves Chinese food. Where would recommend??
[UserT]: Can you
attractions visit?
[User2] Where do you want me to search.
find attirratccionn light, got-ride,
Schema-
Inient Classification
[User]]: Torento please.
find attractions
[User]] found 10, about 3D Torooto Sign, it's tourist attraction.
cearch-screcning. playlist. get weather,
Intent Classiñcationn
[UserT]: nead
Indic playlist.
add-to-playlist
[Userl]: Have you met the new intem?
[User]]: She's 's very enilusiuticc
positive. negitive, neutral
ScenrioS.A
Sentiment Classification
[Usert]: know, I don't trust her.
negative
[User2]: Why? She
ambitious.
[User]]: think she wants my job.
[User]]: Happy hirtiday, Jim Here is present for you.
ordinary life. work, school, tourism. polities, relationship....
DailyDialog
[User2]: Oh. great I love it!
Topic Classilication
relationship
Topic
[Usert]: I'm very glad hear that.
[User2]: Come here, let me introduce some friends
Table 9: Examples from probing tasks.
is designed for each class label, to generate the probability distribution for class label space by cal- culating the dot product between the output of the language model and the class vector. The class vectors are initialized with the pretrained token em- beddings and will be fine-tuned through training,
All models in this paper are optimized through ADAM (Kingma and Ba, 2014) with learning rate and dropout rate optimized through grid search. The number soft tokens is empirically set to 12 our SoltieSouree Probing method. The learning rate is searched within the range of (5 x 10-5, x 10 10-4,2 2 x 10 4,3 3 x 10 5 10 #) and dropout rate within (0.1, 0.3, 0.4, 0.5}. The batch size for Transformer, BlenderBot, and DialoGPT are 128, 64, and 16 respectively. Accuracy and stan- dard deviation data in this paper calculated from the results of replicate experiments. We con- ducted our probing experiments on the NVIDIA V100 Tensor Cores, the average run-time for each probing task about hours.


Reviews:
[{"review_id": "wWrWZM4XtU", "reviewer": "Reviewer_qTdP", "summary": "", "questions": "+ What is the parameter scale of the transformer trained from scratch?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments!\n\n**Q1: Questioning the necessity and motivation of multi-source attention mechanism**\n\nIn our study, we found that a singular attention mechanism often struggles to efficiently extract valuable information from the entire dialogue history while simultaneously giving appropriate weight to critical elements in the most recent interactions. This observation is corroborated by our ablation experiments (see Table 4), which demonstrate the limitations of using a singular attention mechanism for probing tasks in dialogue systems.\n\nIn contrast, our multi-source attention mechanism is specifically designed to address these challenges. By incorporating distinct attention sub-components for different parts of the dialogue, we create a more focused and effective way of gleaning insights from both historical and current conversational data. This approach is not only intuitively aligned with the inherent structure of dialogues, but it is also concise and universally applicable across different dialogue architectures.\n\nTherefore, we contend that the multi-source attention mechanism is not merely an optional refinement but a necessary evolution for improving the probing capabilities of dialogue systems.\u000b\n\n---\n\n**Q2: Concerns about fairness due to fixed decoder parameters in prompt-based probing**\n\nThanks for the very valuable points, we have in fact included this part of the validation experiment in the ablation experiments section. We removed multi-source attention and late fusion from MSP, and this degenerate form is essentially a version of PBP that is consistent with MSP in terms of decoder tunability (see Ablation Experiment MSP w/o MS, Appendix A) .\n\nPrompt-based Probing is a generic probing method that utilizes prompt learning. Being the most primitive generative probing method, it is not fine-tuned in the decoder part, as fixing or removing the decoder is a common approach used in most of the previous works [3][4][5][6][7].  \n\nOne of the key innovations introduced by the MSP is the use of a fine-tunable decoder to carry out probing tasks that are richer in a priori information, while it utilizes the discarded information in a more efficient way. The probing methods commonly used in previous works lack the application of potential understanding capabilities that may be encoded in the decoder parameters. Thus, we introduce MSP to reapply the discarded information and better evaluate the language model's ability to understand and model contextual information.\u000b\n\n---\n\n**Q3: Implementation details of verbalizer**\n\nThank you for bringing attention to the absence of implementation details concerning the verbalizer. We appreciate the importance of this element for a comprehensive understanding of our approach. The verbalizer is a well-established method in prompt learning, and its implementation is based on existing literature [1][2]. Due to space constraints, we did not delve into the specific design of the verbalizer in our paper. However, we acknowledge the need for this information and will provide a more detailed description along with relevant citations in the final draft of the paper.\n\n---\n\n**Q4: Parameter scale of the transformer**\n\nThe parameter scale of the transformer is 37M (Please see Table 8)\n\n---\n\n**References**\n\n[1]https://arxiv.org/pdf/2108.02035.pdf\n\n[2] https://aclanthology.org/2022.acl-demo.10.pdf\u000b\n\n[3]https://aclanthology.org/P17-1080/\n\n[4]https://proceedings.neurips.cc/paper/2017/file/b069b3415151fa7217e870017374de7c-Paper.pdf\n\n[5]https://aclanthology.org/P18-1198/\n\n[6]https://groups.csail.mit.edu/sls/publications/2017/ICLR17_Belinkov.pdf\n\n[7]https://aclanthology.org/2020.nlp4convai-1.15/", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "+ The paper introduces an innovative Multi-Source Probing (MSP) technique, which is both original and pragmatic in nature.\n+ Moreover, the paper presents enlightening empirical discoveries concerning the open-domain dialogue models' aptitude for comprehending dialogues.", "reasons_to_reject": "+ The rationale behind the proposed multi-source attention mechanism remains ambiguous. Is it imperative to devise distinct attention mechanisms for various dialogue sources? Even in cases where pertinent information is distributed disparately within the dialogue history, could not a singular attention mechanism be capable of discerning the more significant segments? The necessity of employing a multi-source attention approach warrants further deliberation.\n+ The freezing of decoder parameters in prompt-based probing, in contrast to the tunability in multi-source probing, could potentially lead to an unfair setting for prompt-based probing. Hence, I recommend that the authors conduct an additional experiment involving a variant of probing where decoder tunability is maintained.\n+ The implementation details concerning the verbalizer are absent. How is the verbalizer designed?\n\n ", "questions_to_authors": "+ What is the parameter scale of the transformer trained from scratch?", "paper_topic_and_main_contributions": "The paper proposes a Multi-Source Probing (MSP) method to probe the dialogue comprehension abilities of open-domain dialogue models. The method aggregates features from multiple sources (dialogue history, current utterance, integrated context) and conducts downstream tasks in a generative manner with the use of soft prompting that is consistent with the model pre-training process. ", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "+ line 998 Bart -> BARTl; line 999 Bert -> BERT"}, "extra_text": {}}, {"review_id": "4JUS6er9C4", "reviewer": "Reviewer_PXn6", "summary": "", "questions": "Question A: Table 1, which lines of data are \"data that passed the significance test\" compared with?\n\nQuestion B: Is there a relationship between the performance of the probing method and the number of fine-tuned parameters of each method? Why is the decoder fixed for Prompt-based Probing while fine-tuned for Multi-Source Probing?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1. Expect more in-depth probing experiments**\n\nThank you for your feedback. While I appreciate the call for more in-depth probing experiments and your opinion of our experimental setup, I'd like to address your points:\n\n* Our experimental setup was designed to follow previous research in this area to maintain consistency and comparability. This allows for a more reliable assessment of how our work stands in relation to existing literature. Importantly, even within this established framework, our findings were not merely a reiteration of known conclusions but provided new and more credible insights.\n\n* We agree that additional probing experiments, such as the analysis of understanding dialogue discourse relations, would add value. However, it's worth noting that our current work already includes extensive experiments, the details of which consumed four pages in the appendix due to their complexity and depth. Given the constraints on article length, we found it prudent to leave further in-depth analyses for future work.\n\n---\n\n**Q2. Suggestions of recent dialogue models (Vicuna, Alpaca, Baize, etc.)**\n\nThe baseline dialogue models we chose follow prior work [7][8][9], with the aim of studying the natural language understanding capability of basic transformer-based dialogue models. In addition, we also compared the understanding capabilities of dialogue models pre-trained on dialogue corpora with different amounts of parameters and different architectures. For recent dialogue models such as Vicuna-chat, Alpaca, and Baize, it is difficult for us to conduct controllable and fair comparisons because their training process includes various instruction data and human preference-aligned training to enhance their semantic understanding capabilities, and there may be risks of data leakage during this process. We have also noticed that various benchmarks such as MMCU, Big-Bench, C-EVAL, and SuperCLUE, appeared to evaluate the capabilities of these models, which will be a focus of our future research.\u000b\n\n---\n\n**Q3. Query about significance test comparison in Table 1**\n\nWe compared the performances of word embeddings and encoder states under one specific probing method. If one of the two above input sources (under the same dataset, with the same probing method) passes the significance test, then it is super-scripted with an asterisk.\n\nThe majority of encoder state results outperform word embedding results, indicating that encoder states contain more semantic features than word embeddings. This result, conversely, can corroborate the rationality of a particular probing methodology.\u000b\n\n---\n\n**Q4. Correlation between probing method performance and fine-tuned parameters**\n\nThank you for your insightful question regarding the relationship between the performance of the probing method and the number of fine-tuned parameters. To address this, we already conducted extension experiments where we scaled the original two-layer MLP to a parameter size comparable with that of the MSP. The detailed results of this experiment are available in the Appendix (see Table 5).\n\nDespite matching the parameter size of the MLP to that of the MSP, we observed that the MLP's performance remained sub-optimal. This underscores the idea that the architecture and operation mode of the MSP are intrinsically more effective than simply scaling up the number of parameters. In essence, the additional experiments validate that the effectiveness of the MSP method extends beyond mere parameter scaling, reinforcing the method's robustness in achieving its predefined objectives.\n\n---\n\n**Q5. Why is the decoder fixed for PBP while fine-tuned for MSP?** \n\nIn our study, the reason for using a fixed decoder in Prompt-based Probing while opting for a fine-tuned decoder in Multi-Source Probing (MSP) is rooted in the specific objectives and empirical context of each approach.\n\nPrompt-based Probing serves as a foundational generative probing method, leveraging the well-established practice of prompt learning. Consistent with much of the prior research in this field [3][4][5][6][7], we opted not to fine-tune the decoder. This is because retaining or removing a fixed decoder has been a standard procedure in previous generative probing methodologies, ensuring that our findings are comparable and validated against extant literature.\n\nIn contrast, Multi-Source Probing (MSP) is an innovative probing approach that brings additional dimensions to the table. Specifically, MSP employs a fine-tunable decoder to handle tasks rich in a priori information more effectively. This approach allows us to capitalize on the latent understanding capabilities potentially encoded within the decoder parameters—something generally overlooked in traditional probing methods. By fine-tuning the decoder in MSP, we unlock the ability to re-utilize information that would otherwise be discarded, thereby providing a more comprehensive assessment of the language model's skill in grasping and modeling contextual cues.\u000b\u000b\n\nThis differentiation in decoder treatment between the two methods serves to both align with existing research practices and introduce novel probing capabilities, fulfilling different aspects of our research objectives.\n\n---\n\n**References**\n\n[3]https://aclanthology.org/P17-1080/\n\n[4]https://proceedings.neurips.cc/paper/2017/file/b069b3415151fa7217e870017374de7c-Paper.pdf\n\n[5]https://aclanthology.org/P18-1198/\n\n[6]https://groups.csail.mit.edu/sls/publications/2017/ICLR17_Belinkov.pdf\n\n[7]https://aclanthology.org/2020.nlp4convai-1.15/\n\n[8]https://arxiv.org/pdf/2104.09574.pdf\n\n[9]https://arxiv.org/pdf/2005.00719.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is easy-to-follow and self-contained with the introduction of previous approaches.\n\n2. Seven probing tasks, three probing methods, and different dialogue models provide a sufficient experiment setting for probing dialogue models.\n\n3. The results are consistent with detailed and clear discussions.", "reasons_to_reject": "1. The conclusions are not surprising. More in-depth probing experiments are expected, such as the analysis of understanding dialogue discourse relations, etc. The experiment setup for the impact of word embedding (using random word embeddings) and dialogue structure (shuffling the order of tokens) are too simple, leading to known conclusions.\n\n2. The probing model may learn some shortcuts with the growing number of trainable parameters, which leads to performance gains.", "questions_to_authors": "Question A: Table 1, which lines of data are \"data that passed the significance test\" compared with?\n\nQuestion B: Is there a relationship between the performance of the probing method and the number of fine-tuned parameters of each method? Why is the decoder fixed for Prompt-based Probing while fine-tuned for Multi-Source Probing?", "paper_topic_and_main_contributions": "This paper aims at examining the dialogue context modeling ability of open-domain dialogue models with probing approaches. To fully utilize the information encoded in the intermediate representations, the authors propose a Multi-Source Probing method, which takes advantage of multiple cross-attention masks to attend to different representations from the pre-trained dialogue encoder through the decoding process with soft prompt tokens, including the history-source, current-source, and integrated source. Then these local features are gathered through a late fusion layer for integrating global features. Finally, the output representations of three consecutive prompt tokens are averaged to get the final prediction with a verbalizer. Extensive experiments covering seven probing tasks, three probing methods, and five dialogue models show the following conclusions: dialogue models learn semantic information from conversations; decoder models are weaker than encoder-decoder models; the pre-trained dialogue models show a strong er capability when scaled up; the pre-trained word-embeddings are necessary; and the token order matters.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Other suggestions:\nThe dialogue models are out-of-date to some extent. More recent models, such as vicuna-chat, alpaca, baize, etc.\n\n[1] https://lmsys.org/blog/2023-03-30-vicuna/\n[2] https://huggingface.co/project-baize/baize-v2-7b\n[3] https://crfm.stanford.edu/2023/03/13/alpaca.html"}}, {"review_id": "X6nj1MCTtf", "reviewer": "Reviewer_vySG", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments!\n\n**Q1. Definition of soft tokens and prompt templates in MSP and PBP**\n\nSoft (Continuous) prompt tokens generally refer to tunable, learnable tokens that are optimized during fine-tuning. Unlike hard tokens such as fixed words, these soft tokens can adapt during the training process to help guide the model toward better performance on specific tasks. \u000b\u000b\n\nAs for prompt templates, they are essentially predefined scaffolds for creating prompts. In this context, a well-designed prompt template can help extract more accurate and nuanced answers from the model. Here our prompt template is a sequence of soft tokens for decoder input, and each soft token in this sequence has a unique word embedding that is constantly adjusted and updated during the training stage.\u000b\n\nBy incorporating these concepts of soft tokens and prompt templates, both our MSP and Prompt-Based methods aim to offer a more effective way of probing dialog models, providing richer insights into their understanding and performance capabilities.\u000b\n\n---\n\n**Q2: Question on the impact of decoder models' inferior performance on human interaction and result alignment**\n\nThank you for asking a critical question that delves into the practical implications of our findings. Our research highlights the superiority of encoder-decoder models like BlenderBot over decoder-only models like DialoGPT in capturing the intricacies of conversational context. This suggests that encoder-decoder models, with their bi-directional attention mechanisms, are better equipped to generate richer and more context-aware dialogues in human interactions.\u000b\u000b\n\nHowever, the real-world application of these models in human-machine dialogue can be influenced by a multitude of factors, such as the characteristics of conversation, and the particular use case. Decoder-only models may still be highly effective in straightforward or less context-dependent scenarios, whereas encoder-decoder models would likely excel in complex, context-rich conversations.\u000b\u000b\n\nAdditionally, as noted in our Extension Experiment section (Sect 5.3.1 and Table 2), the performance gap between decoder-only and encoder-decoder models narrows significantly as the model scales in terms of parameters. In other words, when dealing with large models like LAMA and GPT-4, which are decoder-only structures, their performance is comparably exceptional. This suggests that the sheer volume of parameters could have a more pronounced influence on performance than the specific architectural design.\u000b\u000b\n\nTherefore, while the architecture does have an impact, particularly for smaller models, its importance may be overshadowed by other factors, most notably the scale of the model, when evaluating efficacy in human interactions.\u000b\n\n---\n\n**Q3: Question about running experiments on recurrent models due to their purported superiority over transformers**\n\nThank you for raising the question about the potential relevance of RNNs in our research context. While the debate over RNNs vs. Transformers exists, there are compelling reasons why we focused on Transformer models for our experiments.\n\n1. The transformer models we used, such as BlenderBot and DialoGPT, are currently the most advanced mainstream dialogue model architectures. They offer several advantages, including ease of training deep models and resistance to overfitting. In contrast, RNN models struggle to optimize well at large parameter scales, which means we couldn't find RNN models of comparable scale and performance to BlenderBot and DialoGPT for a fair experiment.\n\n2. Our probing methods have been designed to significantly test the understanding capabilities of dialogue models. Therefore, the primary objective of our study was not to reignite the RNN vs. Transformer debate but to introduce a more effective probing methodology.  Our probing method has demonstrated a significant capacity to assess the understanding ability of dialogue models, thereby highlighting the limitations of previous methods and conclusions, regardless of whether those were based on RNN or Transformer architectures.\n\n3. The paper we referred to[7] already explored experiments on RNNs and reached conclusions that were consistent with those of Transformer models. Given this and the space constraints for our own paper, it was logical for us to utilize Transformers as our baseline.\u000b\u000b\n\nIn summary, while it would be interesting to run experiments on recurrent models, our decision to focus on transformer models was driven by their superior performance and scalability, the effectiveness of our probing methods, and the consistency of previous findings between RNNs and transformer models.\u000b\n\n---\n\n**References:**\n\n[7]https://aclanthology.org/2020.nlp4convai-1.15/", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) The experimental results look convincing that their method is better than previous probing methods based on their selection of datasets and models.", "reasons_to_reject": "(1) The description of their MSP and Prompt Based methods are not super clear. It is not defined what are these soft tokens and prompt templates.\n\n(2) It is mentioned that decoder models are worse than encoder-decoder models. If this is the case how does this affect the actual use of these models when interacting with humans. Do those results align?\n\n(3) There was a mention that one controversy in this problem space is that recurrent methods are better than transformer models. If this is the case then should these experiments be run on recurrent models too?", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper is about determining what is the best way to measure the dialog comprehension ability of open-domain dialog models. There are several methods proposed which involve leveraging the word embeddings along with encoder states to perform a variety of NLP tasks (question answering, nli etc.) \n\nThe main contribution of this work is the proposal of their novel method.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "I would try to put the ablation results in the main text.\n"}, "extra_text": {}}, {"review_id": "Sp1fS6XXje", "reviewer": "Reviewer_i7vA", "summary": "", "questions": "1. The model's performance has already achieved excellent results in certain tasks. In light of this, does probing for this particular task still hold sufficient significance?", "limitations": "", "rating": 2, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time!\n\n**Q1: Concerns about whether incorporating human intuition into complex probing models skews the test results.**\n\nFirst, it's important to clarify that the objective of the MSP technique is to provide a comprehensive and nuanced evaluation of dialogue models. While it may seem complex, the intention behind using multiple data sources and generative tasks is to simulate real-world, multifaceted scenarios more effectively than simpler probing methods.\u000b\n\nAny model or evaluation technique is ultimately based on human-generated benchmarks or intuitions. The goal here is not to entirely eliminate human intuition but to systematically operationalize it. MSP offers a structured approach to incorporate what we consider essential tasks for dialogue comprehension, which aims for a more generalized evaluation as it encompasses multiple sources and tasks. This helps in offsetting the specific biases or limitations that might be ingrained due to human intuition, making it more transparent for the probing to assess the capabilities of the model.\n\n---\n\n**Q2: Unclear novel contributions in the paper; suggests highlighting unique insights for future research.**\n\nFirstly, our Multi-Source Probing (MSP) method represents a significant departure from existing approaches in the open-domain dialogue landscape. Unlike previous MLP-based probing methods, MSP effectively addresses the misalignment between the objectives of downstream probing tasks and the goals of dialogue generation during the pretraining phase. This ensures a more harmonized evaluation of dialogue systems(L230-235).\n\nSecondly, our study is pioneering in its introduction of a prompt-based design into probing methodologies. MSP conducts probing tasks in a generative manner, which is better aligned with the original pretraining tasks of dialogue models. This synchronization is a step forward in the dialogue probing field, offering a more cohesive evaluation approach(L239-254).\n\nIn terms of empirical evidence, our paper presents results across a range of experimental settings (as detailed in Tables 1, 2, and 3). These results corroborate the effectiveness of MSP in providing a more nuanced evaluation for pre-trained dialogue models. The comparative analysis between MSP and traditional MLP-based methods (Table 1) highlights the advantages of our generative probing approach. Furthermore, the ablation studies underscore the importance of innovative MSP modules like late fusion and multi-source attention (Table 4).\n\nOur study reveals three critical insights:\n1. Different from the conclusion reached by vanilla probing method, we find through MSP that encoder hidden states contain more information than original embeddings in pretrained dialogue models, as reflected by the higher accuracy obtained on our probing tasks.\n2. Generative dialogue models with a single decoder have a worse understanding of the conversation than encoder-decoder-based models, as the uni-directional attention mechanism only encodes partial context (content before each token) information for tokens, leading to asymmetric representations of dialogue history and current utterance.\n3. Dialogue models can capture the dialogue structure in conversational understanding. Larger dialogue models have a better understanding of conversational information and achieve higher accuracy on probing tasks.\n\nIn summary, both the architectural design and empirical validation of MSP attest to its innovative nature, offering new avenues for future research in dialogue system evaluation.\n\n---\n\n**Q3: Questioning the necessity of probing given the model's already excellent performance in certain tasks**\n\nThank you for raising an insightful question that speaks to the core considerations behind our probing methodology.\u000b\n\nWhile the model has indeed demonstrated high performance in certain tasks, this doesn't eliminate the necessity for a more nuanced evaluation through probing. Probing serves a dual purpose: not only to assess task-specific performance but also to scrutinize the model's ability to discern between varying levels of informative features. Our probing technique, Multi-Source Probing (MSP), accentuates these differences by providing a more detailed examination of the model's true capabilities and shortcomings in understanding dialogue.\n\nMoreover, our work aims to discern the underlying factors contributing to the model's performance. It's essential to distinguish whether the model's excellence is attributed to its nuanced understanding of the underlying features and informational structures in dialogues, as opposed to mere sensitivity to dataset-specific shortcuts, such as individual vocabulary items or syntactic structures. In other words, we are keen on ascertaining that the model's prowess is rooted in genuine comprehension rather than trivial memorization or exploitation of data-specific cues.\n\nTherefore, probing continues to hold significant value as a comprehensive evaluation tool, offering insights into the model's capabilities that go beyond mere task performance metrics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The writing in the paper is clear and  easy to understand\n2. Extensive experiments were conducted across multiple tasks and settings.", "reasons_to_reject": "1. I have some concerns about the necessity of employing complex modeling for the probing task. If a significant amount of human intuition and experience is incorporated to enhance the model's performance, can we still consider the test results as indicative of the probing model's inherent capabilities?\n2. After reading the entire paper, it is not clear what new experimental phenomena, insights, or differences from prior work have been derived from this study. It would be beneficial to highlight any novel contributions or insights that could inspire further research in the field.", "questions_to_authors": "1. The model's performance has already achieved excellent results in certain tasks. In light of this, does probing for this particular task still hold sufficient significance?", "paper_topic_and_main_contributions": "This paper introduces a Multi-Source Probing (MSP) method to evaluate the dialogue comprehension abilities of open-domain dialogue models. The authors conduct probing experiments on seven tasks based on the internal representations encoded by dialogue models. The experimental results show that open-domain dialogue models can encode  semantic information in their intermediate hidden states, and different models have different conversational understanding capabilities.  The findings encourage a comprehensive evaluation and design of open-domain dialogue models.", "excitement": 2, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a Multi-Source Probing (MSP) method to evaluate the dialogue comprehension abilities of open-domain dialogue models. The reviewers mostly agree on the potential significance and originality of the work, as it introduces an innovative method and presents enlightening empirical discoveries. However, there are some concerns and suggestions for improvement.

One reviewer questions the necessity of employing a multi-source attention mechanism and suggests exploring a variant of probing with decoder tunability. Another reviewer suggests more in-depth probing experiments and analyzing dialogue discourse relations. They also mention the possibility of shortcuts in the probing model due to the growing number of trainable parameters. Additionally, there are requests for clarification on the MSP and Prompt Based methods, the role of decoder models compared to encoder-decoder models, and the use of recurrent models. The implementation details of the verbalizer are also requested.

The reviewers generally find the paper sound, with sufficient support for its major claims, but some minor points need more support or details. The presentation and writing style are considered clear and easy to understand.

In terms of excitement, there is ambivalence among the reviewers. While the paper has merits like reporting state-of-the-art results and presenting a nice idea, there are also concerns about incremental work and the need for further revision.

Overall, the paper shows promise but should address the reviewers' concerns and suggestions to improve its clarity, originality, and significance.
Decision: Accept-Main