Better Correlation and Robustness: A Distribution-Balanced Self-Supervised Learning Framework for Automatic Dialogue Evaluation
Authors: Peiwen Yuan
Xinglin Wang
Jiayi Shi
Bin Sun
Yiwei Li
Kan Li
Venue: NeurIPS 2023 poster (2023)
Keywords: Natural language process
Automatic dialog evaluation
TL;DR: Propose a new framework for more accurate and robust dialogue evaluation.

Abstract: Turn-level dialogue evaluation models (TDEMs), using self-supervised learning (SSL) framework, have achieved state-of-the-art performance in open-domain dialogue evaluation. However, these models inevitably face two potential problems. First, they have low correlations with humans on medium coherence samples as the SSL framework often brings training data with unbalanced coherence distribution. Second, the SSL framework leads TDEM to nonuniform score distribution. There is a danger that the nonuniform score distribution will weaken the robustness of TDEM through our theoretical analysis. To tackle these problems, we propose Better Correlation and Robustness (BCR), a distribution-balanced self-supervised learning framework for TDEM. Given a dialogue dataset, BCR offers an effective training set reconstructing method to provide coherence-balanced training signals and further facilitate balanced evaluating abilities of TDEM. To get a uniform score distribution, a novel loss function is proposed, which can adjust adaptively according to the uniformity of score distribution estimated by kernel density estimation. Comprehensive experiments on 17 benchmark datasets show that vanilla BERT-base using BCR outperforms SOTA methods significantly by 11.3% on average. BCR also demonstrates strong generalization ability as it can lead multiple SOTA methods to attain better correlation and robustness.

Better Correlation and Robustness: A Distribution-Balanced Self-Supervised Learning Framework for Automatic Dialogue Evaluation
Peiwen Yuan, Xinglin Wang, Jiayi Shi, Bin Sun, Yiwei Li, Kan Li' School of Computer Science and Technology, Beijing Institute of Technology, China {peiwenyuan, wangxinglin, shijiayi, binsun, liyiwei, 1ikaa]@biit www.biittddd.... cn
Abstract
Turn-level dialogue evaluation models (TDEMs), using self-supervised learning (SSL) framework, have achieved state-of-the-an performance in open-domain dia- logue evaluation. However, these models inevitably face two potential problems. First, they have low correlations with humans medium coherence samples the SSL framework often brings training data with unbalanced coherence distribution. Second, the SSL framework leads TDEM nonuniform score distribution. There is a danger that the nonuniform score distribution will weaken the robustness of TDEM through our theoretical analysis. To tackle these problems, we propose Better Correlation and Robustness (BCR), a distribution-balancee
self-supervised learning framework for TDEM. Given dialogue dataset, BCR offers an effective training set reconstructing method to provide coherence-balancee training signals and further facilitate balanced evaluating abilities of TDEM. To get uniform score distribution, a novel loss function is proposed, which can adjust adaptively according to the uniformity of score distribution estimated by kernel density esti- mation. Comprehensive experiments on 17 benchmark datasets show that vanilla BERT-base using BCR outperforms SOTA methods significantly by 1 11.3% on average. BCR also demonstrates strong generalization ability as it can lead multi- ple SOTA methods to attain better correlation and robustness. Code and datasets: httpss//giiubb.co////11100222221 Bete:////eeeeeerrrllattteeee Correlation madd-Robsttness
Introduction
Evaluating model-genee generated responses efficiently and accurately facilitate the hyperparameter tuning and comparison among models, which is essential for the research of open- open-domain dialogue system (Bao et al. 2020) Sun al., 2021 Feng al., 202T: Li et al.) 2023). Therefore, economical and practical automatic metrics are widely applied instead of human evaluation during development phase. However, the referenced automatic metrics, assessing dialogue based the golden response (e.g. BLEU (Papineni et al.] 2002) and BERTScore (Zhang et al. 20205)), have been shown to be inaccurate for dialogue evaluation (Liu al. 2016 Deriu al. 202 due the one-to-many nature of dialogue (Zhao et al. 2017). Fortunately, the unreferenced metrics, especially the turn-level dialogue evaluation models (TDEMs) (Mehri and Eskénazi 2020b a Huang al. 2020: Ye et al.
2021 Zhang et al. 2021a a 20226), have achieved a favorable budget between efficiency and accuracy,
Nevertheless, existing TDEMs still perform worse correlations with human judgements medium coherence samples (samples with human ranking scores falling within the interval [0.25, 0.75]) and lack robustness shown in Figure [ We argue that these problems from the self-supervised learning (SSL) paradigm, which generally constructs positive and negative samples to strengthen
*Corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Coherence Label
Density
Test Data
Density
Training
Loss
ca. frinct
X1
Mors slepp can oritance nalotance
aboun my
model scherenoe
score
0.0
labe
R2
Co Congre lationa
0.3
R3
looks to ricet How much
10
1.0
TOEM
Density
low correlation
dathariog
polaracd medium
polariasd
molium
polaroed
poor robustness
Coherence
Coherence
model score
Figure I: Deficiencies of self-supervised TDEM. (a) TDEM possesses low correlations with human judgements conversations of medium coherence due to the lack of such samples of training data. (b) TDEM attains poor robustness due to the nonuniform score distribution caused by discrete label domain of training data together with general fixed loss functions.
the scoring ability of TDEM. Due to the fact that positive samples possess high coherence while negative samples possess low coherence (Sato et al. 2020), TDEM lacks training signals of medium coherence samples, thus leading to bad performance when evaluating such samples (e.g.. (Q. R2) in Figure Ip. Moreover, current discrete label domain together with fixed loss functions generally brings TDEM nonuniform score distribution (Figure 1.3. which will hurt the robustness of TDEM according to our theoretical analysis (demonstrated in $3.4).
To solve the aforementioned problems, we propose Better Correlation and Robustness (BCR), intributtionballncee SSL framework for TDEM. BCR offers Training Set Reconstructing (TSR) method to balance the data coherence distribution of training set and advance high correlation with humans on medium coherence samples, and Dynamic Penalty loss function (DP loss) for more uniform model score distribution. In TSR method, we use two rule-based strategies to expand a conversation into a positive sample and a negative sample respectively to supplement medium coherence samples. Considering that utterances within conversation are internally coherent, strategies are reasonable. We also provide continuous labels for these medium coherence samples through diffdistiilation to alleviate the deviation of discrete two-level labels. DP loss promotes TDEM to obtain more uniform score distribution by adaptively adjusting the penalty strength of samples in different scoring intervals according to the uniformity of score distribution estimated by kernel density estimation (Parzen 1962), thus enhancing the robustness. Our contributions are summarized follows:
We propose Better Correlation and Robustness (BCR), a novel distribution-balancee SSL framework for TDEM, which can effectively alleviate limitations of vanilla SSL framework through Training Set Reconstructing (TSR) and Dynamic Penalty loss function (DP loss).
We theoretically prove that the robustness of TDEM and the uniformity of score distribution are positively correlated, and further verify it through experiments.
Comprehensive experiments on 17 benchmark datasets show that vanilla BERT-base (Devlin et al. 2019) applying BCR can outperform SOTA methods significantly by 11.3% on average. We also demonstrate the strong generalization of BCR as it can lead multiple SOTA methods to attain better robustness and correlations with human judgements.
2 Related work
Referenced metrics have shown to correlate poorly with human judgements (Liu et al. 2016) and thus various unreferenced model-based metrics have been proposed, which can be divided into CDEM (conversation-leve)) and TDEM
CDEMs (Mchri and Eskénazi 2020a Ghazarian et al. 2022 Zhang et al. 2022a) evaluate conversations between real user and dialogue system. As this interactive evaluation occupies too much extra energy researchers, we mainly study TDEM, which can be divided into supervised and self-supervised.
Supervised TDEMs (Lowe et al. 2017 Ye et al. 2021 are currently hard to train due to the lack of human annotated datasets. It is worth mentioning that the reward models used in the RLHF stage of I.LM training (e.g., InstructGPT (Ouyang al., 2022)) are essentially supervised TDEMs. Although existing literature has not published the performance of these models, proved through experiments

---

that as a pre-tinin method, BCR can help supervised TDEMs converge to better performance faster (see Appendix C.2). This may inspire scholars to research and apply pre-trained TDEMs to improve the performance of reward models, thus attaining better L.Mss.
Self-supervised TDEMs generally apply next utterance prediction training task, in which the key points are training set construction and training loss design. Original response and randomly sampled response are generally regarded positive and negative samples respectively for a given context (Tao et al. 2018) Phy et al., 2020). Some literature attains hard negative samples through Word2 Vec
similarity based filterin (Zhang et al. (2022b) speaker- level utterance shuffling (Zhang al. 2021a)
model generation (Lan et al. (020) and adversarial craft (Sai et al. 2020) Ye et al., 2021). However, the lack of medium coherence samples will lead TDEMs unbalanced evaluating ability. General
loss functions are widely used to train TDEMs, such as Triplet loss (Tao et al., 2018). CrossEntropy
loss (Mehri and Eskénazi 2020b), NCE loss (Sinha et al. 2020), etc. (Ye et al. 202T) applies three loss functions and attains a score distribution with three peaks. All of these loss functions together with discrete label domain generally lead to a nonuniform score distribution, which will hurt the robustness of TDEMs, proven in 3.4
3 Preliminary
In this section, we first introduce the task definition and baseline model, followed by verifying the low correlations and poor robustness caused by the SSL framework.
3.1 Task Definition
Given conversation triplet (context, query, response) where response is offered by dialogue system, TDEM required to measure the quality (e.g., coherence) of the response. We can calculate the correlation coefficient (e.g., Spearman Correlation) with human judgements annotated datasets to evaluate the performance of TDEM.
3.2 Baseline Model
Model Score
Our baseline model is shown in Figure 2 Specifi- cally, it contains a BERT-base-unease model (De- ylin al. 2019) feature extractor and three- layers multi-layer perceptron to get a coherence score distributed within interval [0, 1]. Apart from general embeddings (token, speaker, position), we additionally apply class embeddings to distinguish utterances from context, query and response as the coherence between query and response is the dom- inant judging factor (Li et al. 2021). Following GRADE (Huang et al. 2020), we use Triplet loss to train the baseline model on the DiilyDialog' dataset (Li al. 2017). The gap of Triplet loss is set as 0.3 at which our baseline model can at- tain best performance. For each conversation, we randomly select T (T>1) consecutive turns pos- itive sample and replace the last utterance with a randomly chosen utterance get negative sample.
3-layers MLP
H2
H2
H
BERT-base
Eyc
Eut
Exg
Exa
E
Exs
E
E
E
E,
E,
E
E
E
E
E
E.
context
query
response
Figure 2: Baseline model. We apply four kinds of embeddings from top to bottom: token embed- ding, speaker embedding, position embedding and class embedding.
3.3 Coherence Distribution
To verify the impact of unbalanced coherence distribution training data, we test our baseline model subsets of DSTCI0 Zhang et al. (2021b) dataset with different coherence respectively (Table Our bascline model (w/o. TSR) obtains 0. Spearman correlation at the polarized interval while only 0.073 at the medium interval. This proves that TDEM applying general SSL framework performs bad on medium coherence samples.
2).
http://yyanran.i/dalyydiia
3

---

JA
MAE (L1)
(b) CE
MSE (L2)
(d) Triplet
(c) L3
(f) CDP
g
L?
(h) C15
Figure 3: Score distribution of TDEM trained with different loss functions DSTC10 dataset.
3.4 Score Distribution
As shown in Figure widely used loss functions (MAE, MSE, Triplet, CrossEntropy) lead TDEM to a polarized score distribution under the discrete label domain setting. We conjecture that these loss functions continuously give a relatively significant penalty loss the samples already scoring within polarized interval, which aggravate this polarized trend.
We primarily consider the influence of score distribution the robustness of TDEM. Given small disturbance at the sample, the corresponding predicted score changes A, which will affect Spearman correlations r. between the original score and the new score + X. Robust TDEM is supposed to resist noise, thus attains higher rs- Hence, we can use (r(r)). ,) the mathematical expectation of r. on n samples, reflect the robustness of TDEM, and make the following statement:
Theorem For any (x),, the probability density function of TDEM distribution, E(r,) has upper bound after small disturbance:
6E(A)2
B(r.) S
(1)
n 1"
and the equality condition is f(x) Vo E [0, 1].
Proof The ranking difference d(x) before and after disturbance is:
p f(x)dr
d(x)
(2)
According the definition of Spearman correlations, E(rs) can be written as:
6 "D;,d(=;))
E(r.) = E(1
(3)
n (n² 1)
derive the lower bound of [(d(x))² as follows (See detailed derivation in Appendix
B(d(x)²) 1'' (1. f(u)du) f(æ)dæ
(4)
S(u)duf(a)dd
E())2
The equality condition of Eq. ④ is f(x) = 1 for [0, Taking the lower bound of B(d(x)))
into Eq. 3 we conclude the proof.
Note that higher B(r,) denotes better robustness of TDEM. Hence, we can derive that the robustness of TDEM correlates positively with the uniformity of score distribution based on Theorem

---

Dislog Daraset
Original samples
Densty
label
A: Let's talk about NBA
Contest
I think Miclael Jordan the
playes
Augmented samples
A:
agree. Is he your davorite
NBA player?
3 Queyy
N%
label
Alail
1
2 ox 0+f1-cixis
B: Yes, I really admire his talesz.
① Response
18(1-4)600)
Cult C,00 B Ci60
Figure 4: Illustration of BCR Framework.
4 BCR framework
We propose BCR framework (Figure 4 alleviate the low correlations medium coherence samples and poor robustness caused by SSL framework. BCR offers: (1) training sel reconstructing (TSR) attain soherence-balancee training samples with continuous label domain ($4.1) (2) DP
loss to bring TDEM a more uniform score distribution and better robustness ($4.2)
4.1 Balance Coherence Distribution
As training set of SSL framework generally lacks samples of medium coherence, propose TSR to reconstruct coherence balanced training set. Specifically, We replace N% samples the original dataset Dowi with samples of medium coherence generated through two strategies, denoted as Doug-
Given conversation, we suppose that replacing response with an utterance context generally leads to negative sample with medium coherence and disrupting context while maintaining the query and response leads positive sample with medium coherence. Our strategies are motivated by (Zhang et al. 2021a), which shuffles all utterances of speaker to attain hard negative sample. As a comparison, the samples generated by our strategies are more controllable in terms of coherence. 4 for example. As we randomly replace ④ with @, although @ does not answer & they
Take Figure have the same topic (NBA player). Thus, (Q, 2, @ (2) constitute negative sample with medium coherence. As we exchange the order of @ and @, remains positive example since @ still answers @ However, the word lalent 4 seems unnatural as @ is no longer said by Speaker B, which makes (2, @ @, @ positive sample with medium coherence. Further human evaluation experiments show that TSR can provide stronger training signals with medium coherence compared with advanced data augmentation methods (Appendix [B.4).
Assigning discrete two-level labels samples with medium coherence can result in serious deviation. Therefore, we combine the original discrete labels and scores of TDEM to attain soft pseudo labels for the augmented samples and the final labels of the whole reconstructed dataset are as follows:
Labelatiooeete (x), E Dori
LabelrsR(2
(5)
aa Label (x) + (1 a) x M(x), Baug
where M denotes TDEM, Label (x:) 1 if is a positive sample and O otherwise. During training process, pscudo labels are obtained in real time.
4.2 Balance Score Distribution
Based on Theorem we consider adaptively approaching the optimal loss function for uniform score distribution to strengthen the robustness of TDEM. According to Weierstrass Approximation Theorem
(Weierstrass 1885) we consider approximating the optimal loss function in with polynomial loss function C(x):
(.(x) M(x) LabelTsR(z)) l'
(6)
[(x) = Σ (x)
(7)
T=U
5

---

To gain deeper insight of the relationship between the probability density function TDEM score dis- tribution (x) and Li(x) we propose Loss Ratio Curve (Lre) monitor. Taking the difference value (D-value) between label and predicted score as X- axis, the ratio of the corresponding loss and fiducial loss (loss when D-value is 0.5) Y-axis, get Lre shown in Figure 5 Lrc visualizes the penalty strength (loss ratio) at different D-value of certain loss function. The stronger concavity Lre possesses, the smaller penalty the samples with polarized scores attain. Combining Figure and Figure 5 score dis- tribution undergoes transition from polarization centralization with the increasing concavity of Lre and the rising power of (a:). This not only confirms our conjecture in Section but also indicates that Ci(x) with small power (e.g., == 3) generally leads TDEM polarized distribution while (x))) with large power (e.g., == 15) promotes a centralized distribution.
08-
Drodiie
Figure 5: Loss ratio curve of different loss functions
Bared this insight in mind,
simplify Eq. and attain our DP loss function follows:
Lpr (x) = (x) + (Zr(z)
where is smaller than During training, (x) and Ci(x) promote polarized and centralized score
distribution respectively, and / adjusts adaptively according to the score distribution make TDEM converge stably to a more uniform distribution. Specifically, we use kernel density estimation (KDE) estimate the score distribution of TDEM on training set ID at the end of each epoch as follows:
D
f(x) = Dh
K
h
where [D] is the size of the training set, is the bandwidth KDE, X; is the model score of the zth sample and (-) is the kernel function. We divide [0, into polarized interval ([O, 0.25] and [0.75, 1J) and centralized interval ([0.25, 0.75]). the estimated distribution shows polarized trend (the integral of f(x) in polarized interval > 0.6), we update Snew = Bold 10 to enhance the influence of Ci, as alleviate such trend. On the contrary, update Bnew Bold 10. From more intuitive perspective, DP loss can dynamically adjust the penalty strength on different samples to approximate the optimal loss function, thereby enabling TDEM achieve relatively uniform score distribution.
Experiments
We first apply BCR framework on BERT-base, and compare it with the SOTA methods on multiple datasets to verify its effectiveness. Then, we apply BCR framework on existing SOTA TDEMs to verify the generalization of BCR and whether it can lead TDEMs to attain better robustmess and correlations with human judgements. Ablation study and case study are followed further understand the effectiveness of BCR,
5.1 BERT-base With BCR
5.1.1
Experimental Setup
DailyDialog dataset. We set learning rate as 5e-5, batch size as 32, a: in Eq. (5) as 0.8, initial value of B as 1, N in TSR as 20. The default value of and / Eq. (8) are and 7. and we examined the effects of different values in 5.3 In Eq. (9). we use Gaussian kernel and set h as 0.01. AdamW Loshchilov and Hutter 2017) is used as the optimizer (see Appendix B.2 for details).
We apply BCR framework to train the baseline model mentioned in Section 3.2
We conduct experiments on 17 benchmark quality-annotan datasets and compare the results with SOTA methods of each dataset (see Table 5). As DSTC10 datasets contain five sub-datasets and
For each experiment, We four random seeds and report the averaged result.

---

Table 1:
Spearman correlations between TDEMs and human judgements 17 datasets, with
standard deviations in gray. BERT refers to baseline model (See Section[.22 The results of SOTA
TDEMs come from the corresponding papers. All results statistically sigmificant (p-value < 0.05).
Method
FED
GCG
GCR
GDG
GDR GEG
GER
PR
DSTC6
BERT
0.205
0.606
0.500
0.310
0.197 0.202
0.315
0.612
0.224
SOTA
0.264
0.617
0.558
0.358
0.187 0.223
0.338
0.699
0.295
BERT+BCR
0.286
0.642
0.519
0.416 0.487 0.341
0.372
0.669
0.274
Method
UT
UP
JSALT
ESL
NCM
DT
DP
DSTC10
Average
BERT
0.305
0.372
0.166
0.345
0.253
0.220
0.401
0.255
0.323 : 0.012
SOTA
0.419
0.469
0.116
0.414
0.299
0.326
0.456
0.310
0.373
BERT+BCR
0.421
0.435
0.272
0.531 0.336
0.305
0.428
0.316
0.415 d 0.005
corresponding 11 qualities, apart from the appropriateness quality shared by the five sub-datasets, we also report the average results (column DSTC10 in Table of all the 11 qualities. Results of SOTA methods come from the corresponding papers. We also reproduced the results of GRADE (Huang al. 2020) and USR (Mehri and Eskénazi
2020b for convincing comparison (Figure
5.1.2 Experimental Results
As shown in Table [ BERT-bas apply-
ing our BCR framework obtains the high- est correlations with human judgements on 11 of 17 datasets and 11.3% (4.2 points)
rediced Oinesson
higher average Spearman correlations than SOTA methods, which demonstrates the effectiveness of BCR framework (see Ap- pendix [.33 for the results of Pearson corre- lations). We further visualize the encoded features and the predicted scores through Principal Component Analysis (PCA). As shown in Figure 6 the transition range (marked with red box) from high score low score very narrow for baseline model while becomes much broader when applying BCR. This feature distribution can bring benefits: (a)
richaoe dimeriion....
Figure 6: PCA results of baseline model without (left) and with (right) BCR on DSTC10 datasets.
discriminative features bring more accurate
judgments; (b) larger feature space can better resist noise. This sheds light on the reason for the better robustness and correlations brought by BCR from a deeper perspective.
5.2 SOTA TDEMs With BCR
5.2.1
Experimental Setup
We apply BCR on the following two TDEMs that attain best performance in research (Ych et al.) 2021) and test them on DSTC10 datasets.
GRADE (Huang et al. 2020 possesses a BERT branch to get utterance-level contextualized
representations and Graph Reasoning branch to get topic-level graph representations. USR (Mehri and Eskénazi 2020b) employs one mask language model and two retrieval models
to measure sub-qualities of given sample and combines them into overall score.
We replace the original loss with DP loss and apply TSR based on their own training set for both of the two models to apply BCR. DailyDialog dataset used to train all the compared TDEMs for fair comparison. We also measured the difference in Spearman correlations when testing TDEMs on the DSTC10 datasets with and without noise to evaluate the robustness:
Diff(M) Spearman( Add (Dtest))) - Spearman( (Beer)))
(10)
7

---

DUT
0000
DISTC10
0104
DSTC19
5.44
03nt
10%
1SL.
3.402
0230
1387
ISL.
0.71
64th
6428
134
GRADS
0.90
8427
LRLADET BER
De
De
USITBUR
Figure 7: Diff and Spearman correlations of SOTA TDEMs DSTC10 datasets.
where Diest denotes test dataset, MM denotes TDEM and we randomly drop 15% of words in context and replace 10% of words in both context and query with synonyms to add noise. [4]
5.2.2 Experimental Results
As shown in Figure 7 both GRADE and USR attain higher Spearman correlations and Diff when applying BCR framework, which verifies that BCR lead SOTA TDEMs attain better robustness and correlations with human judgements. This also confirms the stable generalization of BCR framework.
5.3
Ablation Study
We perform ablation studies for the main components of BCR to better analyze their relative contri- butions. We conduct experiments on the DSTCI0 datasets based on BCR+BERT.
Training Set Reconstructing. We first verify the effectiveness of TSR. As shown Table BERT+BCR drops 0.006 and 0.055 Spearman correlations without pseudo label and the whole TSR respectively. From a more fine- fine-grained perspective, baseline model achieves 79.4% (0.058) Spearman correlations gain medium coherence samples and 30.4% (0.038) Spearman correlations gain polarized coherence samples respectively when applying TSR. This further proves that TSR can simultaneously strengthen the evaluating ability of TDEM both medium and polarized coherence samples while the former benefits much more.
Table 2: Ablation results (Spearman correlations) TSR on subsets with different coherence of DSTC10 datasets, with standard deviations in gray. The Medium indicates samples with coherence labels ranking within interval [0.25,0.75] while the Polarized indicates the rest.
Metrics
Polarized
Medium
Overall
BERT+BCR
0.163 + 0.003
0.131 0.002
0.316 + 0.002
w/o. Pseudo label
0.161 + 0.002
0.120 + 0.001
0.310 + 0.002
w/o. TSR
0.125 + 0.003
0.073 0.004
0.261 + 0.003
DP Loss Function. To verify the effectiveness of DP loss and the impact different values and I in Eq. (), we apply different loss functions to train TDEM respectively. As shown in Figure g DP loss generally brings better robustness and Spearman correlations compared with other loss functions when applying different (s,1) couples. We use fniforymly of score distribution, where a is the variance of score distribution. We find that the more uniform the score distribution (greater Uni formity) is, the better robustness TDEM (greater Di f) attains. This verifies the theory we proved in $3.4 from an experimental perspective.
lg(or) ( to present the uniformity
WWe use EDA tools nlpaug httpss///gichub....///eeeceeeeee
to add noise.

---

3.92
4.08
4.20
4,15
6.15
4.11
Spearman
3.45
40.00
0.008
0.009
40.50
2.43
0.18
0.017
40014
4.013
-0.1
2,28
0.020
0.316
0.55
0.316
0.308
-0.027
0.200
MAE
CE
MSE
Triplel
C
Ls
C
C27
C32
C67
L2a
Cse
Lea
Figure 8: Ablation results of loss functions.
L.
denotes DP loss applying couple (s,
Table 4:
Comparison of BCR with respect to parameter count, training costs, and computational costs to the compared methods.
Method
BCR+BERT
USR
GRADE
MME-CRS
Parameter Counts (M)
110 2
373.8
469
>435. >4
Training Costs (hours)
6
7
Computational Costs
Thase
3 Those
Thase + TGNN
5 Tosse
5.4 Case Study
Table Two representative examples show the strength of BCR framework. U1 and U2 are two utterances of the conver- sation history and R is the corresponding response.
To illustrate the performance of BCR, two representative examples shown in Table 3 In the first example, the ranking (cumula- tive distribution function value of score distribution) given by USR and USR+BCR both correlates well with human before adding noise. In case of noise, human judgement stays the same since the meaning of response remains unchanged. How- ever, the ranking given by USR changes sharply (0.83 to 0.99) even though the score barely changes (0.995 to 0.999). USR+BCR resists the noise well by converting the po- larized distribution to a more uni- form distribution, which verifies the better robustness brought by BCR. The second example shows BCR can bring better correlations with human judgements medium coherence samples.
U1: you look the mirror?
yeah did.
Score (Human / USR / USR+BCR): 0.444 0.995 0.744 Ranking (Human / USR / USR+BCR): 0.81 / 0.83 0.83
R (add noise): yes did.
Score (USR USR+BCR): 0.999 0.732
Ranking (USR USR+BCR): 0.99 / 0.81
U1: Why aren't you eating anything else?
U2: Well, fruits and vegetables are very healthy.
R: What kind of vegetables do you want to eat?
Score (Human / USR / USR+BCR): 0.620 0.998 / 0.556 Ranking (Human USR / USR+BCR): 0.45 0.87 0.47
5.5 Efficiency Analysis
Table showws the comparison of BCR with respect to parameter count, training costs, and computa- tional costs to the methods we have examined. Specifically, Those denotes the computational cost for a single pretrained model and TaNN denotes a typical graph neural network. In the three dimensions of comparison, BCR used fewer resources but still achieved better results.
Conclusion
This paper proposes BCR, distribuution-bllncee SSL framework for automatic dialogue evaluation. BCR offers two novel technologies: TSR and DP loss. TSR reconstructs coherence distribution

---

balanced training set with continuous label domain. DP loss adjusts adaptively according to the score distribution estimated by kernel density estimation to bring TDEM a more uniform distribution. We prove that the uniformity of score distribution and the robustness of TDEM are positively correlated, which guarantees the better robustness brought by DP loss. Empirical results show that BCR framework brings significant improvements on correlations and robustness to various TDEMs. For future work, we will analyse the pre-training strategies of TDEM to promote the development of dialogue system and LLM training.
Limitation. We notice that different TDEMs may produce score disturbance in different scale for a certain small disturbance at the input, which will affect E()) in Section 3.4 Fortunately, this does not affect the conclusion of Theorem II and we find that for different models with the same backbone network, [()) barely changes. We suppose that this is alleviated by the regularization effect of weight decay. We also notice that though DP loss can approximate the optimal loss function, a completely uniform distribution of model score on the test set has not yet been achieved, which we believe is due to the distribution difference in coherence between the training set and the test set. We will investigate how to better align the distribution of training and test sets in the future.
Acknowledgments and Disclosure of Funding
This work supported by Beijing Natural Science Foundation (No. 4222037, L181010), We thank the anonymous reviewers for their constructive comments.
References
Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. 2020. PLATO: pre-trained dialogue generation model with discrete latent variable, In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACI. 2020, Online, July 5-10, 2020, pages 85-96. Association for Computational Linguistics.
Jan Deriu, Álvaro Rodrigo, Arantxa Otegi, Guillermo Echegoyen, Sophie Rosset, Encko Agirre, and Mark Cieliebak. 2021. (Survyy on evaluation methods for dialogue systems Artif. Intell. Rev., 54(1):755-810.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding) In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Papers), pages 4171-4186. Association for Computational Linguistics.
Alexander Fabbri, Wojeiech Kryscinski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir R. Radev. 2021. (Summeval: Re-evaluating summarization evaluation Trans. Assoc.
Comput. Linguistics, 9:391 409..
Tobias Falke, Leonardo F. Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Ranking generated summaries by correctness: An interesting but challenging application for natural language inference! Proceedings of the 57th Conference of the Association for Computational Linguistics, ACI. 2019, Florence, Italy, July 28- August 2, 2019, Volume I: Long Papers, pages 2214-2220. Association for Computational Linguistics.
Shaoxiong Feng, Xuancheng Ren, Kan Li, and Xu Sun. 2021. Multi-view feature representation for dialogue generation with bidirectional distillation In Thirty-Fifh AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Thine Conference on Innovative Applications of Artificial Intelli- gence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 12812-12820. AAAI Press.
Sarik Ghazarian, Ralph M. Weischedel, Aram Galstyan, and Nanyun Peng. 2020. Predictive en- gagement: An efficient metric for automatic evaluation of open- doen-domain dialogue systems) In The Thirty-Foure AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innova- live Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, FAAI 2020, New York, NY, USA, February 7-12, 2020, pages 7789-7796. AAAI Press.
10

---

Sarik Ghazarian, Nuan Wen, Aram Galstyan, and Nanyun Peng. 2022. DEAM: dialogue coherence evaluation using amr-based semantic manipulations, In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), ACI. 2022, Dublin, Ireland, May 22-27, 2022, pages 771-785. Association for Computational Linguisties.
Karthik Gopalakrishnan, Behnam Hedayatnia, Qinglang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, and Dilek Hakkani-Tür. 2019. [Topical-chat Towards knowledge- grounded open-domain conversations, Interspeech 2019, 20th Annual Conference of the International Speech Communication Association, Graz, Austria, 15-19 September 2019, pages 1891-1895. ISCA.
Chiori Hori and Takaaki Hori. 2017. End-to-end conversation modeling track in DSTC6 CoRR, abs/1706 100.000440
Lishan Huang, Zheng Ye, Jinghui Qin, Liang Lin, and Xiaodan Liang. 2020. GRADE: automatic ggraph-enhanced coherence metric for evaluating open-domain dialogue systems, In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 9230-9240. Association for Computational Linguistics.
Tian Lan, Xian-Ling Mao, Wei Wei, Xiaoyan Gao, and Heyan Huang. 2020. PPONE: A novel
automatic evaluation metric for open-domain generative dialogue systems ACM Trans. Inf. Syst.,
39(1):7:1-7:37.
Yanran Li, Hui Su. Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. Dailydialog
A manually labelled multi-turn dialogue dataset In Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27- December 1, 2017 Volume 1: Long Papers, pages 986-995 Asian Federation of Natural Language Processing.
Yiwei Li, Shaoxiong Feng, Bin Sun, and Kan Li. 2023. Heterogeneouss branch collaborative learning for dialogue generation, In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty- Fifth- Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, pages 13156-1356.. AAAI Press.
Ziming Li, Julia Kiseleva, and Maarten de Rijke. 2021. Improving response quality with backward reasoning in open-domain dialogue systems] In SIGIR 21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, pages 1940-1944. 1944. ACM.
Chia- Wei Liu, Ryan Lowe, Iulian Serban, Michael D. Noseworthy, Laurent Charlin, and Joelle Pincau. 2016. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation, In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 2122-2132. The Association for Computational Linguistics.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. Gpteval: Nlg evaluation using gpt-4 with better human alignment.
Nya Loshchilov and Frank Hutter. 2017. abs/1711.05101
Fixing weight decay regularization in adam
CoRR,
Ryan Lowe, Michael D. Noseworthy, Iulian Vlad Serban, Nicolas Angelard- Yoshua Bengio, and Joelle Pineau. 2017. Towards an automatic turing test: Learning to evaluate dialogue responses In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 August 4, Volume 1: Long Papers, pages 1116-1126. Association for Computational Linguistics.
François Mairesse, Milica Gasic, Filip Jurcícek, Simon Keizer, Blaise Thomson, Kai Yu, and Steve J. Young. 2010. Phrrasebased statistical language generation using graphical models and active learning. In ACI. 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, July 11-16, 2010, Uppsala, Sweden, pages 1552-1561. The A Association for Computer Linguistics.
11

---

Shikib Mehri and Maxine Eskénazi. 2020a. Unsupervised evaluation of interactive dialog with dialogpt In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGdial 2020, 1st virtual meeting, July 1-3, 2020, pages 225-235. Association for Computational Linguisties.
Shikib Mehri and Maxine Eskénazi. 2020b. USR: an unsupervised and reference free evaluation metric for dialog generation In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 681-707. Association for Computational Linguistics.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models follow instructions with human feedback In NeurlPS.
Kishore Papineni. Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic eevaluation of machine translation] In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pages 311-318. ACL.
Emanuel Parzen. 1962. On estimation of a probability density function and mode. Annals of Mathematical Statistics.
Vitou Phy, Yang Zhao, and Akiko Aizawa. 2020. Deconstruct to reconstruct configurable evaluation metric for dren-dommin dialogue systems In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain ( (Online), December 13,, 2020, pages 4178. International Committee on Computational Linguistics.
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset] In Proceedings the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume I: Long Papers, pages 5370-5388 53810 Association for Computational Linguistics.
Ananya B. Sai, Akash Kumar Mohankumar, Siddhartha Arora, and Mitesh M. Khapra. 2020. Improv ing dialog evaluation with multi reference adversarial dataset and large scale pretraining Trans. Assoc. Comput. Linguistics, 8:810-827.
Shiki Sato, Reina Akama, Hiroki Ouchi, Jun Suzuki, and Kentaro Inui 2020. Evaluating dialogue generation systems via response selection| In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, ACL 2020, Online, July 5-10, 2020, pages 593-599, Association for Computational Linguistics.
Koustuv Sinha, Prasanna Parthasarathi, Jasmine Wang. Ryan Lowe, William L. Hamilton, and Joelle Pineau. 2020. Learning an unreferenced metric for online dialogue evaluation) In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 2430-2441, Association for Computational Linguistics.
Bin Sun, Shaoxiong Feng, Yiwei Li, Jiamou Liu, and Kan Li. 2021. Generating relevant and coherent dialogue responses using self-separated conditional variational autoencoders, In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing, ACI/IJCNLP 2021, (Volume I: Long Papers), Virtual Event, August 1-6, 2021, pages 5624-5637. Association for Computational Linguistics.
Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2018. RUBER: an unsupervised method for automatic evaluation of -pmn-n dialog systems, In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artifi- cial Intelligence (1AA1-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18) 18), New Orleans, Louisiana, USA, February 2-7, 2018 pages 722-729. AAAI Press.
12

---

Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the factual consistency summaries, In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, 202. Online, July 5-10, 2020, pages 5008-500 50200. Association for Computational Linguisties.
Jiaan Wang. Yunlong Liang, Fandong Meng, Haoxiang Shi. Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023. Is chatgpt good NLG evaluator? A preliminary study] CoRR, abs/2303.04p .00448
Jason W. Wei and Kai Zou. 2019. EDA: easy data augmentation techniques for boosting performance on text classification tasks) In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 6381-6387. Association for Computational Linguistics.
Karl Weierstrass. 1885. Über die analytische darstellbarkeit sogenannter willkürlicher functionen einer reellen veränderlichen. Sitzungsberichte der Königlich Preußischen Akademie der Wissenschaften zu Berlin, 2:633-639.
Zheng Ye, Liucun Lu, Lishan Huang, Liang Lin, and Xiaodan Liang. 2021. Towards quantifiable dialogue coherence evaluation In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 2718-2711 2729. Association for Computational Linguistics.
Yi-Ting Yeh, Maxine Eskénazi, and Shikib Mehri. 2021. A comprehensive assessment of dialog evaluation metrics CoRR, abs/2106.03706
Chen Zhang, Yiming Chen, Luis Fernando D' Haro, Yan Zhang, Thomas Friedrichs, Grandee Lee, and Haizhou Li. 2021a. Dynaeval: Unifying turn and dialogue level evaluation In Proceedings of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 5676-5689, Association for Computational Linguistics.
Chen Zhang, Luis Fernando D' Haro, Rafael E. Banchs, Thomas Friedrichs, and Haizhou Li. 2020a. Deep AM-FM: toolkit for automatic dialogue evaluation In Conversational Dialogue Systems for the Next Decade- 11th International Workshop on Spoken Dialogue Systems, IWSDS 2020, Madrid, Spain, 21-23 September, 2020, volume 704 of Lecture Notes in Electrical Engineering, pages 53-69. Springer.
Chen Zhang, Luis Fernando D' D'Haro, Qiquan Zhang, Thomas Friedrichs, and Haizhou Li. 2022a. Fined-eval: Fine-grained automatic dialogue-level evaluation CoRR, aabs/2210.13833
Chen Zhang, João Sedoe, Luis Fernando D' Haro, Rafael E. Banchs, and Alexander Rudnicky, 2021b. Automatic evaluation and moderation of open-domain dialogue systems CoRR, abs/2111 1.021 10.
Pengfei Zhang, Xiaohui Hu, Kaidong Yu, Jian Wang. Song Han, Cao Liu, and Chunyang Yuan. 2022b. MME-CRS: multi-metric evaluation based on correlation re-scaling for evaluating open-domain dialogue CoRR, abs/2206.094. 094033.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020b. Bertscore, Evaluating text generation with BERT In 8th International Conference on Learning Representa- lions, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Tiancheng Zhao, Ran Zhao, and Maxine Eskénazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders) In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30- August 4, Volume 1: Long Papers, pages 654-664. Association for Computational Linguistics.
13

---

A Formula Derivation of Theorem
Given
J/ f(z)da 1,
we perform formula derivation as follows:
x (+((1)
f(ujdu)
f(x)dx
(
2 f(u)duv f(x)
dæ
f(u)duy f(x) (
2
dr
f(z)dr
((+(())
(u)duu(()dd
(Cauchy's s Inequality)
(L, B(2) f(x) M(r)do
(E(A) - 0)
=E()²² (S, f(x) f(z)dx
=E())² (L f(x)2
L." 12
(T.' f(x)dz))
2
RB(22
(Cauchy's s Inequality)
=E(2)²
The equality condition of the first inequality is:
E(A)
(a)
*+3(A)
f(u)du=C
E(A) CC
for x [0, 1], where C Constant
Considering that generally B(A) + 0 and
[ f(x)dx= 1,
we further derive that f(x) = 1 for E [0. 17.
The equality condition of the second inequality is:
f(x)/1=0
14

---

for E [0, where C' is Constant
f(x) for [0,
Thus, the overall equality condition of the main derivation is f(x) for [0,
B Detailed Experimental Information
B.1 Detailed Datasets Statistics
The detailed statistics of annotated benchmark datasets we evaluate in the experiments are shown in S T For datasets that contains more than one qualities, select the quality closest to the overall quality for evaluating. For DSTCI0 datasets, we also report the average results on all the 11 qualities this is the final evaluating indicator of DSTC10 track5 subtask !
Table 5: Quality- datasets used in experiments and the corresponding SOTA methods.
Dataset
SOTA method
FED (Mehri and Eskénazi 2020a
DynalEval (Zhang et al.
2021a)
GRADE Huang 2020
ConvAl2-Generatie (GCG)
HallyDialog-GGeeratto (GDG)
GRADE
Fimpathetic Generaato (GEG) Conv-A2-Ranker (GCR)
Daily-ialog-Ranker (GDR) Empathetii-Ranker (GER)
USR (Mehri Eskénazi
2020b
USR
-TopicalChat (CT) Persona (UP)
DSTC6 Hori and Hori 2017
Deep AM-FM Zhang et al.
2020a
Predictive Engagement (PE)
Ghazarian al. 2020)
USL-H Phy
2020
DSTC10 Zhang al.
202
MME-CRS 7hang et al.
2022h
- JSALT ESL NCM
TopicalChat (DT) Persona (DP)
B.2 Detailed Experimental Setup
We set learning rate as 5e-5, batch size as 32. Adam Loshchilov and Hutter (2017) is used as the optimizer with 31 0.9, f2 = 0.999 and eps le-6. We use linear-scheduled learning rate strategy with decay 0.02. We train the model for 40 epochs (except for GRADE and USR) and take the checkpoint of the last epoch. of DP loss finally converges to 100. All experiments are based on a single NVIDIA Geforce RTX 3090 GPU with 24GB memory. We use the following Python code to calculate Spearman correlations and corresponding p-value: from scipy import stats; spearmanr, p_ value stats. taatts.ppaammnnsscorrr label). The p-value here roughly indicates the probability of an uncorrelated system producing datasets that have a Spearman correlation at least as extreme as the one computed from these datasets.
We directly used the loss on the Dailydialog validation set as a criterion for hyperparameter selection except for (s, 1). As for (s, l), we choose (3,7) in our setting as it can bring the highest Uni formity
B.3 Detailed Experimental Results
Apart from Spearman correlations, we also test TDEMs on Pearson correlations to gain a more comprehensive understanding of BCR framework. As some research have not reported Pearson correlations results their methods, comparison can only be performed on 10 datasets. As shown in T BERT-base applying BCR framework obtains the highest correlations with human judgements on 8 10 datasets. This further demonstrates the excellent effectiveness of BCR framework.
https::/dstcc0. dste. community/home
15

---

Table 6: Detailed statistics of quality-annottan datasets used in experiments. Num. Samples column refers to the number of samples the corresponding dataset. Num. Qualities column refers to the number of annotated qualities of the corresponding dataset. Eva. Quality column refers to the quality we evaluate the experiments.
Dataset
Num. Samples
Num. Qualities
Eva. Quality
FED
375
9
Overall
GRADE Conv Conerttt
150
Coherence
GRADE ConvAl2-Rande
150
Coherence
GRADE DailyDialog- -eeerrttor
150
Coherence
GRADE DailyDialog-Rain Ranker
150
Coherence
GRADE Empathetie-Genee Generrttr
150
Coherence
GRADE Empathetic-Raune Ranker
150
Coherence
USR TopicalChat
300
6
Overall
USR Persona
240
6
Overall
DSTC6
40000
Overall
Predictive Engagement
600
Engagement
DSTC10 JSALT
741
Appropriateness
DSTCI0 ESL
1242
Appropriateness
DSTC10 NCM
2461
Appropriateness
DSTC10 TopicalChat
4500
Appropriateness
DSTC10 Persona
5000
4
Appropriateness
Table 7: Pearson correlations between TDEMs and human judgements on benchmark datasets. BERT refers to our baseline model. The results of SOTA TDEMs come from the corresponding papers. All results are statistically significant (p-value 0.05).
Method
FED
GCG
GCR
GDG
GDR
GEG
GER
PE
DSTC6
BERT
0.205
0.616
0.491
0.260
0.261
0.167
0.323
0.612
0.234
SOTA
0.606
0.535
0.368
0.261
0.257
0.375
0.688
0.326
BERT+BCR
0.307
0.636
0.510
0.377
0.444
0.348
0.399
0.666
0.273
Method
UT
UP
JSALT
ESL
NCM
DT
DP
DSTC10
Average
BERT
0.339
0.405
0.171
0.403
0.246
0.297
0.412
0.281
0.336
SOTA
0.422
0,411
BERT+BCR
0.437
0.452
0.266
0.519
0.323
0.341
0.428
0.318
0.414
B.4 Human Evaluation of Coherence Comparison
We conduct human evaluation to examine the coherence level of samples generated by TSR. Compared data augmentation methods include: GRADE (Huang et 2020) applies both lexical sampling and embedding- based sampling to obtain negative samples; (b) PONE Lan et al. (2020) applies EDA (Wei and Zou| 2019) and generative dialogue model obtain positive samples. For given conversation, we use the above two data augmentation methods and medium coherence samples generating strategies (positive and negative) to obtain four samples respectively and ask five human annotators to sort them based on the response coherence. We choose researchers in the field of dialogue as annotators because this work requires certain professional quality. The experiment conducted on 100 conversations of DailyDialog dataset. We calculate the average ranking of each type of the above methods. The average Fleiss's kappa of annotators 0.63, indicating annotators have reached relatively strong agreement. The result of coherence ranking is PONE- CONE-positive (1.3)
16

---

TSR-por 1SR-positive (1.8) > TSR- (3.3) > GRADE-negatin (3.6), indicating that TSR can provide stronger training signals with medium coherence compared with general data augmentation methods.
C More Experimental Results
C.1 Impact Training Dataset
We also evaluate the generalization of BCR on different training datasets. We choose DailyDialog
(Li al. 2017), Empathetic (Rashkin et al., 2019). Topical (Gopalakrishnan et al., 2019) as training datasets and test them DSTC1O dataset respectively. As shown in Table BCR framework brings significant progress to BERT all settings, which further demonstrates the strong generalization of BCR. We also notice that TDEM trained on DailyDialog performs better than other datasets, which explains why most TDEMs choose this dataset as training set.
C.2 BCR for Pre-training
In the future, supervised TDEM will probably achieve higher consistency with human judgements. But self-supervised TDEM can still serve as pre-trained model to help supervised TDEM converge faster better performance under limited labeled data volume conditions. Thus, we compared the performance of supervised TDEM with TDEM that first applied BCR pre-training and then underwent supervised fine-tumine tine-tuning. Specifically, we split FED dataset into train set and test in a ratio of 7:3 and the train set is served for supervised learning. We finetuned vanilla BERT and BERT-BCR trained on DailyDialog on the labeled train and test them respectively. Supervised TDEM converged on the 5th epochs and attained 0.240) Spearman correlations while TDEM underwent both BCR pre-training and supervised finetuning converged on the 3thh epochs and attains 0.372 Spearman correlations. This preliminary experiment may inspire scholars to research and apply pre-trained TDEMs to improve the performance of reward models, thus attaining better LLMs.
C.3 Comparing with GPT4
Recently, some automatic evaluation methods based on LLMs (e.g., GPT-4) (Liu et al. Wang et al.. 2023) have been proposed to explore new paradigms for addressing TDEM. We have compared BCR with Liu al.] in Table D The results of Liu al. is obtained with GPT-4 (version: May 15, 2023). Even in the case of n=20 (self-consistency), BCR+BERT is still able to achieve better average results than G-EVAL. LLM based methods also have the following drawbacks: higher expenses; more time costs; much more computational cost and parameters; bias towards specific content. Therefore, how effectively combine BCR and LLM is of our future research directions.
C.4 Transferability to Other Tasks
We further discuss the possibility of migrating BCR to other self supervupe regression tasks. BCR can be transferred to other tasks, such as Factuality (Falke et al. 2019 Wang et al. 2020), Data to Text (Mairesse et al.) 2010), Text Summarization FFbbri al. 2021). etc.
We take Text Summarization task for an example, As for TSR, in the first step, we can obtain positive samples with medium coherence by randomly deleting or replacing sentences in the text part of the positive samples; we can also obtain negative samples with medium coherence by randomly deleting or replacing sentences in the summarization part of the positive samples. Based on this, TSR can be implemented according to Eq. (5). As for DP loss, we directly use Eq. X to train BCR. It turns out that BERT with DP loss can obtain 0.243 Spearman correlations and -0.013 Diff. f. both better than Triplet loss (0.216 Spearman correlations and -0.030 fif and MSE loss (0.205 Spearman correlations and -0.039 Dif.).
DD Further Discussion
In the task of automatic dialogue evaluation, the mainstream approach is to utilize the SSL framework. This is because constructing a manually annotated training dataset for automatic dialogue evaluation is highly challenging. For instance, once attempted to annotate evaluation dataset for TDEM,
17

---

Table 8: Spearman correlations of BERT and BERT+BCR trained on different datasets.
Training dataset
Method
JSALT
ESL
NCM
DT
DP
DSTC10
BERT
0.166
0.345
0.253
0.220
0.401
0.255
DailyDialog
BERT+BCR
0.272
0.531
0.336
0.305
0.428
0.316
BERT
0.149
0.301
0.129
0.140
0.271
0.157
Empathetic
BERT+BCR
0.215
0.375
0.161
0.166
0.304
0.191
BERT 0.082
0.201
0.113
0.197
0.277
0.152
Topical
BERT+BCR
0.124
0.240
0.135
0.253
0.309
0.194
Table 9; Spearman correlations of BCR and G-EVAL on GRADE datasets. represents GPT4
generating scores for each sample and taking the average (self-consistency).
Method
GDG
GDR
GCG
GCR
GEG
GER
Avg.
G-EVAL n=1
0.427
0,422
0.501
0.464
0.267
0.370
0.408
G-EVAL n=20
0.448
0.448
0.513
0.559
0.296
0.398
0.443
BERT+BCR
0.416
0.487
0.642
0.519
0.341
0.372
0.463
consisting of 600 instances. Each instance comprises a context and multiple responses. Annotators were tasked with assigning scores to these responses individually. Each instance was annotated by individuals reduce noise. Given the longer dialogue contexts, took approximately 2 minutes for single annotator to complete each instance. After data cleaning (removing samples with low internal consistency), approximately 500 instances remained. As result, annotating these 500 instances required a total of around 100 hours of annotation time. Based on this estimation, if we were to annotate a training set of the size of DailyDialog (13118 samples), we would need approximately 2623 hours of annotation time. Even so, this still does not guarantee that such training set would exhibit strong generalization across multiple dialogue domains. The requirement for high inter- annotator consistency and the characteristic of longer dialogue contexts significantly elevate the annotation cost of dialogue evaluation training sets. We believe that achieving better evaluation performance requires self-supervised (BCR) pre- pre-training approach to impart the evaluation model with generalized assessment capabilities. Subsequently, tine-tuning can carried out with limited annotated training data specific the data domain. Thus, look forward to the emergence of high-quality dialogue evaluation training sets the future.
18


Reviews:
[{"review_id": "0SqyjgX8Wh", "reviewer": "Reviewer_qdHL", "summary": "This paper proposes a novel turn-level dialogue evaluation framework, called Better Correlation and Robustness (BCR), which introduces pseudo-negative samples with real-valued pseudo labels to train a model that evaluates responses given a context and query. The authors also propose a new loss function that models a uniform score distribution for increased robustness, named DP loss. Through experiments, the authors demonstrate that the BCR framework can be applied to existing turn-level dialogue evaluation models, leading to improved performance in terms of correlation with human scores.", "questions": "- Can we explore alternative strategies to create augmented samples and soft pseudo labels? For instance, [1] considers not only the speaker but also the conversation partner of the speaker. What are other potential strategies that could be used for the algorithm?\n- Is the BCR framework robust against adversarial attacks, such as removing stopwords, replacing words with synonyms, or copying one of the utterances in the context as the generated response [1, 2, 3]?\n- Is the hyper-parameter beta in the DP loss sensitive? It undergoes multiple updates, so it appears crucial to determine an optimal value for improved performance.\n\n[1] Bak, JinYeong, and Alice Oh. \"Speaker Sensitive Response Evaluation Model.\" In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6376-6385. 2020.\n[2] Sai, Ananya B., Mithun Das Gupta, Mitesh M. Khapra, and Mukundhan Srinivasan. \"Re-Evaluating ADEM: A Deeper Look at Scoring Dialogue Responses.\" (2019).\n[3] Sai, Ananya B., Akash Kumar Mohankumar, Siddhartha Arora, and Mitesh M. Khapra. \"Improving dialog evaluation with a multi-reference adversarial dataset and large scale pretraining.\" Transactions of the Association for Computational Linguistics 8 (2020): 810-827.\n", "limitations": "This paper proposes an evaluation model for generated responses from an AI model, making it difficult to envision the potential negative societal impact of their work. However, I have one question related to this aspect.\nCan we assess social biases, such as gender and race, using the BCR framework? My concern is that if BCR does not consider social biases, it might inadvertently reinforce bias when applied to promote the development of a dialogue system and LLM training, as the authors mentioned in the Conclusion section. Since BCR primarily focuses on aligning the overall human score of the generated response, we need to consider not only total appropriateness but also other aspects, such as context consistency, factual consistency, and social unbiasedness. This question is meant for discussion, and I hope to hear the authors' insights on this matter.\n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "One of the main strengths of this paper is suggesting an interesting and robust method for evaluating generated responses in a conversation. The simplicity of the idea allows other researchers to easily adapt it to their own evaluation models. Furthermore, the authors present numerous experimental results demonstrating that the suggested framework outperforms the baseline model that does not incorporate the BCR framework.\n", "weaknesses": "I have some questions, so I hope to listen to the answers from the authors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive comments as well as constructive suggestions! Below, we discuss each of the reviewer's concerns in detail.\n> **Can we explore alternative strategies to create augmented samples and soft pseudo labels? For instance, [1] considers not only the speaker but also the conversation partner of the speaker. What are other potential strategies that could be used for the algorithm?**\n\nThe rationale behind applying TSR is to balance the model's evaluation capability for samples with different levels of coherence by supplementing training data with medium coherence. In addition to TSR, there may be other methods worth exploring. Here are some of our hypotheses:\n- As you mentioned, [1] considered four scenarios, including SC, SP, SS, and Random, to obtain discriminative hard negatives for training. We believe that combining our pseudo-label generation method with these scenarios could bring these samples closer to the real distribution, potentially resulting in negative samples of medium coherence (which depends on the level of coherence achieved through human evaluation of this data augmentation approach).\n- We can also consider using data from different domains. For example, we know a priori that dialogues in Dailydialog exhibit higher coherence, whereas coherence in collective discussion datasets from forums (such as Twitter) is relatively lower due to weaker targeted responses. Leveraging this prior knowledge, we can attempt to construct a training set with a more balanced coherence level.\n\n> **Is the BCR framework robust against adversarial attacks, such as removing stopwords, replacing words with synonyms, or copying one of the utterances in the context as the generated response ?**\n\nAs shown in lines 252-253, we attempted to test the performance of BCR against adversarial attacks through random deletion and synonym replacement. Experimental results (Figure 7) indicate that after applying BCR, TDEMs achieved significantly higher Diff values (-0.016 -> -0.005 for GRADE and -0.010 -> -0.004 for USR). According to the definition of Formula 10, a higher Diff value indicates lesser performance degradation under adversarial attacks, thereby demonstrating that BCR is effective in countering this type of attack. \n\nWe replaced responses with randomly selected sentences from the preceding context in the FED dataset and tested the average relative scores of BERT and BERT+BCR on such negative examples (average score of the model on these negative examples - average score of the model on the FED dataset). The results showed a relative score of -0.23 for BERT+BCR and a relative score of +0.15 for BERT. This indicates that the BCR framework is more effective in mitigating this type of noise and identifying such negative responses. We believe this is because treat utterances in the context as negative samples for training data construction in TSR, which theoretically enables better resistance against this type of attack.\n\n> **Is the hyper-parameter $\\beta$ in the DP loss sensitive? It undergoes multiple updates, so it appears crucial to determine an optimal value for improved performance.**\n\nWe observed that $\\beta$ tends to reach convergence after several epochs (generally before halfway through training) across different settings (the initial value of $\\beta$, etc). For example, in the experiments shown in Figure 8, among the six different values of ($s$,$l$), five groups of $\\beta$ converged to 100, while another group converged to 10. A $\\beta$ value of 100 appears to be a suitable choice, but for different settings (task, dataset, model), determining the optimal $\\beta$ value in advance is challenging. Therefore, we recommend using DP loss to assist in finding the best $\\beta$ value for a specific setting.\n\n> **Open Discussion: Bias issues in automatic dialogue evaluation.**\n\nIn fact, we are currently engaged in relevant research (the impact of shortcuts on ADEM). Preliminary validation results indicate that existing TDEMs do indeed exhibit some gender and racial biases (models tend to assign lower scores to female soldiers and male nurses, for instance). Therefore, investigating how to automatically assess and mitigate social biases in generated content using automated evaluation models is a valuable avenue for future research. Given the lack of prior work in this area, we are proposing some potential research directions for discussion:\n\n- The first direction involves creating a manually annotated dataset of social biases, training models through regression tasks to automatically score the bias in sentences.\n- If constructing a comparative dataset specifically targeting social biases proves to be labor-consuming or challenging, an alternative could be to perform data augmentation during self-supervised training of ADEM (such as replacing gender and race-related terms in the context-response), in order to mitigate the impact of bias shortcuts on model scoring.\n\nWe are more than willing to engage in further discussions on this matter, and any ideas would be greatly appreciated!"}, {"review_id": "bQ1yaWeRIp", "reviewer": "Reviewer_7oUE", "summary": "This manuscript aims at two problems in turn-level dialogue evaluation models (TDEMs), including lower correlations between human judgments and the self-supervised learning (SSL) framework on medium coherence samples, and the nonuniform score distribution. The authors also provide a theoretical analysis to demonstrate that nonuniform score distribution would hurt the robustness of TDEM. To tackle the problems, this work first presents a training set reconstructing (TSR) method, i.e., constructing hard pos/neg samples, to balance the polarized data distribution, then presents a dynamic penalty loss function (DP loss) to balance the score distribution. The proposed methods are evaluated on 17 benchmark datasets with BERT-based models, and exhibit good performance. More in-depth analyses are also included to verify the effectiveness and generalization of the methods.", "questions": "A small question about the TSR part. You exchange the order of U1 and U2 to create a pseudo-positive sample. Do the additional embeddings, like $E_A E_B E_A$, reorder to $E_B E_A E_A$ accordingly or keep it unchanged?", "limitations": "The limitations are discussed in Section 6. Authors also provide some reasonable explanations. I did not fund any potential negative societal impact.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Good motivation and clear problem formulation. The work studies an important and interesting problem in the field of dialogue evaluation.\n2. The manuscript is well-organized and clearly presented. It is easy for the readers to follow up.\n3. The proposed framework is solid and effective.\n3. Comprehensive experiments and evaluations.\nThe overall quality of this manuscript surpasses the borderline of acceptance.", "weaknesses": "1. The novelty is not very significant. The techniques of TSR are commonly used in dialogue systems.\n2. I'd like to see the experiments with other PrLMs, apart from BERT.\n3. There are seemingly many parameters needing to search, including the initial $\\beta$, N in TSR, the values of $s$ and $l$, etc. How do I select the best parameters on different datasets?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive comments as well as constructive suggestions! Below, we discuss each of the reviewer's concerns in detail.\n> **The novelty is not very significant. The techniques of TSR are commonly used in dialogue systems.**\n\nWe highly value your feedback and would also like to provide the following addition: TSR has been demonstrated in the manuscript as an effective method for reconstructing a training dataset with a balanced coherence distribution. Despite its simplicity and intuitiveness, this method, compared to other methods contrasted in the article, exhibits a better ability to address the issue of insufficient moderately consistent samples mentioned in the article. Additionally, we consider the theoretical proof and empirical validation (via DP loss) of Theorem 1 to be innovative and significant (thanks once again for your acknowledgment). We believe that Theorem 1 and the proposed DP loss hold the potential for future applications in other tasks.\n\n> **I'd like to see the experiments with other PrLMs, apart from BERT.**\n\nUSR adopts RoBERTa as the base model. We conducted comparative tests on the effectiveness of USR combined with the BCR framework. As shown in Figure 7, USR+BCR outperforms USR in both Spearman correlations and robustness, consistent with the conclusion drawn when using BERT as the base model.\n\n> **There are seemingly many parameters needing to search, including the initial $\\beta$, N in TSR, the values of $s$ and $l$, etc. How do I select the best parameters on different datasets?**\n\nWe applied the BCR framework on different training sets and presented the results in Appendix C.1. Throughout this process, we maintained the following hyperparameter settings: initial $\\beta$ set to 1, N set to 20, and $s$ and $l$ set to 3 and 7, respectively. This set of configurations yielded favorable experimental outcomes across all trials. For a new setting, as $\\beta$ can be adaptively adjusted during training, it is not highly sensitive to the initial value; we suggest maintaining N within the range of [10, 30]. For clean dialogue datasets (e.g., Dailydialog), due to the relatively low volume of data with medium coherence, N can be appropriately increased for supplementation. Conversely, for datasets with higher noise levels (e.g., Twitter), where a larger amount of medium coherence data may exists, a slight reduction in N can be considered. As for ($s$ and $l$), we choose (3,7) in our setting as it can bring the highest $Uniformity$ (line 273). If you want to achieve better results for a new setting, we suggest adjusting hyperparameters based on the loss on the validation set and the $Uniformity$ of the model score distribution.\n\nWe will incorporate this section of content in the revised version. Thank you for your valuable suggestion.\n\n> **A small question about the TSR part. You exchange the order of U1 and U2 to create a pseudo-positive sample. Do the additional embeddings, like $E_AE_BE_A$, reorder to $E_BE_AE_A$ accordingly or keep it unchanged?**\n\nWe keep the additional speaker embeddings unchanged when creating a pseudo-positive sample. Otherwise, the only difference between the pseudo-positive sample and the positive sample at the input end will be the change in positional embeddings, which we consider is insufficient to distinguish it from the positive sample."}, {"review_id": "1bUekLEerl", "reviewer": "Reviewer_z9qv", "summary": "This paper proposes a distribution-balanced self-supervised learning framework called Better Correlation and Robustness (BCR) to tackle the challenges of weak performance in medium coherence samples and nonuniform score distribution. Specifically, BCR contains an effective training set reconstructing method to provide coherence-balanced training signals and a loss function which can adjust adaptively based the uniformity of score distribution. Experimental results show the superior performance and robustness of the proposed method.", "questions": "I have included my questions in the weaknesses part.", "limitations": "The authors have adequately addressed the limitations.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed method is overall sound and effective. It may be extended to self-supervised learning (SSL) frameworks for other NLP tasks in addition to dialogue evaluation.\n\n2. Extensive experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed method.", "weaknesses": "1. The research challenges tackled by this paper are limited to a specific framework (i.e., SSL) for a specific task (i.e., dialogue evaluation). Thus, the contribution is somewhat narrow especially for a general machine learning conference like NeurIPS. The selected SSL models including GRADE and USR in the experiment are both proposed in 2020. As we are already in 2023, I think that SSL may not be the most mainstream method for dialogue evaluation tasks. Unsupervised evaluation methods based on large language models (LLMs) like ChatGPT / GPT-4 have shown impressive performance in the evaluation of various natural language generation models [1]. It’s obvious that the proposed method which needs to train the evaluation model (such as BERT-base) on specific datasets cannot adapt to unsupervised methods based on LLMs. Thus, the technical contribution may be further weakened.\n\n2. Many parts in the proposed method need more intuitive explanations or theoretical support. For example, from Line 197 to Line 200, the authors set many numbers such as the boundary of polarized / centralized interval and the multiple to compute new $\\beta$. I wonder whether these numbers are optimal for the method or they are just an empirical try. Also, there are many other important hyper-parameters such as $s$ and $l$ (Equation 8). The authors should further explain how to tune these hyper-parameters.\n\n3. From Table 5, the SOTA baselines in the main results are mostly proposed before 2021, which are outdated. The authors should select recent SOTA baselines for comparison to judge the performance of the proposed method.\n\n\n[1] GPTEVAL: NLG Evaluation using GPT-4 with Better Human Alignment.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments as well as constructive suggestions! Below, we discuss each of the reviewer's concerns in detail.\n\n> **The research challenges tackled by this paper are limited to a specific framework (i.e., SSL) for a specific task (i.e., dialogue evaluation). Thus, the contribution is somewhat narrow especially for a general machine learning conference like NeurIPS.**\n\nWe rigorously proved Theorem 1 through formula derivation and conducted experimental validation. Theorem 1 is universal and makes a significant contribution to enhancing the robustness of regression task prediction models. From this perspective, our contribution is far-reaching. Additionally, efficient and accurate evaluation of LLMs can effectively promote scholars' assessment and research on the performance of LLMs. Therefore, research on automated dialogue evaluation holds significant significance.\n\n> **The selected SSL models including GRADE and USR in the experiment are both proposed in 2020. As we are already in 2023, I think that SSL may not be the most mainstream method for dialogue evaluation tasks. Unsupervised evaluation methods based on large language models (LLMs) like ChatGPT / GPT-4 have shown impressive performance in the evaluation of various natural language generation models [1]. It’s obvious that the proposed method which needs to train the evaluation model (such as BERT-base) on specific datasets cannot adapt to unsupervised methods based on LLMs. Thus, the technical contribution may be further weakened.**\n\nIn fact, we are also tracking the unsupervised evaluation methods based on LLM that you mentioned [1] [2]. In order to align with our model (assessing coherence between responses and context), we reproduced the [1] method on the [3] dataset and obtained the following experimental results with GPT-4 (version: May 15, 2023) (We have stored the data generated during this experiment  in an anonymous link and sent it to AC. According to the meeting requirements, we cannot place this link here, but we are willing to make it public in the future):\n\n|Methods|GDG|GDR|GCG|GCR|GEG|GER|Average|\n| :- | :-:| :-: | :-: | :-: | :-: | :-: | :-: |\n|G-EVAL n=1|0.427|0.422|0.501|0.464|0.267|0.370|0.408|\n|G-EVAL n=20|0.448|0.448|0.513|0.559|0.296|0.398|0.443|\n|BCR|0.416|0.487|0.642|0.519|0.341|0.372|**0.463**|\n\nThe parameter n in represents GPT4 generating n scores for each sample and taking the average (a type of Self consistency). The experimental results indicate that across 6 test sets, BCR have surpassed G-EVAL by an average of 4.5% (0.02) , achieving a higher level of coherence with humans. In the field of automated evaluation, LLM still cannot fully replace medium-scale fine-tuned models (BCR).\n\nFurthermore, LLM encounters the following issues when performing dialogue automated evaluation tasks:\n- Higher expenses\n- More time costs\n- Much more computational cost and parameters\n- Bias towards specific content (e.g., LLM-generated content tends to receive higher scores compared to human responses [1]).\n\nTherefore,  current LLM based evaluation framework still needs further development. In fact, we believe that a more reasonable approach in the future is to transfer the knowledge of LLM to medium-scale models in an unbiased manner. Under this notion, BCR can serve as an appropriate pre-training or fine-tuning strategy. Hence, BCR holds significant importance both now and in the future.\n\n> **Many parts in the proposed method need more intuitive explanations or theoretical support. For example, from Line 197 to Line 200, the authors set many numbers such as the boundary of polarized / centralized interval and the multiple to compute new $\\beta$. I wonder whether these numbers are optimal for the method or they are just an empirical try. Also, there are many other important hyper-parameters such as $s$ and $l$ (Equation 8). The authors should further explain how to tune these hyper-parameters.**\n\n The boundary of the polarized interval ([0, 0.25] and [0.75, 1]) and the centralized interval ([0.25, 0.75]) is a natural (and only) selection to ensure that both the polarized interval and the centralized interval each occupy half and do not lean towards any extreme. Setting the update threshold to 0.6 is a natural trial value. We chose 0.6 based on two principles: firstly, we aim for a uniformly distributed final score distribution. Therefore, when a certain interval deviates from 0.5, we should update the loss formulation to correct the bias. Hence, this threshold should be close to 0.5. Secondly, we want to avoid updating the loss formulation too frequently, so this threshold should not be too close to 0.5. Therefore, we chose 0.6, believing that further improvement in BCR's effectiveness can be achieved through hyperparameter searching. As for ($s$ and $l$), we choose (3,7) in our setting as it can bring the highest $Uniformity$ (line 273). We demonstrate in Figure 8 that different values of $s$ and $l$ generally achieve good results.\n\nWe will incorporate this section of content in the revised version. Thank you for your valuable suggestion.\n\n> **From Table 5, the SOTA baselines in the main results are mostly proposed before 2021, which are outdated. The authors should select recent SOTA baselines for comparison to judge the performance of the proposed method.**\n\nTable 1 includes the first-place solution for the DSTC10 Dialogue Evaluation Track in 2022 (DSTC being a top international competition in the field of dialogue). Furthermore, as mentioned earlier, we have also compared our results with the G-EVAL method (May 2023), which you kindly suggested, and achieved better performance. Therefore, we believe that we have compared BCR to state-of-the-art methods.\n\n[1] GPTEVAL: NLG Evaluation using GPT-4 with Better Human Alignment.\n\n[2] Is ChatGPT a Good NLG Evaluator? A Preliminary Study.\n\n[3] GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems."}, {"review_id": "8EIiG8l1AO", "reviewer": "Reviewer_yNip", "summary": "This paper introduces a turn-level dialogue evaluation model (TDEM) and focuses on the self-supervised learning (SSL) framework, which has recently demonstrated state-of-the-art (SOTA) performance in open-domain dialogue evaluation. \n\nThe authors identify two potential problems with existing SSL methods for training TDEMs. Firstly, these methods tend to have low correlations with humans on medium coherence samples. Secondly, the SSL approach leads to a nonuniform score distribution in TDEM. \nIn response to these drawbacks, the paper presents the \"Better Correlation and Robustness (BCR)\" method, which enables distribution-balanced self-supervised learning for TDEM. \nThis method includes two techniques, TSR and DP, and is applied during the SSL training of TDEM. TSR is responsible for reconstructing a coherence distribution balanced training set with a continuous label domain. DP loss, on the other hand, adjusts adaptively based on the score distribution estimated by kernel density estimation to achieve a more uniform distribution. \n\nThe authors conducted experiments on 17 benchmark datasets and found that the vanilla BERT-base with the proposed BCR method outperforms current SOTA methods by an average of 11.3%. \nAdditionally, the experiments also demonstrate the generalization ability of the proposed method by combining the proposed method with the current SOTA methods.", "questions": "* I am not very knowledgeable about dialogue evaluation, but one potential solution to address the scarce issue of score distribution could be to increase the amount of human evaluation data for the range of scores that are currently scarce, such as medium correlation scores that are discussed in this paper. However, the paper does not delve into the reasons why this straightforward approach cannot be implemented. Nowadays, acquiring a sufficient amount of human judgement data is not particularly challenging, so it may be worthwhile to explore this avenue further. Additionally, it is intriguing to consider the improvements achieved through the proposed method in terms of their equivalence to the number of human annotated samples. If this number is substantial and difficult to obtain, it could further enhance the effectiveness of the proposed method.\n\n* As explained in the weaknesssection", "limitations": "* Given that this paper focuses on evaluating dialogue systems, there is a potential risk of evaluating systems in an inaccurate manner. This risk is evident from the relatively not enough high correlation observed between the model's outputs and human judgments.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The motivation behind this paper is well-founded as it addresses a relevant and significant research gap. \n\n* The paper is generally easy to comprehend (although the mathematical components may require additional effort to fully grasp.)\n\n* The proposed method is captivating and demonstrates novelty, offering a fresh perspective on TDEMs. \n\n* The experimental results presented in the paper seem robust and compelling, providing substantial evidence to support the efficacy of the proposed approach (if we believe that the results are correct).", "weaknesses": "* This paper explores exclusively the narrow topic of turn-level dialogue evaluation. I understand that dialogue evaluation is a crucial research topic in the AI research field. While the paper demonstrates improved performance of the BCR method compared to current state-of-the-art (SOTA) methods in TDEMs, it lacks a discussion on the transferability and practicality of the proposed approach to other similar tasks in the literature. \nWhile this may not be a critical flaw that warrants an immediate rejection of the paper from the conference, it may pose a presentational drawback compared with other papers that explicitly discuss the potential broader applicability across various AI research areas.\n\n* Another limitation is the lack of discussion on the computational costs associated with implementing the proposed method. \n\n* The impact of the proposed method is unclear to me at least in the current status: first, as the Spearman correlation scores for all the methods, including the BCR method, seem to fall within a similar range (around 0.4), which is generally considered to indicate a low or moderate correlation.\nSecond, regarding the statistical test, The captions in Tables 1 and 4 say \"All results are statistically significant (p-value > 0.05).\" However, there is no explicit procedure how the authors confirmed this statistical significance. It should be explained in the paper, at least in the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive comments as well as constructive suggestions! Below, we discuss each of the reviewer's concerns in detail.\n\n> **This paper explores exclusively the narrow topic of turn-level dialogue evaluation. I understand that dialogue evaluation is a crucial research topic in the AI research field. While the paper demonstrates improved performance of the BCR method compared to current state-of-the-art (SOTA) methods in TDEMs, it lacks a discussion on the transferability and practicality of the proposed approach to other similar tasks in the literature.**\n\nBCR can be widely applied to various other tasks, such as evaluation of summarization, QA and so on. To demonstrate that, we choose summarization evaluation set SummEval [1] (relevance) and compared the performance of BERT with MSE,  Triplet, and DP loss respectively. As shown below:\n|Loss|Spearman|$Diff$|\n|:--|:--:|:--:|\n|MSE|0.205| -0.039|\n|Triplet|0.216|-0.030|\n|DP|**0.243**|**-0.013**|\n\nAs can be observed, compared to the commonly used losses for regression tasks, DP loss is capable of significantly enhancing the model's consistency (higher Spearman) and robustness (higher $Diff$) . We will further validate its effectiveness on additional tasks in the future. Thank you for your suggestions.\n\n> **Another limitation is the lack of discussion on the computational costs associated with implementing the proposed method.**\n\nBelow is a comparison of BCR with respect to parameter count, training costs, and computational costs to the methods we have examined. \n\n|Method|BCR+BERT|USR|GARDE|MME-CRS|\n|:-|:-:|:-:|:-:|:-:|\n|Parameter Counts (M)|110| 373.8|469|>435.2|\n|Training Costs (hours)|~2|~6|~7|>4|\n|Computational Costs|$T_{base}$|3*$T_{base}$|$T_{base}$+$T_{GNN}$|5*$T_{base}$|\n\n$T_{base}$ denotes the computational cost for a single pre-trained fine-tuned model in BCR and $T_{GNN}$ denotes graph neural network. We will incorporate this section of discussion in the revised version, thanks.\n\n> **The impact of the proposed method is unclear to me at least in the current status: first, as the Spearman correlation scores for all the methods, including the BCR method, seem to fall within a similar range (around 0.4), which is generally considered to indicate a low or moderate correlation.** \n\nPrior to BCR, SOTA automatic evaluation methods exhibited lower Spearman Correlations (seen Table 1), due to the challenging nature of dialogue evaluation, even for humans (0.63 inter-annotator correlation in [2]). The significance of our work lies in uncovering the inherent issue of uneven training data distribution in TDEM and theoretically demonstrating the relationship between the scoring distribution of evaluation models and robustness. By employing TSR and DP-loss, we elevated the BERT model to the latest SOTA performance and even surpassed GPT4 (see response to Reviewer z9qv), thereby fostering the advancement of future dialogue evaluation tasks.\n\n> **Second, regarding the statistical test, The captions in Tables 1 and 4 say \"All results are statistically significant (p-value > 0.05).\" However, there is no explicit procedure how the authors confirmed this statistical significance. It should be explained in the paper, at least in the appendix.**\n\nWe apologize that we made a typo here that the p-value should be less than 0.05 (generally less than 1e-3 in fact), and we will make corrections in the revised version. According to the definition of the p-value [3], when the p-value is less than the pre-set significance level (usually 0.05 in most papers), it implies that we have sufficient evidence to consider the observed data statistically significant.\n\n> **One potential solution to address the scarce issue of score distribution could be to increase the amount of human evaluation data for the range of scores that are currently scarce, such as medium correlation scores that are discussed in this paper. However, the paper does not delve into the reasons why this straightforward approach cannot be implemented. Nowadays, acquiring a sufficient amount of human judgement data is not particularly challenging, so it may be worthwhile to explore this avenue further. Additionally, it is intriguing to consider the improvements achieved through the proposed method in terms of their equivalence to the number of human annotated samples. If this number is substantial and difficult to obtain, it could further enhance the effectiveness of the proposed method.**\n\nWe highly acknowledge the potential of mitigating the issue of imbalanced training data by augmenting human-annotated data. But this has consistently been challenging. On one hand, extensive annotation incurs high costs in terms of both labor and time. On the other hand, achieving good consistency among human annotators is crucial and hard. This is also why a public labeled training set is practically non-existent in the dialogue evaluation field at present, except for the closed source ADEM[2] dataset in 2017. \n\nWe believe that achieving better evaluation performance requires a self-supervised (BCR) pre-training approach to impart the evaluation model with generalized assessment capabilities. Subsequently, fine-tuning can be carried out with limited annotated training data specific to the data domain.  In Appendix C.2, we measured the effects of fine-tuning BERT+BCR compared to directly fine-tuning BERT. This confirms that the training paradigm of SSL pre-training followed by supervised fine-tuning yields better results than direct supervised training (Spearman 0.372 > 0.240). Hence, exploring automatic evaluation under the SSL paradigm is of paramount significance. We greatly appreciate your interesting suggestion, and we look forward to completing this verification when a publicly available training set appears.\n\n[1] SummEval: Re-evaluating Summarization Evaluation\n\n[2] Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses\n\n[3] Statistical Methods for Research Workers\n"}, {"review_id": "qQtsy2nIFs", "reviewer": "Reviewer_WZUk", "summary": "This paper proposes a new framework to enhance prior TDEMs by utilizing a self-supervised learning approach. The authors introduce a simple and intuitive framework called BCR. The BCR framework employs two approaches: TSR, which reconstructs the training set using a heuristic to balance the coherence distribution, and DP loss, which ensures uniform score distribution. Experimental results demonstrate that their framework improves the correlation between machine judgment and human judgment, outperforming state-of-the-art TDEM approaches when applied to a vanilla BERT-based model.", "questions": "These questions are listed in order of priority:\n\n- How was hyperparameter tuning performed? How were the learning rate, batch size, number of epochs, s, and l for DP loss selected?\n  - I could not find details about the hyperparameter tuning, and if the experiments were not conducted based on a train/validation split, then the experimental results may not be reliable.\n  - If fair hyperparameter tuning was conducted, what criteria were used to select the hyperparameters? Validation loss is not directly suitable for selecting hyperparameters.\n- Figure 7 & Sec 5.2.2: I am having trouble understanding the explanation in Section 5.2.2; aren't the values in Figure 7 only represent \"Diffs,\" not \"Spearman correlations\"?\n- Table 2: In the Spearman correlations of TSR, the table shows that Polarized exhibits better correlations compared to medium. Isn't it natural? The Polarized samples indicate that the coherence is much easier to distinguish. More detailed explanations about the experiments would help in understanding the results.\n- Table 3: The explanation about Example 2 is unclear; \"BCR can bring better correlations with human judgments on medium coherence samples.\" Can you provide further explanation about this? When comparing USR and USR+BCR, I only observe a small gap between them.\n\n", "limitations": "I think the authors have addressed their limitations.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "I believe the motivation of the paper is clearly stated, though I feel confused about the key assumption of this paper (described in Weaknesses). The authors provide sufficient illustrations to demonstrate the limitations of current TDEM approaches. They propose an intuitive approach to address these limitations and support their claims with experimental results. Although their approach may not be technically novel, their insights are meaningful.", "weaknesses": "- The main assumption of this paper is that the test set consists of a uniformly distributed coherence, but I am not clear about the justification for this statement. It would be helpful if the authors could provide more background information on this assumption.\n\n- The keyword used in this work is \"medium coherence,\" which is initially challenging to understand. Both \"medium\" and \"coherence\" are ambiguous terms. What is the definition of \"medium\" and \"coherence\"? This lack of clarity weakens the soundness of the paper. In prior works that address automatic dialogue evaluation, most provide clear definitions of metrics such as coherence, interestingness, etc.\n\n- Some details about the experiments are insufficient and unclear. The paper would be more robust if the authors could clarify these aspects. I will provide detailed questions regarding the experimental details below (Questions).\n\nThough, I am willing to raise the current rating if the authors provide sufficient explanations about these concerns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive comments as well as constructive suggestions! Below, we discuss each of the reviewer's concerns in detail.\n> **The main assumption of this paper is that the test set consists of a uniformly distributed coherence, but I am not clear about the justification for this statement. It would be helpful if the authors could provide more background information on this assumption.**\n\nIn the field of dialogue evaluation, the datasets proposed in previous works are typically balanced (e.g., \"the distributions of human judgements are balanced from score 1 to 5\" from [1]). Intuitively, evaluation sets with more balanced score distributions are better suited for comprehensively assessing the performance of different dialogue models. (Imagine a test set with only 1-point and 5-point ratings, as opposed to evenly distributed ratings of 1, 2, 3, 4, and 5.)\n\n> **The keyword used in this work is \"medium coherence,\" which is initially challenging to understand. Both \"medium\" and \"coherence\" are ambiguous terms. What is the definition of \"medium\" and \"coherence\"? This lack of clarity weakens the soundness of the paper. In prior works that address automatic dialogue evaluation, most provide clear definitions of metrics such as coherence, interestingness, etc.**\n\nHere, \"medium coherence\" refers to samples with human ranking scores falling within the range [0.25, 0.75], indicating a level of consistency between the responses and the preceding context that lies between strong and weak. We appreciate your reminder, and we will incorporate the corresponding definition in the upcoming revisions.  Thank you for your valuable suggestion.\n\n\n> **How was hyperparameter tuning performed? How were the learning rate, batch size, number of epochs, s, and l for DP loss selected?**\n\nWe use the validation loss  and $Uniformity$ (only for ($s$,$l$)) as criterions for hyperparameter selection. As you mentioned, validation loss is not directly suitable for selecting hyperparameters. However, In order to align with the testing performance of previous research work, we did not split the evaluation set we used into a validation / test set (which would result in incomplete testing data). Instead, we directly used the loss on the Dailydialog validation set as a criterion for hyperparameter selection except for ($s$, $l$). As for ($s$, $l$), we choose (3,7) in our setting as it can bring the highest $Uniformity$ (line 273). As a matter of fact, if there is a validation set with high quality human annotations available (outside of evaluation set we used), the performance of model would be further improved after hyperparameter tuning. See Figure 8, if the values of (s, l) are chosen as (3,8), better results in both consistency and robustness can be achieved compared to the (3,7) we selected according to $Uniformity$. However, in order to ensure the rigor of the experiment, we did not choose (3,8) as the default option for our (s, l), but instead retained (3,7).\n\n> **Figure 7 & Sec 5.2.2: I am having trouble understanding the explanation in Section 5.2.2; aren't the values in Figure 7 only represent \"Diffs,\" not \"Spearman correlations\"?**\n\nAs illustrated in Figure 7, DSTC10, NCM, DT, DP, ESL, JSALT represent the model's \"Spearman Correlations\" on the respective datasets. The values under \"Diff\" (highlighted in red) indicate the model's average performance of Diff metric on all these datasets. We apologize for any confusion caused by the unclear description and we will clarify this point in the caption of Figure 7 in our revised version. Thanks for your reminder.\n\n> **Table 3: The explanation about Example 2 is unclear; \"BCR can bring better correlations with human judgments on medium coherence samples.\" Can you provide further explanation about this? When comparing USR and USR+BCR, I only observe a small gap between them.**\n\nRegarding Example 2 in Table 3 (U1: Why aren't you eating anything else? U2: Well, fruits and vegetables are very healthy. R: What kind of vegetables do you want to eat?), the response \"R: What kind of vegetables do you want to eat?\" is considered as medium coherence by human evaluators (with a corresponding ranking of 0.45). In comparison to USR (which has a ranking of 0.87), the alignment between USR+BCR and human evaluators is higher in consistency (with a ranking of 0.47, approaching the human ranking of 0.45), which suggests the conclusion (\"BCR can bring better correlations with human judgments on medium coherence samples.\").\n\n> **Table 2: In the Spearman correlations of TSR, the table shows that Polarized exhibits better correlations compared to medium. Isn't it natural? The Polarized samples indicate that the coherence is much easier to distinguish. More detailed explanations about the experiments would help in understanding the results.**\n\nWe calculated the Spearman scores for the \"polarized\" and \"medium\" samples separately as follows: For the polarized samples, we computed the model's Spearman performance on subsets of samples with labels within [0, 0.25] and [0.75, 1] respectively, and then averaged these values to obtain the polarized score. For the medium sample, we considered the model's Spearman performance on samples within the [0.25, 0.75] score range, and referred it as the medium score. In other words, we evaluated the model's performance for polarized and medium samples within their corresponding score ranges. Therefore, \"Polarized exhibits better correlations compared to medium\" is not a natural conclusion.\n\nIf we have any unclear explanations, please feel free to ask us any questions. We look forward to helping you solve your confusion!\n\n[1] GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems"}]

Author rebuttal:
We sincerely thank all the reviewers for their valuable feedback. We have provided detailed responses to these suggestions, and if there are still any questions that have left you confused, we eagerly look forward to providing further clarification for you.

As demonstrated by our presentation, our work provides a solid theoretical foundation for the robustness of regression tasks based on sequential relationships, and holds significant importance for advancing automatic dialogue evaluation tasks in the era of LLMs.

Looking forward to your further recognition of our work.

Meta-review: In this paper, the authors address issues in turn-level dialogue evaluation models (TDEMs), namely, lower correlations between human judgments and self-supervised learning (SSL) models and nonuniform score distributions. They propose a Training Set Reconstructing (TSR) method to balance data distribution and introduce a Dynamic Penalty (DP) loss function to mitigate score distribution disparities. The proposed methods are evaluated across 17 benchmark datasets with BERT-based models, yielding promising results. One notable strength of this work is its simple yet effective approach, making it adaptable for other researchers. The motivation and problem formulation are clear, and the manuscript's organization facilitates easy comprehension. The robustness and efficacy of the proposed framework are well-supported by comprehensive experiments and evaluations.

However, several weaknesses exist and should be addressed. The novelty of the TSR technique is limited, as one reviewer pointed out that it's commonly used in dialogue systems. The paper lacks guidance on hyper-parameter tuning, and there is uncertainty surrounding certain numerical choices within the method. Comparing the proposed method with up-to-date state-of-the-art baselines are required. While the motivation for this research is sound and the proposed method offers a fresh perspective on TDEMs, it primarily focuses on a narrow scope. A broader discussion on its applicability to other tasks and its computational costs would enhance the paper.
Decision: Accept (poster)