# An Adaptive Contextual Relation Model for Improving Response Generation
## 6 Experimental Results
### Ablation Study  
Our model contains three components: component1 contains two response generators (\(G_{F}\) and \(G_{B}\)) and a future generator \(G_{F2}\); component2 implements the learning of sentence relevance by annotated labels; component3 uses a simple generator Dec to predict the final response. In this part, we analyze the performance of the 3 response generators and the effect of our annotated labels. The performance of response generators. \(G_{F}\), \(G_{B}\), and Dec are 3 response generators fed with history, future, and [history, future, relevance label] as inputs, respectively. We first analyze their performance and then discuss their functionality from the perspective of different dialogue relations. Table 4 shows the results of the 3 generators. We first compare Ours\({}_{G_{F}}\) with BART because they have the same model relation and input, the only difference being that Ours\({}_{G_{F}}\) is also constrained by the final NLL loss from the Dec through back-propagating. Ours\({}_{G_{F}}\) shows significantly better results on BLEU-2\(\sim\)4 (+0.16, +0.3, +0.34), which indicates that the relevance-aware final loss facilitates the context modeling between history and response. Then, we compare Ours\({}_{G_{B}}\) with Ours\({}_{G_{F}}\). The difference between them is that Ours\({}_{G_{B}}\) uses \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Models** & **BLEU-1** & **BLEU-2** & **BLEU-3** & **BLEU-4** & **Dist-1** & **Dist-2** \\ \hline HRED & 17.19 & 7.76 & 4.35 & 2.53 & 1.67 & 6.41 \\ NEXUS & 17.33 & 7.40 & 4.10 & 2.34 & 2.46 & 8.13 \\ Transformer & 19.69 & 10.15 & 6.43 & 4.37 & 2.18 & 13.94 \\ RegDG & 21.14 & 11.04 & 7.39 & 5.48 & 2.12 & 9.74 \\ D2 GPO & 22.17 & 11.98 & 7.86 & 5.55 & 2.08 & 14.87 \\ ProphetChat & 23.21 & 14.17 & 10.62 & 8.25 & 4.05 & 22.67 \\ AdaLabel & 24.16 & 14.80 & 10.85 & 8.63 & 3.95 & 22.20 \\ HDLD & 27.52 & 18.26 & 14.30 & 12.02 & 4.37 & 23.23 \\ iVAE\({}_{MI}\) & 30.90 & 24.90 & - & - & 2.90 & 25.00 \\ PLATO & 39.70 & 31.10 & - & - & 5.30 & 29.10 \\ PLATO\({}_{w/oLatent}\) & 40.50 & 32.20 & - & - & 4.60 & 24.60 \\ Dual & 40.80 & 35.00 & 32.70 & 31.50 & 6.60 & 33.00 \\ Ours & **41.05** & **35.61** & **33.51** & **32.45** & **7.01** & **38.83** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Main results of our model with all evaluation metrics, the bold font in the table represents the best result value. the simulated future generated by \(\text{Ours}_{G_{F2}}\) as input. However, the performance of \(\text{Ours}_{G_{B}}\) is not decreased much, which not only shows that our method of simulating the future by history in \(\text{Ours}_{G_{F2}}\) is feasible but also demonstrates that the extremely difficult backward generation can still perform well via our model design. \(\text{Ours}_{Dec}\) achieves the best results in all metrics, which proves that it can reasonably combine the strengths of both \(\text{Ours}_{G_{F}}\) and \(\text{Ours}_{G_{B}}\) with the help of relevance labels to achieve better results. Further, we compare the 3 response generators on different dialogue relations in Table 5. \(\text{Ours}_{G_{F}}\) obtains better results than \(\text{Ours}_{G_{B}}\) on relation1, while \(\text{Ours}_{G_{B}}\) has better performance on relation2. This shows that with the help of sentence relevance labels, \(\text{Ours}_{G_{F}}\) and \(\text{Ours}_{G_{B}}\) in our framework are more focused on learning the relations that they correspond to. And \(\text{Ours}_{Dec}\) obtains the best performance in all the relations, which indicates that it can make good use of the annotated labels and further handle different dialogue relations flexibly. The effect of sentence relevance labels. To validate the effect of the relevance labels, we design two alternative methods for obtaining \(w_{1}\) and \(w_{2}\) without the help of labels based on our model framework: (1) assigning fixed values 0.5 to \(w_{1}\) and \(w_{2}\), called \(\text{Ours}_{fix}\); and (2) letting the model learn \(w_{1}\) and \(w_{2}\) by itself implicitly, called \(\text{Ours}_{learn}\).