SlowGuess/NAACL_2021_OCR huggingface dataset:
Models	Ubuntu	Douban	E-commerce
\(R_{10}@1\)	\(R_{10}@2\)	\(R_{10}@5\)	MAP	MRR	P@1	\(R_{10}@1\)	\(R_{10}@2\)	\(R_{10}@5\)	\(R_{10}@1\)	\(R_{10}@2\)	\(R_{10}@5\)
TF-IDF (Lowe et al., 2015)	0.410	0.545	0.708	0.331	0.359	0.180	0.096	0.172	0.405	0.159	0.256	0.477
RNN (Lowe et al., 2015)	0.403	0.547	0.819	0.390	0.422	0.208	0.118	0.223	0.589	0.325	0.463	0.775
CNN (Kadlec et al., 2015)	0.549	0.684	0.896	0.417	0.440	0.226	0.121	0.252	0.647	0.328	0.515	0.792
LSTM (Kadlec et al., 2015)	0.638	0.784	0.949	0.485	0.537	0.320	0.187	0.343	0.720	0.365	0.536	0.828
SMN (Wu et al., 2017)	0.726	0.847	0.961	0.529	0.569	0.397	0.233	0.396	0.724	0.453	0.654	0.886
DUA (Zhang et al., 2018)	0.752	0.868	0.962	0.551	0.599	0.421	0.243	0.421	0.780	0.501	0.700	0.921
DAM(Zhou et al., 2018)	0.767	0.874	0.969	0.550	0.601	0.427	0.254	0.410	0.757	0.526	0.727	0.933
IOI (Tao et al., 2019)	0.796	0.894	0.974	0.573	0.621	0.444	0.269	0.451	0.786	0.563	0.768	0.950
ESIM (Chen and Wang, 2019)	0.796	0.894	0.975	-	-	-	-	-	-	0.570	0.767	0.948
MSN (Yuan et al., 2019)	0.800	0.899	0.978	0.587	0.632	0.470	0.295	0.452	0.788	0.606	0.770	0.937
BERT (Gu et al., 2020)	0.808	0.897	0.975	0.591	0.633	0.454	0.280	0.470	0.828	0.610	0.814	0.973
RoBERTa-SS-DA (Lu et al., 2020)	0.826	0.909	0.978	0.602	0.646	0.460	0.280	0.495	0.847	0.627	0.835	0.980
BERT-DPT (Whang et al., 2020)	0.851	0.924	0.984	-	-	-	-	-	-	-	-	-
BERT-VFT (Whang et al., 2020)	0.855	0.928	0.985	-	-	-	-	-	-	-	-	-
SA-BERT (Gu et al., 2020)	0.855	0.928	0.983	0.619	0.659	0.496	0.313	0.481	0.847	0.704	0.879	0.985
UMSBERT+ (Whang et al., 2021)	0.875	0.942	0.988	0.625	0.664	0.499	0.318	0.482	0.858	0.762	0.905	0.986
BERT-SL (Xu et al., 2021)	0.884	0.946	0.990	-	-	-	-	-	-	0.776	0.919	0.991
BERT-FP (diff. %p)	0.911(+2.7)	0.962(+1.6)	0.994(+0.4)	0.644(+1.9)	0.680(+1.6)	0.512(+1.3)	0.324(+0.6)	0.542(+4.7)	0.870(+1.2)	0.870(+9.4)	0.956(+3.7)	0.993(+0.2)
Table 2: Model comparison for the three benchmarks. BERT-SL and UMS  ${}_{BERT + }$  are the previous state-of-the-art models
- UMS $_{BERT+}$ : Whang et al. (2021) proposed a multi-task learning framework consisting of three tasks (i.e., utterance insertion, deletion, and search).
- BERT-SL: Xu et al. (2021) introduced four self-supervised tasks and trained the response selection model with these auxiliary tasks in a multi-task manner.
Table 2 shows the performance of the proposed BERT-FP that is evaluated on three benchmarks. As you can see in the results, the proposed model outperformed all of the other models used as baselines. In comparison to the vanilla model of BERT, our model achieved an absolute improvement in  $R_{10}@1$  by  $10.3\% \mathrm{p}$ ,  $4.4\% \mathrm{p}$ , and  $26\% \mathrm{p}$  on Ubuntu Corpus V1, Douban Corpus, and E-commerce Corpus, respectively. Compared to BERT-DPT, our model achieved an absolute improvement of  $6\% \mathrm{p}$  in  $R_{10}@1$  on the Ubuntu Corpus. These results indicate that fine-grained post-training, which reflects the dialogue's characteristics, is superior to the previous post-training. In comparison to the previous state-of-the-art models,  $\mathrm{UMSBERT+}$  and BERT-SL, our model achieved an improved performance by a large margin in terms of all the metrics for the three benchmarks. These results demonstrate that our method effectively learns the semantic relevance and coherence between the internal utterances, which enhances selection performance significantly.
Figure 3: Performance according to post-training context length
5.1 Performance across Different Lengths of Short Context
Figure 3 shows the performance variations of BERT-FP depending on the length of the short context. In this experiment, we trained the models with  $10\%$  of the training set and evaluated them with the entire test set to perform many experiments. Therefore, they achieved lower performance. For the Ubuntu Corpus and E-commerce Corpus, the best performance in  $R_{10}@\mathbf{1}$  is achieved when the context length is three. For Douban Corpus, we evaluated performance with MAP rather than
Models	R10@1	R10@2	R10@5
BERT-NSP	0.904	0.960	0.994
BERT-SOP	0.865	0.935	0.987
BERT-URC	0.911	0.962	0.994
Table 3: Performance according to the training objective on Ubuntu Corpus V1.
Models	R10@1	R10@2	R10@5
BERT(Gu et al., 2020)	0.808	0.897	0.975
+MLM	0.851 (+4.3%p)	0.925	0.983
+MLM+NSP	0.864 (+1.3%p)	0.934	0.987
+MLM+NSP_SCR	0.904 (+4.0%p)	0.960	0.994
+MLM+URC_SCR (Ours)	0.911 (+0.7%p)	0.962	0.994
$R_{10}@1$  because it may have multiple correct responses in the candidates. The best performance in  $MAP$  on Douban Corpus is achieved when the context length is set to two.
5.2 Performance according to Training Objective
We compared the proposed training objective (URC) with the previous training objectives (NSP, SOP). Table 3 demonstrates that our training objective outperforms the other training objectives. This indicates that learning both topics and the coherence between the internal utterances is important.
We investigated the impact of each part of the fine-grained post-training method through a series of ablation experiments on the Ubuntu Corpus in Table 4. The model without post-training (BERT) is used as the baseline. Then, we gradually applied our methods for post-training. +MLM indicates that the model is post-trained only with the MLM. The "SCR" suffix denotes the model that is post-trained with the short context-response pairs. The comparison between +MLM and +MLM+NSP shows that the NSP during the existing post-training has little effect on the performance. However, as shown in the comparison between +MLM and MLM+NSP_SCRR, the NSP trained with a short context-response pair significantly improved the model performance. The experimental results also showed that using URC instead of NSP enhances performance.
5.4 Comparison with the Data Augmentation
The post-training method has the effect of data augmentation. However, it differs from the usual
Table 4: Ablation study on Ubuntu Corpus V1
Models	R10@1	R10@2	R10@5
BERT (Gu et al., 2020)	0.808	0.897	0.975
BERT-DA	0.880	0.946	0.990
BERT-FP	0.911	0.962	0.994
Table 5: Comparing the data augmentation on Ubuntu Corpus V1
Models	R10@1	R10@2	R10@5
BERT (Gu et al., 2020)	0.808	0.897	0.975
BERT-FP-NF	0.862	0.933	0.986
BERT-FP	0.911	0.962	0.994
Table 6: The effectiveness of fine-grained post-training for response selection on Ubuntu Corpus V1
data augmentation method, which directly augments the data in the fine-tuning step. Therefore, we compared the fine-grained post-training (BERT-FP) method with the typical data augmentation (BERT-DA) on the Ubuntu Corpus. The data augmentation strategy is similar to the method used in Chen and Wang (2019). We considered each utterance as a response and its previous utterances as its context. The experimental results are shown in Table 5. BERT-FP outperforms the data augmentation model (BERT-DA) by a  $3.1\%$  p in  $R_{10}@1$ . The significant improvement demonstrates the effectiveness of the proposed method in comparison to the data augmentation. Our method, including post-training and fine-tuning steps, is about 2.5 times faster than BERT-DA. In particular, the post-trained model takes much less time to fine-tune than BERT-DA, making them easy to adapt to various applications.
5.5 The Effectiveness of Fine-grained Post-Training for Response Selection Task
To demonstrate the effectiveness of the fine-grained post-training method for the response selection task, we compared three different models: BERT, BERT-FP, and BERT-FP-NF (no fine-tuning). BERT-FP-NF is a model that was post-trained and evaluated without fine-tuning. As shown in Table 6, the performance of BERT-FP-NF is close to BERT-FP, which is fine-tuned. These results show that even before fine-tuning to the response selection task, our fine-grained post-training alone could measure the matching degree between the context and the response.
In this paper, we have proposed a new fine-grained post-training method that is suitable for the multiturn dialogue. The proposed method allows the matching model to learn the semantic relevance and the coherence of the utterances in the dialogue, and it improves the model's capability to select the appropriate response. The experimental results on the three benchmark datasets demonstrate our post-training method's superiority for the response selection. From this, our model achieved a new state-of-the-art performance for all three benchmarks.
In the future, we plan to research new posttraining methods that are suitable for a variety of tasks, such as question answering and dialogue generation.
This work was supported by the Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques).
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Debanjan Chaudhuri, Agustinus Kristiadi, Jens Lehmann, and Asja Fischer. 2018. Improving response selection in multi-turn dialogue systems by incorporating domain knowledge. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 497-507, Brussels, Belgium. Association for Computational Linguistics.
Qian Chen and Wen Wang. 2019. Sequential attention-based network for noetic end-to-end response selection. CoRR, abs/1901.02609.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Jia-Chen Gu, Tianda Li, Quan Liu, Zhen-Hua Ling, Zhiming Su, Si Wei, and Xiaodan Zhu. 2020. Speaker-aware BERT for multi-turn response selection in retrieval-based chatbots. In CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020, pages 2041-2044. ACM.
Donghoon Ham, Jeong-Gwan Lee, Youngsoo Jang, and Kee-Eung Kim. 2020. End-to-end neural pipeline for goal-oriented dialogue systems using GPT-2. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 583-592, Online. Association for Computational Linguistics.
Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz, and Richard Socher. 2020. A simple language model for task-oriented dialogue.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 2790-2799. PMLR.
Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Rudolf Kadlec, Martin Schmid, and Jan Kleindienst. 2015. Improved deep learning baselines for ubuntu corpus dialogs. ArXiv, abs/1510.03753.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In SIGDIAL Conference.
Junyu Lu, Xiancong Ren, Yazhou Ren, Ao Liu, and Zenglin Xu. 2020. Improving contextual language models for response retrieval in multi-turn conversation. In Proceedings of the 43rd International ACM
SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, pages 1805-1808. ACM.
Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412-1421, Lisbon, Portugal. Association for Computational Linguistics.
Chongyang Tao, Wei Wu, Can Xu, Wenpeng Hu, Dongyan Zhao, and Rui Yan. 2019. One time of interaction may not be enough: Go deep with an interaction-over-interaction network for response selection in dialogues. In ACL.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pages 5998-6008.
Jesse Vig and Kalai Ramea. 2019. Comparison of transfer-learning approaches for response selection in multi-turn conversations. In Workshop on DSTC7.
Taesun Whang, Dongyub Lee, Chanhee Lee, Kisu Yang, Dongsuk Oh, and Heuseok Lim. 2020. An effective domain adaptive post-training method for BERT in response selection. In Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China, 25-29 October 2020, pages 1585-1589. ISCA.
Taesun Whang, Dongyub Lee, Dongsuk Oh, Chanhee Lee, Kijong Han, Dong-hun Lee, and Saebyeok Lee. 2021. Do response selection models really know what's next? utterance manipulation strategies for multi-turn response selection. Proceedings of the AAAI Conference on Artificial Intelligence.
Xianchao Wu, Ander Martínez, and Momo Klyen. 2018. Dialog generation using multi-turn reasoning neural networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2049-2059, New Orleans, Louisiana. Association for Computational Linguistics.
Yu Wu, Wei Wu, Chen Xing, Ming Zhou, and Zhoujun Li. 2017. Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 496-505. Association for Computational Linguistics.
Hu Xu, Bing Liu, Lei Shu, and Philip Yu. 2019. BERT post-training for review reading comprehension and aspect-based sentiment analysis. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2324-2335, Minneapolis, Minnesota. Association for Computational Linguistics.
Ruijian Xu, Chongyang Tao, Daxin Jiang, Xueliang Zhao, Dongyan Zhao, and Rui Yan. 2021. Learning an effective context-response matching model with self-supervised tasks for retrieval-based dialogues. Proceedings of the AAAI Conference on Artificial Intelligence.
Z. Xu, B. Liu, B. Wang, C. Sun, and X. Wang. 2017. Incorporating loose-structured knowledge into conversation modeling via recall-gate LSTM. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 3506-3513.
Chunyuan Yuan, Wen jie Zhou, MingMing Li, Shangwen Lv, Fuqing Zhu, Jizhong Han, and Songlin Hu. 2019. Multi-hop selector network for multi-turn response selection in retrieval-based chatbots. In EMNLP/IJCNLP.
Zhuosheng Zhang, Jiangtong Li, Pengfei Zhu, Hai Zhao, and Gongshen Liu. 2018. Modeling multi-turn conversation with deep utterance aggregation. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pages 3740-3752. Association for Computational Linguistics.
Xiangyang Zhou, Lu Li, Daxiang Dong, Yi Liu, Ying Chen, Wayne Xin Zhao, Dianhai Yu, and Hua Wu. 2018. Multi-turn response selection for chatbots with deep attention matching network. In ACL.