SlowGuess/NAACL_2022_OCR huggingface dataset:
Model	Recommendation metrics	Language generation metrics
R@1	R@10	R@50	ReR	PPL	Dist2	Dist3	Dist4	Bleu2	Bleu4
ReDial	2.4	14.0	32.0	0.7	28.1	0.225	0.236	0.228	0.178	0.074
KBRD	3.1	15.0	33.6	0.8	17.9	0.263	0.368	0.423	0.185	0.074
KGSF	3.9	18.3	37.8	0.9	5.6	0.289	0.434	0.519	0.164	0.074
CR-Walker	4.0	18.7	37.6	-	-	-	-	-	-	-
CRFR	4.0	20.2	39.9	-	-	-	-	-	-	-
RID	-	-	-	3.1	54.1	0.518	0.624	0.598	0.204	0.110
NTRD	-	-	-	1.8	4.4	0.578	0.820	1.005	-	-
MESE	5.6	25.6	45.5	6.4	12.9	0.822	1.152	1.313	0.246	0.143
Table 2: Results and comparison with the literature on REDIAL.
top k	Ranking Acc	Recall Acc	Final Acc
@1	7.2	0.778	5.6
@10	33.0	0.778	25.6
@50	58.5	0.778	45.5
models by a large margin. We argue in 5.2 that this significant gain of performance is due to the effectiveness of the item encoder. MESE also performs well on the ReR score, which indicates that the filling placeholder tokens can help integrate recommended items into responses. For language generation, MESE also achieves significantly better performance than all other models on distinct ngrams and bleu scores with the exception that the PPL is worse than those of KGSF and NTRD. This indicates that MESE can generate more diverse responses while sticking to the topic.
Table 3: Recall, Ranking and Final Accuracy of MESE.
Model	Fluency	Informativeness
KGSF	24%	19%
MESE	38%	31%
Table 4 presents the results of human evaluation. Our proposed model MESE outperforms KGSF by a large margin on both fluency and informativeness. Responses of MESE have a  $50\%$  more chance of being chosen as the better answer than responses of KGSF. By using the encoded item embeddings and joint training, MESE can better integrate its pretrained weights with the injected item information. Therefore, it generates more fluent responses that contain richer information about the items.
In this section, we first analyze the reason behind the performance gain of our recommendation module by analyzing the embeddings learned by the item encoder.
How much does metadata help recommendation? We argue that our training objectives on recommendation enable the item encoder to selectively extract useful features pertinent to the recommendation task from item metadata and construct item representations that resonate with instructional semantic properties in the dialog histories. For example, in REDIAL dataset, movie genre information is the most frequently mentioned property in dialog histories and human recommenders often make recommendation decisions based on this property. Although other properties like actors also help with recommendations, they do not appear in the corpus as often as genres or movie plots. We designed the following experiments to test our hypothesis. First, we train MESE with movie genre and plot information removed from the metadata, which we refer to as MESE w/o content, and compare its recommendation performance with MESE in Table 5.
Table 4: Human Evaluation of Response Generation
Model	R@1	R@10	R@50
MESE w/o content	3.9	19.5	37.9
MESE	5.6	25.6	45.5
Table 5: Comparison Results of MESE and MESE w/o content.
As we can see from the table, there is a significant performance decrease after we remove genre and plot information, which indicates that MESE depends on the item information to make high-quality recommendations. We also point out that movie titles contain weak genre information but
are not able to provide adequate features for the item encoder to extract from.
How does the item encoder help recommendation? We claim that the item encoder can construct embeddings in a systematic way that aligns matching information between its input and dialog context. We designed an experiment to prove the point. Specifically, we select all movie items with only one genre as our candidates, resulting in a subset of  $\sim 700$  movies. We then select 2 item encoders (section 3.2.2) from MESE, MESE w/o content, and the item encoder before training (MESE raw), respectively, and obtain 3 sets of item embeddings of the selected movie subset. On each set of embeddings, we run a K-means clustering algorithm with K being set to be 3, 4, and 5, respectively. For each cluster obtained, we calculated the proportion of the majority genre among all item candidates. This process is repeated 20 times and the average accuracy is reported in Table 6. Genre information appears most frequently in dialog context and most recommendations are made based on genre attributes. Our item encoder, after joint training, should construct item embeddings that reflect genre information. Hence, the embeddings should be more clustered in terms of genre.
Model	K=3	K=4	K=5
MESE raw	0.492	0.514	0.574
MESE w/o content	0.555	0.589	0.606
MESE	0.695	0.725	0.738
As we can see from the table, without training, MESE raw, being the least sensitive to genre information, achieves the lowest accuracy scores on all clusters. MESE w/o content, although deprived of genre and plot, still has slightly higher accuracy than MESE raw due to its exposure to REDIAL conversations. MESE is most sensitive to genre information. This is an indication that by aligning matching information in both dialog context and item metadata, our item encoder is able to generate meaningful representations, which can facilitate the PLM to produce better rankings through its self-attention mechanism.
What if we remove mentioned entities from dialog context? Mentioned entities are crucial to previous approaches (Chen et al., 2019; Zhou et al., 2020) in terms of recommendations. We train MESE with mentioned entities removed from
dialog history and compare its performance with MESE on REDIAL dataset and INSPIRED dataset in table 7.
Table 6: Item Encoders Clustering Accuracy
Dataset	Model	R@1	R@10	R@50
REDIAL	MESE w/o item	3.4	18.1	38.7
MESE	5.6	25.6	45.5
INSPIRED	MESE w/o item	4.3	11.9	26.7
MESE	4.8	13.5	30.1
Table 7: Results of MESE and MESE w/o on REDIAL and INSPIRED.
We can see removing the entities led to an average of  $26.3\%$  performance drop on REDIAL and an average of  $11.2\%$  performance drop on INSPIRED. The recommendation performance on REDIAL is more impacted by the removal of entities because the conversations in REDIAL are rich with entities and weak in semantic information, whereas INSPIRED is more sparse on entities but contains richer dialog information. In REDIAL, there is 1 mentioned movies among every 21.85 word tokens. The sentence level distinct 1-grams and 3-grams are 0.15 and 2.81. In contract, there is 1 mentioned movies among every 63.54 word tokens in INSPIRED. Its sentence level distinct 1-grams and 3-grams are 0.59 and 6.84. This proves that our model can efficiently infer user interests from texts to make high-quality recommendations without explicitly using mentioned entities. This property could be useful in an e-commerce setting where users tend to convey their requirements more with texts than entities. It could also be useful in a cold start scenario where we don't have many entities in the context.
In this paper, we introduced MESE, a novel CRS framework. By utilizing item encoders to construct embeddings from metadata, MESE can provide high-quality recommendations that align with the dialog history. Our approach yields better performance than existing state-of-the-art models. Ablation studies explain the reason behind this performance gain. As for future work, we will consider applying this approach to a broader domain of CRS datasets. Currently, we only experiment on movie recommendations. However, the encoder of MESE is flexible and can potentially work well with cross-modality tasks. Multi-modal CRSs can be explored in the e-commerce domain with MESE.
Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, and Jie Tang. 2019. Towards knowledge-based recommender dialog system. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1803-1813, Hong Kong, China. Association for Computational Linguistics.
Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems, page 191-198.
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pages 1811-1818.
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3816-3830, Online. Association for Computational Linguistics.
Shirley Anugrah Hayati, Dongyeop Kang, Qingxiaoyang Zhu, Weiyan Shi, and Zhou Yu. 2020. INSPIRED: Toward sociable recommendation dialog systems. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8142-8152, Online. Association for Computational Linguistics.
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web, pages 173-182.
Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR).
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119, San Diego, California. Association for Computational Linguistics.
Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. In Advances in Neural Information Processing Systems (NeurIPS).
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582-4597, Online. Association for Computational Linguistics.
Yu Li, Shirley Anugrah Hayati, Weiyan Shi, and Zhou Yu. 2021. DEUX: an attribute-guided framework for sociable recommendation dialog systems. CoRR, abs/2105.00825.
Zujie Liang, Huang Hu, Can Xu, Jian Miao, Yingying He, Yining Chen, Xiubo Geng, Fan Liang, and Daxin Jiang. 2021. Learning neural templates for recommender dialogue system. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7821-7833.
Ting Liu, Andrew W. Moore, Alexander Gray, and Ke Yang. 2004. An investigation of practical approximate nearest neighbor algorithms. In Advances in Neural Information Processing Systems (NeurIPS), page 825-832.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR).
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Advances in Neural Information Processing Systems (NeurIPS), pages 13-23.
Wenchang Ma, Ryuichi Takanobu, and Minlie Huang. 2021. Cr-walker: Tree-structured graph reasoning and dialog acts for conversational recommendation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1839–1851.
Marius Muja and David G Lowe. 2014. Scalable nearest neighbor algorithms for high dimensional data. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 36(11):2227-2240.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Victor Sanh, Lysandre Debut, Julien Chaumont, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pages 593-607. Springer.
Yue Shi, Martha Larson, and Alan Hanjalic. 2014. Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges. ACM Computing Surveys (CSUR), 47(1):1-45.
Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. 2015. A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pages 553-562.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the Thirty-first AAAI Conference on Artificial Intelligence, page 4444-4451.
Xiaoyuan Su and Taghi M Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Advances in Artificial Intelligence, 2009.
Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. 2019a. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. arXiv preprint arXiv:1908.01000.
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019b. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1441-1450.
Hao Tan and Mohit Bansal. 2019. LXMERT: Learning cross-modality encoder representations from transformers. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5100-5111, Hong Kong, China. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), pages 5998-6008.
Lingzhi Wang, Huang Hu, Lei Sha, Can Xu, Kam-Fai Wong, and Daxin Jiang. 2021. Finetuning large-scale pre-trained language models for conversational recommendation with knowledge graph. arXiv preprint arXiv:2110.07477.
Tianxing Wu, Arijit Khan, Huan Gao, and Cheng Li. 2019. Efficiently embedding dynamic knowledge graphs. arXiv preprint arXiv:1910.06708.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT: Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270-278, Online. Association for Computational Linguistics.
Guorui Zhou, Xiaogiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for clickthrough rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1059-1068.
Jinfeng Zhou, Bo Wang, Ruifang He, and Yuexian Hou. 2021. CRFR: Improving conversational recommender systems via flexible fragments reasoning on knowledge graphs. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4324-4334, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020. Improving conversational recommender systems via knowledge graph based semantic fusion. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1006-1014.
A Candidate Selection With Multiple Sources
To maximize the performance of candidate selection, we blend candidates selected from the nearest neighbor search source with a rule based source. The rule based source is constructed by traversing the training data and maintaining a list of co-occurring items for each individual item. For example, if item A and item B both occurred in a conversation in the training set, we add them to each other's list. During testing, if item A is mentioned in the dialog context, the maintained co-occurring item list is combined with the candidates retrieved by nearest neighbor search as the final selection.
B Response Generation and Recommendation Trade-Off
The trade-off between response generation and recommendation during training. We separate language training objective from the jointly trained loss and compare the trained models on language generation metrics on ReDial in table 8. MESE-LG denotes the model trained with only the language generation objective  $\mathcal{L}_{\mathrm{res}}$
Model	Dist2	Dist4	Bleu2	Bleu4
MESE-LG	1.13	2.25	0.295	0.171
MESE	0.822	1.313	0.246	0.143
Table 8: Results of MESE and MESE-LG on REDIAL.
As we can see, training only on language generation objective yields better results. In the settings of MESE, both tasks update the weights of the same pre-trained language model simultaneously with different goals: In recommendation, the language model needs to construct meaningful scores for generated candidates; In response generation, the language model aims to generate next token given previous contexts. This experiment showed that this joint training approach induces a trade-off between training on response generation and recommendation: back-propagation on the recommendation objective can lead to catastrophic forgetting of information learned from pretraining, which leads to worse language metrics.
C Response Generation Example
We include example dialog of human seeker and MESE model in table 9.
User: Hello, how are you?
System: Good, how are you?
What type of movies do you like?
User: I'm great. thanks!
I'm looking for crime movies to watch.
System: Have you seen Goodfellas?
User: Yes, I love that movie.
Could you recommend another one?
System: If you haven't seen The Rise of
the Krays you should give it a try.
User: I never saw that I should check it out.
Thank you and have a great afternoon!
System: You too! Have a great day!
Table 9: Conversation Example