SlowGuess/NAACL_2022_OCR huggingface dataset:
Bowen Yang†, Cong Han†, Yu Li‡, Lei Zuo†, Zhou Yu†  
†Columbia University
{by2299，ch3212，lz2771，zy2461}@columbia.edu
$^{\ddagger}$ University of California, Davis
{yooli}@ucdavis.edu
A key challenge of Conversational Recommendation Systems (CRS) is to integrate the recommendation function and the dialog generation function smoothly. Previous works employ graph neural networks with external knowledge graphs (KG) to model individual recommendation items and integrate KGs with language models through attention mechanisms for response generation. Although previous approaches prove effective, there is still room for improvement. For example, KG-based approaches only rely on entity relations and bag-of-words to recommend items and neglect the information in the conversational context. We propose to improve the usage of dialog context for both recommendation and response generation using an encoding architecture along with the self-attention mechanism of transformers. In this paper, we propose a simple yet effective architecture comprising a pre-trained language model (PLM) and an item metadata encoder to integrate the recommendation and the dialog generation better. The proposed item encoder learns to map item metadata to embeddings reflecting the rich information of the item, which can be matched with dialog context. The PLM then consumes the context-aware item embeddings and dialog context to generate high-quality recommendations and responses. Experimental results on the benchmark dataset REDIAL show that our model obtains state-of-the-art results on both recommendation and response generation tasks<sup>1</sup>.
An automated conversational recommendation system (CRS) (Li et al., 2018; Zhou et al., 2020) is intended to interact with users and provide accurate product recommendations (e.g., movies, songs, and consumables). It has been a focal point of research lately due to its potential applications in the
e-commerce industry. Traditional recommendation systems collect user preferences from implicit feedback such as click-through-rate (Zhou et al., 2018) or purchase history and apply collaborative filtering (Su and Khoshgoftaar, 2009; Shi et al., 2014) or deep learning models (Covington et al., 2016; He et al., 2017) to construct latent spaces for user preferences. Unlike traditional recommendation systems, CRSs directly extract user preferences from live dialog history to precisely address the users' needs.
Although some progress has been made in this area, there is still room for improvement. First, previous CRSs (Chen et al., 2019; Zhou et al., 2020; Li et al., 2021) track entities mentioned in the dialog context, and then search related items in knowledge graphs to recommend to users. However, these systems require a named-entity recognition (NER) module to extract mentioned entities from the dialog context. Thus we need to collect additional domain-specific data to train the NER module. In practice, such NER modules have deficient performance, leading to a bad accuracy of CRS. Second, existing CRSs built upon graph neural networks (Kipf and Welling, 2017; Schlichtkrull et al., 2018) cannot quickly scale up or respond to rapid changes of the underlining entities. In e-commerce, items for recommendation change frequently due to constant updates of merchants and products. Existing approaches require either re-training the entire system when the structure of knowledge graph changes (Dettmers et al., 2018) or adding complex architectures on top to be adaptive (Wu et al., 2019). A more flexible architecture can help the system react to rapid changes and adapt itself to new items.
Moreover, meta-information about the items can be leveraged. Similar information can be found in both dialog context and item meta-information. For example, in a movie recommendation setting, words like "crime, gangsters, etc." are likely to exist in the dialog context when a user is searching
$^{1}$ Code is available online https://github.com/by2299/MESE
Findings of the Association for Computational Linguistics: NAACL 2022, pages 38 - 48 July 10-15, 2022 ©2022 Association for Computational Linguistics
for crime movies. In the synopsis of a crime movie, such keywords are likely to exist as well. Ideally, an alignment between the semantics of dialog context and the item meta-information can be used to improve system's performance.
Driven by the motivations above, we present a Metadata Enhanced learning approach via Semantic Extraction from dialog context i.e. MESE. The major components of MESE contain a pre-trained language model (PLM) and an item encoder architecture. The item encoder takes item metadata as input and outputs a vector embedding. By jointly training the encoder and the PLM, the entire system can extract co-occurring information between dialog context and item metadata, and item encoders can systematically construct representations reflecting this alignment. Item embeddings are then consumed with dialog context by the self-attention mechanism of the PLM. This mechanism smoothly integrates dialog context and item information well into the recommendation and response generation tasks.
The key contributions of this paper are summarized as follows: This paper presents MESE, a novel CRS framework that considers both item metadata and dialog context for recommendations. Our model employs a simple yet effective item metadata encoder that learns to represent rich item information during training. Such encoder can adapt to database changes quickly and is independent of task-specific architectures. Extensive experiments on standard dataset REDIAL demonstrate that MESE outperforms previous state-of-the-art methods on both response generation and recommendation with a large margin.
The current CRS paradigm contains two major modules: a recommendation module that suggests items based on conversational context and a response generation module that generate responses based on dialog history and the recommended items. Integrating these two modules to perform well on both tasks has been a major challenge. Chen et al. (2019) leverage external knowledge and employees graph neural networks as the backbone to model entities and entity relations in the knowledge graph (KG) to enhance performance. Zhou et al. (2020) introduce a word-level KG (Speer et al., 2017) to the system with semantic fusion (Sun et al., 2019a) to enhance the semantic repre
sentations of words and items. Since item information and dialog context are processed separately in the above approaches, they lose integrated sentence-level information. We propose to condition recommendation on integrated contextual information of both dialog context and mentioned entity information. More recent works adopt pre-trained language models (PLM) (Vaswani et al., 2017; Radford et al., 2019; Zhang et al., 2020) and template-based methods to facilitate response generation. Liang et al. (2021) generate a response template containing a mixture of contextual words and slot locations to incorporate recommended items better. Wang et al. (2021) expand the vocabulary list of the PLM to include items to unify the process of item recommendation with response generation. We propose to enhance our PLM with an item metadata encoder to extract context-aware representations by jointly training on both recommendation and response generation tasks. We also generate response templates with slot locations to better incorporate recommended items into responses.
Our work is also inspired by studies from other areas. Recent works have shown that cross-modality training across vision and language tasks can lead to outstanding results in building multimodal representations (Tan and Bansal, 2019; Lu et al., 2019). In (Tan and Bansal, 2019), a large-scale transformer-based model is adapted with cross-modal encoders to connect visual and linguistic semantics and pre-trained on vision-language pairs to learn cross-modality relationships. Prompt tuning (Li and Liang, 2021; Gao et al., 2021) methods show that PLMs are capable of integrating different sources of information into the same embedding space. In terms of using PLM as a recommendation system, Sun et al. (2019b) train a bidirectional self-attention model to predict masked items and achieve remarkable results. Inspired by the above studies, we propose to use an encoder module to map item meta-information to an embedding space. By jointly training on dialog context and encoded item representations, the system can align these two information streams by fusing the semantic spaces.
In this section, we present our framework MESE that integrates item metadata with dialog context. We first introduce how to encode item metadata and how to blend item information into dialog con
text. We then illustrate how the recommendation module and the response generation module are built. Finally, we describe the training objectives and the testing process.
3.1 Encoding Item Metadata
We propose to use an item encoder to directly map the metadata of each item to an embedding. In the movie recommendation setting, description on title, genre, actors, directors, and plot are collected as metadata and concatenated with a "[SEP]" token for each movie. This concatenated information is the input to the item encoder which produces a vector representation for each item. The item encoder consists of a DistilBERT (Sanh et al., 2019) model that maps the input sequence to a sequence of vector embeddings, a pooling layer that condenses the sequence embeddings to a single vector embedding, and a feed-forward layer to produce the output embedding with a certain dimension. A visualization of this module is shown in Figure 1.
Figure 1: Item Encoder takes in the metadata of an item and outputs an embedding of the item
Next, we discuss how to incorporate items into dialog context with the encoded embeddings and the PLM (Radford et al., 2019). Previous studies have shown that KG-based frameworks cannot always integrate recommended items into generated replies (Wang et al., 2021). To solve this issue, we introduce a special placeholder token "[PH]" to the vocabulary list of the PLM. Every occurrence of item name in the corpus is replaced with this "[PH]" token. This modified dialog sequence is then mapped to a sequence of word token embeddings (WTE) by the vocabulary embedding matrix of the PLM. To include item information into the context, an instance of the item encoder is used to encode item metadata into token embeddings. The item encoder takes in item metadata and outputs an item token embedding (ITE) with the same di
mensionality as a WTE of the PLM. The ITE is then concatenated with the WTEs constructed from the dialog context to be consumed by the PLM. An example is shown in 2.
Figure 2: Dialog context is represented as a concatenation of WTEs and ITEs to be consumed by the PLM.
3.2 Recommendation Module
Similar to (Covington et al., 2016), we pose recommendation as a two-phase process: candidate selection and candidate ranking. During candidate selection, the entire item database is traversed and narrowed down to a few hundred candidates based on a calculated similarity score between the dialog context and the item metadata. During candidate ranking, similarity scores between the dialog context and the generated candidates are recomputed with finer granularity by the self-attention mechanism of the PLM.
3.2.1 Candidate Selection
In this section, we describe the training objective of candidate selection. We add a special token "[REC]" to the vocabulary embedding matrix of PLM. This token is used to indicate the start of the recommendation process and to summarize dialog context. At the end of each turn, a token embedding sequence is created following Figure 2 in the format of an interleaving of word token embeddings (WTE) and item token embeddings (ITE) to represent all previous dialog context. When recommendation is labeled in a conversation turn in the training dataset, the WTE of "[REC]" is appended to the previous token embedding sequence to form a new sequence  $D$ . Next, the PLM takes in  $D$  and produces an output embedding sequence. We denote the last vector of this output embedding sequence as  $D_{R}$  which corresponds to the appended special token "[REC]".  $D_{R}$  summarizes dialog context and can be used to retrieve candidate items.
Figure 3: Overview of MESE. During training, M items are sampled from the database to compute the joint loss  $\mathcal{L}_{\mathrm{select}}$  and  $\mathcal{L}_{\mathrm{rank}}$ , which are then combined with the response generation loss  $\mathcal{L}_{\mathrm{res}}$  and jointly optimized. During testing, the entire metadata DB is stored as a nearest neighbor index (NNI). First, dialog context is condensed into a vector  $D_R$ . An approximate nearest neighbor search is performed on  $D_R$  to get candidate items, which is then passed to the ITE Encoder to compute their ranking scores and the the highest-ranked candidate is used as a prompt to generate responses. We only present the case when there's only one ground truth recommendation in the utterance. However, it's easy to extend the above approach to multiple recommendations.
We randomly sample M items and their metadata from the database as negative examples and combine them with the ground truth item labeled in the dataset to get the training samples. Another instance of the item encoder, is used to create candidate token embeddings for each item in the training samples. The item Encoder takes in the metadata of samples items and outputs a set of candidate token embeddings  $C = (c_0, c_1, \dots, c_M)$ , each with the same dimensionality as  $D_R$ . The recommendation task at this phase is posed as a multi-class classification problem of predicting the ground truth item over the negative samples (Covington et al., 2016). The probability of each candidate item is defined in (1) and optimized by a cross-entropy loss function, denoted as  $\mathcal{L}_{\mathrm{select}}$ :
$$
P (i) = \frac {e ^ {c _ {i} \cdot D _ {R}}}{\sum_ {n = 0} ^ {M} e ^ {c _ {n} \cdot D _ {R}}} \tag {1}
$$
Note that the purpose of this learning objective is to let the model learn how to construct the  $D_{R}$  representation instead of learning the probabilities of candidate items. The  $D_{R}$  representation is later used in an approximate nearest neighbor search (Liu et al., 2004) to select candidates from the entire database in testing 3.5.
3.2.2 Candidate Ranking
In this section, we describe the training objective of candidate ranking. The goal of candidate ranking is to further perform more fine-grained scoring on the similarities between generated candidates and dialog context so that the final rankings of items can better reflect users' preferences. We propose to use the PLM and its self-attention to compute ranking scores.
During training, the same context token embedding sequence  $D$  and the same training sample with M negative examples are used. The ITE encoder from section 3.1 is used to map the metadata of the sample to an ITE set  $T = (t_0, t_1, \dots, t_M)$ , where the subscript of each  $t_i$  corresponds to their index in the database. A concatenation of context sequence  $D$  and  $T$  are created and consumed by the same PLM used above and the output embeddings are computed. The order of candidate items should not make a difference in the values of the outputs. Therefore, we add the same positional encoding to each ITE in  $T$  and remove the attention masks among the ITEs. The output embeddings of PLM that correspond to the ITEs in  $T$  are then passed to a feed-forward layer to reduce each vector from a higher dimension to a single number with dimensionality equals 1. This set of numbers is denoted
by  $\mathbf{Q} = (q_0, q_1, \dots, q_M)$  where the index of each number corresponds to their index in  $T$ . The final ranking score of each candidate item is defined in (2) and optimized by a cross-entropy loss function, denoted as  $\mathcal{L}_{\mathrm{rank}}$ :
$$
R (i) = \frac {e ^ {q _ {i}}}{\sum_ {n = 0} ^ {M} e ^ {q _ {n}}} \tag {2}
$$
3.3 Response Generation Module
In this section, we describe how to train the model to generate responses based on the recommended items' metadata. The same token embedding sequence  $D$  is used as context and current system utterance  $U = (w_{0}, w_{1}, \dots, w_{n})$  is used as targets where each  $w_{i}$  represents a WTE. We only optimize the PLM to reconstruct system utterances.
If the current utterance contains recommendations, we create ITEs by passing metadata of the recommended items through the item Encoder used in 2 and append the ITEs to context token embedding sequence  $D$  to obtain  $D'$ . If the current utterance doesn't contain recommendations,  $D'$  is the same as  $D$ . The PLM is trained to reconstruct the ground truth  $U$  based on  $D'$ . The probability of generated response is formulated as:
$$
P (U | D ^ {\prime}) = \prod_ {i = 1} ^ {n} P \left(w _ {i} \mid w _ {i - 1}, \dots , w _ {0}, D ^ {\prime}\right) \tag {3}
$$
The loss function is set to be:
$$
L _ {r e s} = - \frac {1}{N} \sum_ {i = 1} ^ {N} \log \left(P \left(U _ {i} \mid D ^ {\prime}\right)\right) \tag {4}
$$
Where  $\mathbf{N}$  is the total number of system utterances in one dialog.
Finally, we use the following combined loss to jointly train both the encoders and the PLM:
$$
L o s s = a \cdot \mathcal {L} _ {\text {s e l e c t}} + b \cdot \mathcal {L} _ {\text {r a n k}} + c \cdot \mathcal {L} _ {\text {r e s}} \tag {5}
$$
Where a, b and c are the weights of language training and recommendation training objectives. During training, all weight parameters of the two item encoders, the PLM and relevant feed-forward layers participate in back-propagation. An overview of training is shown in Figure 3.
3.5 Testing
During testing, a candidate embedding set over the entire item database is built by running metadata through the item encoder used in section 3.2.1 and stored with a nearest neighbor index (NNI) (Muja and Lowe, 2014). During response generation, when a "[REC]" token is generated, candidate selection 3.2.1 is activated. An approximate nearest neighbor search is conducted over the NNI and K closest candidates are selected based on their similarities from the  $D_R$  vector<sup>2</sup>. Candidate ranking is then activated and the PLM and the item encoder from Figure 2 are used to generate a score for each candidate. When ranking finishes, the ITE that receives the highest ranking score is appended to the dialog context  $D$  and response generation continues until the end-of-sentence token is generated. After generation is completed, we replace the occurrence of the placeholder token "[PH]" with the title of the recommended item to form the final response. Note that when the turn involves no recommendation, our PLM simply generates a clarification question or a chitchat response with no placeholder tokens. An overview of testing is shown in Figure 3.
In this section, we discuss the datasets used, experimental setup, experimental results on both recommendation and language metrics, and report analysis results with ablation studies.
We evaluated our model on two datasets: ReDial dataset (Li et al., 2018) for comparison with previous models and INSPIRED dataset (Hayati et al., 2020) for ablation studies. Both datasets were collected on Amazon Mechanical Turk (AMT) platform where workers made conversations related to movie seeking and recommending following a set of extensive instructions. The statistics of both datasets are shown in Table 1.
Dataset	dialogs	utterances	avg turns
ReDial	10006	182150	18.2
INSPIRED	1001	35811	10.73
Table 1: Statistics of Datasets
$^{2}$ Multi-Source Selection in Appendix A
4.2.1 Baselines
The baseline models for evaluation on the ReDial dataset is described below:
ReDial (Li et al., 2018): A dialogue generation model using HRED (Sordoni et al., 2015) as backbone for dialog module
KBRD (Chen et al., 2019): The dialog generation module based on the Transformer architecture (Vaswani et al., 2017). It exploits external knowledge to perform recommendations and language generation.
KGSF (Zhou et al., 2020): Concept-net is used alongside knowledge graph to perform semantic-aware recommendations.
CR-Walker (Ma et al., 2021): performs tree-structured reasoning on a knowledge graph and guides language generation with dialog acts
CRFR (Zhou et al., 2021): conversational context-based reinforcement learning model with multi-hop reasoning on KGs.
NTRD (Liang et al., 2021): an encoder-decoder model is used to generate a response template with slot locations to be filled in with recommended items using a sufficient attention mechanism.
RID (Wang et al., 2021): pre-trained language model and knowledge graph are used to improve CRS performance.
4.2.2 Implementation Details
We employed GPT-2 model (Radford et al., 2019) as the backbone of MESE for dialog generation, which contains 12 layers, 768 hidden units, 12 heads, with 117M parameters. We recruited 2 item encoders (Sanh et al., 2019) to encoder items in candidate generation 3.2.1 and candidate ranking 3.2.2, respectively, each has a distil-bert model with 6 layers, 768 hidden units, 12 heads, with 66M parameters. We used the AdamW optimizer (Loshchilov and Hutter, 2019) with epsilon set to  $1e^{-6}$ , learning rate set to  $3e^{-5}$ . The model was trained for 8 epochs on ReDial dataset, and the first epoch was dedicated to warm up with a linear scheduler. We set the sample size M during candidate generation and candidate ranking to be 150. We set  $\mathrm{a} = 0.8$ ,  $\mathrm{b} = 1.0$  and  $\mathrm{c} = 0.28$  as coefficients for 3 loss functions respectively. We chose  $\mathrm{K} = 500$  for the number of candidates during testing.
4.2.3 Evaluation Metrics
We performed two evaluations, recommendation evaluation and dialog evaluation, for the model. For
recommendation evaluation, we used Recall@X (R@X), which shows whether the top X items recommended by the system include the ground truth item suggested by human recommenders. In particular, we chose R@1, R@10 and R@50 following previous works (Chen et al., 2019; Zhou et al., 2020). We also defined recall accuracy of MESE to be the percentage of ground truth items that appear among the 500 generated candidates in the candidate generation phase 3.2.1 and ranking accuracy to be the percentage of items that appear in the top k (k=1, 10, 50) position of the sorted candidates in the candidate ranking phase 3.2.2. The product of the recall and ranking accuracy is the final recommendation accuracy of MESE. We also adopted end-to-end response evaluation following (Wang et al., 2021). We computed response recall (ReR) as whether the final response contains the target items recommended by human annotators. For dialog evaluation, we adopted perplexity, distinct n-grams (Li et al., 2016), and BLEU score (Papineni et al., 2002) for automatic evaluations. Human evaluation (on a random sampling of 100 dialogs from the test set) is also conducted on dialog evaluation in comparison with KGSF. We invite three annotators to score the generated samples in two aspects, Fluency and Informativeness. The annotator is asked to select a better response based on the given context. Ties are allowed if two responses have similar qualities. The score is the percentage of the model's response being selected. The final performance is calculated using the average scores of the three annotators.
We first report recall, ranking, and final accuracy on REDIAL dataset of MESE in table 3. From the results, it can be seen that candidate ranking has remarkable performance gains in scoring the items. It demonstrates that PLMs have great potential in making recommendations. One possible reason behind this is that the PLM and its self-attention mechanism is effective in learning the similarities and discrepancies between item semantics and dialog semantics.
Table 2 compares different models on REDIAL dataset. The superiority of MESE persists across recommendation and language generation. On all recommendation metrics, including R@1, R@10, and R@50, MESE outperforms the state-of-the-art