Quotation Recommendation and Interpretation Based on Transformation from Queries to Quotations (2021)

Abstract: To help individuals express themselves better, quotation recommendation is receiving growing attention. Nevertheless, most prior efforts focus on modeling quotations and queries separately and ignore the relationship between the quotations and the queries. In this work, we introduce a transformation matrix that directly maps the query representations to quotation representations. To better learn the mapping relationship, we employ a mapping loss that minimizes the distance of two semantic spaces (one for quotation and another for mapped-query). Furthermore, we explore using the words in history queries to interpret the figurative language of quotations, where quotation-aware attention is applied on top of history queries to highlight the indicator words. Experiments on two datasets in English and Chinese show that our model outperforms previous state-of-the-art models.

## introduction quotations are essential for successful persuasion and explanation in interpersonal communication. however, it is a daunting task for many individuals to write down a suitable quotation in a short time. this results in a pressing need to develop a quotation recommendation tool to meet such a demand. to that end, extensive efforts have been made to quotation recommendation, which aims to recommend an ongoing conversation with a quotation whose sense continues with the existing context @xcite . as quotations are concise phrases or sentences to spread wisdom, which are always in figurative language and difficult to understand, they are assumed written in a different pseudo-language @xcite . intuitively, we the code is available at https://github.com/ lingzhi-wang/quotation-recommendation [t1]: save your money. scuf is the biggest ripoff in gaming. to illustrate our motivation, figure 1 shows a reddit conversation with some history queries associated with quotation q, "a fool and his money are soon parted". from the queries (t 5 and h 1 to h 3 ), we can infer the meaning of quotation q is "a foolish person spends money carelessly and won't have a lot of money." based on the contexts. from h 3 , we can also know the implication behind the words, which is "do a marketing research before buying". humans can establish such a relationship between quotations and queries and then decide what to quote in their writings, so can machines (neural network). therefore, we introduce a transformation matrix, in which machines can learn the direct mapping from queries to quotations. the matrix is worked on the outputs of two encoders, conversation encoder and quotation encoder, encoding conversation context and quotations respectively. furthermore, we can use the words in the queries to interpret quotations. h 1 to h 3 in figure 1 are denoted as history queries, and the words on wavyunderline are denoted as indicators to quotations. it can be seen that we can interpret quotations by highlighting the words in the queries. therefore, we compute quotation-aware attention over all the history queries (after the same transformation as we mentioned before) and then display indicators we learned, which also reflects the effectiveness of the transformation. in summary, we introduce a transformation between the query semantic space and quotation semantic space. to minimize the distance of their semantic space after transformation mapping, an auxiliary mapping loss is employed. besides, we propose a way to interpret quotations with indicative words in the corresponding queries. the remainder of this paper is organized as follows. the related work is surveyed in section 2. section 3 presents the proposed approach. and section 4 and 5 present the experimental setup and results respectively. finally, conclusions are drawn in section 6. ## our model this section describes our quotation recommendation model, whose overall structure is shown in figure 2 . the input of the model mainly contains the observed conversation c and the quotation list q. the conversation c is formalized as a sequence of turns (e.g., posts or comments) {t 1 , t 2 , ..., t nc } where n c represents the length of the conversation (number of turns) and t nc is the query turn. t i represents the i-th turn of the conversation and contains words w i . the quotation list q is {q 1 , q 2 , ..., q nq }, where n q is the number of quotations and q k is the k-th quotation in list q, containing words w k . our model will output a label @xmath0, 2, ..., n q }, to indicate which quotation to recommend. ## experimental setup datasets. we conduct experiments based on datasets from two different platforms, weibo and reddit, released by @xcite . to make our experimental results comparable to @xcite , we utilize their preprocessed data directly. parameter setting. we first initialize the embedding layer with 200-dimensional glove embedding @xcite for reddit and chinese words embedding @xcite for weibo. for transformer layers, we choose the number of layers and heads as (2, 3) for reddit and (4, 4) for weibo. for the hidden dimension of transformer layers and bigru layers (each direction), we set it to 200. we employ adam optimizer (kingma and ba, 2015) with initial learning rate with 1e-4 and early stop adoption @xcite in training. the batch size is set to 32. dropout strategy @xcite and l 2 regularization are used to alleviate overfitting. and the tradeoff parameter λ is chosen from {1e-4, 1e-3}. all the hyper-parameters above are tuned on the validation set by grid search. ## conclusion in this paper, we propose a transformation from queries to quotations to enhance a quotation recommendation model for conversations. experiments on weibo and reddit datasets show the effectiveness of our model with transformation. we further explore using indicative words in history queries to interpret quotations, which shows rationality of our method.