Continuity of Topic, Interaction, and Query: Learning to Quote in Online Conversations (2020)

Abstract: Quotations are crucial for successful explanations and persuasions in interpersonal communications. However, finding what to quote in a conversation is challenging for both humans and machines. This work studies automatic quotation generation in an online conversation and explores how language consistency affects whether a quotation fits the given context. Here, we capture the contextual consistency of a quotation in terms of latent topics, interactions with the dialogue history, and coherence to the query turn’s existing contents. Further, an encoder-decoder neural framework is employed to continue the context with a quotation via language generation. Experiment results on two large-scale datasets in English and Chinese demonstrate that our quotation generation model outperforms the state-of-the-art models. Further analysis shows that topic, interaction, and query consistency are all helpful to learn how to quote in online conversations.

## introduction quotations, or quotes, are memorable phrases or sentences widely echoed to spread patterns of wisdom @xcite . they are derived from the ancient art of rhetoric and now appearing in various daily activities, ranging from formal writings @xcite to everyday conversations @xcite , all help us present clear, beautiful, and persuasive language. however, for many individuals, writing a suitable quotation that fits the ongoing contexts is a daunting task. the issue becomes more pressing for quoting in online conversations where quick responses are usually needed on mobile devices @xcite . to help online users find what to quote in the discussions they are involved in, our work studies how to recommend an ongoing conversation with * xingshan zeng is the corresponding author. to predict quotations, our work explores semantic consistency of what will be quoted and what was given in the contexts. in context modeling, we distinguish the query turn (henceforth query) and the other turns in earlier history (henceforth history), where topic, interaction, and query consistency work together to determine whether a quote fits the contexts. here topic consistency ensures that the words in quotation reflect the discussion topic (such as "fool" and "money" in figure 1 ). interaction consistency is to identify the turns in history to which the query responds (e.g., t 1 and t 4 in figure 1 ) and guide the quote to follow such interaction. query consistency measures the language coherence of quote in continuing the story started by the query. for example, the quote in figure 1 is to support the query's argument. in previous work of quotation recommendation, there are many methods designed for formal writings @xcite ; whereas much fewer efforts are made for online conversations with informal language and complex interactions in their contexts. @xcite use a ranking model to recommend quotes for twitter conversations. different from them, we attempt to generate quotations in a word-by-word manner, which allows the semantic consistency of quotes and contexts to be explored. concretely, we propose a neural encoderdecoder framework to predict a quotation that continues the given conversation contexts. we capture topic consistency with latent topics (i.e., word distributions), which are learned by a neural topic model @xcite and inferred jointly with the other components. interaction consistency is modeled with a turn-based attention over the history turns, and the query is additionally encoded to initialize the decoder's states for query consistency. to the best of our knowledge, we are the first to explore quotation generation in conversations and extensively study the effects of topic, interaction, and query consistency on this task. our empirical study is conducted on two largescale datasets, one in chinese from weibo and the other in english from reddit, both of which are constructed as part of this work. experiment results show that our model significantly outperforms both the state-of-the-art model based on quote rankings @xcite and the recent topic-aware encoder-decoder model for social media language generation @xcite . for example, we achieve 27.2 precision@1 on weibo compared with 24.0 by @xcite . further discussions show that topic, interaction, and query consistency can all usefully indicate what to quote in online conversations. we also study how length of history and quotation affects the quoting results and find that we perform consistently better than comparison models in varying scenarios. ## related work our work is in the line with content-based recommendation @xcite or cloze-style read-ing comprehension @xcite , which learns to put suitable text fragments (e.g., words, phrases, sentences) in the given contexts. most prior studies explore the task in formal writings, such as citing previous work in scientific papers @xcite , quoting famous sayings in books @xcite , and using idioms in news articles @xcite . the language they face is mostly formal and welledited, while we tackle online conversations exhibiting noisy contexts and hence involving quote consistency modeling with turn interactions. @xcite al. (2016) also recommend quotations for conversations. however, they consider quotations as discrete attributes (for learning to rank) and hence largely ignore the rich information reflected by a quotation's internal word patterns. compared with them, our model learns to quote with language generation, which can usefully exploit how words appear in both contexts and quotations. for methodology, we are inspired by the encoder-decoder neural language generation models @xcite . in dialogue domains, such models have achieved huge success in digesting contexts and generate microblog hashtags @xcite , meeting summaries @xcite , dialogue responses @xcite , etc. here we explore how the encoder-decoder architecture works to generate quotations in conversations, which has never been studied in existing work. our study is also related to previous research to understand conversation contexts @xcite @xcite , where it is shown to be useful to capture interaction structures @xcite ) and latent topics @xcite . for latent topics, we are benefited from the recent advance of neural topic models @xcite ), which allows end-toend topic inference in neural architectures. nevertheless, none of the above work attempts to study the semantic consistency of quotes in conversation contexts, which is a gap our work fills in. ## our quotation generation model this section describes our neural encoder-decoder framework that generates quotations in conversations, whose architecture is shown in figure 2 . the encoding process works for context modeling of turn interactions (described in section 3.1) and latent topics (presented in section 3.2). for turn 1 turn encoder turn 2 turn encoder turn n turn encoder gru gru gru … … query turn encoder gru history encoder query encoder 𝒔𝒐𝒇𝒕𝒎𝒂𝒙(𝒇 𝜽 (𝒛)) 𝒔𝒐𝒇𝒕𝒎𝒂𝒙(𝒇 𝝓 𝜽 ) 𝜽 𝜷 𝒙 𝒃𝒐𝒘 0 𝒍𝒐𝒈𝝈 𝝁 𝒛 𝒙 𝒃𝒐𝒘 𝑵(𝝁，𝝈 6 ) neural topic model @xmath0: our encoder-decoder framework for conversation quotation generation. it encodes turn interactions (for both query and earlier history) and latent topics in contexts. the decoder predicts quotes in aware of topic, interaction, and query consistency. the decoding process to be discussed in section 3.3, we predict words in quotes taking topic, interaction, and query consistency into consideration. the learning objective of the entire framework will be given at last in section 3.4. ## experimental results in this section, we first show the main comparison results in section 5.1. then section 5.2 discusses what we learn to represent consistency. finally, section 5.3 presents more analysis to characterize quotations in online conversations. ## conclusion we present a novel quotation generation framework for online conversations via the modeling of topic, interaction, and query consistency. experiment results on two newly constructed online conversation datasets, weibo and reddit, show that our model outperforms the previous state-of-theart models. further discussions provide more insights on quoting in online conversations.