Identifying Implicit Quotes for Unsupervised Extractive Summarization of Conversations (2020)

Abstract: We propose Implicit Quote Extractor, an end-to-end unsupervised extractive neural summarization model for conversational texts. When we reply to posts, quotes are used to highlight important part of texts. We aim to extract quoted sentences as summaries. Most replies do not explicitly include quotes, so it is difficult to use quotes as supervision. However, even if it is not explicitly shown, replies always refer to certain parts of texts; we call them implicit quotes. Implicit Quote Extractor aims to extract implicit quotes as summaries. The training task of the model is to predict whether a reply candidate is a true reply to a post. For prediction, the model has to choose a few sentences from the post. To predict accurately, the model learns to extract sentences that replies frequently refer to. We evaluate our model on two email datasets and one social media dataset, and confirm that our model is useful for extractive summarization. We further discuss two topics; one is whether quote extraction is an important factor for summarization, and the other is whether our model can capture salient sentences that conventional methods cannot.

## introduction as the amount of information exchanged via online conversations is growing rapidly, automated summarization of conversations is in demand. neuralnetwork-based models have achieved great performance on supervised summarization, but its application to unsupervised summarization is not sufficiently explored. supervised summarization requires tens of thousands of human-annotated summaries. because it is not realistic to prepare such large datasets for every domain, there is a growing requirement for unsupervised methods. previous research proposed diverse methods of unsupervised summarization. graph-centrality figure 1 : example of a post and a reply with a quote and a reply with no quote. implicit quote is the part of post that reply refers to, but not explicitly shown in the reply. the premise of these methods is that important topics appear frequently in a document. therefore, if important topics appear only a few times, these methods fail to capture salient sentences. for more accurate summarization, relying solely on the frequency is not sufficient and we need to focus on other aspects of texts. as an alternative aspect, we propose "the probability of being quoted". when one replies to an email or a post, a quote is used to highlight the important parts of the text; an example is shown in figure 1 . the reply on the bottom includes a quote, which generally starts with a symbol ">". if we can predict quoted parts, we can extract important sentences irrespective of how frequently the same topic appears in the text. thus, we aim to extract quotes as summaries. previous research assigned weights to words that appear in quotes, and improved the centroidbased summarization @xcite . however, most replies do not include quotes, so it is difficult to use quotes as the training labels of neural models. we propose a model that can be trained without explicit labels of quotes. the model is implicit quote extractor (iqe). as shown in figure 1 , implicit quotes are sentences of posts that are not explicitly quoted in replies, but are those the replies most likely refer to. the aim of our model is to extract these implicit quotes for extractive summarization. we use pairs of a post and reply candidate to train the model. the training task of the model is to predict if a reply candidate is an actual reply to the post. iqe extracts a few sentences of the post as a feature for prediction. to predict accurately, iqe has to extract sentences that replies frequently refer to. summaries should not depend on replies, so iqe does not use reply features to extract sentences. the model requires replies only during the training and not during the evaluation. we evaluate our model with two datasets of enron mail @xcite , corporate and private mails, and verify that our model outperforms baseline models. we also evaluated our model with reddit tifu dataset @xcite and achieved results competitive with those of the baseline models. our model is based on a hypothesis that the ability of extracting quotes leads to a good result. using the reddit dataset where quotes are abundant, we obtain results that supports the hypothesis. furthermore, we both quantitatively and qualitatively analyzed that our model can capture salient sentences that conventional frequency-based methods cannot. the contributions of our research are as follows: summarization methods can be roughly grouped into two methods: extractive summarization and abstractive summarization. most unsupervised summarization methods proposed are extractive methods. despite the rise of neural networks, conventional non-neural methods are still powerful in the field of unsupervised extractive summarization. the graph-centrality-based method @xcite @xcite and centroid-based method @xcite have been major methods in this field. other models use reconstruction loss @xcite @xcite , kullback-leibler divergence @xcite or path score calculation @xcite based on multi-sentence compression algorithm @xcite . these methods assume that important topics appear frequently in a document, but our model focuses on a different aspect of texts: the probability of being quoted. that is, our model can extract salient sentences that conventional methods fail to. a few neural-network-based unsupervised extractive summarization methods were proposed @xcite @xcite . however, these methods use pretrained neural network models as a feature extractor, whereas we propose an end-to-end neural extractive summarization model. as for end-to-end unsupervised neural models, a few abstractive models have been proposed. for sentence compression, @xcite employed the task to reorder the shuffled word order of sentences. @xcite employed the reconstruction task of the original sentence from a compressed one. for review abstractive summarization, @xcite revealed parent nodes of tree structures induce summaries, chu and liu post … reply candidate coattention matrix split to sentences decomposable attention predict if reply candidate is an actual reply bilstm bilstm bilstm encoder lstm extractor extracted sentences predictor (only used during training) ## model we propose implicit quote extractor (iqe), an unsupervised extractive summarization model. figure 2 shows the structure of the model. the inputs to the model during training are a post and reply candidate. a reply candidate can be either a true or a false reply to the post. the training task of the model is to predict whether a reply candidate is true or not. the model comprises an encoder, an extractor, and a predictor. the encoder computes features of posts, the extractor extracts sentences of a post to use for prediction, and the predictor predicts whether a reply candidate is an actual reply or not. we describe each component below. ## experiment we train and evaluate the model on two domains of datasets. one is a mail dataset, and the other is a dataset from the social media platform, reddit. ## results and discussion our model outperforms the baseline models more with the eps dataset than the ecs dataset. the overview of the datasets in ## conclusion this paper proposes implicit quote extractor, a model that extracts implicit quotes as summaries. we evaluated our model with two mail datasets, ecs and eps, and one social media dataset tifu, using rouge as an evaluation metric, and validated that our model is useful for summarization. we hypothesized that our model is more likely to extract quotes and that ability improved the performance of our model. we verified these hypotheses with the reddit tifu dataset, but not with the email datasets, because few emails included annotated summaries, and those emails did not have replies with quotes. for future work, we will examine whether our hypotheses are valid for emails and other datasets.