Self-Supervised and Controlled Multi-Document Opinion Summarization (2021)

Abstract: We address the problem of unsupervised abstractive summarization of collections of user generated reviews through self-supervision and control. We propose a self-supervised setup that considers an individual document as a target summary for a set of similar documents. This setting makes training simpler than previous approaches by relying only on standard log-likelihood loss and mainstream models. We address the problem of hallucinations through the use of control codes, to steer the generation towards more coherent and relevant summaries.

## introduction recent progress in unsupervised methods has created breakthroughs in natural language processing applications, such as machine translation @xcite . those have been mostly based on a bootstrapping approach, which consists in iteratively alternating between two representations, and optimizing a reconstruction loss. beyond machine translation, other applications include question-answering @xcite and parsing @xcite . while similar ideas have been applied as well for video summarization @xcite , such a bootstrapping approach seems less suited for summarization, because of the inherent information loss when going from the full text to the summarized one. existing unsupervised approaches for summarization therefore relied mostly on extractive graph-based systems @xcite . only recently have there been proposals for unsupervised abstractive summarization, using auto-encoders @xcite . however, these set-ups are complex and require a combination of loss functions @xcite or hierarchical latent variables @xcite to ensure that the generated summaries remain on-topic. in this paper, we investigate a self-supervised approach for multi-document opinion summarization. in this setting, there are multiple opinions (reviews), one entity (products, venues, movies, etc) and the goal is to extract a short summary of those opinions. our approach is based on self-supervision and does not require any gold summaries. we train a supervised model on examples artificially created by selecting (i) one review that will act as a target summary and (ii) a subset of reviews of the same entity that acts as a document collection. neural models have a known problem of hallucination @xcite , which can be misleading in natural language generation tasks as the fluency of those models often distract from the wrong facts stated in the generated text. to reduce this effect, we propose to use control tokens @xcite @xcite . control tokens are discrete variables that are used to condition the generation. different from previous work, our goal is not to allow users to control the generated text, but instead to steer the generated text to produce an output which is consistent with the input documents to be summarized. our main contributions are therefore three-fold: ## related work unsupervised opinion summarization extractive summarization consists in selecting a few sentences from the input documents to form the output summary. the centroid method @xcite @xcite consists in ranking sentences according to their relevance to the whole input. graph-based methods, such as lexrank @xcite or @xcite , use the pagerank algorithm to find the most central sentences in a graph of input sentences, where edge weights indicate word overlap. in contrast to these methods, we focus on abstractive summarization methods. non-neural abstractive methods @xcite are also graph-based, but work on word-type graphs. @xcite introduced a self-supervised model for sentence compression: they use an unsupervised extractive system to generate training data for a supervised sentence compressor. their system works on single sentences whereas our end-to-end approach summarizes multiple reviews. recently, a few approaches for neural unsupervised abstractive summarization have been proposed. @xcite introduced a summarization system based on a review autoencoder. at inference time, meansum encodes every review for a product to a vector, computes the centroid of reviews and uses this centroid to seed the decoder and generate a summary. however, averaging representations of statements that are sometimes contradictory tends to confuse the decoder, and might lead it to ignore the input signal. to deal with this limitation, @xcite add a clustering step to group similar reviews and to generate one sentence per such found cluster. @xcite proposed to solve the problem of unsupervised opinion summarization with an auto-encoder with latent variables. they use latent variable for products and reviews to address the hallucination issue, while at the same time allowing it to capture information from the set of reviews on the same entity. in contrast, we argue that our self-supervised setting is simpler as it relies on training with standard models. in addition, the use of transformer (as opposed to gru in their case) makes it possible to apply separate attentions to each input. probably most similar to our self-supervised proposal is the recent work of amplayo and lapata (2020), in particular their document noising sub-strategy. compared to it, our simple selection criteria of the dataset avoids any use of (domain-specific) noise generator. in addition, our use of control tokens allows to easily include existing (or inferred) meta-information. a similar approach is also used by @xcite , which trains a seq2seq model by clustering reviews and using the medoid as target summary. another work that has recently shown the promise of self-supervision for summarization is @xcite , in which masked-out sentences are predicted from the surrounding text. our self-supervision training mechanism can be seen as a multi-document version of that. controlled generation controllable text generation has been previously investigated to apply global constraints on text generation, by directly optimizing evaluation metrics through policy gradient methods @xcite @xcite @xcite or continuous approximation methods @xcite . other methods applied control only at inference time. weighted decoding @xcite was shown to be challenging, and often detrimental to fluency and coherence @xcite . constrained beam search @xcite @xcite is slower, requires very large beam sizes, and does not enable soft constraints. finally, updating the decoder hidden states @xcite ) requires an extra training step. control codes have been introduced in generation as an early form of copy mechanism @xcite to address the problem of rare words. they were widely adopted to steer language models towards specific features, such as aspects @xcite or structured outputs @xcite . in prior work, controlled language models rely on a predefined set of control tokens, collected manually @xcite or from dictionaries @xcite , which can lead to low domain coverage. @xcite and elsahar and el-beltagy (2015) construct lexicons by exploiting the feature selection ability of sentiment classifiers, an approach that produces more relevant lexicons than classical topic models (e.g. lda, @xcite . in our work, we also rely on classifiers us-ing the categories of reviews provided as meta-data. without meta-data, we could have relied instead on unsupervised or weakly supervised aspect extractors @xcite . hierarchical encoding. in order to allow a neural summarizer to read several sections, @xcite proposes a hierarchical lstm that works at two level. similar to our proposal, @xcite extends a transformer network to read several ranked paragraphs as input, avoiding a retrievethen-read pipeline. in multi-document summarization, the paragraphs are not ranked but independent. this entails a significant change model-wise. we propose to encode each review independently (avoiding inter-paragraph self-attention) and only adapt the decoder-encoder attention. ## self-supervision in order to create our training dataset we assume that a review s i for an entity (venue or product) can serve as a summary for a set of other similar reviews d i . this simple intuition allows us to create training points (d i , s i ) in a very similar way to what the model will experience at inference time. however, there are two issues with this approach. first, the potential set of training points is too large to be explored exhaustively. given the set of all reviews d the total number of possible input-output pairs is 2 |d|-1 × |d|. second, the assumption that any review is fit to serve as a summary for any set of other reviews is obviously not true, and might yield a very noisy training dataset. to solve the combinatorial explosion, we limit the size of d i to k, and from a given s i , we look for a set of k good reviews d i , for which s i serves as a good summary. fixing k also simplifies training, and enables comparison with previous work where the number of input reviews is fixed @xcite . both s i and all members of d i are reviews of the same entity. fixing first the target summaries turns traditional approaches upside down. in particular, a recently proposed theoretical model of importance in summarization @xcite defines the importance of a summary based on three aspects: (i) minimum redundancy, (ii) maximum relevance with the input document, and (iii) maximum informativeness. in that line of work d i is considered fixed: redundancy and informativeness are not dependent on d i and can therefore be ignored when s i is fixed. in this setting @xcite reduces then to eq. 1 then, we sort the data-points (d i , rel(d i )) according to the value of the relevance depending on the desired size of the target dataset, we keep the top-t pairs for training. limiting t inherently increases informativeness, since it limits the creation of training examples where input and outputs are repetitive similar reviews that might be very prominent on corpora level (e.g. "great restaurant."). this method is simple and fast, thanks to nearest neighbour search libraries @xcite . for all our experiments we defined sim to be the cosine similarity over a tf-idf bag-of-word representation @xcite . ## controlling hallucinations hallucinations are pieces of generated text that bear no relationship to the text they were conditioned on. they are likely to happen in our self-supervised setting, due to the noise from the construction of training instances. this might happen if the synthetically created training data contains contradictory signals, or because certain types of review are overly present (e.g. "great movie"). the model might default to those frequent patterns if it finds itself in a unfrequent state during decoding. to alleviate the problem of hallucinations, we propose to use control tokens that represent desired traits of the output text to steer the generated text towards more input-coherent summaries. these control tokens are inferred from each review, and used as prompts at inference time. we use two types of codes as follows: to do so, we rely on the intrinsic feature selection capabilities of regularized linear classification models. for each category ℓ in the meta-data associated with each review we train a linear support vector machine (svm) classifier @xcite foot_0 that learns to classify between reviews from this category and negative examples sampled randomly from the rest of the corpus. the features of the svms are parameterized by the weight vector θ ℓ ∈ r d , where d is the number of features (in our experiments: all unigrams and bigrams present in the corpus). we used a squared hinge loss with l1 regularization over θ ℓ -the latter to increase sparsity and force feature selection @xcite . finally, we trim the feature list into those who correspond to positive weights and renormalize the weights. the output of this step is a ranked list of n-grams that represent the distinctive aspects of each category. when creating training data for summarization, we enrich each review with the top weighted ngrams of their corresponding categories as follows. for a given review d about entity p, we consider all m labels of p and use the weights of the corresponding classifiers θ ℓ (p) i (for each label ℓ (p) i of p). we only consider those n-grams actually occurring in d, and keep the top 8 such features. note that these features could come from different classifiers, as we consider all m labels. during training, we enrich each review with its tailored control codes. in particular, the reviews acting as summary also contain them, and by construction those are n-grams present in the text. at inference time -when the target side and its control codes are not available -we select the most repeated control tokens from the input side and feed them as a prefix to the decoder. there is clearly a risk that the model just learns to copy the control codes it has seen somewhere in the text. we check whether this is the case in sect. 7. ## multi-source transformer model previous work for multi-document summarization @xcite built multi-source input representations through a simple mean over the last hidden states of the encoder. an intrinsic limitation of this method is that the full set of reviews is represented as a single vector. this aggregation might cause information distortion especially when some input reviews are expected to have conflicted opinions in between. standard transformer models @xcite consider only a single input to the decoder part of the model. aggregating all input reviews into a single input (junczys-dowmunt, 2019) with special tokens to represent document boundaries might be slow and impractical due the o(n 2 ) complexity of the self-attention mechanism. we therefore experiment with several input combination strategies of the transformer cross-attention @xcite . parallel. at each cross-attention head, the decoder set of queries q attend to each of the encoded inputs separately from which the set of keys (k @xmath1:m ) and values (v @xmath2:m ) are generated and then the yielded context is averaged and followed by a residual connection from the previous decoder layer (box c in fig. 1 ). mean. we also propose a simpler and less computationally demanding input combination strategy. it does not apply the cross-attention with each encoder separately. instead, the set of keys and values coming from each input encoder are aggregated using the average at each absolute position. afterwards the decoder set of queries attend to this aggregated set of keys and values. this combination can be seen as a more efficient variation of the flat combination strategy @xcite with mean instead of concatenation. fig. 2 depicts this strategy, which replaces box (c) in fig. 1 . in practice, we share the parameters across all encoders, this can be also seen as a single encoder used to encode each input document independently. we believe that this is an appropriate design choice as the order of the input document doesn't matter. furthermore, this is necessary to allow variable number of input documents during different training batches or during inference. in sect. 7, we compare both approaches through an ablation study, focusing on summary quality as well as empirical training times. ## experimental setup experimental details all our models are implemented with pytorch @xcite and fairseq @xcite libraries, as well as scikit-learn @xcite for the classifiers used either for inferring control tokens or for evaluation. for all our models we use sentence piece (kudo and richardson, 2018) as a tokenizer with a vocabulary size of 32 000. we use the same hyperparameters as the transformer big model described by @xcite (d @xmath3, n @xmath4, n @xmath5, @xmath6.1). we optimize them with a nesterov accelerated sgd optimizer with a learning rate of 0.01. we train all models for a total of 80 000 steps across 25 epochs, with linear warm-up for the first 8 000 steps. we select the best model checkpoint based on perplexity on the validation set. all models were trained on one machine with 4 nvidia v100 gpus, the ## evaluation results automatic evaluation with the rottentomatoes dataset we only benchmarked the graph-based unsupervised methods, since the released pretrained meansum model does not cover the domain of movie reviews. we attribute the lower score in sentiment accuracy to the fact that the "summaries" in rottentomatoes are critical reviews, written in a very different style than the original reviews. fig. 3 shows summaries generated by different models from the same input. we notice that our model learned to copy aspects of the input documents such as restaurant names "capricotti's" and menu items "the bobbie", possibly due to the crossattention mechanism. we provide more examples as supplementary material. human evaluation existing natural language generation systems are known to generate very fluent language, that looks very natural to native speakers. on the other side, current neural models are known to generate factually incorrect data, something which was less of a concern in preneural methods but also much harder to detect. as mentioned by @xcite : "neither of the methods explicitly examines the factual consistency of summaries, leaving this important dimension unchecked." inspired by @xcite we decided to focus the human evaluation on those aspects of the summarization evaluation in which existing models risk failing the most, the one of faithfulness. we annotated 94 summaries through a crowdsourcing platform, comparing 3 systems (gold, meansum and ours). workers were asked if "the summary contains correct information given the original reviews". in total we had 282 tasks (94×3) and each task was labeled by 3 annotators and paid @xmath015 / hour) and restricted to experienced, englishspeaking workers. a full description of the campaign, including the filtering of the annotations, is detailed in the supplementary material. ## conclusion the promise of unsupervised multi-document abstractive summarization is been hampered by the complexity of those models and the problem of hal-lucinations. our proposed model has the advantage of being very simple to train compared to previous proposals. in addition, the combined use of control-codes to steer the generation and of multiinput transformers results in summaries that are better (as measured by automatic measures), and produce more faithful summaries (as measured by human evaluation). while the generated reviews are more factual than those generated by other models, we want to stress that inaccuracies can still appear. generated summaries are often conjugated in first person, which could lend to believe that an actual human wrote those. we recommend strongly that any use of such algorithms to be accompanied by a clear disclaimer on its true nature. figures 6, and 7 include a set of samples generated from our model and baselines, full generations our model can be downloaded from https://www.dropbox.com/s/ w6eqviy5fnda11f/hypos and refs.zip?@xmath7