Comparative Snippet Generation (2022)

Abstract: We model products’ reviews to generate comparative responses consisting of positive and negative experiences regarding the product. Specifically, we generate a single-sentence, comparative response from a given positive and a negative opinion. We contribute the first dataset for this task of Comparative Snippet Generation from contrasting opinions regarding a product, and an analysis of performance of a pre-trained BERT model to generate such snippets.

## introduction the proliferation of opinions on the web has transformed the way users express their opinions and experiences about aspects of products and services. online user reviews contribute personal opinions, and when aggregated together, these reviews play a crucial role in purchasing decisions. however, due to the large volume of reviews, it can be infeasible for customers to skim all such sources. as users have to navigate through a large pool of opinions to make decisions, opinion mining and summarization grows in importance. as such, this area of work has received significant attention. many e-commerce platforms provide functionalities to compare products. these functionalities may be template-based and compare products on the basis of information provided by sellers. comparative opinions from customers, who are experienced users of the product or service, are largely missing from such template-based comparison. on the other hand, question answering systems based on reviews, such as amazonqa @xcite , often only tell one side of the story in the response: either positive or negative. in our opinion, there is a demand for compact representations of both positive and negative opinions of products. such compact textual representation could be enunciated by dialogue agents or shown as a succinct summary to drill-down on in a mobile interface. to the best of our knowledge, no such work has been done yet to provide comparative responses regarding a product to a user. we attempt this novel task. we take as input a positive and a negative opinion regarding a product and generate a comparative, single-sentenced fused response, which we call a comparative snippet (fig. 1 ). we extract single-sentence summaries of positive and negative opinions, separately, from reviews of 3,269 products mentioned by the amazon reviews dataset @xcite . we chose to base our corpus on this existing dataset to help spur future research on our task that can leverage existing work on the parent dataset. we then combine these positive and negative opinions to generate comparative responses. our final dataset contains 174, 394 training instances, 19, 725 validation instances, and 21, 397 test instances foot_0 . we also successfully model positive and negative opinions to generate a comparative response expressing both positive and negative opinions about a target product. opinion summarization. opinion summarization differs from other summarization tasks in two aspects. first, it cannot rely on reference summaries for training, as it is infeasible to get such meta-reviews. to produce a reference summary for a single product, a reviewer may have to go through hundreds of reviews. second, due to the subjectivity and conflicting nature of reviews, the notion of information importance applies differently. in this task, output summaries are based on the popularity of opinions. moreover to be viable, approaches must be flexible with respect to input size as products can be reviewed frequently, resulting in increasing amounts of review content. opinion summarization can be either abstractive or extractive. in abstractive summarization, summaries are generated token-by-token to generate new sentences that articulate prevalent opinions from the inputs. these generated summaries offer a solution to the lack of reference summaries, and can be written in the style of the input reviews. however, prior work have used unrealistically small number of input reviews -10 or fewer -to generate output summaries @xcite ; amplayo and lapata (2021). due to these shortcomings, we chose the alternative style of extractive summarization, which generates summaries by selecting phrases from the inputs. as a foundation, we base our method on @xcite , who used the vector-quantized variational autoencoder (vq-vae) in their extractive opinion summarization. first introduced by @xcite , vq-vae is used to learn discrete latent variables. it passes encoder output through a discretization bottleneck by lookup in the space of latent code embeddings. specifically, we use the quantized transformer (cf. § 3.1), an unsupervised neural model inspired by vq-vae, to generate popularity-driven opinions. this method does not depend on vector averaging, nor does it suffer from information loss, which motivates us to use it as it easily accommodates large numbers of reviews. sentence fusion. sentence fusion combines multiple sentences, which may contain redundancies, into one coherent sentence. the output sentence not only should preserve input information but also any semantic relationships among sentences. sentence fusion requires understanding the discourse semantics between the input sentences. previously, feature-based approaches were used to combine sentences due to the lack of annotated data. recently, a large-scale sentence fusion dataset, dis-cofuse @xcite , was introduced, which has enabled the training of neural network-based models for the fusion task. the authors also train the sequence-to-sequence model to fuse the input sentences and find that the trained model succeeds in combining the sentences through structural constructions, but performs badly when fusion involves inserting discourse connectives. recently, @xcite uses a bert-based encoder-decoder model. although this work improves the accuracy, it struggles in detecting the semantic relationships correctly between the input sentences. predicting discourse markers or connecting strings is a sister task of sentence fusion. it is typically utilized as an intermediate step to improve downstream tasks. @xcite train a model to learn both the discourse relation and discourse connective together in a multi-task framework. in our work, similar to @xcite , we fuse two sentences together by training a model to learn the appropriate insertion of a discourse connective. ## dataset generation an instance of our dataset contains positive and negative opinions as an input and a comparative response as an output as shown in ## model architecture given a positive opinion o @xmath0, and a negative opinion o @xmath1, our task is to generate a response r as shown in ## evaluation and results dataset. we use reviews of products from the "electronics" category of amazon reviews dataset ( 2018 ) @xcite . we generate our dataset in two phases, following the steps in § 3. in the first phase, we consider reviews of 74 products only and hand-curate segments in the generated summaries. we consider syntactically and semantically valid segments only. in the second phase, we scale the number of products and consider reviews of 3, 269 products. after extracting segments from these reviews, we consider only those segments for the summary generation which have part-of-speech patterns similar to hand-curated segments extracted in the first phase. thus, we can ensure syntactic validity of the segments. our final dataset contains 174, 394 training instances generated from reviews of 2, 569 products, 19, 725 validation instances generated from reviews of 321 products, and 21, 397 test instances generated from reviews of 379 products. ## conclusion and future work we introduced a novel task of generating a comparative response (or "snippet") regarding a product that combines positive and negative opinions together in a single sentence. as such comparative responses are not easily found in natural review environments, we generate such comparative responses through extractive summaries of product reviews using an unsupervised approach. to spur future research in this area, we have also made our incorrect mixing pattern distribution however the other hand, [first word from the negative opinion] few users have complained that 9.85% however [first word from the negative opinion] there are people who have complained that 7.83% on the other hand, [first word from the negative opinion] few users have complained that 5.83% on [first word from the negative opinion] there are people who have complained that 7.38% on, some users have also mentioned that 8.15% however the other hand, 6.40% expected although the retractil system works fine, according to a few users the pads are sort of squarish. predicted the the retractil system works fine, according to a few users the pads are sort of squarish. insertion of "on" expected the 415's are a great upgrade from the oem earbuds. but, it is super uncomfortable. predicted the 415's are a great upgrade from the oem earbuds. on, it is super uncomfortable. information modification expected sound is pretty good. but, the movement is actually more like a saw. predicted sound is pretty good. however, the movement is actually more like a see.