U ni COQE : Unified Comparative Opinion Quintuple Extraction As A Set (2023)

Abstract: Comparative Opinion Quintuple Extraction (COQE) aims to identify comparative opinion sentences in product reviews, extract comparative opinion elements in the sentences, and then incorporate them into quintuples. Existing methods decompose the COQE task into multiple primary subtasks and then solve them in a pipeline manner. However, these approaches ignore the intrinsic connection between subtasks and the error propagation among stages. This paper proposes a unified generative model, UniCOQE, to solve the COQE task in one shot. We design a generative template where all the comparative tuples are concatenated as the target output sequence. However, the multiple tuples are inherently not an ordered sequence but an unordered set. The pre-defined order will force the generative model to learn a false order bias and hinge the model’s training. To alleviate this bias, we introduce a new “predict-and-assign” training paradigm that models the golden tuples as a set. Specifically, we utilize a set-matching strategy to find the optimal order of tuples. The experimental results on multiple benchmarks show that our unified generative model significantly outperforms the SOTA method, and ablation experiments prove the effectiveness of the set-matching strategy.

## introduction as an essential branch of opinion mining, comparative opinion mining aims to explore the information of comparisons in product reviews. @xcite first proposed the concept of comparative opinion mining and introduces two primary subtasks: comparative sentence identification (csi) and comparative element extraction (cee). the former aims to identify whether a given sentence is a comparative sentence, while the latter aims to extract all the comparative elements in a given comparative sentence. @xcite further figure 1 : an example of the comparative opinion quintuple extraction (coqe) task. given a product review sentence, coqe aims to identify whether it is a comparative sentence, extract all the comparative elements (if existing), and incorporate them into quintuples. in order to integrate various subtasks of comparative opinion mining, @xcite first proposed the comparative opinion quintuple extraction (coqe) task (shown in fig. 1 ). coqe aims to identify comparative opinion sentences in product reviews and extract five comparative opinion elements in the sentences, i.e., comparative subject (sub), comparative object (obj), comparative aspect (ca), comparative opinion (co) and comparative preferences (cp), and then incorporate them into a quintuple (sub, obj, ca, co, cp). @xcite adopted a multi-stage model, decomposing the coqe task into primary sub-tasks (csi, cee, and cpc formerly mentioned), and then solving them one by one in a pipeline manner. however, the pipeline model ignores the internal connection between multiple subtasks of comparative opinion mining, and the error propagation between each stage heavily strains the model's performance. to this end, we employ a generative extraction model called unicoqe for the first time on the coqe task. we utilize t5 @xcite as the backbone and propose a generative template to adapt to the coqe task, identifying comparative sentences and extracting all quintuples once and for all. in the generation paradigm, we concatenate all the golden comparative tuples together as the target output sequence of the model. however, multiple tuples are essentially not an ordered sequence but an unordered set. if a pre-defined order is imposed, it will introduce an order bias, forcing the generative model to learn the bias, which hinders the model's training. taking fig. 1 as an example, there are four target tuples: t 1 , t 2 , t 3 , and t 4 . theoretically, a 4 4 = 24 types of permutations of target tuples are all correct. during training, the model would get "confused": why t 1 ; t 2 ; t 3 ; t 4 is correct but t 4 ; t 3 ; t 2 ; t 1 is unacceptable? in order to alleviate this order bias problem, we introduce a "predict-and-assign" training paradigm to the generative model. during the training phase, we first let the model autoregressively predict comparative tuples in the given sentence. subsequently, we model the golden tuples as a set and use the hungarian algorithm @xcite to match the set of golden tuples with the predicted sequence to find the optimal order of golden tuples. finally, we validate the performance of our approach on three coqe benchmarks. experimental results show that our model significantly outperforms sota methods, and the effectiveness of the set-matching strategy is demonstrated through ablation experiments. the contributions of this paper can be summarized as follows: as an important subtask of opinion mining, the task of comparative opinion mining was first proposed by @xcite , which aims to identify comparative sentences in product reviews and extract all the comparative opinion elements (entities, features, and comparative keywords). specifically, it used class sequential rules @xcite to identify comparative sentences and label sequential rules to extract comparative elements. some subsequent studies concentrated on the comparative sentence identification (csi) task. @xcite used diverse features (e.g., keywords and sequential patterns) to recognize comparative sentences. park and blake (2012) exploited semantic and grammatical features to explore the task of identifying comparative sentences in scientific texts. @xcite recognized comparative sentences on chinese documents based on keywords, sentence templates, and dependency analysis. on the comparative element extraction (cee) task, @xcite used semantic role labeling (srl) to analyze the structure of comparative sentences and trains a conditional random field (crf) to extract comparative features. some studies @xcite @xcite also used crf as the extraction model. @xcite further explored the application of existing srl methods to comparative element extraction. @xcite proposed applying deep learning methods to comparative opinion mining, mainly using an lstm-crf framework to extract comparative elements. considering the early comparative opinion mining tasks did not include the author's comparative preference, ganapathibhotla and @xcite proposed the comparative preference classification (cpc) task for the first time, aiming to predict which entity is preferred given a comparative sentence and its comparative elements. it utilized a keyword-based approach to identify comparative preferences. @xcite used a pretrained encoder to encode sentences and classified sentences' comparative preference based on xg-boost @xcite . @xcite employed a graph attention network to model the syntactic parsing information of comparative sentences to better predict comparative preferences. nevertheless, the premise of the cpc task is that the two entities to be compared are annotated in advance, which is challenging to apply in real-world scenarios. liu et al. (2021 we utilize t5 as the backbone of our generative model and employ a "predict-and-assign" training paradigm to alleviate the order bias of the vallina generative model. during training, we temporarily turn off the model's gradient and let the model autoregressively predict the tuples. then we model the golden tuples as a set and use the hungarian algorithm to match the set of golden tuples with the predicted sequence to assign the optimal order of golden tuples. in addition to subtasks such as csi, cee, cpc, and coqe, some research directions are also closely related to comparative opinion mining. comparative question answering system @xcite allows the machines to automatically answer the comparative question "is x better than y with respect to z?'. opinion tuple extraction @xcite and quadruple @xcite extraction in traditional aspect-based sentiment analysis aim to extract fine-grained opinion information in the text. several studies have also explored the use of setmatching strategies for generative models. in the keyphrase extraction task, @xcite concatenate all the keyphrases as target outputs of transformer @xcite without predefining an order. in the event argument extraction task, @xcite introduces a scheme for optimal span assignments of bart @xcite . these studies demonstrate the effectiveness of set matching strategies in generative models, highlighting their potential for improving the performance of generative lms. ## methodology this section introduces the unicoqe framework in detail (as shown in 2). in this framework, we model the coqe task as a natural language generation task. we use the generative pre-trained language model t5 @xcite as the backbone model and adopt a generation template to directly identify comparative sentences and output the comparative quintuples therein in an end-to-end manner. to further alleviate the order bias problem of the generative models, we introduce the "predict-and-assign" training paradigm. ## conclusion in this paper, we investigate the task of comparative opinion quintuple extraction. to overcome the error propagation problem of previous pipeline models, we propose an extraction model based on the generative paradigm. we further introduce a set-matching strategy based on the hungarian algorithm to alleviate the order bias of the generative model during training. the experimental results show that our model significantly outperforms the sota models, and we verify the effectiveness of the set-matching strategy through in-depth experiments.