CORT : A New Baseline for Comparative Opinion Classification by Dual Prompts (2022)

Abstract: Comparative opinion is a common linguistic phenomenon. The opinion is expressed by comparing multiple targets on a shared aspect, e.g., “camera A is better than camera B in picture quality”. Among the various subtasks in opinion mining, comparative opinion classification is relatively less studied. Current solutions use rules or classifiers to identify opinions, i.e., better, worse, or same, through feature engineering. Because the features are directly derived from the input sentence, these solutions are sensitive to the order of the targets mentioned in the sentence. For example, “camera A is better than camera B” means the same as “camera B is worse than camera A”; but the features of these two sentences are completely different. In this paper, we approach comparative opinion classification through prompt learning, taking the advantage of embedded knowledge in pre-trained language model. We design a twin framework with dual prompts, named CORT. This extremely simple model delivers state-of-the-art and robust performance on all benchmark datasets for comparative opinion classification. We believe CORT well serves as a new baseline for comparative opinion classification.

## introduction comparative opinion classification @xcite aims to find the relative opinion preference on a specific aspect towards two or more compared targets. sentences containing comparative opinions may not express a direct positive or negative opinion, but a comparison. in this example sentence, "bmw's handling is better than that of mercedes-benz.", there are two targets: bmw and mercedes-benz. the aspect in comparison is handling, and the opinion is target t 1 (bmw) is better than target t 2 (mercedes-benz). note that, the comparison does not imply that the opinion towards mercedes-benz is negative. hence, performing typical sentiment classification as a whole is less applicable to comparative text. comparative opinion plays a vital role in consumers' purchasing decisions. it is common that a consumer identifies a few candidate products and makes a comparison on all aspects of his/her interest. comparative sentences are also widely observed in product reviews and online forums. the research problem. in comparative opinion mining, there are a predefined set of opinions @xmath0, o 2 , . . . , o n }. given a comparative sentence, denoted by @xmath1, w 2 , . . . , w n ], the task is to predict a four-tuple (t 1 , t 2 , a, o). here, t 1 and t 2 are the targets to be compared, a denotes the aspect, and o is the opinion. this task can be decomposed into two subtasks: (i) comparative elements extraction to extract targets t 1 , t 2 in comparison and aspect a, and (ii) comparative opinion classification to predict opinion o with the assumption that targets t 1 , t 2 and the aspect a are given. in this paper, we focus on the second subtask. that is, we assume that targets and aspects are pre-extracted. generally, the opinion set includes better, worse, same and incomparable. however, incomparable can be filtered in the element extraction stage, so we only consider the remaining three opinions. existing studies for comparative opinion are mainly rule-based or machine-learning methods. in general, comparative elements extraction is first performed, to identify comparative sentences and to extract compared targets and aspects. @xcite first identify comparative sentences from review. @xcite ; @xcite ; @xcite extract comparative elements from the identified comparative texts. with comparative text and its comparative elements, @xcite design six rules based on context and pre-defined pros and cons in review to classify comparative opinion. @xcite evaluate a few classifiers with features for comparative opinion classification. despite that deep learning based solutions have significantly advanced the area of sentiment analysis in recent years, to the best of our knowledge, no dedicated deep learning models have been proposed for comparative opinion classification. a major challenge in comparative opinion classification is that, opinion o depends on the order of targets t 1 , t 2 (i.e., t 1 is better than t 2 means that t 2 is worse than t 1 ) except when o is same. for this reason, sentiment analysis models that directly predict positive or negative, cannot well handle comparative opinion classification. to overcome this problem, we design a novel twin framework to detect comparative opinion. in this framework, a primary channel and a mirror channel are designed to capture both the original (i.e., for the order t 1 , t 2 ) and the reversal (for order t 2 , t 1 ) comparative opinions. both channels are realized by prompt-based learning in our framework. specifically, our proposed cort (comparative opinion representations from twin network) model contains two opinion channels (i.e., primary channel and mirror channel), and a comparative module. each channel includes three cells: prompter, encoder, and classifier. given an input in the form of (text, target t 1 , target t 2 , aspect), the prompter generates a template like "[target t 1 ] is [mask] than [target t 2 ] in [aspect]". then the encoder encodes the original input text and the template to get a global representation (i.e., encoding at the [cls] position) and the opinion representation (i.e., encoding at the [mask] position). lastly, the comparative opinion is predicted by a classifier. mirror channel shares the same configurations as the primary channel. the only difference is that the two targets are swapped in the generated template. to the best of our knowledge, this is the first attempt to design a prompt-based learning framework for comparative opinion classification. we demonstrate that cort achieves state-of-the-art performance against all existing baselines on three public datasets, namely camerareview, compsent-19, and compsent-08. more importantly, our cort model is robust and is insensitive to the order of targets in comparison. ## related work comparative opinion expresses opinions by comparing similar targets, which is different from directly expressing an opinion about targets and their aspects @xcite . simply put, comparative opinion mining is the analysis of the contrast between multiple targets/objects @xcite . generally, there are two main subtasks: (i) elements extraction, to extract comparative sentences, targets, aspects, and (ii) comparative opinion classification. this paper focuses on the latter. very few studies consider both subtasks @xcite . as our model is built on top of prompt-based learning, we also briefly review pretrained language models for sentiment analysis. ## cort model the comparative opinion representations from twin network (cort) has its root in prompt learning. we first brief pre-trained language model with prompt (plm w. prompt) for comparative opinion classification. then we detail the design of cort model and its optimization method. ## experiment we now evaluate the proposed base model plm w. prompt and cort for comparative opinion classification, against baselines. ## conclusion in this paper, we focus on comparative opinion classification, a specific sentiment analysis subtask. built on the top powerful pre-trained language models, we show that comparative opinion classification can be addressed by prompt learning with promising accuracy. in our proposed cort, we designed two channels for comparative targets arranged in either order, to facilitate the model to learn the semantics behind the comparative opinion, e.g., a is better than b vs. b is worse than a. experiments show that the proposed cort achieves state-of-the-art performance compared to various baselines, on all comparative datasets. we show that cort based on the twin framework with different pre-trained language models performs beautifully on both the original and reversal data. we also show that the model achieves good performance in cross-dataset setting, demonstrating its effectiveness and robustness. we believe, as a simple and effective model, cort well serves as a new baseline for comparative opinion classification.