A Unified Span-Based Approach for Opinion Mining with Syntactic Constituents (2021)

Abstract: Fine-grained opinion mining (OM) has achieved increasing attraction in the natural language processing (NLP) community, which aims to find the opinion structures of “Who expressed what opinions towards what” in one sentence. In this work, motivated by its span-based representations of opinion expressions and roles, we propose a unified span-based approach for the end-to-end OM setting. Furthermore, inspired by the unified span-based formalism of OM and constituent parsing, we explore two different methods (multi-task learning and graph convolutional neural network) to integrate syntactic constituents into the proposed model to help OM. We conduct experiments on the commonly used MPQA 2.0 dataset. The experimental results show that our proposed unified span-based approach achieves significant improvements over previous works in the exact F1 score and reduces the number of wrongly-predicted opinion expressions and roles, showing the effectiveness of our method. In addition, incorporating the syntactic constituents achieves promising improvements over the strong baseline enhanced by contextualized word representations.

## introduction opinion mining (om), which aims to find the opinion structures of "who expressed what opinions towards what." in one sentence, has achieved much attention in recent years @xcite @xcite . the opinion analysis has many nlp applications, such as social media monitoring @xcite ) and e-commerce applications @xcite . the commonly used benchmark figure 1 : an example of om, where the blue, yellow, and green blocks denote the opinion expressions, holders, and targets, respectively. mpqa @xcite uses span-based annotations to represent opinion expressions and roles. figure 1 gives an example of its opinion structures with two opinion expressions and related roles. previous om works @xcite @xcite ) mainly treat it as a bmeso-style tagging problem, which converts opinion expressions and opinion roles (holder/target) into bmesobased labels and uses a linking module to connect the predicted expressions and roles. the b, m, and e represent the beginning, middle, and ending word of a role, s denotes a single-word role, and o denotes other words. however, this kind of method is not perfect for the end-to-end om setting, because one word can only belong to one opinion role (one word has only one label), while there exist overlapping opinion structures between different expressions in one sentence. figure 1 gives an example, in which some overlapped opinion relations have been discarded by previous works @xcite , such as [happy, he loves being enderly park, target] and @xcite . there are also other works which focus only on predicting opinions roles based on the gold-standard expressions, which also follow the bmeso-based method @xcite . however, they also suffer from some weaknesses: 1) the expressions are usually fed into the model input as indicator embeddings (1 if the current word belongs to an expression, 0 otherwise), thus one sample is expanded n times if one sentence has n expressions, which is inefficient @xcite . 2) the bmesobased method is weak to capture long-range dependencies and prefers to predict shorter opinion role spans @xcite . motivated by the span-based representations of opinion expressions and roles, we propose a unified span-based opinion mining model (spanom) that can solve or alleviate the aforementioned weaknesses. first, we treat the identification of opinion expressions and roles as two unified binary span classification problems, i.e., judging whether the word span is an expression (or role) or not. then, we allocate the opinion relations on the predicted expression-role pairs. this strategy converts the overlapped opinion role identification of different expressions into classifying different expressionrole pairs. for example, predicting [happy, he loves being enderly park, target] and [loves, he, holder] is infeasible in bmeso-based method, while it is feasible in our span-based method. benefit from the model architecture, the proposed model only needs to train once for one sample in one epoch, which is very efficient for training. besides, the unified model can be easily adapted to the givenexpression setting by using gold-standard expressions. furthermore, inspired by the same spanbased formalism between the syntactic constituents and opinion roles, we explore two types of methods to encode the syntactic knowledge to improve the role spans recognition for two motivations, i.e., multi-task learning (mtl) for enhancing the model representative ability and graph convolutional networks (gcn) @xcite for encoding the constituent structures. we conduct extensive experiments on the commonly used mpqa2.0 dataset and demonstrate that our proposed unified model achieves superior performance compared with previously proposed bmeso-based works. our contributions are: (i) we propose a unified span-based model for opinion mining in the end-to-end fashion that also supports the given-expression setting, (ii) we successfully integrate syntactic constituents knowledge into our model with mtl and gcn, achieving promising improvements, (iii) detailed analyses demonstrate the effectiveness of our unified model and the usefulness of integrating constituent syntactic knowledge on the long-distance opinion roles. ## related work there are several task settings for opinion mining in the community: 1) @xcite ; @xcite focus on labeling the expressions. 2) @xcite ; @xcite ; @xcite discover the opinion structures in the end-to-end setting, i.e, based on the systematic expressions. 3) @xcite ; @xcite identify the opinion roles based on the given expressions. our work follows the end-to-end setting and also supports the given-expression setting. most of the previous opinion mining works treat it as a bmeso-tagging problem, which can be handled by the typical sequence labeling model, such as bi-directional long-short term memory network conditional random field (bilstm-crf). @xcite propose to use traditional feature-based crf model to predict the bmesobased opinion role labels. @xcite propose a bilstm-crf model to first predict the word-wise opinion role label and then determine the relationship with the expression by the role label and distance to the expressions. @xcite propose a transition-based model for opinion mining, which identifies opinion expressions and roles by the human-designed transition actions. @xcite integrate bert representations into a bilstm-crf model, but they do not distinguish different expressions in one sentence. as aforementioned, it is trivial for the sequence labeling style models to handle the overlapped opinion roles belonging to different expressions in one sentence. due to the issue of data scarcity, several kinds of external knowledge have been investigated to improve om performance. @xcite propose several mtl frameworks with semantic role labeling (srl) to utilize semantic knowledge. @xcite extract the semantic representations from a pre-trained srl model and feed them into the opinion mining model, achieving substantial improvements. @xcite incorporate the powerful contextual representations of bi-directional encoder representations from transformers (bert) @xcite and external dependency syntactic knowledge. to solve or alleviate the weaknesses of the previously proposed bmeso-based models, we propose a new method to unifiedly model the opinion expressions and roles, which treats the expres-sion identification, role identification, and opinion relation classification as an mtl problem. besides, to boost the opinion mining performance and motivated by the span-based task formalism, we explore to incorporate syntactic constituents into our model. utilizing span-based representations have been investigated for many other nlp tasks, such as named entity recognition (ner) @xcite , constituency parsing @xcite , and semantic role labeling (srl) @xcite . generally, ner is a single span classification problem, constituency parsing is a span-based structure prediction problem, and srl is a word-span classification problem. different from them, in our methodology, om is a span-span classification problem. ## syntactic constituents since the data scale is relatively small, previous works usually try to integrate external knowledge to enhance the basic om model and improve its performance @xcite . previous sequence tagging models usually incorporate word-wise external information, such as dependency parsing @xcite . we try to investigate the integration of constituent knowledge, which is motivated by their unified span-based formalism. two different methods are explored in our work, i.e., mtl and gcn. ## analysis in this section, we conduct detailed analyses to gain more insights into our unified om model and the effectiveness of integrating syntactic constituents. ## conclusion in this paper, we propose a unified span-based opinion mining model that can handle the overlapped opinion roles, providing a new methodology. our proposed model outperforms previously proposed bmeso-based models in terms of exact match metric on both the end-to-end and givenexpression settings. furthermore, integrating syntactic constituents knowledge with mtl and gcn brings substantial improvements over our bertenhanced baseline model. detailed analyses show the difference between the span-based model and the bmeso-based model and the effectiveness of incorporating syntactic constituents on the determination of opinion role spans.