OPI at S em E val-2022 Task 10: Transformer-based Sequence Tagging with Relation Classification for Structured Sentiment Analysis (2022)

Abstract: This paper presents our solution for SemEval-2022 Task 10: Structured Sentiment Analysis. The solution consisted of two modules: the first for sequence tagging and the second for relation classification. In both modules we used transformer-based language models. In addition to utilizing language models specific to each of the five competition languages, we also adopted multilingual models. This approach allowed us to apply the solution to both monolingual and cross-lingual sub-tasks, where we obtained average Sentiment Graph F1 of 54.5% and 53.1%, respectively. The source code of the prepared solution is available at https://github.com/rafalposwiata/structured-sentiment-analysis .

## introduction structured sentiment analysis (ssa) can be formulated as an information extraction task in which one attempts to find all of the opinion tuples @xmath0, ..., o n in a text. each opinion o i is a tuple (h, t, e, p) where h is a holder who expresses a polarity p towards a target t through a sentiment expression e, implicitly defining pairwise relationships between elements of the same tuple @xcite . an example of such tuples as a structure sentiment graph was shown in figure 1 . this problem is relatively new and there has been little work published on the subject to date. to stimulate interest in this issue among the nlp community the semeval-2022 task 10: structured sentiment analysis @xcite competition was organized. the contest consisted of two sub-tasks: monolingual and cross-lingual. in the monolingual sub-task, the systems were trained and then tested on the datasets in the same languages. in the cross-lingual sub-task, systems had to be prepared for catalan, basque and spanish datasets, while data in these languages could not be used for training. this setup is often known as zero-shot cross-lingual transfer @xcite . in this paper we present our system for this competition. we mainly focused on the solution for the monolingual track, however, it has also been successfully applied to the cross-lingual. the rest of the paper is organized as follows. section 2 briefly describes related work. section 3 shows an overview of used datasets. section 4 elaborates on our solution. experiments showing the effectiveness of the created system performed on development and test sets are presented in section 5. the next section briefly describes the mistakes and limitations of our system. finally, section 7 concludes this paper. ## related work the recently proposed, aspect sentiment triplet extraction (aste) fill this gap @xcite . the task is to extracting all aspects terms with their corresponding opinion terms and sentiment polarity (tasks a, b, d and e). @xcite propose two stage model. in the first stage, it extracts opinions and aspects along with sentiment using sequence tagging based on the unified bio scheme. the second stage pairs up the predicted terms from the first stage to output triplets. aste is most similar to ssa, missing only the holder extraction. for ssa, the subject of the competition, there are few solutions. @xcite cast the structured sentiment problem as dependency graph parsing. @xcite extend this work and propose a sparse and fuzzy attention scorer with pooling layers which improves parser performance. ## datasets seven structured sentiment datasets in five languages were selected for the competition. the mpqa dataset @xcite contains news documents from the world press in english. ds unis @xcite are english reviews of online universities and e-commerce. opener en and opener es @xcite consist of hotel reviews in english and spanish, respectively. multib eu and multib ca @xcite are also hotel reviews, but in basque and catalan. the last dataset is norec fine @xcite , a multi-domain dataset of professional reviews in norwegian. the statistics of each dataset are sum- some others give the new umuc 5 stars -don't believe them . transformer-based sequence tagging (extraction module) b-holder i-holder o b-targ i-targ i-targ b-exp-pos i-exp-pos o b-exp-neg i-exp-neg b-targ i-targ some others give the new umuc 5 stars -don't believe them . create pairs expression 5 stars target / holder some others 5 stars the new umuc r-bert relation classifier (relation classification module) 5 stars them . don't believe some others don't believe the new umuc don't believe them . 5 stars 5 stars not_related not_related not_related related related related 5 stars some others 5 stars the new umuc 5 stars them . don't believe some others don't believe the new umuc don't believe them . create output 5 stars some others the new umuc don't believe them . positive negative output expression polarity target holder input marized in ## system overview the architecture of our solution is shown in figure 2 . this solution was inspired by the works of @xcite ; @xcite , and especially the work of @xcite . it consists of two main components: extraction module and relation classification module. the first module is based on sequence tagging and is used to extract targets, holders and expressions with polarity. this is accomplished by using a suitable tagset which is a modification of the bio scheme, consisting of the following tags: {b-holder, b-targ, b-exp-neg, b-exp-neu, b-exp-pos, i-holder, i-targ, i-exp-neg, i-exp-neu, i-exp-pos, o}. transformer-based language model with a linear classification layer was used as an implementation. having already extracted entities, the role of the second module is to classify whether there is a relationship between them. specifically, it is about verifying that there is a holder and/or target associated with a particu-lar expression. we utilized the r-bert (wu and he, 2019) model to accomplish this task. based on a sentence with two appropriately marked entities (expression and holder/target), it determines whether or not they are related. 3 entities that are related are combined and form an output. extraction and relation classification modules are trained independently. ## errors analysis as a result of the used architecture, most errors are due to incorrect tagging. in particular, this is relevant to expressions where a correct sentiment is additionally required. the results were significantly worse for expressions limited in a given set, e.g., neutrals in the mpqa or ds unis sets. furthermore, by using a single extraction model, the solution is not able to correctly handle more complicated cases such as mixed or nested tags or opposite polarity expressions. this is most noticeable in the norec fine dataset.