UFRGS ent at S em E val-2022 Task 10: Structured Sentiment Analysis using a Question Answering Model (2022)

Abstract: This paper describes the system submitted by our team (UFRGSent) to SemEval-2022 Task 10: Structured Sentiment Analysis. We propose a multilingual approach that relies on a Question Answering model to find tuples consisting of aspect, opinion, and holder. The approach starts from general questions and uses the extracted tuple elements to find the remaining components. Finally, we employ an aspect sentiment classification model to classify the polarity of the entire tuple. Despite our method being in a mid-rank position on SemEval competition, we show that the question-answering approach can achieve good coverage retrieving sentiment tuples, allowing room for improvements in the technique.

## introduction opinions abound on the internet nowadays. they are a valuable source of information since people often rely on them for making purchases. companies can also benefit from this vast amount of opinions, as they do not need to conduct opinion polls or focus groups to measure the acceptance of a particular product @xcite . the large volume of opinions available becomes hard for humans to process. this leads to the study of ways of automating the processing of opinions, in order to summarize them. sentiment analysis is the field of study which aims at processing the information conveyed by unstructured texts, providing structured information that facilitates the understanding of the opinions, attitudes, or emotions towards a particular entity @xcite . sentiment analysis can be performed at different levels of granularity (entire review, sentence, or aspect). aspect-based sentiment analysis (absa) aims to identify and rate the features (or aspects) of the entity being evaluated. typically, absa involves the following phases: (i) identify and extract entities in reviews; (ii) identify and extract the aspects of an entity; (iii) cluster similar aspects; and (iv) determine the polarity of the sentiment over the entities and the aspects. most of the research in sentiment analysis focuses on solving only one of these phases at a time. task 10 in semeval 2022 -structured sentiment analysis @xcite proposes a new approach to tackle the sentiment analysis problem, where the elements that constitute an opinion are identified together, in a structured way through a graph. thus, structured sentiment analysis can be seen as an information extraction task since we want to find the text spans where opinions about a particular feature are expressed @xcite . in this paper, we describe ufrgsent, a multilingual approach that relies on a question answering system to find the elements of an opinion present in review texts and a fine-tuned model to classify the polarity of the sentiment tuple. our average results ranked 20 th out of 31 participating systems. nevertheless, we believe there is room for improvement in our technique. ## background and related work an opinion can be defined as a tuple @xmath0, t, e, p), where h represents the opinion holder (person who emits the opinion), t is the aspect target of the entity being reviewed, e is the opinion that is being expressed, and p is the sentiment related to the aspect expressed on the review @xcite . while many works treat each opinion component separately, some approaches extract them all together, taking advantage of the components being interconnected. graph neural networks @xcite , transition-based neural models @xcite , and multi-task learning @xcite ) can be used to accomplish this task. there are also works applying co-extraction to find correlated opinion com-ponents, such as aspect words and corresponding polarities @xcite , aspect and opinion terms @xcite , or aspects, opinions, and polarities @xcite . question answering is a challenging and wellstudied problem in the natural language processing field, gaining attention in the last years due to the use of pre-trained language models. one of the most popular subtasks of question answering is the machine reading comprehension task. this task consists of, from a text piece (also known as context) and a question, finding the answer to the question in context @xcite . @xcite proposed using a machine reading comprehension system to solve the problem of aspect sentiment triplet extraction. they use three-turn questions to extract aspects and opinions (in the first two turns) and sentiments (in the last turn), achieving state-of-the-art performance on standard aspect-based sentiment analysis (absa) datasets. ## results considering the darmstadt_unis dataset, the best result was when we just considered the aspect questions and generated one answer. on the other hand, the best results for mpqa and multibooked_eu datasets were obtained using the three types of questions and one, five, or ten answers. we noticed a significant loss in performance when generating three answers. we conclude that this happened because the first answer generated is the expected response most of the time. producing additional answers tends to introduce noise in the candidate tuples. however, increasing the number of answers makes the correct answer be generated more times, yielding the right tuple choice. in comparison with other participants, our average score was the 20 th result out of 31 participating systems. the dataset in which we achieved the best rank was multibooked_ca (15 th ), while our worst results were darmstadt_unis and mpqa datasets (20 th out of 31). although our team did not submit results for the cross-lingual task, we emphasize that our solution is entirely multilingual -there is only one model, which extracted sentiment tuples for all datasets simultaneously. therefore, our solution can be extended to any other language among the 104 languages present in multilingual bert. in order to assess the quality of our candidate tuple extraction, we measured the coverage of the question-answering results (i.e., the percentage of gold tuples present in the set of candidate tuples). this measurement was done on the development dataset, for which we know beforehand the gold sentiment graphs. the results of the experiment can be seen in we set the hyper-parameter k to one, extracting only one answer per question. the evaluation was made in two ways -the exact match between gold tuple and candidates and the overlap, in which a pair of tuples containing an overlap between their tokens is considered a correct match. the experiment was repeated, varying the type of questions the results show that our question-answering technique provides good coverage of sentiment tuples. for darmstadt_unis and mpqa, we have an exact match with over 65% coverage. we also improved the measure on datasets with less coverage in the exact match experiments, considering tuple overlap. all datasets achieved at least 70% coverage in the overlap experiments. using only one component to extract candidate tuples reduces coverage for the question answering for all datasets. while the mpqa dataset was less affected by removing the components (loss of 2.9% in coverage using targets, 3.9% using opinion expressions, and 2.4% using holder questions). other datasets had significant drops in coverage, especially when just holder questions were considered. ## conclusion in this paper, we described our system submitted to semeval-2022 task 10. we designed a multilingual approach that relies on a qa system and an asc model to find the sentiment graphs. the key idea is that the joint and incremental extraction of holder, target, and opinion helps to achieve a good coverage for ufrgsent. in these preliminary experiments, we could not establish the quality of our tuple ranking and selection methods and we leave it for future work. additionally, we are interested in understanding how well our multilingual model performs against a monolingual version of our technique, and how other state-of-the-art qa models can improve the extraction of sentiment tuples.