Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with R o BERT a (2021)

Abstract: Aspect-based Sentiment Analysis (ABSA), aiming at predicting the polarities for aspects, is a fine-grained task in the field of sentiment analysis. Previous work showed syntactic information, e.g. dependency trees, can effectively improve the ABSA performance. Recently, pre-trained models (PTMs) also have shown their effectiveness on ABSA. Therefore, the question naturally arises whether PTMs contain sufficient syntactic information for ABSA so that we can obtain a good ABSA model only based on PTMs. In this paper, we firstly compare the induced trees from PTMs and the dependency parsing trees on several popular models for the ABSA task, showing that the induced tree from fine-tuned RoBERTa (FT-RoBERTa) outperforms the parser-provided tree. The further analysis experiments reveal that the FT-RoBERTa Induced Tree is more sentiment-word-oriented and could benefit the ABSA task. The experiments also show that the pure RoBERTa-based model can outperform or approximate to the previous SOTA performances on six datasets across four languages since it implicitly incorporates the task-oriented syntactic information.

## introduction aspect-based sentiment analysis (absa) aims to do the fine-grained sentiment analysis towards aspects @xcite . specifically, for one or more aspects in a sentence, the task calls for detecting the sentiment polarities for all aspects. take the sentence "great food but the service was dreadful" for example, the task is to predict the sentiments towards the underlined aspects, which expects to get polarity positive for aspect food and polarity negative for aspect service. generally, absa contains aspect extraction (ae) and aspect-level sentiment classification (alsc). we only focus on the alsc task. early works of alsc mainly rely on manually designed syntactic features, which is laborintensive yet insufficient. in order to avoid designing hand-crafted features @xcite , various neural network models have been proposed in alsc @xcite @xcite @xcite @xcite . since the dependency tree can help the aspects find their contextual words, most of the recently proposed state-of-the-art (sota) alsc models utilize the dependency tree to assist in modeling connections between aspects and their opinion words @xcite @xcite . generally, these dependency tree based alsc models are implemented in three methods. the first one is to use the topological structure of the dependency tree @xcite @xcite @xcite ; the second one is to use the treebased distance, which counts the number of edges in a shortest path between two tokens in the dependency tree @xcite @xcite ); the third one is to simultaneously use both the topological structure and the tree-based distance. except for the dependency tree, pre-trained models (ptms) @xcite , such as bert @xcite , have also been used to enhance the performance of the alsc task @xcite @xcite . from the view of interpretability of ptms, @xcite ; @xcite ; @xcite try to use probing methods to detect syntactic information in ptms. empirical results reveal that ptms capture some kind of dependency tree structures implicitly. therefore, two following questions arise naturally. q1: will the tree induced from ptms achieve better performance than the tree given by a dependency parser when combined with different tree-based alsc models? to answer this question, we choose one model from each of the three typical dependency tree based methods in alsc, and compare their performance when combined with the parser-provided dependency tree and the off-the-shelf ptms induced trees. q2: will ptms adapt the implicitly entailed tree structure to the alsc task during the finetuning? therefore, in this paper, we not only use the trees induced from the off-the-shelf ptms to enhance alsc models, but also use the trees induced from the fine-tuned ptms (in short ft-ptms) which are fine-tuned on the alsc datasets. experiments show that trees induced from ft-ptms can help tree-based alsc models achieve better performance than their counterparts before finetuning. besides, models with trees induced from the alsc fine-tuned roberta can even outperform trees from the dependency parser. last but not least, we find that the base roberta with an mlp layer is enough to achieve state-ofthe-art (sota) or near sota performance on all six alsc datasets across four languages, while incorporating tree structures into roberta-based alsc models does not achieve concrete improvement. therefore, our contributions can be summarized as: a myriad of works attempt to fuse dependency tree into neural network models in alsc. @xcite propose to convert the dependency tree into a binary tree first, then apply the adaptive recursive neural network to propagate information from the context words to aspects. despite the improvement of aspect-oriented feature modeling, converting the dependency tree into a binary tree might cause syntax related words separated away from each other. in general, owing to the syntax parsing errors, early dependency tree based alsc models do not show clear preponderance over models without the dependency tree. however, the introduction of the neural network into the dependency parsing task enhances the parsing quality substantially @xcite . recent advances, leveraging graph neural network (gnn) to model the dependency tree @xcite @xcite @xcite , have achieved significant performance. among them, @xcite ; @xcite attempt to convert the dependency tree into the aspect-oriented dependency tree. instead of using the topological structure of dependency tree, @xcite al. (2018); zhang et al. (2019b) ; phan and ogunbona (2020) exploit the tree-based distance between two tokens in the dependency tree. ptms-based dependency probing over the past few years, the pre-trained models (ptms) have dominated across various nlp tasks. therefore, many researchers are attracted to investigate what linguistic knowledge has been captured by ptms @xcite @xcite . @xcite try to use a single or a combination of head attention maps of bert to infer the dependencies. since bert has many attention heads, this method can hardly fully reveal the dependency between two tokens. @xcite propose a small learnable probing model to probe the syntax dependencies encoded in bert. despite very few parameters been added, it may still be very hard to tell if the syntactic information is encoded by bert itself or by the additional parameters from the probing model. therefore, the parameterfree dependency probing method proposed in @xcite might be more preferred. ## method in this section, we first introduce how to induce trees from ptms, then we describe three tree-based alsc models, which are selected from three representative methods of incorporating the dependency tree in alsc task. ## experimental setup in this section, we present details about the datasets, the tree structures used in experiments, as well as the experiments implementations. we conduct experiments on all six datasets across four languages. but due to the limited space, we present our experiments on the non-english datasets in the appendix. ## conclusion in this paper, we analyze several tree structures for the alsc task including parser-provided dependency tree and ptms-induced tree. specifically, we induce trees using the perturbed masking method from the original ptms and alsc finetuned ptms respectively, and then compare the different tree structures on three typical tree-based alsc models on six datasets across four languages. experiments reveal that fine-tuning on alsc task forces ptms to implicitly learn more sentimentword-oriented trees, which can bring benefits to glove based alsc models. benefited from its better implicit syntactic information, the fine-tuned roberta with an mlp is enough to obtain sota or near sota results for alsc task. our work can lead to several promising directions, such as ptmssuitable tree-based models and better tree-inducing methods from ptms. english datasets. moreover, we find that the results of the ft-roberta induced tree usually have more stable f 1 scores. (2) subjected to the quality of the parser of non-english languages, models using the plms induced trees achieve slightly better performance compared to "dep.". this illustrates that the dependency tree could be very sensitive to parser and quality of corpus. (3) similarly, from "roberta induced tree" and "ft-roberta induced tree", we conclude that fine-tuning can substantially enhance the alsc performance through trees induced from plms.