Structural Bias for Aspect Sentiment Triplet Extraction (2022)

Abstract: Structural bias has recently been exploited for aspect sentiment triplet extraction (ASTE) and led to improved performance. On the other hand, it is recognized that explicitly incorporating structural bias would have a negative impact on efficiency, whereas pretrained language models (PLMs) can already capture implicit structures. Thus, a natural question arises: Is structural bias still a necessity in the context of PLMs? To answer the question, we propose to address the efficiency issues by using an adapter to integrate structural bias in the PLM and using a cheap-to-compute relative position structure in place of the syntactic dependency structure. Benchmarking evaluation is conducted on the SemEval datasets. The results show that our proposed structural adapter is beneficial to PLMs and achieves state-of-the-art performance over a range of strong baselines, yet with a light parameter demand and low latency. Meanwhile, we give rise to the concern that the current evaluation default with data of small scale is under-confident. Consequently, we release a large-scale dataset for ASTE. The results on the new dataset hint that the structural adapter is confidently effective and efficient to a large scale. Overall, we draw the conclusion that structural bias shall still be a necessity even with PLMs.

## introduction aspect sentiment triplet extraction (aste) is a task central to fine-grained opinion mining. compared to aspect sentiment classification that only aims to predict sentiment polarities for various aspects, aste instead extracts descriptive opinion units in the form of triplets (i.e., aspect-opinionsentiment tuples). for example, (food, great, pos) and (service, dreadful, neg) are aspect sentiment triplets for the sentence in figure 1 (top), where {pos, neg, neu} respectively represent {positive, negative, neutral}. while aste can be generally tackled with neural models in either a pipeline manner @xcite or a multi-task manner @xcite @xcite @xcite , the aspect sentiment triplets can be rather derivable from dependency structures (e.g., syntactic dependency trees) with hand-crafted rules @xcite . for the example in figure 1 (bottom), the triplets can be recognized via certain structural dependency relations. 2 various studies are motivated by this intuition and exploit dependency bias to enhance neural aste models @xcite ), yet without a necessary comparison with pretrained language models (plms). on the other hand, recent advances find that using plms can already achieve compelling performance @xcite @xcite owing to implicit structures captured by plms @xcite . it signals that, compared with plms, explicit structural biases such as dependency bias, may become cumbersome @xcite due to parameter inefficiency and latency inefficiency. that is, combining dependencies into models can require redundant parameters to achieve structure encoding, while producing structures can also require increased latency to achieve external parsing. therefore, a critical question naturally arises: is structural bias still a necessity for aste in the context of plms? in this paper, we aspire to answer the question from two perspectives: 1) whether structural bias can be incorporated into plms in a flexible way in terms of both parameter and latency efficiency; and 2) whether structural bias can enhance plms for aste. to boost the parameter efficiency, we develop the idea of adapter and put forward a parameterefficient adapter that can incorporate structural bias. the adapter @xcite was proposed initially to integrate additional modules into plms and enable plms to leverage inductive bias efficiently. although feasible, such adapters can be far from lightweight. for example, @xcite introduces a series of linear transformations in their proposed adapter which involve numerous parameters. in contrast, instead of introducing carefullydesigned plugins, we propose to use structured attention maps induced with structures, to additively impact the raw attention maps in self-attention, thus requiring only a tiny amount of incremental parameters. to improve the latency efficiency, we argue that dependency distance is a sufficient simplification of the dependency graph since the simplification has been proven equally powerful in downstream tasks like aspect sentiment classification @xcite . on this basis, we further propose to use relative distance as an alternative to the dependency distance. the intuition lies in the observation that opinions predominantly locate closely to their corresponding aspects @xcite , and thus we posit that using relative distance bias would suffice for the purpose of aste. in fact, the relative distance is also exhibited to bring merits to the transformer architectures in previous work @xcite . as the relative distance can be obtained with cheap operations in lower latency, the latency efficiency issue is thereby resolved. we conduct a benchmarking comparative study on the semeval datasets @xcite . the results show that models with the proposed structural adapter achieve the state-of-the-art (sota) performance compared with an array of strong baselines, indicating that incorporating structural bias is beneficial to plms. we also conduct a further study on how the relative distance-derived structural adapter overwhelms its alternatives. the results demonstrate that the structural adapter is an appealing choice. specifically, our structural adapter realizes a 1,000× scale-down in terms of incremental parameters and a 1,000× speed-up of distance derivation. in summary, structural bias can be flexibly incorporated into plms and improve both parameter and latency efficiency. the structural adapter is imposed with only a light parameter demand. the relative distance can be implemented with low latency. moreover, the structural adapter vastly improves the sota performance. therefore, structural bias is still be a necessity even in the context of plms to achieve a better aste performance. last but not least, in the view that current benchmarks are of small scales, we create a large-scale aste dataset termed lasted. lasted is collected from one of the largest review platform in china, namely dianping. 3 the dataset will be released to facilitate a more confident evaluation for aste and other possible research directions. the results on lasted hint that structural adapter confidently improves the performance. furthermore, compared with the results on the semeval datasets, the model performance generally tends to be lower on this dataset, suggesting that the large-scale deployment of aste systems is still challenging. ## conclusion in this paper, we are concerned about the parameter and latency inefficiency issues of incorporating structural bias to plms for aspect sentiment triplet extraction, and raise the question on whether structural bias is a necessity. to answer the question, we propose to use an adapter to integrate the relative position structure into plms for a light parameter demand compared with incremental layers and low latency compared with the syntactic dependency structure. we carry out benchmarking experiments on semeval benchmarks and large-scale experiments on our newly released lasted dataset as a supplementary. the results in two rounds of evaluations show that the structural adapter is an appealing choice regarding its effectiveness, parameter efficiency, and latency efficiency, implying the structural bias, in the form of the structural adapter,