End-to-end Aspect-based Sentiment Analysis with C ombinatory C ategorial G rammar (2023)

Abstract: End-to-end Aspect-based Sentiment Analysis (EASA) is a natural language processing (NLP) task that involves extracting aspect terms and identifying the sentiments for them, which provides a fine-grained level of text analysis and thus requires a deep understanding of the running text. Many previous studies leverage advanced text encoders to extract context information and use syntactic information, e.g., the dependency structure of the input sentence, to improve the model performance. However, such models may reach a bottleneck since the dependency structure is not designed to provide semantic information of the text, which is also important for identifying the sentiment and thus leave room for further improvement. Considering that combinatory categorial grammar (CCG) is a formalism that expresses both syntactic and semantic information of a sentence, it has the potential to be beneficial to EASA. In this paper, we propose a novel approach to improve EASA with CCG supertags, which carry the syntactic and semantic information of the associated words and serve as the most important part of the CCG derivation. Specifically, our approach proposes a CCG supertag decoding process to learn the syntactic and semantic information carried by CCG supertags and use the information to guide the attention over the input words so as to identify important contextual information for EASA. Furthermore, a gate mechanism is used in incorporating the weighted contextual information into the backbone EASA decoding process. We evaluate our approach on three publicly available English datasets for EASA, and show that it outperforms strong baselines and achieves state-of-the-art results on all datasets.

## introduction end-to-end aspect-based sentiment analysis (easa) is an important task that provides the † corresponding author. 1 the code involved in this paper is released at https: //github.com/synlp/asa-ccg . figure 1 : an example sentence for easa where the sentiments of two aspect terms, namely, "environments" and "bar service", are positive and negative, respectively. the supertag and easa label of each word are also presented for better illustration. the easa labels aggregate the bio labels for aspect term extraction with the positive/negative labels for sentiment polarity. most of the previous studies @xcite @xcite @xcite @xcite @xcite on easa are generally categorized into three categories, namely, pipeline, multi-task, and joint-label approaches, based on how they formalize the task. among the three types of approaches, the joint-label approaches aggregate labels for the sub-tasks rather than directly conducting them, and they achieve the best performance. figure 1 shows the aggregated easa labels where the first part indicates the position of a word in an aspect term following the bio schema and the second part indicates the sentiment of the aspect term. 2 in performing easa, previous studies try advanced encoders, such as lstm, transformer @xcite , and bert @xcite , to capture contextual information and achieve outstanding performance. since most aspect terms are noun phrases and the syntactic structure of the text is able to provide additional clues about the sentiment expressed towards an aspect, previous studies leverage syntactic information, e.g., the dependency of the sentence, to further enhance sentiment analysis @xcite @xcite . however, approaches enhanced by conventional syntactic information, especially the dependencies, may reach a bottleneck since it is unable to provide semantic information about the sentence, which is also important for easa. combinatory categorial grammar (ccg) offers an alternative to phrase structure grammar for describing the syntax and build transparent connections for syntax and semantics @xcite @xcite . in using combinatory rules over simple syntactic categories, a key feature of ccg is its use of type-logical semantics, which provides a systematic way of associating meanings with the syntactic structures generated by the grammar. this allows for a more precise and intuitive representation of the meanings of sentences as well as each word in it. particularly, the lexical category (which is also known as ccg supertags) associated with each word conveys both syntactic and semantic information of the word. therefore, learning the ccg supertagging process allows the model to learn the syntactic and semantic function of each word in the running text and thus shows its effectiveness in many tasks @xcite @xcite . in this study, we hypothesize that ccg could be useful in enhancing the performance of the model for easa as well. we propose a joint-label approach following the encoding-decoding paradigm to enhance easa with ccg supertags. in doing so, we enhance easa with a ccg supertag decoding process to learn from the ccg supertags automatically annotated by an off-the-shelf ccg supertagger. an attention mechanism is performed over all input words to identify the ones that contribute to the easa task, where the attention weights are guided by the supertag decoding process. this allows our model to learn the syntactic and semantic information carried by ccg supertags through the ccg decoding process rather than using them as additional input features and thus makes our model run faster in inference. furthermore, considering that there could be noise in the auto-generated ccg supertags, we introduce a gate mechanism to balance the contributions between the context information obtained from the text encoder and the attention module. experimental results on three english benchmark datasets for easa present the effectiveness of our approach, where our approach outperforms strong baselines and achieves state-ofthe-art performance. ## related work the easa task has drawn much attention in recent years and existing approaches can be categorized into three groups, namely, pipeline approaches, multi-task approaches, and joint approaches. specifically, the pipeline approaches @xcite @xcite contains two steps, where the first performs the aspect term extraction and the second aims to predict the sentiment polarities of the extracted aspect terms. the multi-task approaches @xcite @xcite normally use a text encoder to model the input and employ multi-task learning with two separately decoding processes to extract the aspect terms and predict the sentiments. joint-label approaches @xcite @xcite perform the aspect term extraction and the sentiment polarity prediction tasks simultaneously through a unified labeling scheme. most recent approaches to easa apply advanced encoders (e.g., lstm, transformer, and bert) and achieve promising performance. to further enhance easa, chen et al. (2020) proposed a joint-label model to leverage the dependencies between words through graph-based models; @xcite used a hierarchical architecture to perform multi-task learning and outperforms existing multi-task based approaches. overall, the pipeline approaches often suffer from error propagation issues in the cascade steps and the multi-task approaches have label mismatching problems, where the decoding results usually do not match each other. in comparison, joint-label approaches are superior in avoiding error propagation and label mismatch problems. compared with previous studies, this paper proposes a joint-label approach for easa with the enhancement of using ccg information (rather than the dependencies which are widely used in previous studies) through ccg supertag decoding, attention mechanism, and gate modules. ## the approach our approach for easa follows the standard sequence labeling paradigm, whose architecture is illustrated in figure 2 . specifically, our approach consists of four parts: (1) the backbone model to predict easa labels, (2) the supertag decoding process to learn syntactic and semantic information carried by supertags, (3) the attention module to weigh different contextual information, and (4) the gate mechanism to balance the contribution of the backbone model and the attention module to the easa task. the process of our approach (which is denoted as f ) for easa can be formalized as ## conclusion in this paper, we propose to enhance joint-label easa by leveraging ccg information, which contains both syntactic and semantic information of the running text and thus shows its superior to the conventional phrase and dependency style grammars. specifically, we learn the ccg supertag information through a ccg supertag decoding process, where such information is used to guide the attention weights over the input words in the attention module so that important context information is distinguished and leverage accordingly. to further enhance the model performance, we employ the gate module to balance the contribution of the context information obtained from the backbone text encoder and the attention module. experimental results and further analysis on three english benchmark datasets demonstrate the effectiveness of the proposed model, where our model outperforms strong baselines and achieves state-of-the-art performance on all datasets. for future study, we plan to explore effective approaches to leverage the ccg derivation of the input text to improve easa.