Generative Aspect-Based Sentiment Analysis with Contrastive Learning and Expressive Structure (2022)

Abstract: Generative models have demonstrated impressive results on Aspect-based Sentiment Analysis (ABSA) tasks, particularly for the emerging task of extracting Aspect-Category-Opinion-Sentiment (ACOS) quadruples. However, these models struggle with implicit sentiment expressions, which are commonly observed in opinionated content such as online reviews. In this work, we introduce GEN-SCL-NAT, which consists of two techniques for improved structured generation for ACOS quadruple extraction. First, we propose GEN-SCL, a supervised contrastive learning objective that aids quadruple prediction by encouraging the model to produce input representations that are discriminable across key input attributes, such as sentiment polarity and the existence of implicit opinions and aspects. Second, we introduce GEN-NAT, a new structured generation format that better adapts pre-trained autoregressive encoder-decoder models to extract quadruples in a generative fashion. Experimental results show that GEN-SCL-NAT achieves top performance across three ACOS datasets, averaging 1.48% F1 improvement, with a maximum 1.73% increase on the LAPTOP-L1 dataset. Additionally, we see significant gains on implicit aspect and opinion splits that have been shown as challenging for existing ACOS approaches.

## introduction aspect-based sentiment analysis (absa) is the task of extracting fine-grained sentiment information from text. applications of this task range from social media opinion mining and empathetic dialog assistants to product review analysis @xcite . common absa subtasks involve identifying aspect term mentions and their corresponding aspect categories, associating them with supporting opinion terms, and/or sentiment polarity that is implicitly or explicitly expressed within the text. while over 30% of sentiment expressions contain implicit language @xcite , existing methods struggle with these, performing significantly worse on examples with implicit aspects and/or opinions. these cases, such as "it took an hour to be seated", are more difficult for models as they lack common indicative explicit aspect and opinion terms, e.g., "the service" and "slow". in this work, we address the acos quadruple extraction task, particularly for formulations supporting implicit aspects and opinions. we build off @xcite , a state-of-the-art t5-based technique outputting parseable structured acos quadruple predictions for a given text. we introduce gen-scl-nat foot_0 , consisting of two novel modifications to this approach. first, we modify the model training objective, adding gen-scl, an auxiliary supervised contrastive learning objective that tasks the model to discriminably represent examples across key characteristics such as the existence of implicit aspects and opinions, and the expressed sentiment polarity. a supervised contrastive loss is used to maximize the margin between inconsistent examples and minimize it between consistent examples. second, we introduce a modified generation target format with gen-nat, where we improve the naturalness of the target format in three ways: by (1) replacing existing aspect category labels with human-readable category descriptions, (2) enforcing a reproducible "scan-based" inter-quadruple ordering for multiquad cases, and (3) modifying the intra-quadruple format to reflect a natural causal ordering of quadruple components that aligns with auto-regressive decoding. figure 1 outlines the acos task and our gen-scl-nat approach. leveraging these techniques, we show improved performance on acos benchmarks, with respective acos f1 improvements of 1.64%, 1.08% and 1.73% on the rest, laptop, and laptop-l1 datasets. we also see significant gains on examples with challenging implicit language, averaging 1.47% improvement on this subset. finally, we conduct an ablation study that shows the complementary behavior of gen-nat and gen-scl. ## acos quadruple extraction task formulation we follow @xcite and @xcite in formulating absa as a joint quadruple extraction task, where the goal is to extract an unordered set of acos quadruples q 1 , q 2 , ...q n in text t , where q @xmath0, c i , o i , s i ) contains a corresponding aspect term, aspect category, opinion term, and sentiment polarity. quadruples can lack clear supporting aspect and/or opinion spans, and these cases are marked as implicit. figure 1 displays the acos task and quadruple components. ## methodology in this section we (1) introduce gen-scl, a taskspecific supervised contrastive learning (scl) objective, and (2) propose gen-nat, an enhanced acos quadruple generation target format that addresses weaknesses in existing methods. ## results we evaluate on the task of exact quadruple extraction using the f1 metric, where a correct extraction requires all components are correct. overall performance ## related work acos quadruple prediction is an emerging absa task with structured generation techniques producing top results on the acos task @xcite @xcite . @xcite empirically demonstrate the importance of output ordering and formatting in generative unordered set prediction techniques for keyphrase generation, motivating our exploration of this direction. supervised contrastive learning @xcite ) is a popular technique for representation learning and has proven useful for nlp tasks @xcite . @xcite apply it to absa tasks during pre-training, but only for representing positive vs negative sentiment. our novel methods combine generation format improvements with a task-specific supervised contrastive learning objective that learns to represent key acos characteristics. ## conclusion in this work, we introduce gen-nat, a modified acos generation output format encompassing three methods for improving the naturalness of the decoded output sequences. we combine this with gen-scl, our novel task-specific application of supervised contrastive learning that learns improved example representations leading to downstream gains. our proposed gen-scl-nat method demonstrates state-of-the-art results, both overall and for challenging implicit-sentiment splits.