Opinion-based Relational Pivoting for Cross-domain Aspect Term Extraction (2022)

Abstract: Domain adaptation methods often exploit domain-transferable input features, a.k.a. pivots. The task of Aspect and Opinion Term Extraction presents a special challenge for domain transfer: while opinion terms largely transfer across domains, aspects change drastically from one domain to another (e.g. from restaurants to laptops). In this paper, we investigate and establish empirically a prior conjecture, which suggests that the linguistic relations connecting opinion terms to their aspects transfer well across domains and therefore can be leveraged for cross-domain aspect term extraction. We present several analyses supporting this conjecture, via experiments with four linguistic dependency formalisms to represent relation patterns. Subsequently, we present an aspect term extraction method that drives models to consider opinion–aspect relations via explicit multitask objectives. This method provides significant performance gains, even on top of a prior state-of-the-art linguistically-informed model, which are shown in analysis to stem from the relational pivoting signal.

## introduction sentiment analysis is one of the most widely used applications of natural language processing. a common fine grained formulation of the task, termed aspect based sentiment analysis, matches the terms in the text expressing opinions to corresponding aspects. for example, in the restaurant review in figure 1 , great, calm and quiet are opinion terms (ots) referring to the aspect term (at) ambience. following the semeval shared tasks @xcite , the preliminary task of at and ot extraction has attracted significant research attention @xcite , especially for its domain adaptation setup, the sound is very good ( and loud ) cop nsubj advmod det conj cc the ambience was great , calm and quiet . where a model trained on one domain is tested on another, unseen domain. considering each product or service as a "domain", domain adaptation is crucial for making models of this task widely applicable. yet performance on cross-domain aspect term extraction is still low, reflecting that it poses a special challenge to common domain adaptation paradigms. in most domain adaptation settings, some features of the input are domain specific, while others -also known as pivot features @xcite -do transfer into unseen domains. hence, cross-domain generalization concerns focusing the model's learning on the latter. however, aspect terms across domains share little direct commonalities. essentially, their common denominator is being the target topic referred to by opinion terms. for this reason, prior works suggested using handcrafted syntactic rules @xcite , or alternatively, injecting a full syntactic analysis into the model @xcite , aiming to capture the transferable relation-based properties of aspects. our first contribution is establishing the relational pivoting approach for cross-domain at extraction on quantitative, data driven analysis ( §3). we utilize four different linguistic formalisms (i.e., syntactic and semantic dependencies) to characterize ot-at relations, and empirically confirm their domain transferability and importance for the task. following, we propose an auxiliary multi-task learning method with specialized relation-focused tasks, designed to teach the model to focally capture these relations during ot and at extraction training ( §4). our method improves cross-domain at extraction performance when applied over both vanilla bert @xcite and the stateof-the-art sa-exal @xcite models. we conclude with a quantitative analysis of model predictions, ascribing observed performance gains to enhanced relational pivoting. foot_0 ## background following the semeval aspect based sentiment analysis shared tasks @xcite , recent works have formulated the ot and at extraction task: given an opinionated text, identify the spans denoting ots and ats. we adopt the benchmark dataset that was used by recent works @xcite , which consists of three customer-review domains -(r)estaurants, (l)aptops and digital (d)evices -and was aggregated from the semeval tasks jointly with several published resources @xcite . while promising at extraction performance has been demonstrated for in-domain settings @xcite , it does not scale to unseen domains, where state-of-the-art models exhibited small incremental improvements and struggle to surpass f1 scores of 40-55 (for the different domain pairs). previous works have conjectured that aspect and opinion terms maintain frequent syntactic relations between them. subsequently, @xcite , followed by @xcite , crafted a handful of simple syntactic patterns for in-domain at extraction based on ots. motivated by the hypothesized domain transferability of syntactic ot-at relations, @xcite employed pseudo labeling of at based on the aforementioned patterns, which was used as auxiliary supervision for domain adaptation setup. we, however, extract our patterns from the data rather than manually crafting them. in a related line of work, syntax was leveraged more broadly for the same relational pivoting mo- aspects opinions @xmath0.3 78.6 @xmath1.3 83.2 @xmath2.2 59.1 @xmath3.4 @xmath4.3 65.4 @xmath5.1 68.3 mean 20.5 69.3 ## motivating data analysis the relational pivoting hypothesis is jointly entailed from two observations: (1) opinion terms are similar across domains. ## multi-task learning method to propagate the relational pivoting signal into an ot and at extraction model, we apply auxiliary multitask learning (amtl). we experimented with prior multitask learning approaches for enriching models with syntax @xcite @xcite have pushed them to encode a full syntactic analysis, possibly including irrelevant information. in contrast, our auxiliary tasks form a "partial parsing" objective, specialized in the relevant terms and their multifarious relations. we use both vanilla bert @xcite and stateof-the-art sa-exal @xcite as base models, where the latter may imply whether our relation-focused signal is subsumed by sa-exal's awareness to the full syntactic parse ( §2). ## results and analysis following @xcite , we run each model on 3 random data splits and 3 different random seeds, presenting the mean f1 (and standard deviation) of the 9 runs. detailed results are shown in ## conclusion we establish an opinion-based cross-domain at extraction approach, by analyzing the domain invariance of linguistic ot-at path pattern. we consequently propose a relation-focused multitask learning method, and demonstrate that it enhances models results by utilizing relational features.