K in GDOM : K nowledge- G uided DOM ain A daptation for S entiment A nalysis (2020)

Abstract: Cross-domain sentiment analysis has received significant attention in recent years, prompted by the need to combat the domain gap between different applications that make use of sentiment analysis. In this paper, we take a novel perspective on this task by exploring the role of external commonsense knowledge. We introduce a new framework, KinGDOM, which utilizes the ConceptNet knowledge graph to enrich the semantics of a document by providing both domain-specific and domain-general background concepts. These concepts are learned by training a graph convolutional autoencoder that leverages inter-domain concepts in a domain-invariant manner. Conditioning a popular domain-adversarial baseline method with these learned concepts helps improve its performance over state-of-the-art approaches, demonstrating the efficacy of our proposed framework.

## introduction sentiment analysis (sa) is a popular nlp task used in many applications @xcite . current models trained for this task, however, cannot be reliably deployed due to the distributional mismatch between the training and evaluation domains @xcite . domain adaptation, a case of transductive transfer learning, is a widely studied field of research that can be effectively used to tackle this problem @xcite . research in the field of cross-domain sa has proposed diverse approaches, which include learning domain-specific sentiment words/lexicons @xcite , co-occurrence based learning @xcite , domainadversarial learning @xcite , among concepts that enhance their semantic understanding. for example, for a target sentence from electronics domain, the software came with decent screen savers, comprising domainspecific terms like screen saver or wallpaper, conceptnet helps connecting them to general concepts like design, thus allowing a network better understand their meaning. furthermore, inter-domain conceptual bridge can also be established to connect source and target domains (wallpaper-sketch have similar conceptual notions under the link design). augmenting neural models with external knowledge bases (kb) has shown benefits across a range of nlp applications @xcite @xcite @xcite . despite their popularity, efforts to incorporate kbs into the domain-adaptation framework has been sporadic @xcite . to this end, we identify multiple advantages of using commonsense kbs for domain adaptation. first, kbs help in grounding text to real enti-ties, factual knowledge, and commonsense concepts. commonsense kbs, in particular, provide a rich source of background concepts-related by commonsense links-which can enhance the semantics of a piece of text by providing both domainspecific and domain-general concepts @xcite @xcite (see fig. 1 ). for cross-domain sa, word polarities might vary among different domains. for example, heavy can be a positive feature for a truck, but a negative feature for a smartphone. it is, however, difficult to assign contextual-polarities solely from data, especially when there is no supervision @xcite . in this domain-specific scenario, commonsense knowledge provides a dynamic way to enhance the context and help models understand sentimentbearing terms and opinion targets through its structural relations @xcite . they also often aid in unearthing implicitly expressed sentiment @xcite . second, domains often share relations through latent semantic concepts @xcite . for example, notions of wallpaper (from electronics) and sketch (from books) can be associated via related concepts such as design (see fig. 1 ). multi-relational kbs provide a natural way to leverage such inter-domain relationships. these connections can help models understand target-specific terms by associating to known domain-general or even source-specific concepts. following these intuitions, we propose a twostep modular framework, kingdom (knowledge-guided domain adaptation), which utilizes commonsense kb for domain adaptation. kingdom first trains a shared graph autoencoder using a graph convolution network (gcn) on conceptnet, so as to learn: 1) inter-domain conceptual links through multiple inference steps across neighboring concepts; and 2) domain-invariant concept representations due to shared autoencoding. it then extracts document-specific sub-graph embeddings and feeds them to a popular domain-adversarial model dann @xcite . additionally, we also train a shared autoencoder on these extracted graph embeddings to promote further domain-invariance @xcite . our main contributions in this work are: in the remaining paper, §2 explains related works and compares kingdom to them; §3 presents task definition and preliminaries; §4 introduces our proposed framework, kingdom; §5 discusses experimental setup followed by results and extensive analyses in §6; finally, §7 concludes this paper. ## related work domain adaptation methods can be broadly categorized into three approaches: a) instanceselection @xcite @xcite , b) self-labeling @xcite and c) representation learning @xcite @xcite . our focus is on the third category which has emerged as a popular approach in this deep representation learning era @xcite . ## our proposed method kingdom aims to improve the dann approach by leveraging an external knowledge source i.e., conceptnet. such a knowledge base is particularly useful for domain adaptation as it contains both domain specific and domain general knowledge. unlike traditional word embeddings and semantic knowledge graphs (e.g. wordnet), conceptnet is unique as it contains commonsense related information. we posit that both these properties of conceptnet will be highly useful for domain adaptation. kingdom follows a two-step approach described below: step 2: after the graph autoencoder is trained, a) we extract and pool document-relevant features from the trained graph for each instance in the dataset ( §4.3). b) the corresponding graph feature vector is then fed into the dann architecture for adversarial training @xcite . to further enforce domain invariance, we also introduce a shared autoencoder to reconstruct the graph features ( §4.4). ## results and analysis as mentioned in §5.3, we reimplemented the baseline dann model using adam optimizer and observed that its results has been notably underreported in many of the unsupervised domain adaptation literature for sentiment analysis (see next, we look at in ## conclusion in this paper, we explored the role of external commonsense knowledge for domain adaptation. we introduced a domain-adversarial framework called kingdom, which relies on an external commonsense kb (conceptnet) to perform unsupervised domain adaptation. we showed that we can learn domain-invariant features for the concepts in the kb by using a graph convolutional autoencoder. using the standard amazon benchmark for domain adaption in sentiment analysis, we showed that our framework exceeds the performance of previously proposed methods for the same task. our experiments demonstrate the usefulness of external knowledge for the task of cross-domain sentiment analysis. our code is publicly available at https://github.com/declare-lab/kingdom .