Improving Cross-Lingual Sentiment Analysis via Conditional Language Adversarial Nets (2021)

Abstract: Sentiment analysis has come a long way for high-resource languages due to the availability of large annotated corpora. However, it still suffers from lack of training data for low-resource languages. To tackle this problem, we propose Conditional Language Adversarial Network (CLAN), an end-to-end neural architecture for cross-lingual sentiment analysis without cross-lingual supervision. CLAN differs from prior work in that it allows the adversarial training to be conditioned on both learned features and the sentiment prediction, to increase discriminativity for learned representation in the cross-lingual setting. Experimental results demonstrate that CLAN outperforms previous methods on the multilingual multi-domain Amazon review dataset. Our source code is released at https://github.com/hemanthkandula/clan .

## introduction recent success in sentiment analysis @xcite @xcite is largely due to the availability of large-scale annotated datasets @xcite @xcite . however, such success can not be replicated to lowresource languages because of the lack of labeled data for training machine learning models. as it is prohibitively expensive to obtain training data for all languages of interest, cross-lingual sentiment analysis (clsa) @xcite @xcite @xcite @xcite offers the possibility of learning sentiment classification models for a target language using only annotated data from a different source language where large annotated data is available. these models often rely on bilingual lexicons, pre-trained cross-lingual word embeddings, or machine translation to bridge the gap between the source and target languages. clidsa/clcdsa @xcite is the first end-to-end clsa model that does not require cross-lingual supervision which may not be available for low-resource languages. in this paper, we propose conditional language adversarial network (clan) for end-toend clsa. similar to prior work, clan performs clsa without using any cross-lingual supervision. differing from prior work, clan incorporates conditional language adversarial training to learn language invariant features by conditioning on both learned feature representations (or features for short) and sentiment predictions, therefore increases the features' discriminativity in the crosslingual setting. our contributions are three fold: adversarial training for domain adaptation our approach draws inspiration from domain-adversarial training of neural networks @xcite and conditional adversarial domain adaptation (cdan) @xcite . dann @xcite trains a feature generator to minimize the classification loss, and a domain discriminator to distinguish the domain where the input instances come from. it attempts to learn domain invariant features that deceive the domain discriminator while learning to predict the correct sentiment labels.cdan @xcite additionally makes the discriminator conditioned on both extracted features and class predictions to improve discriminativity. ## conditional language adversarial networks for sentiment analysis figure 1 shows the architecture of clan. it has three components: a multilingual language model (lm) that extracts features from the input sentences, a sentiment classifier built atop of the fea-tures extracted by the lm, and a conditional language adversarial trainer to force the features to be language invariant. all three components are jointly optimized in a single end-to-end neural architecture, allowing clan to learn cross-lingual features and to capture multiplicative interactions between the features and sentiment predictions. the resulting cross-lingual features are specialized for each sentiment class. clan aims at solving the cross-lingual multidomain sentiment analysis task. formally, given a set of domains d and a set of languages l, clan consists of the following components: sentiment classifier we use a linear classifier that takes the average final hidden states 1 |x| |x| @xmath0, and then a softmax to output sentiment labels. the objective is: conditional language adversarial training to force the features to be language invariant, we adopted conditional adversarial training @xcite : a language discriminator is trained to predict language id given the features by minimizing the cross-entropy loss, while the lm is trained to fool the discriminator by maximizing the loss: a key innovation is the conditional language adversarial training: the multilinear conditioning enables manipulation of the multiplicative interactions across features and class predictions. such interactions capture the cross-covariance between the language invariant features and classifier predictions to improve discriminability. the full model putting all components together, the final objective function is the following: ## conclusion we present conditional language adversarial networks for cross-lingual sentiment analysis, and show that it achieves state-of-the-art performance.