Cross-lingual Aspect-based Sentiment Analysis with Aspect Term Code-Switching (2021)

Abstract: Many efforts have been made in solving the Aspect-based sentiment analysis (ABSA) task. While most existing studies focus on English texts, handling ABSA in resource-poor languages remains a challenging problem. In this paper, we consider the unsupervised cross-lingual transfer for the ABSA task, where only labeled data in the source language is available and we aim at transferring its knowledge to the target language having no labeled data. To this end, we propose an alignment-free label projection method to obtain high-quality pseudo-labeled data of the target language with the help of the translation system, which could preserve more accurate task-specific knowledge in the target language. For better utilizing the source and translated data, as well as enhancing the cross-lingual alignment, we design an aspect code-switching mechanism to augment the training data with code-switched bilingual sentences. To further investigate the importance of language-specific knowledge in solving the ABSA problem, we distill the above model on the unlabeled target language data which improves the performance to the same level of the supervised method.

## introduction aspect-based sentiment analysis (absa) is the task of extracting mentioned aspects from a given sentence and predicting their corresponding sentiment polarities 1 @xcite ). consider the following example, "the food is great, but the service is kinda disappointing", we can detect two mentioned aspect terms "food" and "service", and judge their corresponding sentiments as positive and negative, respectively. given its wide application scenarios, it has attracted lots of attention in the nlp community in recent years @xcite @xcite @xcite @xcite . the majority of existing absa studies are conducted on english texts. however, in real-world scenarios such as the e-commerce website, users' opinions are usually expressed in different languages @xcite . manually annotating a large quantity of absa data for every language can be extremely costly. in this work, we investigate the unsupervised cross-lingual transfer for the absa task, where we only have labeled data in the source language and aim to transfer the knowledge to target languages whose labeled absa data is unavailable. existing works on cross-lingual absa mainly focus on its subtasks, including cross-lingual aspect term extraction and aspect sentiment classification. early studies usually adopt a translate-then-align strategy: a machine translation system is first used to translate the source sentence to the target language. then, word alignment algorithms @xcite are applied to project the labels (i.e., the position of the aspect term) to obtain labeled target language data @xcite . later methods make use of the cross-lingual word embeddings @xcite trained on large parallel corpus to allow the model to be used in a language-independent manner, by simply switching the word embedding layer while keeping the model unchanged @xcite @xcite when adopted for different languages. recently, employing multilingual pre-trained models such as the multilingual bert @xcite and xlm-roberta @xcite has become the de-facto approach to tackle the cross-lingual transfer for many nlp tasks @xcite . typically, the model is first fine-tuned on labeled source language data and then can be directly used for inference on the target language data (i.e., zero-shot approach), thanks to the language knowledge learned in the pre-training stage @xcite . there are some challenges for adopting such a paradigm to solve the cross-lingual absa task. the language-specific knowledge plays an essential role in tackling the absa problem, since the concerned texts are often written by ordinary users with all kinds of abbreviations or slang. the aspect terms and the opinion expressions may also be language-dependent. however, the languagespecific knowledge of the zero-shot method purely comes from the pre-training process where the lowresource languages might be under-represented @xcite . utilizing the translated target language data with projected labels is a plausible idea to compensate the language-specific knowledge @xcite . but the performance of such translation-based methods largely depends on the quality of the translation and label projection. the task-specific knowledge in the translated data would also be limited if the projected label quality is unsatisfactory. to this end, we propose an alignment-free label projection method to obtain high-quality pseudolabeled target language data. different from the previous translate-then-align paradigm, our method does not rely on any word alignment tool for projecting the labels from the source to the translated target sentence, which avoids the mis-alignment issue brought in by this step. the high-quality labeled target data thus preserves more task-specific knowledge, helping establish a strong baseline by purely training on such pseudo-labeled data. previous findings suggest that training on the bilingual corpus (i.e., labeled source data and translated target data) often leads to better performance in crosslingual transfer tasks @xcite . inspired by this finding and to further enhance the interactions between the two languages, we propose an aspect code-switching (acs) mechanism, which switches the aspect terms between the source and translated target sentences to construct two bilingual sentences. by training on the combinations of the monolingual sentences of the source/target languages and the code-switched bilingual sentences, the embedding space between the source and target languages can be better aligned with aspects as the anchor. it is natural to further extend our acs method to the multilingual setting, assuming multiple translation engines are available. in this case, the target languages can benefit from the task-specific knowledge contained in different translations. to further verify the importance of the languagespecific knowledge for the cross-lingual absa task, we exploit the usage of the unlabeled target language data via knowledge distillation @xcite . concretely, we treat the proposed acs method as the teacher model for predicting the probability distributions on the unlabeled target data. then, a student model is trained with such soft-labeled data. the distilled student model can thus utilize both the task-specific knowledge from the teacher model and the language-specific knowledge from the unlabeled target language data. in summary, our main contributions are: (1) we establish a strong translation-based baseline for the cross-lingual absa task based on an alignmentfree label projection approach, which outperforms previous translate-then-align paradigm by a large margin. (2) we propose an aspect code-switching mechanism that makes better use of the source data and the translated target data via aspect terms as anchors. (3) we show that language-specific knowledge is essential for tackling the cross-lingual absa task. by distilling the proposed model on the unlabeled target data, the performance can be further improved. (4) we conduct extensive experiments on benchmark datasets of five languages and our method achieves new state-of-the-art results under both cross-lingual and multilingual settings. foot_0 2 methodology ## related work existing works on cross-lingual absa mainly focus on its sub-tasks including the cross-lingual aspect term extraction @xcite and aspect sentiment classification @xcite . to obtain language knowledge of the target languages, translation systems are used to obtain pseudo-parallel data @xcite . a word or phrase alignment algorithm such as fastalign @xcite is then utilized to project the label from the source to the target sentence. since the performance of such methods heavily depends on the quality of the translation and alignment, different strategies are proposed to further improve the data quality. @xcite conduct an instance selection process to filter out low-quality target data. @xcite propose a span-to-span mapping heuristics to refine the aspect boundary. another line of work uses the cross-lingual word embeddings trained on large parallel bilingual corpus @xcite . by switching the word embeddings between different languages, the model can be used in a language-agnostic manner @xcite @xcite . @xcite propose a transition-based aspect extraction model which aligns the representations in different languages through an adversarial network. jebbara and cimiano (2019) consider the zero-shot transfer for aspect term extraction task with two types of cross-lingual embeddings. recently, transformer-based models pre-trained example: labeled target data given by different label projection methods source english sentence [mermaid inn] pos is an overall good restaurant with really good [seafood] pos . labeled french sentence with translate-then-align method [mermaid inn] pos est un bon restaurant dans l' ensemble avec de très bons [fruits] pos de mer . labeled french sentence with alignment-free method [mermaid inn] pos est un bon restaurant dans l' ensemble avec de très bons [fruits de mer] pos . on large multilingual corpus, such as the multilingual bert @xcite and xlm-roberta @xcite , have shown significant improvements for various cross-lingual nlp tasks. thanks to the language knowledge learned in the pre-training process, fine-tuning the model on the labeled source language data and directly conducting the inference on the target data can achieve impressive cross-lingual adaptation performance @xcite @xcite . some studies further utilize the translation system together with the pre-trained models, for example, by direct data transfer @xcite , data augmentation @xcite , and parameter warm-up @xcite . ## conclusions we tackle the cross-lingual absa task in this paper. to obtain high-quality labeled target data, we design an alignment-free label projection method which establishes a strong translation-based baseline. we further propose an aspect code-switching strategy to enhance the cross-lingual alignment and distill our method on the unlabeled target data to verify the importance of the language-specific knowledge for the absa problem. experiments with five languages show the effectiveness of our methods.