MT -Speech at S em E val-2022 Task 10: Incorporating Data Augmentation and Auxiliary Task with Cross-Lingual Pretrained Language Model for Structured Sentiment Analysis (2022)

Abstract: Sentiment analysis is a fundamental task, and structure sentiment analysis (SSA) is an important component of sentiment analysis. However, traditional SSA is suffering from some important issues: (1) lack of interactive knowledge of different languages; (2) small amount of annotation data or even no annotation data. To address the above problems, we incorporate data augment and auxiliary tasks within a cross-lingual pretrained language model into SSA. Specifically, we employ XLM-Roberta to enhance mutually interactive information when parallel data is available in the pretraining stage. Furthermore, we leverage two data augment strategies and auxiliary tasks to improve the performance on few-label data and zero-shot cross-lingual settings. Experiments demonstrate the effectiveness of our models. Our models rank first on the cross-lingual sub-task and rank second on the monolingual sub-task of SemEval-2022 task 10.

## introduction structured sentiment analysis (ssa) is an important task in sentiment analysis @xcite @xcite . the goal of ssa is to extract all opinion tuples from given texts. the opinion tuple (h, t, e, p) consists of a holder (h) which expresses a polarity (p) towards a target (t) by a textual sentiment expression (e). benefiting a variety of business applications, such as humanmachine dialogue and recommendation systems, ssa has attracted much more attention from both academia and industry @xcite @xcite @xcite . the mainstream method for ssa is to adopt a pipeline approach by separately performing the subtasks including holder extraction and target extraction. however, such methods can not capture dependencies of multiple sub-tasks. to address this problem, @xcite leverages graphbased dependency parsing to capture the dependencies among opinion tuples, where sentiment holders, targets and expressions are the nodes, and the relations of them as the arcs. this model has obtained state-of-the-art performance on ssa. however, the aforementioned methods still suffer from some important issues. firstly, the knowledge of the pre-trained language models (plms) has not been fully exploited. in fact, the crosslingual plms contain rich knowledge of the interactions among different languages. secondly, the above data-driven models rely on a large amount of annotation data, but there is insufficient or even no annotated data in the real scene. for example, in semeval-2022 shared task 10 ( @xcite , the multib eu @xcite dataset has only 1215 sentences and the multib ca @xcite dataset have only 1341 sentences, and there is no training data for the target language in the cross-lingual setting, which heavily hinders the performance on ssa. to address the above problems, we propose a unified and end-to-end model for ssa, which performs data augmentation and adopts auxiliary tasks with cross-lingual plms. specifically, we employ xlm-roberta @xcite as the backbone encoder to make use of its multilingual/cross-lingual knowledge. to alleviate the problem of insufficiency or lack of annotated data, we adopt two data augmentation methods: the one is to add in-domain annotated data of the same task under the training stage, and the other is to employ masked language model (mlm) @xcite for generating similar texts. furthermore, in addition to predicting each tuple in the dependency parsing graph simultaneously, we add two auxiliary tasks: 1) sequence labeling to predict the span of the holder / target / expression, and 2) sentiment polarity classification. note that both of them do not need additional we conduct experiments on subtask 1 and subtask 2 of semeval-2022 shared task on ssa. experimental results demonstrate that our method outperforms strong baselines. we rank first on the cross-lingual sub-task and rank second on the monolingual subtask in semeval-2022 task 10 1 . to summarize, our contributions are as follows. ## method we incorporate the dependency graph parsing approach @xcite into our model. the general architecture is a pre-trained language model (e.g bert @xcite , roberta @xcite , etc..) followed by a three-layers bilstms @xcite ; cross and 1 https://competitions.codalab.org/ competitions/33556 huang, 2016) and the bilinear @xcite attention as the decoding component. hence, we can take advantage of the knowledge of largescale plms @xcite @xcite @xcite and deep semantic dependency parsing @xcite @xcite . ## conclusion this paper studies the task of structured sentiment analysis. in order to deal with the problems of poor interactions of different languages and lack of annotation data, we adopt the cross-lingual pretrained language model and adopt data augmentation and auxiliary tasks. specifically, we em-ploy xlm to capture the interactive information in the pre-training stage. furthermore, we leverage two data augmentation strategies and two auxiliary tasks to improve the performance for lack of training data. experiments demonstrate the effectiveness of our models. our models rank first on the cross-lingual sub-task and rank second on the monolingual sub-task of semeval-2022 task 10.