Improving Sentence Classification by Multilingual Data Augmentation and Consensus Learning (2020)

Abstract: Neural network based models have achieved impressive results on the sentence classification task. However, most of previous work focuses on designing more sophisticated network or effective learning paradigms on monolingual data, which often suffers from insufficient discriminative knowledge for classification. In this paper, we investigate to improve sentence classification by multilingual data augmentation and consensus learning. Comparing to previous methods, our model can make use of multilingual data generated by machine translation and mine their language-share and language-specific knowledge for better representation and classification. We evaluate our model using English (i.e., source language) and Chinese (i.e., target language) data on several sentence classification tasks. Very positive classification performance can be achieved by our proposed model.

## introduction sentence classification is a task of assigning sentences to predefined categories, which has been widely explored in past decades. it requires modeling, representing and mining a degree of semantic comprehension, which are mainly based on the structure or sentiment of sentences. this task is important for many practical applications, such as product recommendation @xcite , public opinion detection @xcite , and human-machine interaction @xcite , etc. recently, deep learning has achieved state-of-the-art results across a range of computer vision (cv) @xcite , speech recognition @xcite , and natural language processing tasks (nlp) @xcite . especially, convolutional neural network (cnn) has gained great success in sentence modelling. however, training deep models requires a great diversity of data so that more discriminative patterns can be mined for better prediction. most existing work on sentence classification focuses on learning better representation for a sentence given limited training data (i.e., source language), which resorts to design a sophisticated network architecture or learning paradigm, such as attention model @xcite , multi-task learning @xcite , adversarial training @xcite , etc. inspired by recent advances in machine translation (mt) @xcite , we can perform an input data augmentation by making use of multilingual data (i.e., target language) generated by machine translation for sentence classification tasks. such generated new language data can be used as the auxiliary information, and provide the additional knowledge for learning a robust sentence representation. in order to effectively exploit multilingual data, we further propose a novel deep consensus learning framework to mine the language-share and language-specific knowledge for sentence classification. since the machine translation model can be pre-trained off-the-shelf with great generalization ability, it is worth noting that we do not directly introduce other language data comparing to existing methods in the training and testing phase. our main contributions are of two-folds: 1) we first propose utilizing multilingual data augmentation to assist sentence classification, which can provide more beneficial auxiliary knowledge for sentence 2 related work ## methodology we aim to learn a deep feature representation model for sentence classification based on languagespecific input, without any specific feature transformation. figure 1 depicts our proposed framework, which consists of two stages. the first stage performs multilingual data augmentation from an offthe-shelf machine translator; and the second one feeds the source language data and generated target language data to our deep consensus learning model for sentence classification. ## experiment and analysis in this section, we investigate the empirical performance of our proposed architecture on five benchmark datasets for sentence classification.