SPACL : Shared-Private Architecture based on Contrastive Learning for Multi-domain Text Classification (2022)

Abstract: “With the development of deep learning in recent years, text classification research has achieved remarkable results. However, text classification task often requires a large amount of annotated data, and data in different fields often force the model to learn different knowledge. It is often difficult for models to distinguish data labeled in different domains. Sometimes data from different domains can even damage the classification ability of the model and reduce the overall performance of the model. To address these issues, we propose a shared-private architecture based on contrastive learning for multi-domain text classification which can improve both the accuracy and robustness of classifiers. Extensive experiments are conducted on two public datasets. The results of experiments show that the our approach achieves the state-of-the-art performance in multi-domain text classification.”

## introduction text classification is one of the most basic tasks among the many tasks of natural langugae processing(nlp). in recent years, the research work of text classification has produced a large number of applications and achieved remarkable results. with the continuous release of a large number of pretrained language models in recent years, such as bert @xcite , albert @xcite , roberta @xcite and other pretrained models, text classification problems have been able to achieve good results on the basis of neural network and pretrained models. however, most text classification problems are highly domain-dependent in that the meaning of the same word may transform in different domains. for example, the word apple expresses the fruit in kitchen review (e.g., i have shifted to an apple for lunch ), while in electronics review, it means a brand of electronic products(e.g., i can't understand how apple sell so much ipod video). a common strategy, training multiple classifiers for different domains, is used to solve above problems. however, text data in reality often have characteristics of multiple domains and the cost of labeling a large number of multi-domain data is too high. therefore, it is very important and practical meaningful to improve the accuracy of text classification in multiple related domains. multi-domain text classification(mdtc) @xcite is proposed to solve above problems, it aims to utilize textual information in different domains to improve the performance of model architecture, but there is no need to train a separate classifier for each domain. in recent years, deep learning has been widely used in mdtc problems, and has achieved excellent results @xcite . the method used in most studies is shared-private architecture. private modules are used to capture domain-specific knowledge for each domain, and shared modules are used to capture domain-invariant knowledge @xcite .however, these researches only pay attention to how to obtain the shared knowledge of multiple domains and domain-specific knowledge better, but ignore the representation of the samples in the representation space. in order to solve the problems above, in this paper, we propose shared-private architecture based on contrastive learning(spacl), which uses contrastive learning to improve the representations of different types of samples in the representation space, thereby improving the performance level of downstream tasks. different from previous studies, our architecture can not only use conditional adversarial training to extract domain-invariant features, but also generate better sample representations for mdtc. the contributions of this paper are summarized as follows: 1)in order to strengthen the alignment representations of data in different domains, we propose a shared-private architecture based on contrastive learning for multi-domain text classification which can improve both the accuracy and robustness of the text classfier. 2)we adopt a conditional adversarial network to interact domain-shared features and classification labels, which can be better adapted to multi-domain text classification. 3)experiments are carried out on two public multi-domain datasets, and the experimental results compared with multiple baselines show that our proposed model architeture has achieved state-of-the-art results. ## conclusion in this paper, we proposed a shared-private architecture based on contrastive learning to use across different domains of all the available resources for multi-domain text classification. the model architecture expands the distance between shared-representations of samples of different categories in the sample space by introducing contrastive learning, thereby further improving the discriminative ability of the model architecture. in addition, the model architecture uses a conditional adversarial network to establish the correlation between domain shared features and classification prediction labels which improves the overall performance of the model architecture. the experimental results on two benchmarks show that the spacl model architecture can effectively improve the performance of the system on the multidomain text classification task. in the future, we will explore a better solution to transfer knowledge from different domains for multi-domain text classification.