Semi-Supervised Domain Adaptation for Emotion-Related Tasks
Year: 2023

Abstract:
Semi-supervised domain adaptation (SSDA) adopts a model trained from a label-rich source domain to a new but related domain with a few labels of target data. It is shown that, in an SSDA setting, a simple combination of domain adaptation (DA) with semi-supervised learning (SSL) techniques often fails to effectively utilize the target supervision and cannot address distribution shifts across different domains due to the training data bias toward the source-labeled samples. In this paper, inspired by the co-learning of multiple classifiers for the computer vision tasks, we propose to decompose the SSDA framework for emotion-related tasks into two subcomponents of unsupervised domain adaptation (UDA) from the source to the target domain and semi-supervised learning (SSL) in the target domain where the two models iteratively teach each other by interchanging their high confident predictions. We further propose a novel data cartography-based regularization technique for pseudo-label denoising that employs training dynamics to further hone our models’ performance. We release our code.1

heading: 1 Introduction
text: Large pre-trained language models (Devlin et al., 2019; Liu et al., 2019) have significantly improved many natural language processing (NLP) task performances with the help of large quantities of labeled training data and have become the de facto model for NLP applications. However, obtaining vast troves of annotated data for training in many real-world scenarios is costly and challenging. For example, reliable large annotated emotion or empathy data might not exist in a computer-assisted therapy session (Hosseini and Caragea, 2021a,b, 2023). On top of that, it is shown that a shift in data distribution can substantially affect the performance of such text classification models (Ngo et al., 2022; Blitzer et al., 2006); a model trained
1https://github.com/Mahhos/CotrainingTrainingDynamics
on a source domain does not perform well on a dataset from another domain. This deficiency is due to the domain shift across the datasets (Tzeng et al., 2017), which is a problem that is commonly encountered in NLP. To relieve the unsupervised domain adaptation (UDA) bottleneck for textual tasks, recent works attempt to align distributions between source and target domains by extracting the domain-invariant representations (Ganin et al., 2016). However, despite the recent progress, the UDA methods are still impractical as they may yield different new domain-sensitive particularities for large-scale language models.
Recent works showed that the presence of few labeled samples from the target domain in a semisupervised domain adaption (SSDA) setup can positively impact and significantly boost the performance of the neural models (Qin et al., 2020; Kim and Kim, 2020; Saito et al., 2019). There exists prior work on supervised domain adaptation (Daumé III, 2007) and multi-task learning (Sun et al., 2011) for natural language processing (NLP) tasks. However, despite the importance of semisupervised domain adaption, in NLP, only a few studies have focused on this problem (Daumé III et al., 2010; Cheng and Pan, 2014). Daumé III et al. (2010) expanded an existing fully supervised domain adaptation technique (Daumé III, 2007) to semi-supervised domain adaptation settings using co-regularization, which originated in the context of multi-view learning (Rosenberg and Bartlett, 2007; Sindhwani and Rosenberg, 2008). Cheng and Pan (2014) also framed the semi-supervised domain adaptation problem as learning with a transformation function and a prediction model under manifold constraints.
Given a large number of labeled data provided in the source domain and only a few target labeled data with the inherent distributional difference, it is shown that in an SSDA setting, a single classifier may likely be dominated by the source domain
5402
(Yang et al., 2021). In this paper, we extend the co-training strategy (Blum and Mitchell, 1998), a semi-supervised learning approach for multi-view data, to a single-view setting for NLP tasks to effectively use unlabeled target data. Co-training trains two classifiers from each view and employs the most confident predictions of the unlabeled data for the two classifiers to teach each other. Inspired by co-training, we propose to decompose the SSDA framework and learn two distinct classifiers to teach each other so that both classifiers can excel in the target domain. Particularly, we employ an unsupervised domain adaptation setup where we leverage the labeled source data and the unlabeled target data to learn one classifier. Furthermore, we employ a semi-supervised learning setup where we learn another classifier using the labeled target data together with the unlabeled target data. We further propose a novel data cartography-based regularization technique for pseudo-label denoising that employs training dynamics (Swayamdipta et al., 2020) to further hone our models’ performance.
Our preliminary results on three emotion-related NLP tasks show that transferring knowledge between the two classifiers and incorporating training dynamics to help denoising the generated pseudolabels can effectively improve the performance on the target domain.

heading: 2 Approach

heading: 2.1 Co-training with Task Decomposition
text: Co-training (Blum and Mitchell, 1998) is a wellknown semi-supervised learning (SSL) technique that employs two different views of an example (e.g., audio and video) and learns two predictive models that are trained separately on each view. Assuming that each view is sufficient for correct classification, co-training confers the models the capability to teach each other by adding the highly confident predictions of one model (on new unlabeled examples) to the training set of the other. In this way, co-training helps boost a learning algorithm’s performance when only a small set of labeled examples is available. Recently, Chen et al. (2011) and Qiao et al. (2018) proposed techniques to perform co-training using single-view data, but they still need additional tasks or objective functions to apply co-training. Along these lines, Yang et al. (2021) proposed to use only single-view data (i.e., only images) to perform co-training in a semisupervised domain adaptation setup for computer
vision tasks, where they leverage the supervision of the labeled data from the source and target domains and combine them with the unlabeled samples from the target domain.
Here we propose our co-training with task decomposition and cartography-based mixup approach, which greatly benefits the text-based emotion-related classification tasks in the fewshot setting. Task decomposition in a co-training paradigm has been initially introduced for computer vision tasks by Yang et al. (2021) to enhance the performance of image classification models. With that, our work is greatly inspired by Yang et al. (2021).
Task Setup. Given the labeled data from the source DS = {(si, yi)}NSi=1 and the target DT = {(ti, yi)}NTi=1 domains such that |DS | ≫ |DT | and |DT | = K × |y|2 in the few-shot setting, and the unlabeled data from the target domain DU = {(ui)}NUi=1, we first construct two sub-tasks in the semi-supervised domain adaptation setup: one using our DS and DU to train an unsupervised domain adaptation (UDA) model θuda, and one using our DT and DU to train a semi-supervised learning (SSL) model θssl. We conduct our tasks using mini-batch SGD to update the models’ weights in our experiments. In each iteration, we make predictions on U = {ub}Bb=1 (which is sampled from our unlabeled set DU with mini-batch size B) using our two models θuda and θssl, and generate pseudolabel sets Ussl and Uuda that will be filtered based on a threshold τ to update θssl and θuda:
Ussl = {(ub, y′b = argmax c p(c|ub; θuda));
if max c p(c|ub; θuda)) > τ} Uuda = {(ub, y′b = argmax
c p(c|ub; θssl));
if max c
p(c|ub; θssl)) > τ}
where p(c|ub; .) is the predicted probability of the unlabeled sample ub ∈ U for a class c. In essence, if the prediction confidence of one model (e.g., θssl) on ub is greater than our pseudo-label selection threshold τ ,3 it would be added to the Uuda to train the other model θuda. In other words, a model imparts the other model with confident pseudolabels to learn from and uses the other model’s confident pseudo-labels to learn from.
2|y| represents the number of classes in y. 3We experiment with a range (i.e., [0.5, 0.6, 0.7]) for the pseudo-label selection threshold and select τ = 0.7 as our final confidence threshold.

heading: 2.2 Data Cartography-based Regularization for Pseudo-label Denoising
text: It is inevitable to obtain noisy pseudo-labels from each model. First due to the domain shift (in our UDA component), and second given that only a few labeled samples (i.e., K = [4, 8, 16] per class) are available from the target domain (in our SSL component), which in turn impacts the supervision process. To mitigate noise in the generated pseudolabels and hone our models’ performance, we propose a cartography-based mixup strategy that helps to effectively denoise an incorrect pseudo-label.
Mixup Training. Mixup augments the training data by linearly interpolating training samples and their corresponding labels, based on a simple rule proposed by Zhang et al. (2018):
(x̃ij , ỹij) := (λxi + (1− λ)xj , λyi + (1− λ)yj) (1)
where λ is a mixing ratio sampled from a Beta(α, α) distribution with a hyper-parameter α and (xi, yi) and (xj , yj) are two input examples that are randomly drawn from the training set.
Proposed Approach. Few-shot learning methods generally assume that the training sets always include accurately labeled samples. However, this assumption can sometimes be unrealistic. No matter how small, training sets can still contain mislabeled samples (Liang et al., 2022). In other words, it could not be guaranteed that the few-shot training sets were carefully selected to represent their class. In fact, even carefully annotated and selected datasets often hold mislabeled samples (Northcutt et al., 2021; Yang et al., 2020) due to several reasons like ambiguity, automated weakly supervised annotation, or human error. Here, we propose to use a novel Mixup data augmentation technique on the target training data and the pseudo-labels generated by the source domain that is informed by training dynamics to further surpass the noisy data bottleneck and improve the target domain performance in few-shot setting. Our proposed mixup creates vicinal distribution steered by the data maps (Swayamdipta et al., 2020) as described below.
We first characterize each training sample of our few-shot target domain training set DT into three groups of easy-to-learn, ambiguous, and hard-tolearn, based on how they contribute to the model learning (i.e., training dynamics). We then sample examples with specific characteristics (emanated from the previous step) to interpolate with the generated pseudo-labels by the source domain Ussl
during our cartography-based mixup process. In our experiments, we measure the statistics using a RoBERTa-base model. Sample (xi, yi) training dynamics are measured as statistics called confidence and variability computed across the E epochs (Swayamdipta et al., 2020). Confidence is computed as the mean model probability of the true label yi across epochs, µ̂i = 1E E∑
e=1
pθe(yi|xi);
where θ indicates the model parameters and pθe denotes the model’s probability at the end of the eth epoch. Variability is measured as the standard deviation of the ground-truth probabilities pθe(yi|xi) across different epochs, σ̂i = √∑E e=1(pθe (yi|xi)−µ̂i)
E .
Intuitively, samples to which the model confidently (i.e., high confidence) and constantly (i.e., low variability) assigns the true, and the same label corresponds to easy-to-learn examples (for the model). On the other hand, samples with low confidence and low variability resemble hard-to-learn examples (for the model), which usually are referred to as mislabeled samples, and examples with high variability that the model is uncertain about during training are ambiguous (to the model). Using these statistics, we select the easy-to-learn samples (i.e., samples that the model consistently predicts correctly across epochs) to interpolate with the pseudo-labels generated by the source domain. By employing such particularities, our goal is to ensure that we effectively denoise an incorrect pseudo-label by mixing it with the most informative data samples (Swayamdipta et al., 2020), samples with high confidence and low variability which are detected to be actually correct. Our mixup approach combines samples at the level of the hidden state representations generated by the task-specific layer on top of the pre-trained language model.

heading: 3 Experiments

heading: 3.1 Datasets
text: We perform evaluations on three text classification tasks of emotion detection, sentiment analysis, and empathy detection. We analyze tasks with challenging domain shifts where out-of-domain performance is considerably lower. Furthermore, it is shown that detecting empathy or emotions from text without visual or acoustic information is challenging due to the subjective nature of the annotations (Hosseini and Caragea, 2021b), which makes it difficult to accurately label and interpret the emotions or empathy expressed. Additionally,
most datasets, particularly in empathy detection, are limited in size, with only a few exceptions. However, given the significance of these tasks and the profound impact emotions have on our behavior and daily lives, our objective is to enhance the performance of such tasks and achieve improved detection of emotion-related information from text. We explain our source and target domains datasets below. Empathy Detection. NewsEmp is a dataset of empathic reactions to news stories, including empathy binary labels released by Buechel et al. (2018) which we use as our source domain dataset. TwittEmp Dataset (Hosseini and Caragea, 2021a) contains perceived empathy annotated by empathy direction (seeking vs. providing) in the health domain, which we use as our target domain dataset.
Emotion Detection. GoEmotions is an emotion detection dataset from Reddit comments where we use the six basic emotions (joy, anger, fear, sadness, disgust, and surprise) and neutral as our source domain dataset. Meld (Poria et al., 2019) contains dialogues from the popular Friends TV series annotated with the same set of emotion labels, which we use as the target domain dataset.
Sentiment Analysis. Yelp (Zhang et al., 2015) is a dataset for binary sentiment classification, consisting of reviews from Yelp, which is used as our source domain. Our target domain dataset is IMDB movie reviews (Maas et al., 2011), containing sentences of movie reviews and their sentiment.

heading: 3.2 Baseline Methods
text: The details of the experiments are as follows. We use the BERT-base model and K = [4, 8, 16] in
all the experiments.4 We contrast our proposed approach on emotion, empathy, and sentiment classification tasks with the following baselines: (1) Source-Only, which uses the source domain for fine-tuning BERT (the training portion) and the target domain for the evaluation (the test portion), which is the same for all our three settings; (2) Target-Only, uses the target domain for both training and evaluation of BERT with few-shot data; (3) UDA unsupervised domain adaptation where source domain training set is used to train a model and make predictions on unlabeled data from the target domain. Then, the generated pseudo-labels are added to the source domain training set iteratively based on the selection threshold; (4) SSL semi-supervised learning, where the target domain training set is used to train a model and make predictions on unlabeled data from the target domain. Then, the generated pseudo-labels are added to the target domain training set iteratively based on the selection threshold; (5) MixText (Chen et al., 2020) which guesses low-entropy labels for unlabeled target data and uses Mixup to interpolate labeled and unlabeled samples; (6) FliText leverages convolution networks to achieve faster and lighter semi-supervised text classification; (7) Unsupervised data augmentation enhances training by augmenting unlabeled data and promoting consistency between augmented versions; (8) Co-training with task decomposition where the SSDA is decomposed to two components of UDA and SSL; (9) Cotraining with task decomposition and mixup (Yang et al., 2021) where the generated pseudo-labels with the source domain classifier are interpolated
4BERT-base yields the best results in our experiments, so we only report the results using this model.
with the target training data; and (10) Standard supervised learning with the full training and test sets of the domains separately (as an upper bound of performance).

heading: 3.3 Results
text: Table 1 compares our proposed approach and baseline methods on our classification tasks for different few-shot settings. We report the average performance on 3 distinct randomly sampled training and development splits with three random seeds to provide a robust measure of our few-shot performance. We make a few remarks below.
As we can see from Table 1, our proposed method achieves higher accuracy on all the fewshot settings than any baseline using task decomposition and cartography-based mixup. The results suggest that incorporating cartography-based mixup to effectively denoise the generated pseudolabels (see Ours in the tables) results in constant improvement over all the few-shot settings. For example, on empathy (i.e., News to Health) with K = 4, our proposed approach increased the performance by a factor of 1.63% compared to the Target-Only and by 1.55% compared to the SSL. Interestingly, we also observe that our proposed approach outperforms standard mixup (i.e., Cotraining with TD + Mixup), which signifies the importance and effectiveness of our proposed strategy in using training dynamics to characterize data and identify the correctly-labeled samples (easy-tolearn samples) for denoising the generated pseudolabels through the mixup process. In the standard mixup, the interpolation process occurs between the generated pseudo-labels and all examples in the target labeled set, where it is possible for some of these examples to have incorrect labels. Table 2 shows examples with erroneous labels from all of our few-shot target labeled sets. Erroneous labels can occur due to human errors in annotations,
even with small datasets when using crowdsourcing techniques or relying on human annotators. It is apparent from the table that the standard supervised learning using the full training and test sets from the respective domains results in an increase in performance.

heading: 4 Conclusion
text: In this work, we extend the co-training strategy as a semi-supervised learning approach for multiview data to a single-view setting for NLP tasks and propose to decompose the SSDA framework and learn two distinct classifiers (one in an semisupervised setup and another one in a domain adaptation setup) to teach each other so that both classifiers can excel in the target domain. We further propose a novel data cartography-based regularization technique for pseudo-label denoising that employs training dynamics to further hone our models’ performance. Our preliminary results show that denoising the pseudo-labels of unlabeled target data using high-quality labeled target data within a cotraining framework yields improvements in performance over multiple baselines.

heading: 5 Limitations
text: One potential limitation of our method is that it induces an extra cost of estimating training dynamic statistics of the data samples to characterize them (e.g., easy-to-learn or ambiguous) based on how they incorporate into the model’s learning. This may be more expensive for tasks and datasets with a large number of classes. In the future, we will focus on approaches to characterize the training examples on the fly.

heading: Acknowledgments
text: We acknowledge NSF for support from grants IIS1912887, IIS-2107487, and ITE-2137846. We also thank our reviewers for their insightful feedback and comments.

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
5
7 A2. Did you discuss any potential risks of your work? Our work does not have any potential risks

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Section 1 for introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? 3

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 3
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 2

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 3.3

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 3.2
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In the future, we will focus on approaches to characterize the training examples on the fly