SlowGuess/NAACL_2022_OCR huggingface dataset:
Simiao Zuo\*, Yue Yu\*, Chen Liang, Haoming Jiang\*, Siawpeng Er\*, Chao Zhang, Tuo Zhao and Hongyuan Zha
$^{\text{©}}$ Georgia Institute of Technology  $\square$  Amazon
†The Chinese University of Hong Kong, Shenzhen
{simiaozuo,yueyu,cliang73,ser8,chaozhang,tourzhao}@gatech.edu
jhaoming@amazon.com zhahy@cuhk.edu.cn
Self-training achieves enormous success in various semi-supervised and weakly-supervised learning tasks. The method can be interpreted as a teacher-student framework, where the teacher generates pseudo-labels, and the student makes predictions. The two models are updated alternatingly. However, such a straightforward alternating update rule leads to training instability. This is because a small change in the teacher may result in a significant change in the student. To address this issue, we propose DRIFT, short for differentiable self-training, that treats teacher-student as a Stackelberg game. In this game, a leader is always in a more advantageous position than a follower. In self-training, the student contributes to the prediction performance, and the teacher controls the training process by generating pseudo-labels. Therefore, we treat the student as the leader and the teacher as the follower. The leader procures its advantage by acknowledging the follower's strategy, which involves differentiable pseudolabels and differentiable sample weights. Consequently, the leader-follower interaction can be effectively captured via Stackelberg gradient, obtained by differentiating the follower's strategy. Experimental results on semi- and weakly-supervised classification and named entity recognition tasks show that our model outperforms existing approaches by large margins.
Self-training is a classic method that was first proposed for semi-supervised learning (Rosenberg et al., 2005; Lee, 2013). It is also interpreted as a regularization method (Mobahi et al., 2020), and is extended to weakly-supervised learning and domain adaptation (Meng et al., 2018). The approach has gain popularity in many applications. For example, in conjunction with pre-trained language models (Devlin et al., 2019), self-training has
demonstrated superior performance on tasks such as natural language understanding (Du et al., 2021), named entity recognition (Liang et al., 2020), and question answering (Sachan and Xing, 2018).
Conventional self-training can be interpreted as a teacher-student framework. Within this framework, a teacher model generates pseudo-labels for the unlabeled data. Then, a student model updates its parameters by minimizing the discrepancy between its predictions and the pseudo-labels. The teacher subsequently refines its parameters based on the updated version of the student using predefined rules. Such rules include minimizing a loss function (Pham et al., 2020), copying the student's parameters (Rasmus et al., 2015), and integrating models from previous iterations (Laine and Aila, 2017; Tarvainen and Valpola, 2017). The above procedures are operated iteratively.
Computationally, the alternating update procedure often causes training instability. Such instability comes from undesired interactions between the teacher and the student. In practice, we often use stochastic gradient descent to optimize the student, and the noise of the stochastic gradient can cause oscillation during training. This means in a certain iteration, the student is optimized towards a certain direction; while in the next iteration, it may be optimized toward a drastically different direction. Such a scenario renders the optimization ill-conditioned. Moreover, the student model's gradient is determined by the pseudo-labels generated by the teacher. Because of the training instability, a small change in the pseudo-labels may result in a substantial change in the student.
To resolve this issue, we propose DRIFT (differentiable self-training), where we formulate self-training as a Stackelberg game (Von Stackelberg, 2010). The concept arises from economics, where there are two players, called the leader and the follower. In a Stackelberg game, the leader is always in an advantageous position by acknowledging
*Equal contribution. Corresponding authors.
†Work was done at Georgia Institute of Technology.
Findings of the Association for Computational Linguistics: NAACL 2022, pages 933 - 949 July 10-15, 2022 ©2022 Association for Computational Linguistics
ing the follower's strategy. Within the self-training framework, we grant the student a higher priority than the teacher. This is because the teacher serves the purpose of generating intermediate pseudolabels, such that the student can behave well on the task. The student (i.e., the leader) procures its advantage by considering what the response of the teacher (i.e., the follower) will be, i.e., how will the follower react after observing the leader's move. Then, the leader makes its move, in anticipation of the predicted response of the follower. We remark that the Stackelberg game formulation has also been used in other domains such as adversarial training (Zuo et al., 2021).
We highlight that in DRIFT, the student has a higher priority than the teacher. In contrast, in conventional self-training, the two models are treated equally and have the same priority. When using conventional self-training, the student only reacts to what the teacher has generated. In differentiable self-training, the student recognizes the teacher's strategy and reacts to what the teacher is anticipated to response. In this way, we can find a better descent direction for the student, such that training can be stabilized.
To facilitate the leader's advantage, our framework treats the follower's strategy (i.e., pseudolabels generated by the teacher) as a function of the leader's decision (i.e., the student's parameters). In this way, differentiable self-training can be viewed solely as a function of the student's parameters. Therefore, the problem can be efficiently solved using gradient descent.
Besides pseudo-labels, the teacher can also generate sample weights (Freund and Schapire, 1997; Kumar et al., 2010; Malisiewicz et al., 2011). Sample reweighting associates low-confidence samples with small weights, such that the influence of noisy labels can be effectively reduced. Similar to pseudo-labels, sample weights and the student model are also updated iteratively. As such, we can further equip DRIFT with differentiable sample weights. This can be achieved by integrating the weights as a part of the follower's strategy. We remark that our method is flexible and can incorporate even more designs to the follower's strategy.
We evaluate the performance of differentiable self-training on a set of weakly- and semi-supervised text classification and named entity recognition tasks. In some weakly-supervised learning tasks, our proposed method achieves com
petitive performance in comparison with fully-supervised models. For example, we obtain a  $97.3\%$  vs.  $96.2\%$  classification accuracy on Yelp, and we do not use any labeled training data from the Yelp dataset.
We highlight that our proposed differentiable self-training approach is an efficient substitution to existing self-training methods. Moreover, our method does not introduce any additional tuning parameter to the teacher-student framework. Additionally, DRIFT is flexible and can combine with various neural architectures. We summarize our contributions as the following: (1) We propose a differentiable self-training framework DRIFT, which employs a Stackelberg game formulation of the teacher-student approach. (2) We employ differentiable pseudo-labels and differentiable sample weights as the follower's strategy. Our method alleviates the training instability issue. (3) Extensive experiments on semi-supervised node classification, semi- and weakly-supervised text classification and named entity recognition tasks verify the efficacy of DRIFT.
$\diamond$  Self-training for semi-supervised learning. Self-training is one of the earliest and simplest approaches to semi-supervised learning (Rosenberg et al., 2005; Lee, 2013). The method uses a teacher model to generate new labels, on which a student model is fitted. Similar methods such as self-knowledge distillation (Furlanello et al., 2018) are proposed for supervised learning. The major drawback of self-training is that it is vulnerable to label noise. A popular approach to tackle this is sample reweighting (Freund and Schapire, 1997; Kumar et al., 2010; Malisiewicz et al., 2011), where high-confidence samples (Rosenberg et al., 2005; Zhou et al., 2012) are assigned larger weights. Data augmentation methods (Berthelot et al., 2019; Chen et al., 2020) are also proposed to further enhance self-training.
$\diamond$  Self-training for weakly-supervised learning. Weak supervision sources, such as semantic rules and knowledge bases, facilitate generating large amounts of labeled data (Goh et al., 2018; Hoffmann et al., 2011). The weak supervision sources have limited coverage, i.e., not all samples can be matched by the rules, such that a considerable amount of samples are unlabeled. Moreover, the generated weak labels usually contain excess-
sive noise. Recently, self-training techniques are adopted to weakly-supervised learning. In conjunction with pre-trained language models (Devlin et al., 2019; Liu et al., 2019), the technique achieves superior performance in various tasks (Meng et al., 2018, 2020; Niu et al., 2020; Liang et al., 2020; Yu et al., 2021).
For both semi-supervised and weakly-supervised learning problems, we have labeled samples  $\mathcal{X}_l = \{(x_i,y_i)\}_{i = 1}^{N_l}$  and unlabeled samples  $\mathcal{X}_u = \{x_j\}_{j = 1}^{N_u}$ . Here  $N_{l}$  is the number of labeled data, and  $N_{u}$  is the number of unlabeled data. Note that in weakly-supervised learning, we have unlabeled data because of the limited coverage of weak supervision sources. The difference between semi- and weakly-supervised learning is that in the former case, the labels  $\{y_i\}_{i = 1}^{N_l}$  are assumed to be accurate, whereas in the latter case, the labels are noisy. The goal is to learn a classifier  $f:\mathcal{X}\to \mathbb{R}^C$  where  $\mathcal{X} = \mathcal{X}_l\cup \mathcal{X}_u$  denotes all the data samples,  $\mathcal{Y} = \{1,\dots ,C\}$  is the label set, and  $C$  is the number of classes. The classifier  $f$  outputs a point in the  $C$ -dimensional probability simplex, where each dimension denotes the probability that the input belongs to a specific class.
3.1 Differentiable Self-Training for Semi-Supervised Learning
Self-training can be interpreted as a teacher-student framework. Within this framework, the teacher first generates pseudo-labels  $\widetilde{\mathbf{y}}$  (see (6)) for the data samples. Then, the student updates itself by minimizing a loss function (see (8)), subject to the generated pseudo-labels. Such two procedures are run iteratively.
We remark that self-training behaves poorly when encountering unreliable pseudo-labels, which will cause the student model to be updated towards the wrong direction. To alleviate this issue, we find a good initialization  $\theta^{\mathrm{init}}$  for the models. In semi-supervised learning,  $\theta^{\mathrm{init}}$  is found by fitting a model on the labeled data  $\mathcal{X}_l$ . Concretely, we solve
$$
\min  _ {\theta} \mathcal {L} _ {\sup } (\theta) = \frac {1}{N _ {l}} \sum_ {\mathcal {X} _ {l}} \ell_ {\sup } \left(f \left(x _ {i}, \theta\right), y _ {i}\right). \tag {1}
$$
Here  $(x_{i},y_{i})\in \mathcal{X}_{l}$ , and  $\ell_{\mathrm{sup}}(\cdot ,\cdot)$  is the supervised loss, e.g., the cross-entropy loss. (1) can be efficiently optimized using stochastic gradient-type algorithms, such as Adam (Kingma and Ba, 2015).
Algorithm 1: Differentiable Self-Training.
Input:  $\mathcal{X}_l$ : labeled dataset;  $\mathcal{X}_u$ : unlabeled dataset;  $\alpha$ : parameter of exponential moving average;  $\theta^{\mathrm{init}}$ : initialization; Optimizer: optimizer to update  $\theta^S$ .
Initialize:  $\theta_0^T = \theta_0^S = \theta^{\mathrm{init}}$
for  $t = 1,\dots ,T - 1$  do
Sample a labeled minibatch
$$
\mathcal {B} _ {l} = \left\{x _ {i} \right\} _ {i = 1} ^ {| \mathcal {B} _ {l} |} \text {f r o m} \mathcal {X} _ {l};
$$
Sample an unlabeled minibatch
$$
\begin{array}{l} \mathcal {B} _ {u} = \left\{x _ {i} \right\} _ {i = 1} ^ {| \mathcal {B} _ {u} |} \text {f r o m} \mathcal {X} _ {u}; \\ \widetilde {\mathbf {y}} \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) \gets (6) \text {o n} \mathcal {B} _ {u}; \\ \omega \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) \leftarrow (7) \text {o n} \mathcal {B} _ {u}; \\ \mathcal {L} \left(\theta_ {t} ^ {S}\right) \leftarrow (8) \text {o n} \mathcal {B} _ {u} \cup \mathcal {B} _ {l}; \\ g = \mathrm {d} \mathcal {L} \left(\theta_ {t} ^ {S}\right) / \mathrm {d} \theta_ {t} ^ {S} \leftarrow (4); \\ \theta_ {t + 1} ^ {S} = \operatorname {O p t i m i z e r} \left(\theta_ {t} ^ {S}, g\right); \\ \theta_ {t + 1} ^ {T} = \alpha \theta_ {t} ^ {T} + (1 - \alpha) \theta_ {t + 1} ^ {S}; \\ \end{array}
$$
Output: Student model  $\theta_T^S$  for prediction.
At time  $t$ , denote the student's parameters  $\theta_t^S$ , and the teacher's parameters  $\theta_t^T (\theta_t^S)$ . We set both the student's and the teacher's initial parameters to  $\theta^{\mathrm{init}}$ , i.e.,  $\theta_0^S = \theta_0^T (\theta_0^S) = \theta^{\mathrm{init}}$ . Note that the teacher model depends on the student. We adopt an exponential moving average (Laine and Aila, 2017; Tarvainen and Valpola, 2017) approach to model such a dependency:
$$
\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right) = \alpha \theta_ {t - 1} ^ {T} + (1 - \alpha) \theta_ {t} ^ {S}. \tag {2}
$$
Recall that in our differentiable self-training framework, the student acknowledges the teacher's strategy. This meets the definition of a Stackelberg game (Von Stackelberg, 2010), and we propose the following formulation:
$$
\begin{array}{l} \min  _ {\theta_ {t} ^ {S}} \mathcal {L} \left(\theta_ {t} ^ {S}\right) = \mathcal {L} _ {\sup } \left(\theta_ {t} ^ {S}\right) \tag {3} \\ + \frac {1}{N _ {u}} \sum_ {x _ {i} \in \mathcal {X} _ {u}} \ell_ {S} \left(x _ {i}, F \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right), \theta_ {t} ^ {S}\right), \\ \end{array}
$$
$$
\mathrm {s . t .} F \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) = \left[ \widetilde {\mathbf {y}} \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right), \omega \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) \right].
$$
Here recall that  $\mathcal{X}_u$  is the unlabeled data samples, and  $N_u$  is the size of  $\mathcal{X}_u$ . In (3),  $F(\theta_t^T (\theta_t^S))$  is the teacher's strategy, which contains differentiable pseudo-labels (i.e.,  $\widetilde{\mathbf{y}} (\theta_t^T)$  in (6)) and differentiable sample weights (i.e.,  $\omega (\theta_t^T)$  in (7)). The loss function  $\ell_S$  is defined in (8). Note that we still include the supervised loss  $\mathcal{L}_{\mathrm{sup}}$  in (1) in the objective function  $\mathcal{L}$ . Following conventions, in (3), the minimization problem solves for the leader, and we call
$F(\theta_t^T)$  the follower's strategy. Note that the Stackelberg game formulation (3) has also been adopted in adversarial training (Zuo et al., 2021).
The Stackelberg game formulation is fundamentally different from conventional self-training approaches, where the teacher  $\theta^T$  is not treated as a function of the student  $\theta^S$ . In our differentiable self-training framework, the leader takes the follower's strategy into account by considering  $F(\theta_t^T (\theta_t^S))$ . In this way, self-training can be viewed solely in terms of the leader's parameters  $\theta_t^S$ .
Consequently, the leader problem can be efficiently solved using stochastic gradient-type algorithms, where the gradient is
$$
\begin{array}{l} \frac {\mathrm {d} \mathcal {L} \left(\theta_ {t} ^ {S}\right)}{\mathrm {d} \theta_ {t} ^ {S}} = \frac {1}{N _ {l}} \sum_ {\left(x _ {i}, y _ {i}\right) \in \mathcal {X} _ {l}} \frac {\mathrm {d} \ell_ {\sup } \left(\theta_ {t} ^ {S}\right)}{\mathrm {d} \theta_ {t} ^ {S}} \tag {4} \\ + \frac {1}{N _ {u}} \sum_ {x _ {i} \in \mathcal {X} _ {u}} \frac {\mathrm {d} \ell_ {S} (x _ {i} , F (\theta_ {t} ^ {T} (\theta_ {t} ^ {S})), \theta_ {t} ^ {S})}{\mathrm {d} \theta_ {t} ^ {S}} \\ = \underbrace {\frac {1}{N _ {l}} \sum_ {\mathcal {X} _ {l}} \frac {\mathrm {d} \ell_ {\operatorname* {s u p}} (\theta_ {t} ^ {S})}{\mathrm {d} \theta_ {t} ^ {S}} + \frac {1}{N _ {u}} \sum_ {\mathcal {X} _ {u}} \frac {\partial \ell_ {S} (x _ {i} , F , \theta_ {t} ^ {S})}{\partial \theta_ {t} ^ {S}}} _ {\text {l e a d e r}} \\ + \underbrace {\frac {1}{N _ {u}} \sum_ {x _ {i} \in \mathcal {X} _ {u}} \frac {\partial \ell_ {S} \left(x _ {i} , F \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) , \theta_ {t} ^ {S}\right)}{\partial \theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)} \frac {\mathrm {d} \theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)}{\mathrm {d} \theta_ {t} ^ {S}}} _ {\text {l e a d e r - f o l l o w e r i n t e r a c t i o n}}. \\ \end{array}
$$
In (4), we have  $\mathrm{d}\theta_t^T (\theta_t^S) / \mathrm{d}\theta_t^S = 1 - \alpha$  because of (2). Note that a conventional self-training method only considers the "leader" term, and ignores "leader-follower interaction". This causes training instabilities, which we demonstrate empirically in Fig. 1 and Fig. 2.
The proposed differential self-training algorithm is summarized in Algorithm 1. In the next two sections, we spell out the two components of the follower's strategy, namely differentiable pseudolabels and differentiable sample weights.
We remark that Algorithm 1 adopts a Stackelberg game formulation of self-training. That is, the loss terms in (3) (soft-labels and sample weights) are well-established techniques, and the proposed method is a novel optimization algorithm.
3.2 Differentiable Pseudo-Labels
In a self-training framework, the teacher model labels the unlabeled data. Concretely, at time  $t$ , for each sample  $x \in \mathcal{X}_u$  in the unlabeled dataset, a hard pseudo-label (Lee, 2013) is defined as
$$
\widetilde {y} _ {\text {h a r d}} = \underset {j \in \mathcal {Y}} {\operatorname {a r g m a x}} \left[ f \left(x, \theta_ {t} ^ {T}\right) \right] _ {j}. \tag {5}
$$
Here  $f(x,\theta_t^T)\in \mathbb{R}^C$  is in the probability simplex, and  $[f(x,\theta_t^T)]_j$  denotes its  $j$ -th entry.
There are two problems with the hard pseudo-labels. First, differentiable self-training requires every component of the follower's strategy (3) to be differentiable with respect to the leader's parameters. However, (5) introduces a non-differentiable argmax operation. Second, the hard pseudo-labels exacerbates error accumulation. This is because  $\widetilde{y}_{\mathrm{hard}}$  only contains information about the most likely class, such that statistics regarding the prediction confidence  $f(x,\theta_t^T)$  is lost. For example, suppose in a two-class classification problem, we obtain  $f(x,\theta_t^T) = [0.51,0.49]$  for some  $x$ . This prediction result indicates that the model is uncertain to which class  $x$  belongs. However, under the hard pseudo-label  $\widetilde{y}_{\mathrm{hard}} = 0$ , the student model becomes unaware of such uncertainty.
To resolve the above two issues, we propose to employ soft pseudo-labels (Xie et al., 2016, 2020; Meng et al., 2020). Concretely, for a data sample  $x \in \mathcal{B}$  in a batch  $\mathcal{B}$ , the  $j$ -th entry of its soft pseudolabel  $\widetilde{\mathbf{y}}(\theta_t^T) \in \mathbb{R}^C$  is defined as
$$
\left[ \tilde {\mathbf {y}} \left(\theta_ {t} ^ {T}\right) \right] _ {j} = \frac {\left[ f \left(x , \theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) \right] ^ {1 / \tau} / f _ {j}}{\sum_ {j ^ {\prime} \in \mathcal {Y}} \left[ f \left(x , \theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) \right] ^ {1 / \tau} / f _ {j ^ {\prime}}}, \tag {6}
$$
where  $f_{j} = \sum_{x^{\prime}\in \mathcal{B}}[f(x^{\prime},\theta_{t}^{T}(\theta_{t}^{S}))]^{1 / \tau}$ , and  $\tau$  is a temperature parameter that controls the "softness" of the soft pseudo-label. Note that when the temperature is low, i.e.,  $\tau \rightarrow 0$ , the soft pseudo-label becomes sharper and eventually converges to the hard pseudo-label (5).
In (6), the soft pseudo-label  $\widetilde{\mathbf{y}} (\theta_t^T)$  is a function of the teacher's parameters  $\theta_t^T$ , which in turn is a function of the student's parameters  $\theta_t^S$  (2). Therefore,  $\widetilde{\mathbf{y}}$  is differentiable with respect to  $\theta_t^S$ , and fits in the differentiable self-training framework. The gradient of  $\widetilde{y}$  with respect to  $\theta^S$  can be efficiently computed by a single back-propagation using deep learning libraries.
Notice that (6) emphasizes the tendency of  $x$  belonging to a specific class, instead of to which
The "leader" term is written as  $\partial \ell_{S}(x_{i},F,\theta_{t}^{S}) / \partial \theta_{t}^{S}$  instead of  $\partial \ell_{S}(x_{i},F(\theta_{t}^{T}(\theta_{t}^{S})),\theta_{t}^{S}) / \partial \theta_{t}^{S}$  because the partial derivative is only taken with respect to the third argument in  $\ell_S(x_i,F,\theta_t^S)$ . We drop the  $\theta_t^T (\theta_t^S)$  term in  $F(\theta_t^T (\theta_t^S))$  to avoid causing confusion.
class  $x$  belongs. Therefore, even when  $\widetilde{y}_{\mathrm{hard}}$  is wrong, the soft version of it is still informative.
3.3 Differentiable Sample Weights
Sample reweighting is an effective tool to tackle erroneous labels (Freund and Schapire, 1997; Kumar et al., 2010; Malisiewicz et al., 2011; Liang et al., 2020). Specifically, pseudo-labels that have dominating entries are more likely to be accurate than those with uniformly distributed entries. For example, a sample labeled [0.9, 0.1] is more likely to be classified correctly than a sample labeled [0.6, 0.4]. With this intuition, for a sample and its soft pseudolabel  $\widetilde{\mathbf{y}} (\theta_t^T)$ , we define its sample weight  $\omega$  as
$$
\omega \left(\theta_ {t} ^ {T}\right) = 1 - \frac {H \left(\widetilde {\mathbf {y}} \left(\theta_ {t} ^ {T}\right)\right)}{\log (C)}, \tag {7}
$$
where  $H(\widetilde{\mathbf{y}} (\theta_t^T)) = -\sum_{j = 1}^{C}\widetilde{\mathbf{y}}_j\log (\widetilde{\mathbf{y}}_j)$  is the entropy of  $\widetilde{\mathbf{y}} (\theta_t^T)$  that satisfies  $0\leq H(\widetilde{\mathbf{y}} (\theta_t^T))\leq$ $\log (C)$ . Note that if the pseudo-label is uniformly distributed, then the corresponding sample weight is low, and vice versa. Similar to the pseudo-label  $\widetilde{\mathbf{y}} (\theta_s^T)$ , the sample weight  $\omega (\theta_t^T)$  is a function of the teacher's parameters  $\theta_t^T$ , and further a function of the student's parameters  $\theta_t^S$ .
With the differentiable pseudo-labels and the differentiable sample weights, we define the student's loss function as
$$
\begin{array}{l} \ell_ {S} \left(x _ {i}, F \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right), \theta_ {t} ^ {S}\right) = \omega \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) \\ \operatorname {K L} \left(\widetilde {\mathbf {y}} \left(\theta_ {t} ^ {T} \left(\theta_ {t} ^ {S}\right)\right) \| f \left(x _ {i}, \theta_ {t} ^ {S}\right)\right), \tag {8} \\ \end{array}
$$
where  $\mathrm{KL}(p||q) = \sum_{k} p_k \log (p_k / q_k)$  is the Kullback-Leibler (KL) divergence.
3.4 Weakly-Supervised Learning
Recall that in weakly-supervised learning, we have both labeled data  $\mathcal{X}_l$  and unlabeled data  $\mathcal{X}_u$ . Note that weak supervision sources often yield noisy labels. This is because data are annotated automatically by, for example, linguistic rules, which have limited accuracy. As such, the supervised loss  $\mathcal{L}_{\mathrm{sup}}$  in (3) only exacerbates the label noise issue.
We address this problem by discarding the noisy weak labels in  $\mathcal{X}_l$  after obtaining the initialization  $\theta^{\mathrm{init}}$  (1). Accordingly, we adopt the following formulation for weakly-supervised learning: