Bridging the Gap between Semi-supervised and Supervised Continual Learning via Data Programming
Authors: Pengyuan Lu
Seungwon Lee
Amanda Annette Watson
David Kent
Insup Lee
ERIC EATON
James Weimer
Venue: Submitted to ICLR 2023 (2023)
Keywords: continual learning
lifelong learning
semi-supervised learning

Abstract: Semi-supervised continual learning (SSCL) has shown its utility in learning cumulative knowledge with partially labeled data per task. However, the state-of-the-art has yet to explicitly address how to reduce the performance gap between using partially labeled data and fully labeled. In response, we propose a general-purpose SSCL framework, namely DP-SSCL, that uses data programming (DP) to pseudo-label the unlabeled data per task, and then cascades both ground-truth-labeled and pseudo-labeled data to update a downstream supervised continual learning model. The framework includes a feedback loop that brings mutual benefits: On one hand, DP-SSCL inherits guaranteed pseudo-labeling quality from DP techniques to improve continual learning, approaching the performance of using fully supervised data. On the other hand, knowledge transfer from previous tasks facilitates training of the DP pseudo-labeler, taking advantage of cumulative information via self-teaching. Experiments show that (1) DP-SSCL bridges the performance gap, approaching the final accuracy and catastrophic forgetting as using fully labeled data, (2) DP-SSCL outperforms existing SSCL approaches at low cost, by up to $25\%$ higher final accuracy and lower catastrophic forgetting on standard benchmarks, while reducing memory overhead from $100$ MB level to $1$ MB level at the same time complexity, and (3) DP-SSCL is flexible, maintaining steady performance supporting plug-and-play extensions for a variety of supervised continual learning models.

Under review conference paper ICLR 2023
BRIDGING THE GAP BETWEEN SEMI-SUPERVISED AND SUPERVISED CONTINUAL LEARNING VIA DATA PRO- GRAMMING
Anonymous authors Paper under double-blind review
ABSTRACT
Semi-supervised continual learning (SSCL.) has shown its utility in learning cumu- lative knowledge with partially labeled data per task. However. the state-of-the- of-the-art has yet to explicitly address how to reduce the performance gap between using par- tially labeled data and fully labeled. In response, SSCL framework, namely DP-SSCL, that uses data programming (DP) pseudo- label the unlabeled data per task, and then cascades both ground-trutte soud---utt-labeled and pseudo-labeled data to update downstream supervised continual learning model. The framework includes feedback loop that brings mutual benefits: On one hand, DP-SSCL inherits guaranteed pseudo-labeling quality from DP techniques to improve continual learning. approaching the performance of using fully super- vised data. On the other hand, knowledge transfer from previous tasks facilitates training of the DP pseudo-labeler, taking advantage of cumulative information via self- eelf-teaching. Experiments show that (1) DP-SSCL bridges the performance gap, approaching the final accuracy and catastrophic forgetting as using fully labeled data, (2) DP-SSCL outperforms existing SSCL approaches at low cost, by up to 25% higher final accuracy and lower catastrophic forgetting standard bench- marks, while reducing memory overhead from 100 MB level to MB level the same time complexity, and (3) DP-SSCL is flexible, maintaining steady perfor- mance supporting plug-and-play extensions for variety of supervised continual learning models.
propose general-purpor purpose
INTRODUCTION
Lifelong machine learning, also known as continual learning (CL), a machine learning paradigm that accumulates knowledge over sequential tasks (Ruvolo & Eaton 2013a [Silver et al., 2013 Chen & Liu 2016 Liu 2017). It empowers machine learning at the application level such that an agent does not need to he trained from scratch with large amounts of data for every new task, as well as enabling the agent's self-improvement on previously-earnee tasks by continuing to learn post-deployment. Nevertheless, researchers have identified that obtaining labeled training data is expensive (Olivier et al. 2006, Settles] 2009). which semi-supervised continual learning (SSCL) addresses (Baucum addresssesBauuuum et al. 2017] Wang el al. | 202T; [mmihh et al. 202T). As the name suggests, SSCL
utilizes not only labeled data, but also leverages unlabeled task data to construct a cumulative knowl- edge base for learning agents, reducing labeling cost in applied machine learning.
Despite all the research efforts on SSCL, the state-of-the-art of SSCL (Baucum et al. 2017 Wang et al. 2021 Smith al., [221]] has yet to address an elephant in the room: closing the performance gap between supervised and semi-supervised CL. Ideally, learning from ny labeled data and ny unlabeled data per task should provide the same lifelong performance as all the Yy, + ny data are labeled, but state-of-the-ane SSCL frameworks have not approached this goal, and rarely consider computational cost required to do so. Moreover, multiple supervised CL tools have matured Lee [ at (2019): Yoon et al. (2018); Bulat et al. (2020) and would likely benefit by extending them to the semi-supervised setting, but current SSCL approaches are architecture-specte specific and such extension non-trivial.

---

Under review as conference paper ICLR 2023
Motivated by the challenges above, we propose data programming (DP) (Ratner et al. 601 as a solution. DP is an automatic psuedo-labeling approach that collectively generates labels from noisy labeling functions, with some methods providing probabilistic guarantees pseudo-label accuracy (Ratner al. 2016a Varma & Ré 2016). Ideally, the more diverse noisy labelers are sampled, the higher quality pseudo-labels can be produced approaching perfect labeling accuracy. Therefore, upon every task in SSCL, by training a pseudo-labeler via DP and then cascading both ground-truth- labeled and pseudo-labeled data into supervised CL model, we are able to approach high quality pseudo-labeling and decrease the performance gap between semi-supervised and supervised CL. This procedure is also benefited from the small overhead of DP in terms of both time and memory, lowering resource costs on large amounts of unlabeled data and long task sequences. Furthermore, the cumulative knowledge along CL can assist the aseudo-labeler performance, leveraging transfer- ability analysis metrics (Nguyen et al. 2020; Tan et al., [021] Pandy et al. 2022] Tran et al. 2019). Intuitively, the more similar two tasks, the more similar ways they should handle the unlabeled data to shrink the gap. In practice, noisy labeling functions from previous tasks can be retained and transferred to new tasks hased on task transferability, utilizing cumulative knowledge to self-teach the pseudo-labeler throughout the lifelong sequence. This framework design also allows supervised CL approaches to be extended in a plug-and-play fashion, by decoupling the pseudo-labeling and continual learning modules.
Experiments on standard image classification benchmarks show DS-SSCL achieves final accuracy and catastrophic forgetting comparable to supervised CL on fully laheled data. Moreover, DP- SSCL outperforms existing SSCL tools with up to 25% higher final accuracy and lower catastrophic forgetting, while reducing the memory overhead for unlabeled data processing from the 100 MB to the MB level with the same time complexity. Additionally, ablation studies show DP-SSCL maintains steady continual performance at increasing sizes of unlabeled data per tasks, over longer task sequences, and using different knowledge transfer mechanisms.
2 BACKGROUND AND RELATED WORK
2.1 LIFELONG LEARNING/CONTINNUN LEARNING (CL)
The primary goal of continual or lifelong learning is to learn tasks consecutively. exploiting for- ward transfer facilitate the learning of new tasks while retaining performance on previous tasks without catastrophic forgetting. The vast majority of research focuses on supervised methods, using techniques such as weight importance vectors (Fernando et al. 2017 Aljundi et al. (219) to cache critical pathways and prevent catastrophic forgetting, factorized transfer decompose the model parameter space (Ruvolo Faton [2033)] [Bulat et al. 20200 Lee et al.] 2019), deconfficting projec- tions to ensure that new tasks are trained using unused capacity within the deep network (Farajtabar al. 2019) Zeng al.] 2019) [aaha et al. 202T), and dynamically expanding networks that grow accommodate tasks (Veniat et al. (021).
2.2 SEMI-SUPERVISED CONTINUAL LEARNING (SSCL)
Recently, techniques have been developed for CL in semi-supervised settings to take advantage of unlabeled data.
common procedure of SSCL is to pseudo-label these unlabeled data for training set augmenta- tion. For instance, CNNL (Baucum et al.] (017)) fine-tunes a lifelong learning model by repeatedly pseudo-labeling unlabeled data using the model itself, and then augments its training set with the newly-labeled dltaa. Alternatively, DistillMatch (Smith al. 202T) identifies unlabeled data points that are possibly seen in previous tasks by an out-of-distribution detector, and pscudo-labels them us- ing distilled accumulated knowledge. A third example is ORDisCo (Wang et al. [202]). which trains a GAN-based pseudo-labeler in parallel with lifelong learning model by using three-branch net- work, which enables it to learn the joint distribution of data and labels simultaneously. Similarly, Semi-ACGAN (Brahma et al. 2021) utilizes GAN for training task desk-dependent classifiers while us- ing the unlabeled data only train the discriminator of GAN for the source of data (real fake). The last example (Ho et al.]
2022) combines prototypical learning for pscudo-labeling with meta learning to achieve both the label generation on the unlabeled data and fast adaptation to any task in the continual learning scenario. Under pseudo-labeling. bridging the gap towards supervised CL.

---

Under review as conference paper ICLR 2023
becomes simple: the higher pseudo-labeling accuracy, the closer performance CL on fully labeled data. Straightforwardly, to shrink the gap is to improve the pseudo-labeler.
The methods above all generate labels for the unlabeled data by using the classifiers that are trained for task objectives, and their novelty comes from supplementary design decisions that tackle issues of SSCL, such as catastrophic forgetting and pseudo-label consistency. these approaches arises due to the frequently observed tendency of neural network models to have overconfidence in their predictions (Guo et al. 2017a Nguyen et al., 2015 Hein et al., (201);; as result ceuural-ne--base labelers may confidently generate incorrect pseudo-labels, especially when encountering novel tasks or out-of- of-distribution data. On the other hand, our DP-SSCL method uti- lizes DP, which provides theoretical guarantees the quality of labels, making the process more robust to new data from the novel tasks that arise in CL settings. Moreover, these ex- isting SSCL tools require large additional memory and computation overhead for pseudo- such as storage of a ResNet-34 backbone (Smith et al. 2021), GAN networks (Wang et al.) 2021] Brahma et al. 202T) or MAMI. (Ho et al. 2022). while DP-SSCI. minimizes this overhead by storing light- labeling functions facilitated by data programming, which we describe below.
major shortcoming
Additionally, techniques that do not depend directly on pseudo- Labuls have been proposed for stable semi-supervised continual learning. For instance, pseudo-gradient learners (Luo et al. 2022] are trained instead of pscudo-labelers to provide auxiliary gradients for the model update from the un- labeled data. This is useful for the case that the unlabeled data may include instances of unknown classes. The other work, CCIC (Smith et al.] 202T), adapts MixMatch to use pseudo-labels of the unlabeled data as consistency regularization target rather than training target. Still, these methods avoid the usage of pseudo-labels as direct training target, but lack theoretical guarantees on the benefit of these approaches as provided by our DP-SSCI..
2.3 DATA PROGRAMMING (DP)
Data programming (DP) an approach to automatically produce pseudo-labels on an unlabeled data set Xu. given a labeled data set (Xz. yL) (Ratner et al. 2016b)). The idea is to ensemble a set of noisy weak labeling functions (WLFs) with each performing only slightly better than random guess- ing on their own, such that the ensembled pseudo-labels, or strong labels, achieve high accuracy. Therefore, DP papers generally discuss two problems in cascading order: (1) how to generate these WLFs and (2) how to do the ensembling.
To address the upstream problem on WLF generation, one simple way is collect manually de- signed functions by experts (Ratner et al. 20166). One succeeding tool, namely Snuba (Varma & Ré 2016). iteratively trains multiple sets of WLFs, and then from every set selects the top h WLFs ranked by score
(l) where 인L and tu are the predicted label vectors by a WLF on the labeled and unlabeled data re- spectively. The score consists of a performance metric (FI) on the labeled set, as well as a diversity metrie (Jaccard distance (Jaccard (902)) with weighting factor usually 0.5. The selected WLFs that pass this pruning step form the committed labeler set F.
= w FyLL, 0z) + (1 - w) + Jaccard((u)
Then, for the downstream problem on ensembling, multiple techniques can be applicd. For example, majority voting is one of the brute force methods. Other existing techniques such as repeated label- ing could also apply (Ipeirotis et al., 2014). Among these methods, Snorkel (Ratner et al., 2016a) learns a generative model on top of committed WL.Fs in form of
T (Yr, Yu)
Z% exp( YuYo)
(2)
where Po and Yu are the aggregated label matrices of all committed WLF labels and ground-truth labels, di is the parameters and Z4 is a normalization factor. Snorkel trains this generative model such that it labels ye with high accuracy. This ensembling requires each WLF to have higher accuracy than random guessing, which a low requirement for learners.
When the downstream ensembling uses generative model as in equation 2 Snuba provides a prob- abilistic guarantee that the accuracy of the generative model on labeled data and unlabeled data has a maximum difference of e with probability 1- 8. This guarantee exists because Snuba checks an exit

---

Under review conference paper ICLR 2023
condition on WLFs, such that each committed WLF before termination must have certain level confidence on d data points, with
2/ 8
d 2(7 c] log
(3)
Here, 7 is the measured error, i.e. the difference of accuracy between the generative model and WLFs on the labeled data. Please refer to the original paper for a more detailed proof. We also extend the proof of this guarantee to the continual learning setting in Appendix A
2.4 TRANSFERABILITY
In transfer learning, transferring trained model to target task that shares no common features with the source task typically hurts the performance of the model, even worse than learning only the target task. As such, understanding the similarity of tasks and measuring transferability of learning model from one task to another a key aspect of not only transfer learning but also continual learning. An intuitive metric of transferability is the accuracy of the source model on the target data (Trn et al. 2019 Dhillon et al. (020):: measuring how well the trained model performs on the target task. LEEP (gguyen et al. 2020) extends this metric by weighting the likelihood of the model with empirical conditional distribution of the target label given the source label. OTCE (Tan et al. (02T) quantifies the distance between two classification tasks as a sum of domain difference and task difference. The domain difference the difference of the data distribution is computed by optimal transport theory with entropic regularization, and the task difference the difference the classification objectives such as a set of classes - is derived from conditional entropy using the optimal coupling matrix of the optimal transport problem. GBC (Pandy al. 2022) adopts the Bhattacharyya coefficient that measures the amount of overlap between two distributions in the feature space of the source model. Based on the positive results by using, transferability score in transfer learning, this score can help figuring out the reusable knowledge of earlier tasks more effectively.
3 PROBLEM FORMULATION
We aim to solve the following problem: How to design an SSCL framework, such that it able
(1) minimize the performance gap between using partially labeled data and fully labeled data per task at low cost, (2) allow cumulative knowledge transfer to assist handling of unlabeled data, and (3) extend from arbitrary existing supervised CL frameworks?
Formally, in an SSCL problem, a lifelong learner will face a sequence of classification tasks (g(), g(2)....). with each task 2 having data space X S RR,, label space y {l,..., .......Co G... and a joint distribution D(1), (2) x (06)) + [0, governing the data-label pairs. For task 2 we are given ni labeled training data (X (x/,)) VV.
(4) (4) D(0), ny unlabeled training data x
200
and nlf testing data (xx) VT D())
From this setting, we consider a two-part SSCL framework consisting of an upstream pseudo-labeler and a downstream supervised CL module that are isolated from each other's 's processes. Specifically, at task (0), a pseudo-labeling function (1()) : xx + y700 first trained. Then, the unlabeled data x will be labeled with V0 by this function. Next, labeled data (x(*) ( .VL and pseudo-labeled
data (X(() (8) (l()) will be given to supervised CL module for continual learning. With this design, users are able to plug-and-pll existing well-developed supervised CL. tools and augment them into
SSCL, addressing sub-problem (3).
The design of the pseudo-labeling procedure addresses sub-problems (1) and (2). Intuitively, the higher the pseudo-label quality, the smaller the gap is between using partially labeled and fully
labeled data (as if each task ( has all ni, (8) + n labeled). Hence, (1) asks for a labeler that ideally approach perfect labeling by learning every underlying distribution D(i), preferably with guaranteed label quality, and low time and memory overhead. Furthermore, (2) asks for a cumulative knowledge base and its input/output algorithm specifically for the pseudo-labeler In the next section, we provide concrete framework that meets these requirements.

---

Under review conference paper ICLR 2023
Task
Meta Init
DP-SSCL: Seml-Supervised Continual Learning
2
Inperpurenater
Search Space
Pruning
Pscudo Label Generation
Supervised CL Update
2(3)
Model lemplates
Randirs
Labeld
Updated Mode
Data 3
fo Twnk 210
Caneet
2
Ensemblee
pupervieed Unikkng
Figure 1: The DP-SSCL framework, our contribution marked as red and adopted modules blue.
4 THE DP-SSCL FRAMEWORK FOR SEMI-SUPERVID CONTINUAL LEARNING
In this section, we introduce our proposed framework, Data Programming- SSCL,, DP- SSCL. The overall workflow illustrated in Figure [ As shown, after meta initialization, upon every new task, four modules are called in cascading order: (1) WLF initialization and training, (2) WLF pruning, (3) pscudo-label ensembling and (4) supervised CL model updating. Our framework also includes light-weight storage for all committed WLFs throughout all the tasks.
4.1 META INITIALIZATION
In typical CL setting, the tasks are not available at once, but are able to assume some general knowledge for the type tasks (Liu 2017 Ruvvooo Eaton 2013a Silver et al. 2013). For exam- ple, what input dimensions and what latent space that encodes representative features. Therefore, we perform meta initialization step allow users imput these prior information before the first task. Such prior knowledge includes a training hyperparameter search space, in which hyperparameters such as learning rate, hatch sizes and number of epochs for WLF training will be grid-searched later new tasks arise. Other prior knowledge includes model templates, which define the architectures for WLFs, which are typically decision trees, regressors, small neural networks not exceeding 5 layers or 1000 parameters. The WLF storage is initialized to be empty.
4.2 WLF INITIALIZATION VIA KNOWLEDGE TRANSFER
Upon the arrival of a new task, DP-SSCL first acquires set of WLFs, each form of fweek x(0) - y00)) U {0). where an additional 0 label means the confidence of fweak on a data point is lower than a given threshold such that labeling is abstained. To produce WLFs that can ensemble to an accurate aseudo-labeler, we consider two sources of WLF initialization: (1) random sampling the pre-defined model templates in meta initialization, leveraging the user's prior knowledge of the task and (2) transfer from previous tasks, leveraging knowledge accumulated throughout the continual learning process.
Knowledge is represented in the form of previously committed WLFs, that is, WLFs that enters the pseudo-labeler ensembling procedure in previous tasks z(1) g())..... (i-1)), Upon task g DP-SSCL evaluates the suitability of the previously committed WLFS for transfer to the current task using
transferability measures, such the LEEP OTCE transferability score introduced in Section 2.4
and selects from the cached WLFs using the procedure shown in Algorithm
Algorithm returrs one of the previously committed WLFs to initialize a new weak labeling func- tion. In this algorithm, the suitability score and the committed WLF selection criteria can be de- signed based on prior data or task knowledge. For example, one use only the task transferability score to sample tasks relevant to the current task, and then select a weak labeling function at random by treating WLFs of the sampled task uniformly. On the other hand, it is also possible to consider the labeling accuracy of the earlier WLFs in addition task transferability for the selection of WL.ss Since there is no guarantee that all previously encountered tasks are closely related to the current task, suitability score threshold ó excludes negatively correlated tasks from the pool of WLFs for transfer. Similarly, introduce another parameter probability of initializing WL_F by transfer p

---

Under review conference paper ICLR 2023
Algorithm WLF Transfer
Input: WLF storage W, CL model M, and the current task data (XL²) x/(2)
Parameter: Suitability score threshold &
Output: selected WLF swook the storage W
1: SWLF compuuteSuiiaailiyyysoneWWW, M, xL (8)
V/
2: PWLF f comernSecccioorubbbilllyyys..... 3: Sweak + randomSSloction(WV, PWLF)
4:
return Furea
to control the ratio the transferred WLFs to newly generated WLFs for the current task, which allows for the some WLFs to be initialized from scratch to maintain diversity in the WLF pool.
4.3 WLF TRAINING AND PRUNING
After initialization, the framework obtains set of both transferred and randomly generated WLFFs, which fine-tuned by labeled data current task. The training hyperparameters such learning rate, batch size and epochs are selected from within the bounds specified during the meta initial- ization phase. The trained WLFs then pass through Snuba pruner (Varma & Rd 2016), which commits only the top functions ranked by score computed by Equation equation DJ To improve the diversity of each WLF, we adopt bootstrapping on the training data, such that each WLF trained on a randomly selected subset of the labeled data, with the boostrapped size specified during meta initialization. The initialization and training procedure repeated until either maximum size of committed WLFs is met, or if the condition equation 3 will become violated the next iter- ation. Consequently, although we include transferred functions in this procedure, our framework still maintains the Snuha guarantee on pseudo-labeler quality. A detailed proof of how DP-SSCI. maintains this guarantee in the continual learning setting is presented in Appendix
4.4 PSEUDO-LABELER ENSEMBLING AND CONTINUAL MODEL UPDATE
The committed WLFs task ()) enter ensembling, where different aggregators can be used. For instance, majority voting is a simple method to combine WLFs, as well as repeated labeling (Ipcirotis 2014). More advanced ensembling methods available, such training Snorkel (Ratner al.
et al. 2016a) generative model (()) in the form of equation 2 We empirically evlauate different ensembling techniques in an ablation study in Section Via the ensembled labeler, the framework obtains pseudo-labels ((4) After ensembling. if the confidence of the pseudo-labels available, the pscudo-labols can be further adjusted by confidence calibration (Platt) 1999 Guo et al.) (017b)).
Last, together with labeled data (x(² M.. ). the pscudo-labeled data (X() Mir enters a supervised
(8)
continual learning module to update its model to lear a function in the form of f U/=11 X00) +
-}=1 yoo), ing approaches, such as DF-CNN (Lee al. 2019). TF (Bulat et al. 2020). and DEN (Yoon et al. 2018), which we demonstrate below.
Vاا)) For the underlying model, framework supports many state-of-the-arr continual leam-
5 EXPERIMENTAL EVALUATION
In this section, we first explain our instantiation of DP-SSCL from ablation study results on en- sembling methods and WLF transfer. Then, we explore how DP-SSCL's lifelong performance with partially labeled data compares to that supervised CL using fully labeled data sets, as well as how DSCS performs compared to existing SSCL methods. More experiments and discussion presented in Appendix D

---

Under review conference paper ICLR 2023
Table 1: Comparison of different pseudo-labeler ensembling methods.
MNIST Final Accuracy
Continual Learner
Labeling Method
By
(8)
60
By
(6)
120
R4 (2)
240
Majority Voting
0..5
89.1
12.6
94.
DP-SSCL (TF)
Repeated Labeling
1.1
84.7
0..
Snorkel
97.0 0.5
97.0=00
96.5 0.2
Majority Voting
9.
9..6=
93.3+10
DP-SSCL (DF-CNN)
Repeated Labeling
0.5
1.
0..
Snorkel
96.5=0.3
=6.4=0.a
95.2
5.1 SETUP
We show that DP-SSCI is general framework for SSCL by using different combinations of algo- rithms for strong label generation (majority voting, repeated labeling, and Snorkel) and supervised CL (TF Bulat et al. (2020). DEN Yoon et al.] (2018). and OF-CNN et al. (2019)). Our exper- iments employ continual learning versions of standard image classification benchmarks, including MNIST LeCun & Cortes 2010). CIFAR-10, and CIFAR-100 CIFAR-100]Knzhvsskk) (2009). For each task, we hold out 10% of labeled and unlabeled data for validation. We evaluate performance using (1) peak per-task performance metries, (2) final task performance metrics, and (3) forgetting metrics. with forgetting metrics measured ward-aertree (Ruvolo & Eaton 2013h Lopez-Paz & Ranzato 2017). The detailed experimental setup, including hyperparameter se selection and evaluation metrics,
is explained in the following subsections as well as in Appendix
5.2 ABLATION STUDIES
The following two ablation studies are evaluated on validation data select components for the main experiments in Section [.3] which is then evaluated separate testing data. More ablation studies are detailed Appendix
Ensembling Methods We evaluate three methods of pseudo-laheler ensembling discussed in Sec- tion 4.4: majority voting, repeated labeling, and Snorkel. As shown in Table [ (refer Table X in Appendix foro details), we see that Snorkel obtains consistently strong performance across contin- ual learning algorithms, and amounts of unlabeled training data. Consequently, we use Snorkel as the label generator within DP-SSCI for all other experiments.
WLF Transfer We instantiate our framework with knowledge transfer in WLFs by using DF-CNN (CL model), LEEEP score (suitability score) and binary CIFAR 10 experiment. As depicted Fig- @ both peak per-task accuracy and final accuracy slightly increase in comparison to DP-SSCL without WLF transfer (black dashed lines) when approximately half of WLFs are initialized via transfer. Due to the fact that increment is small, we pick high LEEP score threshold that results in no transfer for all the remaining experiments, without a loss in generality. Though the accura- cies within 95% confidence interval of the baseline, these enhanced CL performances show its positive potential of knowledge transfer weak labeling functions.
5.3
MAIN RESULTS
5.3.
SSCL PERFORMANCE
Based on the instantiation of Snorkel ensembler and no LEEP transfer. we compare DP-SSCL to three state-of-the-ard SSCL algorithms: CNNL (Baucum et al. al. (017), ORDisCo (Wang et al. 202 and DasthllMatch (Smith et al. (021).. To enable fair comparisons, we replicate the experimental con- ditions and compare our approach to the best results reported in each algorithm's 's original publica- tion. Specifically,
replicate the instance-ineremental learning experiments of CNNL on MNIST and CIFAR- 10, and the class-ine insremencemmttall learning experiments of ORDisCo and Masth on CI and CIFAR- 1000 respectively. The replicated experimental protocols are summarized below; please refer the original papers for detailed setups.

---

Under review conference paper ICLR 2023
100
Ratio Transferred WLFs
Final Accuracy
78.5
82.0
81.8
78.0
60
81.6
77.5
40
LEEP Threshold
LEEP Threshald
LEEP Threshold -0.5
LEEP Threshold -0.5
20
81.2
81.0
46
60
80
100
60
80
100
40
60
100
WLF Transter Prob.
WLF Transfer Prob.
WLF Transfer Prob.
(a) Peak Per-task Accuracy
(b) Final Accuracy
Ratio Transferred WLFs
Figure 2: SSCL performance when transferring WLFFs of earlier tasks.
Table 2:
SSCL results.
MAIST
CIFAR-10
Approach
Final
Batches to
Final
Batches to
Accuracy
Saturation
Accuracy 45.7
Saturation
CNNL
90.0
25
DP-S Labeled
90.0
3. 5.. 26.5+11
Fully Labeled
99.0#a1
17.0=40
57.6=a5 27.1+17
experiments, all tasks are present at every epoch, but subsequent epochs contain different batches of unlaheled data. Each experiment was run over 10 random seeds. As shown in Table DP-SSCI achieves 90% of final testing accuracy if fully labeled data used, comparable to CNNL in MNIST and higher CIFAR- 10. Moreover, on both benchmarks DP-SSCL has comparable or higher sample efficiency measured in batches to saturation, which means the first epoch where 3-batch sliding window average meets exceeds the final accuracy.
For
For Mlass-incrementa experiments, the model sequentially presented with tasks containing new sets of classes. Each experiment is run with 10 random seeds. As shown in Table when equipped with certain supervised module (DF-CNN), DP-SSCL is able to achieve comparable final testing accuracy to supervised CL using fully labeled data, strictly exceeding that of ORDisCo and Dis- tillMatch. In terms of catastrophic forgetting, we depict backward transfer as percentage, and lies in the range [-100%, 100%). and the higher value the less forgetting occurs. We that DP-SSCL also produces similar forgetting if fully labeled data is used. This result shows that DP-SSCI is able to capture the properties of continually shifting data and tasks, and generate appropriate pseudo-labels for the lifelong learners.
Table 3: Class-incremental SSCL results.
Final Acc. (%)
Backward Transfer (%)
Approach
CIFAR-10 CIFAR-100
CIFAR-10
CIFAR-100
Labeled
DF-CNN
78.6=07
44.8
6.4122
-3.9=
Only
DEN
60.1=29
24.6
-2.1122
-24.3
TF
78.4=as
45.6
7.8
-2.1=04
ORDisCo
74.8
not reported
not reported not reported
not reported not reported
DistillMatch
reported
04.4+
SSCL
Ours (DF-CNN)
16.8+1.3
50.0
6.3
-7.2+0.8
Ours (DEN)
61.4123
24.1
-6.4
-29.6 0.7
Ours (TF)
82.1
28.9
1.8
-7.8 2.4
Fully
DF-CNN
86.8-14 14
52.4+
6.64
-8.4
Labeled
DEN
61,4=37
23.7+00
-6,3
-3 06
TF
82.5
51.0
2.1450
8.74

---

Under review conference paper ICLR 2023
5.3.2 SSCL COMPUTATIONAL COST
We analyze DP-SSCL's 's computational cost required leverage unlabeled data compared to state- of-the-art SSCL methods, shown in Table $ where m is the size of one data point, b is the total number of data batches in a task, and is the number of iterations in CNNL's pseudo-laheling loop. Each scalar parameter in machine learning model is assumed to take 4B floating point. We conclude that our DP-SSCL has small overhead in terms of both memory and time for unlabeled data processing, which we elaborate on below.
For memory, CNNL maintains constant-length queue of data, such that the queue size is propor- tional to the size of data point. A typical queue length is 1000 as used their experiments, with each MNIST data costing 784 B and CIFAR 3072 B. ORDisCo, DistillMatch and DP-SSCL. all have constant (1)) overhead once the architectures are fixed. Nevertheless, in implementation, ORDisCo requires storage of its generator (G) and discriminator (D), which approximately costs 100 MB for parameters, estimated from the last table in (Li et al. (017)) reporting ORDisCo's archi- tecture. DistillMatch requires storage for its OoD detector, which is 254 MB for ResNet34 based DeConf network (Gao et al. 2021). On the other hand, DP-SSCL utilizes small weak labelers at 0.1MB-level, with 0.083 MB and 0,108 MB for MNIST and CIFARI0/100 benchmarks, respec- tively. We set buffer of maximally 25 weak labelers and reallocate the buffer upon each task, the overhead 0.083 x 2.7 MB for MNIST and 0.108 25 2.75 MB for CIFARIO/100.
Since the existing SSCL. papers do not present timing measures, we compare them using time com- plexity. Upon every batch of data, the CNNL algorithm repeatedly labels the unlabeled data for an unbounded number of iterations, denoted r. until it is confident of the labels. For ORDisCo, DistillMatch and DP-SSCL, the + D, OoD detector and weak labelers are trained on all data batches sequentially, so the time overhead is proportional to the number of batches. The conclusion is that DP-S has either lower the same time complexity to utilize unlabeled data for learning compared to existing SSCL. tools.
Table 4: Memory and time overhead to process unlabeled data in different SSCL methods.
ORDisCo
CNNL
DistillMatch
DP-SSCL
Source of mem overhead
G and D in GAN
data queue
OoD detector
WLFS
Mem complexity
((1) not reported ~ 1000B
((m)
O(1)
O(1)
Mem overhead (MNIST)
0.78MB
not reported
2.7MB
Mem overhead (CIFAR-10)
3.07MB
not reported
2.75MB
Mem overhead (CIFAR-100 100)
not reported
3.07MB
254MB
2.75MB
repeatedly labeling queue
training
training
Source of time overhead
training GAN O(b)
OoD detector
WLFs
Time complexity
O(b x r)
O(b)
O(b)
More experiments are run to show that DP-SSCL improves its performance on larger size un- labeled data per task, maintains stable learning on increasing number of tasks, and is sensitive to corrupted pscudo-labels. These studies are detailed in Appendix
6 CONCLUSTON
We designed an SSCL framework, namely DP-SSCL, that leverages a Da-based pseudo-labeler and a supervised CL module in cascade manner with feedback loop. This design allows us to obtain high quality pseudo-labels, shrinking the performance gap between SSCL. and supervised CL. Our framework also shows how CL can improve DP by allowing knowledge transfer from previous tasks to improve pseudo-labeling quality based on transferability metrics. Furthermore, the framework is compatible with many existing mature supervised CI. approaches, enabling trivial extension from supervised to sami-supervised CL. Our ablation studies show that the framework's 's performance de- pend on component selection, and succeeding research shall focus on exploring different component settings more thoroughly. Nevertheless, experiments show DP-SSCL is able output high lifelong learning accuracy and low forgetting, approaching that of supervised CL on fully labeled data, and outperforming existing SSCL approaches by up to 25% higher final accuracy and lower forgetting. with only 1% of memory overhead the same time complexity.

---

Under review conference paper ICLR 2023
REPRODUCIBILITY STATEMENT
Our experiment code is pushed to an anonymous GitHub repository bttpss//gihub.ccmddsssssaaa DPSSCL). All readers are more than welcomed to replicate our experiments.
REFERENCES
R. Aljundi, K. Kelchtermans, and T. Tuytelaars. Task-free continual learning. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 11246-11255, 2019.
M. Baucum, D. Belotto, S. Jeannet, E. Savage, Mupparaju, and C. W. Morato. Semi- supervise deep continuous learning. In Proceedings of the International Conference on Deep Learning Technologies, pp. 11-18, 2017.
Dhanajit Brahma, Vinay Kumar Verma, and Piyush Rai. Hypernetworks for continual semi- supervised learning. arXiv preprint a/Xiv:211000886, 2021.
A. Bulat, J. Kossaifi, G. Tzimiropoulos, and M. Pantic. Incremental momai-domain learning with net- work latent tensor factorization. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 2020.
A. Chaudhry, M. Rohrbach, M. Elhoseiny, T. Ajanthan, P. K. Dokania, H. S. Torr, and M. Ranzato. On tiny episodic memories continual learning. CoRR, abs/aab2..1495000 10486, 2019. URL http:////// org/abs/1902.110 10486
Z. Chen and B. Liu. Lifelong Machine Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Morgan & Claypool Publishers, 2016.
G. S. Dhillon, P. Chaudhari, A. Ravichandran, and S. Soatto. A baseline for few-shot image clas- sification. In Proceedings of the International Conference on Learning Representations (ICIR), 2020.
M. Farajtabar. N. Azizan, A. Mott, and A. Li. Orthogonal gradient descent for continual learning. CORR, abs/1910.07104. 2019. URL http://arrixoor////1111110001110
C. Fernando, D. Banarse, C. Blundell, Y. Zwols, D. Ha, A. A. Rusu, A. Pritzel, and D. Wierstra. Pathnet: Evolution channels gradient descent super neural networks. CoRR, abs/1701.08734, 2017. URL http://arxiv.org/////771..00
M. Gao, D. Qi, H. Mu, and Chen. A transfer residual neural network based on resnet-34 for detection of wood knot defects. Forests, 12(2):212, 2021.
C. Guo. G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. In Pro- ceedings of the International Conference on Machine Learning (ICML), pp. 1321-1332 2017a.
C. Guo. G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. Pro- ceedings of the International Conference on Machine Learning (ICML), 1321-1113 2017b.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
M. Hein, M. Andriushchenko. and J. Bitterwolf. Why relu networks yield high-confidence predic- tions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Stella Ho, Ming Liu, Lan Du, Yunfeng Li, Longxiang Gao, and Shang Gao. Semi-supervised con- tinual learning with meta self training. In Proceedings of the 3 ACM International Conference on Information & Knowledge Management, pp. 4024 4028, 2022.
P. G. Ipeirotis, F. Provost, V. Sheng, and J. Wang. Repeated labeling using multiple noisy labelers. Data Mining and Knowledge Discovery, 28(2):402444, 2014.
10

---

Under review conference paper ICLR 2023
Jaccard. Lois de distribution florale dans zone alpine. Bull Soc Vaudoise Sci Nat, 38:69-1300 130. 1902.
A. Krizhevsky, Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
LeCun and C. Cortes. Mnist handwritten digit database. 2010. URL http:://yyaan.eccm..oo//eeb mnist/
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient tradient-based learning applied to document recog- nition. Proceedings of the IEEE, 86(11), 1998.
S. Lee, J Stokes, and E. Eaton. Learning shared knowledge for deep lifelong learning using deconvo- lutional networks. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 2837-2844, 2019.
C. Li, T. Xu, Zhu, and B. Zhang. Triple generative adversarial nets. Advances neural informa- tion processing systems (NeurIPS), 30, 2017.
B. Liu. Lifelong machine learning: a paradigm for continuous learning. Frontier of Computer Science, 11(3):3993611, 2017.
D. Lopez-Paz and M. A. Ranzato, Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems (NeurlPS), volume 30. Curran Associates, Inc., 2017.
Yan Luo, Yongkang Wong, Mohan Kankanhalli, and Qi Zhao. Learning to predict gradients for semi-supervised continual learning. arXiv preprint arXiv:220..09196 2022.
A. Nguyen, J. Yosinski, and J. Clune. Deep neural networks easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 427-436, jun 2015.
C. V. Nguyen, T. Hassner, M. Seeger, and C. Archambeau. Leep: new measure to evaluate transferability of learned representations. In Proceedings of the International Conference on Machine Learning (ICML), 2020.
C. Olivier, S. Bernhard, and 7. Alexander. Semi-supervised learning. IEEF Transactions on Neural Networks, 20, 2006.
M. Pandy, A. Agostinelli, J. Uijlings, V. Ferrari, and T. Mensink. Transferability estimation using bhattacharyya class separability. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
J. C. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likeli- hood methods. Advances in Large Margin Classifiers, 10(3):61-74, 1999.
A. Ratner, S. H. Bach, H. Ehrenberg. J. Fries, S. Wu, and C. Ré. Snrokel: rapid training data creation with weak supervision. Proceedings of the VLDB Endowment, 11(3), 2016a.
A. Ratner, S. De Sa, C.and Wu, D. Selsam, and C. Ré. Data programming: Creating large training sets, quickly. Advances in Neural Information Processing Systems (NeurIPS), pp. 3574-3582, 2016b.
Ruvolo and E. Eaton. Active task selection for lifelong machine learning. Proceedings of the AAAI Conference on Artificial Intelligence, 27(1), 2013a.
P. Ruvolo and Eaton. ELLA: An efficient lifelong learning algorithm. Proceedings of the Inter- national Conference on Machine Learning (ICML.), 28(1), 2013b.
G. Saha, I. Garg, and K. Roy. Gradient projection memory for continual learning. Proccedings of the International Conference Learning Representations (ICLR), 2021.
B. Settles. Active learning literature survey. Technical report, Department of Computer Science, University of Wisconsin-Maddioon 2009.

---

Under review conference paper ICLR 2023
D. Silver, Q. Yang, and Li. Lifelong machine learning systems: Beyond learning algorithms. In 2013 AAAI spring symposium series, 2013.
J. Smith, Balloch, Hsu, and Z. Kira. Memory-effinn efficient semi- supervsue continual learning: The world is its own replay buffer. In Proceedings of the International Joint Conference on Neural Networks, 2021.
Y. Tan, Y. Li, and S. Huang. Otce: A transferabilityy metric for cross-domain cross-task representa- tions. Pruceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 15779-15788, June 2021.
A. T. Tran, C. V. Nguyen, and T. Hassner. Transferability and hardness of supervised classification tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), October 2019.
Varma and C. Ré. Snuba: automating weak supervision label training data. Proceedings of the VLDB Endowmens, 12, 2016.
Veniat, L.. Denoyer, and M. Ranzato. Efficient continual learning with modular networks and task- driven priors. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
L. Wang, K. Yang, C. Li. L. Hong. Li, and J. Zhu. Ordisco: Effective and efficient usage of incre- mental unlabeled data for semi-supervised continual learning. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5383-5392, June 2021.
J. Yoon, E. Yang. Lee, and S. J. Hwang. Lifelong learning with dynamically expandable networks. Proccedings of the International Conference Learning Representations (ICLR), 2018.
G. Zeng, Y. Chen, B. Cui, and S. Yu. Continual learning of context-dependent processing in neural networks. Nature Machine Intelligence, 18)):364-372, 2019.
Appendices
A PROOF OF SNUBA THEORETICAL GUARANTEE IN A LIFELONG SETTING
This section extends Snuba's 's guaranteed performance, equation in the main paper, to lifelong setting. For completeness, start by stating proposition:
Proposition 1 (Snuba's guarantee (Varma & Rě 2016). adapted to a lifelong setting). Consider consecutive tasks Z(1), 212)... .....or -for which Snuba was used obtain sets of committed weak labelers F()), F()) .... for tasks with corresponding empirical accuracies on x(6 as a vector a For each task, before ensembling. a factor graph-based generative model trains a set of fine-tuned weak labelers from F(t), denoted as B(0 and [ = =IF(),, that have accuracies on x(() (() and x(()) ay. with ai aL dg R plo and al is unknown. We have measured error ||a az 2(0) 11,0 S e00) If each labeler labels a minimum of
(8)
(8)
1 2 2(7 -(i log 22
1
5
data points in (5)
L with above some given confidence threshold v for all iterations, we can guarantee that la a/ 11o0 < for all iterations and all tasks with probability - 6.
Proof. We know that Snuba ensures the following bound the labeling performance of an individ- ual task 2 learned in isolation; the following guarantee is restatement from the original Snuba publication (Varma& & Ré 2016), modified only to include superscripts for the task index
12

---

Under review conference paper ICLR 2023
Proposition (Snuba's guarantee (Varma Ré 2016 an individual task 20)), Given a set F() of committed weak labelers by Snubu for task z(0, denote their empirical accuracies on x(,) as a vector of Before ensembling, a factor graph-based generative model trains set of fine-tuned weak labelers from 1o65, denoted as 1o65) and | | that have accuracies on x_ as a and on x as a with of ai 0i E TRF and di is unknown. We have a measured error [|a (8) L loo S e. If each labeler labels a minimum of
(
2 5
d 2 2(7 19)2 log
data points in x with above some given confidence threshold for all iterations, we guarantee that |a a 1l < ffor all iterations with probubility d.
See Varma Ré 2016) for the proof Proposition 2
We now need to show that this same guarantec holds a lifelong setting. Assume we have a sequence 'f consecutive tasks z(1, z(2) ..... Z Since the first task z is learned in isolation to produce the set of weak labelers F()) using Snuba, by Proposition we know that we have a measured error for the first task of
|1a - 21=11=0 (e(1
and that
| (1) al. 11o <
for iterations with probability 1 1- 8. Let us assume that bound holds for each f00) after learning ,0, 2 ..... 20;; we will show that this bound also holds for task 2(7+1),
From tasks z Z(T), Pnnbaa has learned set of weak labelers F = U'_, F(O, When creating the set of weak labelers I:(T+1) incrementally, Snuha has two choices at each iteration j. Either it can choose to add F(T+1) an existing weak labeler f E F F(T++1) or it can add previously unused weak labeler f' U - F(T+1), where U is the pool of all candidate weak labelers. Snuba chose f' over f. then weak labeler f" was assigned higher score than all other Similarly, if Snuba chose a particular f E F to add, then that f was assigned a higher score than all others. Let fy be the weak labeler chosen to add to F(T+1) at iteration j. either or f', which we know has maximum score of all weak labelers U F(T+1) Snuba obtains an accuracy on the labeled data set x ((T+1)
F.
(T+1) a Ld
Σ 100 (T+1)
0 (T-1))
(+(++L)
(T++1) such that fg achieves confidence greater than equal the confidence threshold each data point in X (T(++1) (T+1) is the true label, and o/
where x C X
(T+1) is the predicted label
1k
by the weak labeler fy.
Varma & Re
2016) (see their Equation 4) show that the probability of Snuba's failure to maintain loo < y one iteration
Pr[lla (T+1) 1 1100 +E
(T+1)
(4)
<2 2 exp(-2(1 c)² min( X
(T
T L.j
Following (Varma & Ré, 2016) and applying the union bound over the sequence of iterations to bound the probability of failure over all iterations used acquire F(T+1), we can obtain that
8 (T+1)) exp(-2(y ci (((++))
(5)
where d(T+1) min (I8
L.1 b..., ")). and so consequently ||a[ [T[TT1)) - oi ((T++))|
||x0 <
similarly holds for z (T++) for all iterations used to obtain c(T++1) with probability 6. By induction, this holds for the entire lifelong sequence.

---

Under review conference paper ICLR 2023
Task configurations used in the experiments in Section 5 of the main paper, with one additional committed WLF per iteration, 25 committed WLFs in total at maximum. An interval [a, b] means the hyperparameter is searched within this interval. Notice that 10% of the labeled and unlabeled data are hold-out for validation. Please refer to Figure for the WLF architectures.
Table 5:
Lafeking Samurio
Saguence
Tasks
of (9)
Labeler Architecture
Leaminn rate
Ratches
Epoctt
Bocosrup Size
Semmi-letenngseenss
Binary MNIST
45
18001800000000
Arch A Arch
[1e4, Te-2]
10
(00 100
30
1.0 2,
Hinryy CIBAR-10
45
[0.8e-3, 1e.5
350
incromental
5-way MNIST
1502575005000
Arch
[0.8633, 1.5e-3
220)
10-way CHEAR-10
20005000010000
Arch tout 110
Clas- ieseeinerematall
5-way CIFARR-10
Bizary CIEAR-10
20
40099000/000
50002000000
Arch B lout 5)
Arch
[0.8e-3, le-2
00.8e-3, le:3
30
20
60, 140
30, 60)
330
200
DETAILED EXPERIMENTAL SETUP
This section details the experimental setup used in Section of the main paper. Our lifelong learning experiments uses the MNIST, 10, and CIFAR- 100 benchmark datasets (LeCun & Cortes
2010 Krizhevsky 2009)
To transform these benchmarks into the lifelong setting, we created tasks as described Table S For example, to re-create the binary MNIST task sequence, one needs to first arrange the 45 tasks {0 1, 0 vs 2, .... 8 9]. Then, split the labeled training data, unlabeled training data, and testing data into /20/1 1880/2000 per task, ensuring that all data splits have balanced classes. We then hold out 10% of training data for validation. That is, only 108 labeled and 10692 unlabeled data participate in training, with 12 labeled and 1 1188 used for validation. Finally, perform our procedure described in Section to generate pseudo-labels as well as train a lifelong learner.
1.0
0.8
Accuracy
0.7
0.6
0.5
0.4
Bin MNIST
Bin CIFAR-10
10-way CIFAR-10
5-way CIFAR-100
5-way MNIST
Figure 3: Pseudo-labeling accuracies produced by Snorkel-ensembled labeler.
To train WLFs for data programming, keeps fine-tuning hyperparameters representing the WLF architectures, learning rate, and Snuba configurations in the given search spaces. Since the hyper- parameters keep adapting the current task even throughout single lifelong sequence, we report these hyperparameters and their constrained search spaces in Table 5 and Figure The search spaces were inspired by various previous works on CNN designs (Lecun et al., 1998 He et al. 2016). Nonetheless, the final architectures are much smaller.
Figure presents the pseudo-labeling accuracy by Snorkel-eneembled pseudo-labeler Ratner et al. (2016a) which is the hest labeler according to our ablation studies. The mean accuracies of hinary MNIST, binary CIFAR-10, and 5-way MNIST are all around 90%. Although the accuracies of
10-way CIFAR-10 and 5-way CIFAR-IOK are 60 65% due to difficulty, we can still outperform existing SSCL frameworks as demonstrated in Section [3.3]
We then detail our metric selection for the experiments. The performance of CL can be quantitatively
measured by (1) peak per-task performance metrics, (2) final task performance metries, and (3)
14

---

Under review conference paper ICLR 2023
cony pMo sigmee
Inppodd
cony pnd pe). sgreed
Mnspidd
Bimna aggnodd
diiaa aggnod
desac sutthnas
relo-022
1+b003
Eadoo
Architecture A: used for MNIST
caid, joct, pet aet
blokk cuLo116
blook 0ML
block_ ouL.
sugnoet 1 1oe2 peo, sec, rorten
cuted
berzhnorm
(h) Architecture B: used for CIFAR- experiments
axt pitakk out bakdremm.
con 1-8 p-t, s-t. bekenom
yotke aut baconom
cow, E Sel." bulcrom
ML solko JMLS bekenore
baldhror
(c) Block structure of Architecture where block out the out input into the block
Figure 4: WLF model templates at meta initialization of. Notations: out_c: output channels/number of filters, k size (width and height) of a filter, pt padding and stride.
forgetting metrics. After training on task 2(1), we measure the accuracy aj of the updated model when evaluated on the testing data of all known tasks {2 j s i]. Peak per-task accuracy (ee et al. 2019) at task (t) measures the average accuracy upon the first encounter of each task:
04 - Ej=1 aj and final accuracy (Lopez-Paz & Ranzato 2017) at task measures the average
(.) The retention of
performance of the current model on all tasks seen so far: 0
knowledge is measured by backward transfer (Ruvolo & Eaton 2013b Lopez-Paz & Ranzato 2017)
task (+) bt, = D 1-1 (8) - -a Note that òt, [-1, is the negative of the forgetting
metric from (Chaudhry et al. 2019). where positive values indicate improvement and negative values
indicate forgetting.
In addition the lifelong learning metries, measure pseudo-labeling accuracy and system head of DP-SSCL terms for memory cost. We also report the proportion weak labelers using knowledge of earlier tasks and the transferability score to analyze the effect of knowledge sharing on data programming.
C ADDITIONAL EXPERIMENT ANALYSIS OF LEARNING
CONTINUAL
This appendix details and provides additional results that complement Section and Section 5.3. We start with the ablation studies that are not covered in the main paper.
100
Plak Purtoos Acarory
1,00
22
0000
120
245
302
200
1205
(a) Peak Per-task Accuracy (b) Peak Per-task Accuracy (c) Relative Performance Binary CIFAR-10
(d) Relative Performance on Binary CIFAR-10
Binary MNIST
on Binary MNIST
Figure 5: SSCL performance different sizes of unlabeled data per task.
15

---

Under review conference paper ICLR 2023
1.00
1.000
0.99
0.979
0.68
0.350
0.97
00.925
0.96
0.900
k//00
0.34
0.95
13-33
<5 -3-53
-2-31 Tunk
0.887
10-10
-0--00
45 Treek
d 33 TrceL
0.93
0.850
0.92
<2-
13
0.91
Task
Task
(a) TF Binary MNIST
(b) DF-CNN Binary MNIST
0.90
0.85
0.85
0.80
0.80
0.75
0.75
0.70
0.70
0.63
0.60
-222
<
<0/=200
<8
0.55
<7 -223
0.55
r0/-200
<0 -32
r01-000
0.50
12'=1200
1233
0.50
<2/=2233
Truet
0.45
15
25
35
46
45
0.45
15
30
25
40
Task
(c) TF on Binary CIFAR-I0
(d) DF-CNN on Binary CIFAR-10
Figure 6: Learning Curve Comparisons. Dotted lines display accuracy of continual learning models trained on unlabeled data Xy with true labels (TrueL).
Sizes of Unlabeled Data We measure the performance of continual learners with the groundrruhhggenertted labels on different supervised methods while varying the quantity of unla- beled training data. Table 6 summarizes the empirical results, providing details in addition to Fig- 5re Figure 6 visualizes the learning curves of final accuracy these experiments, showing stable performance of lifelong learners regardless of the size of unlabeled data, number of tasks, and su- pervised modules.
Peak per-task accuracy of both continual models increases with more DP-SSCI- data. This supports the capability of DP-SSCL to capture the data distribution and assign representative labels. is well documented that continual models typically have increased negative transfer (i.e., infer- ence catastrophic forgetting) when trained more data. However, negative transfer increase is not amplified by using DP-SSCL--eneraaee labels place of true labels the continual performance with DP-SSCL- S------geeeeededd pseudo-labele achieves at least 96% of the continual performance when models are trained the same training data but with true labels instead. Consequently, DP-SSCL maintains stable performance increasing scale of unlabeled data. In real-world deployment, la- beled data has limited availability but unlabeled data is easy to be collected. Therefore, DP-SSCL's scalability supports practical continual learning.
Number of Tasks Figure 6 also demonstrates that DPCSS preserves relatively stable continual learning performance on increasing number of tasks under different supervised CL. modules and task sequences. This result entails that the quality of pscudo-labels generated by DP-SSCL is stable with respect to the number of tasks.
Pseudo-label Noise Levels We also examined the effect of introducing noise DP-SSCL's label generation process: randomly corrupting a portion of the generated labels among the unlabeled data. Figure 7 shows an inverse relationship between manually added pseudo-labe) noises and lifelong performance. This shows that CL sensitive to inaccurate pseudo-labels and using more accurate labeling, such as more diverse WLFs on DP, is necessary.
We next detail the ablation study on ensembling methods in Section 5.2
Ensembling Methods (More) We measure the performance of two continual learners, TF (Bulat al. 2020) and DF-CNN (Lee al. 2019), binary MNIST and binary CIFAR- experiments
16

---

Under review conference paper ICLR 2023
Table 6: Supervised continual learning on binary MNIST (top) and CIFAR-10 (bottom), showing mean + standard deviation. Performance is assessed using three metries, along with accuracy met- ries relative continual models trained on the same data with ground-truth labels instead. Note: here, backward transfer is depicted as percentage, and so is scaled by a factor of 100 to lie in range 1-100, 100] as compared to the metric definition in Appendix B
Relative to True Labels
MNIST Performance
ny
Per-Task Acc.
Final Acc.
Backward
Per-
Final
Transfer*
Task
Acc.
0 95.6+
96.8
1.9
30
0.
97.0=0.4
1.9001
0.99
1.00
TF
60
9..1
96.8
1.8
0.99
0.99
120
9..2
96.9
1.8
0.99
0.99
240
96.5+0.2
96.4
1.7
0.98
0.98
360
9..4
96.4
1.8
0.98
0.97
0
93.8
9..4
0.8
DF-CNN
30
04.61...3
96.210.4
1.510.2
0.98
1.00
60
9.
96.1
1.2
0.98
0.99
120
9..
95.8=0.9
0.8
0.98
0.99
240
9.2
94
0.
0.98
0.97
360
96.1
92.7
0.7
0.97
0.96
Relative to True Labels
CIFAR-10 Performance
(4)
YUU Per-Task
Final Acc.
Backward
Per-
Final
Acc.
Transfer"
Task
Acc.
81.5
76.
0.6
200
83.4
76.8+14
1.
0.99
1.00
TF
400
84.3
76.71113
16.6110
0.98
0.99
800
03.7
78.4+0.6
-..7
0.98
1.00
1200
86.1
77.5+13
17.0+14
0.97
0.98
0 50.7
80.4
0.240.7
DF-CNN
200
82.7
80.0+05
-2.++0.6
0.99
0.99
400
84.0
80.0
-2.7+0.4
0.99
0.99
800
85.3
80.5
-3.310.
0.98
0.98
1200
0..
80.1
-4.340.4
0.98
0.98
(45 binary classification tasks of pairs classes such as O vs 1, vs 2, ..., 8 vs 9). ran each MNIST and CIFAR-10 experiment over 10 and 5 random seeds and task sequences, respectively.
Table 7 shows all the peak per-task accuracy. final accuracy and backward transfer with respect to the escudo-labeling methods and the amount of training data. We see that Snorkel obtains consis- tently strong performance across continual learning algorithms, data sets, and amounts of unlabeled training data.
Finally, we discuss the learning curves in these experiments. More detailed experiment result statis- ties are listed Tables
and 10
Learning Curve We include learning curves for the instance-inerementaa semi-supervised lifelong experiments (Figure { and the class-incremental semi-supervised lifelong experiments (Figure 9 All training curves show final accuracy at task Z as defined Appendix B where task ( is the current task being trained. For easier tasks with high labeling accuracy, such as CIFAR-10, the DP-SSCL-ennhie semi-supervised continual learning methods perform similarly the equivalent
17

---

Under review conference paper ICLR 2023
98
80
96
Final Accuracy
Final Accuracy
75
94
92
65
60
0% Noise
0% Noise
88
20% Noise
55
20% Noise
50% Noise
50% Noise
45
30
60
120 nu
240
200
400
800
1200
ny
(a) Binary MNIST
(b) Binary CIFARIO
Figure 7: Semi-supervised DF-CNN with three levels of DP-SSCL. label noise, showing mean final accuracy.
Table 7: Comparison between pseudo-labeling methods, showing mean final accuracy = standard deviation. The training set for each task is a combination of the labeled data (120 and 400 images per task for MNIST and CIFAR-10, respectively) and specified quantity of unlabeled data. Models are trained in continual learning setting with labels the unlabeled data generated by one of three weak supervision methods.
MNIST Final
Accuracy
Continual Learner
Labeling Method
60
(
120
(i)
240
Majority Voting
96.5
89.1+ 12.6
94.8
DP-SSCL (TF)
Repeated Labeling
89.7
84.7
7..
Snorkel
97.0=
97.0+0.2
96.5
Majority Voting
96.0
95.6
13.3
DP-SSCL (DF-CNN) Repeated Labeling
91.
86.1
77.4
Snorkel
0.3
96.410.3
95.2
CIFARIO Final Accuracy
Continual Learner
Labeling Method
20
400
"e = 800 nu
1200
Majority Voting
7..41
7..0
7..5
DP-SSCL (TF)
Repeated Labeling
76.3
7..9
1..9
Snorkel
77.3
76.1
75.6
Majority Voting
78.1
77.0
76.0
DP-SSCL (DF-CNN) Repeated Labeling
7..0
75.2
74.4
Snorkel
7..1
78.610.7
7..510.8
supervised fulle-labeled task sequence. For further breakdown of results, include task-specific performance measures for all of the semi-supervised lifelong experiments Tables [L
18

---

Under review conference paper ICLR 2023
1.0
0.6
0.8
Accuracy
Accuracy
0.4
0.4
Mako-labeled
0.2
lakko-labeled
Fully-labeled
Fulle-l
0.2
10
20
30
batch
Instance-incrementaa batch
(a) 5-way MNIST Experiment
(b) 10-way CIFAR-I0 10 Experiment
Figure 8: Instance-ineremental semi-supervised vs fully supervised learning curve comparisons
0.9
0.9
Final Accuracy (
Accuracy
0.8
0.8
Final Accuracy
0.8
0.7
0.7
0.5
0.5
0.4
0.4
0.4
0.30
0.3
0.3 0
Task
Task
Task
(a) DF DF-CNN/Binary CIFAR-10
(b) TF/Binary CIFAR-10
(c) DEN/Binary CIFAR-10
0.7
0.7
0.6
0.6
0.6
Accuracy
Final Accuracy
Final Accuracy
0.5
0.5
0.5
0.4
0.4
0..4
0.3
0.3
0.3
0.2
0.2
0.2
0.1
10
15
20
0.1
10
15
20
0.1
10
15
20
Task
Task
Task
(d) DF-CNNNS--AA way CIFAR- 100
(c) TFy5-way CIFAR 100
DEN/5 way CIFAR-100
Figure 9: Class-incremental semi-supervised (blue) vs fully supervised (orange) learning curve com- parisons, showing standard deviation with shaded regions.
19

---

Under review conference paper ICLR 2023
Table Class-incremental Binary CIFAR-I0 experiment results broken down by task, showing mean i standard deviation
DP-SSCL (Semi- -umii-superviee)
Fully-labeled (Supervised)
Approach
Task
Final Ace.
Back. Transfer (%)
Final Acc.
Back. Transfer (%)
88.1 1.4
-4.3 1.2
86.7 5.3
-5.1
75.9 + 4.8
-4.6 5.2
76.6 + 1.7
-4.4 1.9
DF-CNN
84.4 2.0 2.0
-13
84.7 1.1
-1.1
93.2 0.5
-
93.5
0.5
92.5 + 1.1
0.0
92.6
0.0
80.5
7.0
79.0
13.5
67.8
6.7
+.0
4..1
TF
78.1
9.7
7..4
9.0
335
3.7
92.4
-18
4
92.9
0.0
92.8
0.0
0
0.7
0.9
41.7
-38.2 4.5
50.2
3.3
2..1
5.
-24.9
DEN
58.1 9.7
23.7
5.7
4.6
7.5
7.5
5..9
5.7.7
87.4
0.0
9.6
0.0
Table 9; Class- -lass-incremental 5-way CIFAR-10K experimental results with the DF-CNN, broken down by task, showing mean + standard deviation
DP-SSCL (Semi-superrvisel)
Fully-labeled (Supervised)
Approach
Task
Final Ace.
Back. Transfer (%)
Final Acc.
Back. Transfer (%)
33.9 + 5.4
-17.0 + 4.5
31.8 6.0
-21.6 + 6.4
35.1 + 6.5
-29.4 + 6.5
33.8 + 6.3
-35.8 6.5
2
32.8 + 4.0
-27.6 +: 4.6
32.8 5.7
-32.5 - 6.1
41.6 6.6
-23.4
3..
6.5
7.5
-33.5
4..0
730
5
4.4
5.3
5..7
7.3
49.3
-.0
51.5
-.2.7
46.7
-143432
53.6
-.4
57.2+3.1
-6.0
58.2+5.11
4..3
DE-CNN
64.2+27
2.
2.3
-5.5+3.2
3..1
-40041
7..1
-5.222.
11
3.5
337
2.5
-24
1.77
12.9
6..2
-.442.5
13
14
-17
55.4188
-4.9227
14
3.70
-1.2
36.7
-1.2+2.1
15
1.3
0.0
5.5
-1.4
48.1
-0.2
50.0
-10+24
17 5..
0.1
5..1
0.5
62.7
0.1
6..7
0.0
19
1.
0.0
1.9
0.0
20

---

Under review conference paper ICLR 2023
Table 10: Class- intremenatt 5-way CIFAR-100 experimental results with TF, broken down by task.
showing mean standard deviation
DP-SSCL (Semi-supervised)
Fully-labeled (Supervised)
Approach
Task
Final Ace.
Back. Transfer (%)
Final Acc.
Back. Transfer (%)
0
36.9 + 6.7
-13.5 + 7.5
38.8 + 9.1
-13.2 9.2
43.4 4.8
-19.8 = 5.6
49.9
-16.5 1.8
2
36.2 + 7.0
-20.5 + 6.8
6.8
3.0
23.9
3
5.2
5.2
5.5
4.99
40.6 13.2
-
42.4
-
5
5.4
7
50.9
-16.1 13.5
6 46.3
7.6
5.0
-.8.0
4.3
8.7
50.8
13.
8
8.7
7.3
8.5
+.0
9
60.4+7.1
7.0
58.5
-1211
TF
10
17.
11.8
14.
-11.3
11
4.3
7.5
46.9
6.9
12
53.
-7.
3.8
4.5
13
4..
-6.
8.2
9.0
14
3..0
2.0
31.6
-.44
15
41.5
6.1
48.1
-3.7
16
4..
2.7
45.2
4.3
17 4.3
5.2
2.22
18
62.8
-1.7 2.6
67.8 4
2.71
2.2
0.0
2.7
0.0
Table II: Class-incremental 5-way CIFAR-I00 experimental results with the DEN, broken down by task, showing mean + standard deviation
DP-SSCL (Semi- supeiisupervised)
Fully-labeled (Supervised)
Approach
Task
Final Acc.
Back. Transfer (%)
Final Acc.
Back. Transfer (%)
21.1 + 2.9
28.2 + 3.5
19.6 + 1.1
-3188 1.4
18.6 + 2.4
-41.6 = 2.3
22.7 2.3
-38.4 2.8
20.3 + 2.2
-35.6 = 1.6
2.4
3.5
3
20.1 + 1.6
-40.9 2.8
20.5 3.00
-43.0+33.3
4
24.5 4.7
-42.3 +.3 4.8
2.5
-47.4
1.4
1.5
2.58
2.3
1..
-3.8
19.6
-.5.1
22.
2.2
21.8
2.9
1.
3.77
21.8+1.3
1...0
24.0
222
22.0+27
44.1
DEN
10
1.87
2.3
23.1
-.0.4
2..9
-31.4229
22.8+2.5
-33
12 21
-3..1
2..3+-14
3.45
13
24.4
228.8
24.4+3.3
-31.9
2.
2.3
20.2
-10
15
22.8 3.0
-26.0
2..9
1.86
16
2.8
-26.3
21.1
-27.2
2.
-32
21.6+177
-3..6
18 23.1
2.6
23.0
-9.8
19
65.2
0.0
6.21
0.0

---

Under review conference paper ICLR 2023
D ADDITIONAL EXPERIMENT ANALYSIS OF DP-SSCL WITH KNOWLEDGE TRANSFER IN WEAK LABELERS
This appendix details and provides additional results that complement Section 5.2 with respect to the transfer of weak labeling functions across tasks. As described in Section 122 wc can utilize the previously trained weak labelers for the current task if the current task has enough similarity with some earlier tasks. The key component is the suitability score in Algorithm determining which earlier tasks or which earlier WLFs are sufficiently related to the current task.
The main paper showed experimental results using the LEEP transferability score as the suitability score, In this appendix, we experiment with other suitability scores. Specifically we compare the following similarity measures:
the LEEP transferability score, as used in the original paper. LEEP provides task-level information on the similarity of carlier tasks to the current onc.
LEEP Snuba score. Instead of only relying task transferability, this measure incorporates Snuba's s-score (Equation (T) measure how earlier WLFs perform on the current task prior to fine-tuning, Specifically, we computed it as swLe(w)
W is individual WLF, M is CL model and {az, aF,aya are weights of the sum. In this experiment, all terms were weighted identically (az = ap = a_ = 1/3), and the LEEP score was projected to be in the range [0. 1] for compatibility with the other terms via the exponential function ((- inf, 0) (0,1)).
og LEE,,,,M x VE FarrFl x()) -v2") + aa (w, x VE.
(8)
(8) where
the OTCE transferability score. OTCE(Tan et al. (2021) measures the similarity of earlier tasks the current with respect the distance between probability distributions.
OTCE + Snuba score. Similar to LEEP Snuba score, this measure incorporates the indi-
vidual performance of carlier WLFs well as task wisc relationships. We computed this (() (9)
score as swLr (w) ag OTCE(( (6) (8)
-V LLL (2xx0())) V2 (1:-11) ar
Vi
a jaccaradd ((0 ( where w E W is an individual WLF, x x(1a-1) and Vi. (14-1) are labeled data of earlier tasks, and {ag, at,, a are weights of the sum. In this experiment,
the OTCE score projected be in the range [0, by linear transformation, and all terms were weighted identically (as, = oF = =a 1/3).
We evaluated the different suitability scores in our approach on the CIFAR- 10 dataset, using 45 binary classification tasks formed by all pairs of two image classes (see Table 5}. As in the periments from Section 5.2 DF-CNN is used as the continual learner. For both experiments (n (8) (400, 200) and (m n (i)) = (50, 400). transferring WLFs makes the continual learner achieve equal or better peak per-task accuracy and final accuracy compared to the perfor- mance of DP-SSCL without WLF transfer. This experiment shows that this result is robust to the
choice of suitability score.
CIFAR-IO Experiment
Approach
Performance
Per-Task Acc.
Final Acc.
No Transfer
81.910.1
77.41
nL. 400
(4)
LEEP (ø- -0.5, p 0.6)
82.1+0.2
78.9
T 200
LEEP
Snuha Score (Ø = 0.7, p 0.6)
$2.1+02
7..8
OТCE (Φ = 0.8, p 0.6)
82.310.3
7..3
OTCE + Snuba Score (Ø 0.75, P 0.6)
82.510.2
7.4
n
No Transfer
7..
1.7
11
400
LEEP (ø- = -0.3, p = 0.4)
75.3+
72.4+0.7
LEEP
Snuba Score (Ø = 0.7. 0.6)
75.8
7..4
Table 12: SSCL performance when initializing WLFs via knowledge transfer
22


Reviews:
[{"review_id": "zXLS9og3i-0", "reviewer": "Reviewer_SCJ5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors have proposed a  Data Programming (DP) based Semi-supervised Continual Learning (SSCL)  framework which they call as DP-SSCL. Data programming (DP) is a known technique and its objective is to assign pseudo labels to unlabelled data points as a part of the semi supervised learning (SSL) process. The main idea of this paper is to first label unlabelled data points using DP and  then use those as additional (to already available labeled data) labels to further consolidate the model.\n\nThe proposed DP-SSCL framework follow the following steps (in the cascading order) including a feedback loop: (1) Meta (parameter) initialization, (2) Weak Labeling Function (WLF) initialization via knowledge transfer, (3) WLF training and pruning, and (4) Pseudo-labeler ensembling and continual model update. The proposed framework has been tested on  MNIST and CIFAR benchmark datasets and  compared against (a) some existing SSCL methods such as CNNL, ORDisCo, DistillMatch, and (b) Fully supervised Continual Learning with labeled data only.  A set of the performance metrics: 1) peak per-task performance, (2) final task performance, and (3) forgetting scores have been also researched and used in model comparisons.  Reported result is found to be promising. ", "summary_of_the_review": "I have gone through the paper more than once including the appendices. Overall, the idea is quite sound, well articulated through the document. The experiment is thorough and the reported results are encouraging. \n\nThere has been a major flaw identified in the experiment section 5 as some of the model comparisons were done using parameters that were selected based on the same data (see the weakness section). This might have over judged/scored the model performance. \n\nThe authors are suggested to redo their experiments with a proper setup for a valid comparison. I think this work has some value if this problem can be resolved.\n\n\n", "strength_and_weaknesses": "Strengths and weakness:\nStrengths: The paper is well written clearly covering the problem definition, the scope, the experimental and evaluation protocol, the evaluation metrics and the setup to compare with existing techniques in the field. It provides a set of thorough experiments that highlight the effectiveness of approach and the methodology. The reported results look not only better than existing approaches but sometimes close to the completely supervised setup (although tested on simple and small datasets). Also, DP-SSCL reports less memory and time complexity when compared to other existing approaches. In addition, the proposed framework is compatible with other Continuous Learning (CL) approaches so it can be easily evaluated and compared with others. \n \nWeakness: As reported in the paper, the Ensembling Methods and the WLF Transfer are two very important steps of the DP-SSCL framework. It looks the corresponding parameters of these two steps (section 5.3) were learned/chosen using the same data that has been used to report and compare (with the competitors) DP-SSCL performance in section 5.4. These underlying data should have been exclusive for steps 5.3 and 5.4, and therefore the results of section 5.4 may not be valid. It is suggested that authors redo these experiments by setting a disjoint set of data for these two steps. Same comments are applicable for the setup in Appendix B.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "sgSYKKgP_X", "reviewer": "Reviewer_HUWP", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a semi-supervised continual learning technique which leverages data programming to probabilistically label unlabeled data. The paper leverages methods proposed in Snuba to automatically generate labeling functions for new tasks. ", "summary_of_the_review": "I think the paper is strong and would benefit from comparison with some related work, some more experiments. I would also like the authors to highlight the novelty of their approach.", "strength_and_weaknesses": "*Strengths*\n1. The paper solves an important problem and has impressive results, beating previous reported state-of-the-art methods. \n2. I like the fact that the authors have taken time to analyze the memory and time complexity of their methods with baselines. \n3. The authors have a theoretical justification of the quality of labeling function proposed by the mechanisms used by Snuba under the continual learning settings.\n\n*Weaknesses*\n1. A quick google scholar search brings up a number of papers on semi-supervised continual learning [1, 2, 3, 4] which seem to not have been covered by the authors in either the literature review or as baselines. \n2. Comparison on Tiny-imagenet: Previous methods such as DistillMatch also compared their performance on Tiny-Imagenet. I believe that the paper would benefit from experiments on more datasets. \n3. I believe that the paper does not have extremely novel methodology. Having said that, I believe that the paper solves an important problem and does it well using practical methods.\n\nReferences:\n[1] Brahma, Dhanajit, Vinay Kumar Verma, and Piyush Rai. \"Hypernetworks for Continual Semi-Supervised Learning.\" arXiv preprint arXiv:2110.01856 (2021).\n[2] Luo, Yan, et al. \"Learning to Predict Gradients for Semi-Supervised Continual Learning.\" arXiv preprint arXiv:2201.09196 (2022).\n[3] Boschini, Matteo, et al. \"Continual semi-supervised learning through contrastive interpolation consistency.\" Pattern Recognition Letters 162 (2022): 9-14.\n[4] Ho, Stella, et al. \"Semi-supervised Continual Learning with Meta Self-training.\" Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "VPgt5NH0vqP", "reviewer": "Reviewer_5ri5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to apply Data Programming to semi-supervised Continual Learning (SSCL). The core idea is to first generate weak labeling functions (WLF) with existing tools like Snuba, where the labeling functions are then used to pseudo-label the unlabeled data points that are eventually all fed into training the downstream continual learning models. Experimental results show that this pipeline leads to improved performance over existing SSCL methods and performs more closely to the fully-supervised continual learning methods.", "summary_of_the_review": "I like the idea of applying Data Programming to SSCL, the current empirical results also show that this is a promising approach. However, I believe more comprehensive analysis on the proposed DP-SSCL pipeline can make the results stronger, for example, answering questions like what are the important aspects on selecting the transferability metric.", "strength_and_weaknesses": "Strengths:\n- The idea of applying data programming to continual learning is new. The connection and synergy between the two are natural, i.e., similar tasks in continual learning could share similar WLFs, making data programming specifically suitable for CL.\n- By adopting data programming, the underlying CL methods do not have to be modified. This makes the proposed pipeline flexible and able to enjoy improvements as new CL methods are developed.\n\nWeaknesses:\n- While the combination of CL and data programming is novel, the technical contribution appears to be more limited. “Transferability” could be one aspect where the authors could more carefully look into to improve over previously used metrics like LEEP. However, this seems to be not studied in depth.\n- Since the combination of DP and SSCL is new, Sec 5.3 should be one main focus of the paper, but the current set of experiments shown appears to be somewhat weak. For example, only one Transferability metric, LEEP, is tested in the experiments.\n- The entire DP-SSCL pipeline requires many different hyperparemeters to be set, including $\\phi$ and $\\rho$. How are the hyperparameters selected in the experiments?\n- Table 3 lacks one baseline that readers would be interested in seeing, i.e., CL with only the labeled data. It is also encouraged to show the dataset statistics, i.e., size of the labeled set, unlabeled set, test set, etc.\n- In the introduction, it is mentioned that applying DP to CL can be more robust to distribution shifts. However, it is not clear to me why this is the case, and I also didn’t find any experimental results supporting this.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "UNAtMxcnmj", "reviewer": "Reviewer_LGhA", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to use data programming to address the semi-supervised continuous learning, achieving the performance close to the fully supervised continual learning.", "summary_of_the_review": "Please refer to Strength And Weaknesses.", "strength_and_weaknesses": "Pros:\n* They consider a more realistic continual learning with only a few labeled data, and adopt a novel technique, data programming, to annotate these unlabeled data.\n\n* From the perspective of engineering, they greatly reduce the computational cost.\n\nCons:\n* This paper seems to lack a type of important baseline. They can incorporate the existing semi-supervised learning training strategies into the continual learning methods to overcome the weak supervision problem. They need to justify the advantages of DP-SSCL over such straightforward solutions.\n\n* I have a doubt about the meta initialization process. They use the task sequence as the prior knowledge to initialize hyperparameters and model architectures. However, for continual learning, could we obtain the full task sequence at once? If yes, why do not we use the full task sequence to perform standard supervised learning.\n\n* For more severe scenarios, we could not obtain qualified WLFs. Under such a setting, whether DP-SSCL is still useful?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject