Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: Authors state that the efficiency of traditional active learning (AL) should be significantly increased using approaches from continual learning (CL), when we consider past datapoints already used for training in AL as previous tasks. Authors test several standard CL approaches and newly proposed CL methods specifically designed for AL in the AL setting. On datasets FMNIST, CIFAR-10, MedMNIST, and Amazon Polarity, author demonstrate the ability to accelerate AL up to 2-6 times in some settings.
strength_and_weaknesses: + The paper is well written, the motivation is very clear.
- The main problem is the insufficient review of related work and previous approaches. I need not only a citation of previous work, but also a thorough review of the novelty of the proposed algorithm over the existing literature.  
clarity,_quality,_novelty_and_reproducibility: The paper is well written, the motivation is very clear.

The main concern is the seemingly poor review of related work and previous approaches. There are several papers that include the same terms active, continual, learning in their titles. Even though they can have different research goals, they previously considered CL and AL in the same contexts. I do not think that one can state the contribution “We first demonstrate that active learning can be viewed as a continual learning problem and propose the CAL framework” and, at the same time, do not even refer previous papers like "Continual Active Learning for Efficient Adaptation of Machine Learning Models to Changing Image Acquisition", "Continual Active Learning Using Pseudo-Domains for Limited Labelling Resources and Changing Acquisition Characteristics" by Perkonigg et al. and "Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts" by Zhou et al.

The idea of similarities between the CL and AL is not novel itself, and given that the idea of the paper is straightforward at some extent, the review of previous work should be definitely improved. At the current point, it is very difficult to asses the novelty of the paper.
summary_of_the_review: This paper is not ready for publication.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper suggests a set of selection criteria to training points in each AL step to avoid the so-called catastrophic forgetting which degrades model performance. A variety of heuristics is proposed some motivated by prior work and 2 new ones proposed by the authors (CAL-SD and CAL_SDS2).
strength_and_weaknesses: Strengths
1.	The problem tackled in this work is important improving the running time and performance of Active  Learning techniques.
2.	The Authors show in their experiments that indeed it is important to do continual learning and that acceleration can be obtained in this manner.
3.	The paper is well organized
Weaknesses
1.	My main concern is that the empirical results show the simple CAL-ER is the actual winner in acceleration in most cases, and often even the trade-off with accuracy makes the favorable one. Well,…that means that there is no point in any of the ‘intelligent’ criteria, but just to have a diverse set (which can be obtained by the simple uniform sampling of CAL-ER). I would like to get the authors opinion on this observation, please.
2.	There are some parts that are unclear. For example, 
a.	the setting of experiment in 3.3 isnt clear and confusing: its not clear if the model is fully trained first and with what data? What is the accuracy measured for? what does the statement ‘the model performs considerably worse on all of the tasks that is does on the test set? what are you measuring the performance on, if not the test set.
b.	what are “reasonable choices of m’ “?
3.	I don’t understand how the CAL-MIR selection of m’ is done. Is it done exhaustively over all the loss minimizer subsets? That prohibitive at some point, right?
4.	How is the selection of alpha and beta are done for (2) in the experiments?

clarity,_quality,_novelty_and_reproducibility: I find the paper Clarity problematic at some places, please see my comments above.
The novelty has a problematic aspect to it in the sense that the empirical results don't seem to support novelty of the newly proposed CAL criteria.
There is clearly some originality here but the empirical results dont seem to support its contribution.
I find the quality of the work satisfying.
summary_of_the_review: To this end I feel the paper still needs to be refined, and the empirical test show a trend that reduces the novelty and significance of the proposed sampling criteria for CAL. Nevertheless, this is an important problem and I would like to authors to continue its pursuance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes to investigate the computational efficiency of Active Learning (AL). The authors argue that typical AL methods are computationally intensive, as models are retrained from scratch at every round. They propose to reframe the AL setting as a Continual Learning one, which allows them to leverage the vast literature on replay-based CL. The authors conduct a series of experiments testing a variety of existing and novel rehearsal based methods in their new Continual Active Learning setup. Interestingly, the authors show that using CL methods in Active Learning not only provides significant (2-6x) computational gains, but does so without incurring a loss in performance. 
strength_and_weaknesses: ### Strengths 
1. The paper tackles an important, yet understudied, problem of active learning : the computation cost. Given the resources required for training every larger models, I agree with the authors, and believe that this cost should be addressed. This way, practitioners can compare the cost of labeling vs the cost of training models. 
2. The authors make an interesting (and novel to the best of my knowledge) connection with Continual Learning. By doing so, they show that standard CL techniques can be applied in the AL setting with success. 
3. The authors provide some experiments over different modalities and labelling budgets.

### Weaknesses
1. The paper is in dire need of a better discussion of the results. I say this for the following reason : The authors **show that using replay (CAL-ER) consistently outperforms regular Active Learning **. My understanding is that this finding directly contradicts the base experiment (figure 1) in (Ash & Adams) [1], where they found that doing full-replay from a warm-started model (CAL-ER), still underperforms a model trained from scratch on all the data (AL). In other words, I think the authors should provide insights (or at least discuss) how a CL model manages to outperform a model trained from scratch on all the data seen so far, which has long been seen as an upper bound in performance. 
2. On Catastrophic forgetting in AL (fig 1). How are the models cross-validated at every round in CAL ? I don't quite agree with using the word "catastrophic forgetting" to describe the behavior in figure 1, since the test accuracy (in general) gets better. It seems more like the the model is overfitting on the training samples of each round. 
3. The paper would greatly benefit from larger scale experiments. I think the question of compute efficiency is much more relevant in such settings. I understand the increased cost that's required to run these. 

[1] On Warm-Starting Neural Network Training, https://arxiv.org/pdf/1910.08475.pdf, Neurips 2020
clarity,_quality,_novelty_and_reproducibility: The writing is clear and easy to follow. The quality of the writing is good. That said, as mentioned before, a bigger focus needs to be put in the discussion of the results. Moreover, experimental details, like how the cross-validation of models was executed, and how many seeds were used in the experiments should go in the main paper. I appreciate that the authors detailed the hyperparameters in the appendix. 
The work and its perspective on the connection between an AL and CL is novel and original. 
summary_of_the_review: In summary, the authors address an important limitation in the field of Active Learning. I believe that with a more detailed analysis of the results, and a clear explanation as to why replay based methods can actually be more efficient and performant than their iid counterpart, would make this a strong paper. 
For now, given that several key things are missing from the paper, I think this paper still needs work. Should the authors provide more insight and a clear response to the issue raised above, I believe the paper will be worthy of acceptance. 


edit : after the rebuttal I have increased my score to a weak accept.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper introduces a continual learning technique for the active learning. In particular, a set of the continual learning techniques are adopted and integrated into each model training procedure at each cycle of active learning. The proposed CAL regards the newly-queried sample at each round a new task as in the continual learning setting, then apply continual learning techniques combined with warm-starting to active learning. As a result, CAL accelerates the active learning process by improving computational efficiency at the model training stage in active learning. The contribution is demonstrated with extensive experiments in benchmarks across different domains (NLP, vision, medical imaging, and computational biology).
strength_and_weaknesses: The idea of this paper is well-received.

My main concern is listed as follows.
1. the novelty. It is yet to be answered if the performance gain is truly brought by the active learning framework, or the promotion from the classifiers.
2. this paper may seem a bit incremental to me where a lot of existing techniques were combined into a whole.
3. following 2, the integration of the existing continual learning techniques are generally loose. In other words, how can we attain better querying policy via the integration of continual learning remains untapped.
4. the establishment of the new setting needs further explanation.

The other more detailed weaknesses are:

1. With all due respect, I may disagree with the author on the simple statement, "active learning can be viewed as a continual learning problem". While indeed that the active learning and continual learning seem to have something in common (i.e. new sample being added into the pool at every cycle), crucially a more vital stage of the active learning is the querying policy of attaining new batch of unlabeled data that are gauged to bring maximal performance gain. The paper didn't explain this aspect adequately.

2. Section 3 is occupied by the existing settings or techniques. It seems to me that the section 3.1 and 3.2 fit the background interpretation while section 3.3 describes the existing continual learning techniques. These, however, seems to very much orthogonal to the active learning.

3. Following 2, the major methodological innovation lies in the proposition of CAL-SD and CAL-SDS2. It generally lacks essential details that can correlate active learning and continual learning. For example, this paper emphasize a "stability-plasticity" problem in CAL-DS, but I don't see a clear connection with the active learning. The sub-modular sampling in CAL-SDS2 also seems orthogonal to active learning. To summarize, this paper seems to simply utilize the continual learning to an active learning setup incrementally.

4. The empirical analysis in section 4.2 is a bit hard for me to understand, with a surprisingly short piece of analysis. I suggest the authors to focus on CAL-SD and CAL-SDS2 and offer an ablation analysis accordingly, such as the impact of $\mathcal{L}_c$ and $\mathcal{L}_{\rm replay}$.

5. A major problem for active learning is the costly querying process, which is not shown in this paper because the authors only apply continual learning to uncertainty sampling strategies, while there are many existing. I suggest that the authors should consider other types of baselines. 
clarity,_quality,_novelty_and_reproducibility: Have met the standards.
summary_of_the_review: Based on the above comments, I recommend a reject on this paper.

This paper should significantly strengthen the integration of the active learning and continual learning, rather than simply superficially coupling both.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a method to accelerate active learning. Standard active learning requires retraining the model from scratch in every round after acquiring the additional annotated data. This paper proposes using ideas from continual learning to make retraining efficient as well as less forgetful as the model observes more and more annotated data (possibly from a shifting distribution).

While the paper is indeed looking at an interesting and important problem, the reviewers raised a number of concerns related to the experiments (results as well as the setting), lack of clarity, and novelty in general (note: one of the reviewers gave a score of 1 but their main point of concern was missing literature; this review was discarded from the consideration).

The authors provided a response which was discussed. However the concerns still remained and it was felt that, in its current form, the paper does not meet the acceptance criterion. The authors are advised to take into account the reviewers's feedback to improve the paper and resubmit to another venue.
justification_for_why_not_higher_score: The reviewers identified several issues which remained unresolved even after the authors' response and discussion.
Decision: Reject