Paper: Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling (ICLR-2024)

Reviewer comment (Theory):
If the algorithm is theoretically motivated, why is there no theories or lemmas in the main body?

Author response:
> If the algorithm is theoretically motivated, why is there no theories or lemmas in the mainbody?

The theoretical motivation for our algorithm is prominently presented in the main body of the paper, specifically in Section 3.3 through Proposition 1. The proofs for Proposition 1 are rigorously established by Lemma 1 and Lemma 2, both of which can be found in Appendix A.1. While we recognize the importance of detailed theoretical foundations, due to space constraints, we had to place Lemma 1 and 2 in the appendix.

Reviewer comment (Theory):
I would suggest adding more theoretical analysis to the main body for a simple method and moving a part of related work to the Appendix.

Author response:
We value your insightful feedback. After reviewing the post-rebuttal comment, we remain uncertain about the basis for the 'reject' rating.

As highlighted in our common response and individual response to the reviewer, we explained that simplicity is a strength, not a weakness. However, if the reviewer still looks at it differently, we are open to further discussion.

Additionally, we've included a theoretic motivation in Proposition 1, supported by a clear proof and additional insights in Appendix A.1. If the reviewer seeks more theoretical analysis, we kindly ask for specific aspect our work may lack in this regard.

We kindly request a reconsideration of the overall rating. If the suggestion to move related work to the appendix is the remaining concern, we believe a 'reject' may be too severe. Your re-evaluation would be greatly appreciated.

Reviewer comment (Experiments):
Missing reference to [Al-Shedivant et al. 2021](https://arxiv.org/pdf/2102.00127.pdf). This paper studies active learning for meta-learning at training time and proposes a hybrid informative and diverse clustering labeling approach using k-means++. I encourage the authors to include this active learning approach as an additional baseline in their experiments.

Author response:
Thanks for alerting us to this paper. Although they indeed focus on active learning in meta-training time, it is quite related to our work. We compared experimentally to their approach (along with other hybrid methods); results are discussed in the common response, but in short we substantially outperform their approach and other hybrid methods, probably because uncertainty methods perform so poorly in this very-low-budget regime.

Most uncertainty measures should be high near decision boundaries and/or outlier points; this is undesirable for low-budget settings, since they are hard to generalize from in “initial” stages of learning. Methods like our GMM-based proposal ensure that the context set points are “near” most of the target set, helping in the early phases of learning which are all that meta-learning considers. Hybrid methods considering both uncertainty and diversity are likely to help in middle-budget regimes, but not in extremely-low budget settings like meta-learning. We’ve added discussion of this point to Appendix L of the revised paper. 

Note that for this experiment, we applied their method only in meta-test time, for fair comparison to other approaches. Applying this category of approach in meta-training, including theirs as well as ours, requires a substantial computational overhead, since we must run clustering for each task considered in meta-training; we saw in Appendix J that doing so does not seem helpful in general.

We provide further analysis on Al-Shedivat et al. in the reviewer’s last question.

Reviewer comment (Novelty):
Limited technical novelty since theory and approach are straightforward. However, I am not placing too much weight on this since experimental results are strong.

Author response:
We appreciate the reviewer for valuing our strong experimental results. We agree that our proposed method is simple and straightforward, however, we want to emphasize that simple methods are not a limitation but rather a strength. Many existing active learning approaches, despite their improvement, require tuning several hyperparameters and complicated implementation details. For instance, as we discussed in Appendix F, the recent low-budget active learning technique ProbCover is highly sensitive to its difficult-to-tune radius value. On the other hand, GMM is extremely straightforward and easy to apply, with high-quality implementations widely available.

We would like to also emphasize that despite its simplicity and strength, GMMs have not been broadly used in the active learning community of late (including outside the very-low-budget regime): many papers use $k$-means variants, but our experiments show that GMMs often outperform $k$-means in these settings (please refer to the response to the fourth question of Reviewer r9Lv for details). Separately from the choice of clustering method, it also seems that most work “defaults” to hybrid methods (as did e.g. Al-Shedivat et al.) without further considering the tradeoffs. We believe that hybrid methods are indeed likely the best choice in medium- or high-budget settings, but as we demonstrate here, in the very-low-budget setting it seems that very simple selection criteria substantially beat other approaches. We believe these observations can bring new insights to the active learning community.

Reviewer comment (Theory):
Theoretical justification is very basic and presumed to hold in meta-learning case with ad-hoc justification.

Author response:
It is true that our theorem is of limited scope. We believe it is useful as an illustrative example for “why” the method can be a good idea, rather than presuming that the exact mechanism elucidated in the theorem applies for practical meta-learning regimes: indeed, Figure 5 shows a simple setting where the theorem fails. (Almost no theorems in machine learning theory precisely apply to practical deep learning settings.) We do believe, however, that the reasoning described in Proposition 1/Figure 4 and in the discussion around Figure 5 provide good motivation for the method.

Reviewer comment (Reproducibility):
For Tables 1, 2, 3, how many runs are used to compute error bars?

Author response:
As specified in the last sentence of the second paragraph of Section 4.1, we evaluated a single meta-learned algorithm on 600 meta-test tasks, and reported $95%$ confidence intervals across those tasks. This follows the widely-used convention in the meta-learning literature, including MAML, ANIL, SimpleShot, and more.

Reviewer comment (Evaluation):
Is there a hypothesis for why GMM helps at test time but not at meta-train time? This is an important discrepancy that is worth understanding deeper.

Author response:
We do agree that a deeper understanding of this problem is important. We have a hypothesis as to why this might be the case.

The GMM-based method provides context sets which give better-performing predictors. Doing this at meta-training time, however, makes a given task easier. With optimal context set selection, even mediocre features might yield a good meta-learned model on that task; the problem becomes “too easy,” and the meta-learned features will not give such clear classifications on harder problems. With worse selection of context sets, however, the meta-learner must learn better features, such that even mediocre labelings still achieve reasonable task performance.

We conjecture the improvement of Al-Shedivat et al. in meta-training time might be partially explained by this hypothesis. Recall that Al-Shedivat et al. create clusters of unlabeled samples using k-means++, then select the points from each cluster with the highest entropy. As discussed previously, this will tend to select points near cluster boundaries; the meta-learner should then find features that separate clusters very well, so that these points remain informative enough for the final classifier. 

One might be curious about the combination of Al-Shedivat et al. and our GMM method, since their method is for meta-training time and ours is for meta-test time. We thus trained a ProtoNet on MiniImageNet using Al-Shedivat et al., denoted as [1], rather than the Random selection as in the experiments in the paper. We then employed Random, [1], and GMM methods at meta-testing time.
As shown in the table below, models meta-trained with [1] perform somewhat better with [1]-based meta-testing than random meta-testing. When the model is meta-trained with random selection, the two meta-testing methods are about the same. For both meta-training methods, however, meta-testing with the GMM method substantially outperforms the other two approaches.

Usually ProtoNet is trained for 5,000 iterations; these results use only 1,000 or 2,000 iterations, since meta-training with [1] requires substantial computational overhead (mainly, running k-means++ for every task considered during training).


| Train $\rightarrow$ Test | [1] $\rightarrow$ Random | [1] $\rightarrow$ [1]    | [1] $\rightarrow$ GMM    | Random $\rightarrow$ Random | Random $\rightarrow$ [1]  | Random $\rightarrow$ GMM |
| ------------ | ------------ | ------------ | ------------ | --------------- | ------------- | ------------ |
| 1000 iter    | 33.40 $\pm$ 0.22 | 34.76 $\pm$ 0.24 | 40.65 $\pm$ 0.28 | 34.94 $\pm$ 0.24   | 33.64 $\pm$ 0.25 | 38.94 $\pm$ 0.26 |
| 2000 iter    | 36.17 $\pm$ 0.25 | 36.87 $\pm$ 0.25 | 40.76 $\pm$ 0.28 | 35.43 $\pm$ 0.26   | 36.30 $\pm$ 0.2)  | 42.49 $\pm$ 0.28 |

**Table. MiniImageNet using ProtoNet trained with [1] or Random selection**


[1] M. Al-Shedivat, L. Li, E. Xing and A. Talwalkar, “On data efficiency of meta-learning”,  AISTATS 2021.

Reviewer comment (Evaluation):
The primary technical contribution of this paper lies in the GMM-based acquisition function, which is suggested to outperform other acquisition functions in scenarios with extremely limited annotation budgets, as required by meta-learning. However, the results presented in Figure 2 do not convincingly demonstrate a substantial performance improvement of the proposed GMM-Based method over Typiclust.

Author response:
> The primary technical contribution of this paper lies in the GMM-based acquisition function, which is suggested to outperform other acquisition functions in scenarios with extremely limited annotation budgets, as required by meta-learning. However, the results presented in Figure 2 do not convincingly demonstrate a substantial performance improvement of the proposed GMM-Based method over Typiclust.

We would like to emphasize that our goal in Figure 2 was not to claim that the GMM-based method significantly outperforms other methods such as Typiclust; indeed, we described the results as saying that it “matches or outperforms other low-budget methods.” Figure 2 is a motivational experiment for active meta-learning, justifying that the GMM method is a reasonable active learning scheme in the very-low-budget but non-meta setting. In our main experiments for active meta-learning, in Section 4, we believe we do convincingly demonstrate a substantial performance improvement over Typiclust: for instance, the first results in Tables 1 and 2 show improvements over Typiclust varying between two and eight percentage points of accuracy.

Reviewer comment (Novelty):
Selecting the samples close to the cluster centre can better capture the diversity. However, several hybrid active learning approaches already consider both uncertainty and diversity. For example, BEMPS [1] integrates these aspects. To better highlight the advantages of the proposed methods, a more comprehensive examination within the context of active learning is advisable.

Author response:
> Furthermore, selecting the samples close to the cluster centre can better capture the diversity. However, it's important to note that several hybrid active learning approaches already consider both uncertainty and diversity. For example, BEMPS [1] integrates these aspects. To better highlight the advantages of the proposed methods, a more comprehensive examination within the context of active learning is advisable.

Thank you for this important suggestion. We added an experiment comparing our approach with hybrid active learning methods, with results listed in the common response. These results show that the proposed GMM-based method significantly outperforms hybrid methods, which we think was somewhat expected since uncertainty-based methods perform so poorly (often much worse than random selection) in these settings.

Most uncertainty measures should be high near decision boundaries and/or outlier points; this is undesirable for low-budget settings, since they are hard to generalize from in “initial” stages of learning. Methods like our GMM-based proposal ensure that the context set points are “near” most of the target set, helping in the early phases of learning which are all that meta-learning considers. Hybrid methods considering both uncertainty and diversity are likely to help in middle-budget regimes, but not in extremely-low budget settings like meta-learning. We’ve added discussion of this point to Appendix L of the revised paper. 

For BEMPS in particular, as we mentioned in the common response, it is hard to apply to typical meta-learning settings, since it requires a posterior over models. Bayesian active learning papers mostly compare only to other Bayesian active learning methods. We have added a citation to Section 3.1, however.

Reviewer comment (Reproducibility):
The reviewer has a question about the setup of active learning in meta-testing. Was the acquisition just run to acquire N*K samples? Or was the acquisition run multiple times until the total annotation budget N*K was exhausted? If the latter case, was the meta-trained model retrained after each acquisition iteration? And what was the batch size (number of samples acquired) used in each iteration?

Author response:
> Was the acquisition just run to acquire NK samples? Or was the acquisition run multiple times until the total annotation budget NK was exhausted? If the later case, was the meta-trained model retrained after each acquisition iteration? And what was the batch size (No. Samples acquired) used in each iteration?

Thanks for raising the question. As briefly mentioned in Section 3.1, we mostly considered the former scenario where we sample $N\times K$ samples at once. Although iterative sampling is more common in active learning, we focused on this scenario for the following two reasons:


1. Even when we iteratively label additional samples, for most meta-learning algorithms (other than MAML), this does not change the features we will use for our meta-learning methods. Even for other optimization-based methods such as ANIL, since the feature extractor is not updated during adaptation on a context set, the features stay the same for iterative process of active learning. As we demonstrated with ProtoNet in Figure 9 (c)-(d) in Appendix K (details about experiments in [1] below), when we iteratively add more labeled samples, the performance does not change much as the features do not change. In this case, selecting $N \times K$ samples at once is not very different from the iterative process, but it is computationally cheaper.
2. If we iteratively add labeled samples, we will quickly move beyond the few-shot regime of meta-learning, which is often not very practical in real world settings. Suppose we have a meta learner trained in $5$-way $1$-Shot. It is reasonable to add $5$ samples per iteration, since that is the minimum number to cover all the classes. But only after $5$ iterations, it will reach few-shot regime where we typically have $25$ labeled context samples. This is even less practical for $5$-Shot case.

We added the above explanation and experiments in Appendix K. If this is still not clear, we are eager to elaborate it further.

[1] We used MAML and ProtoNet trained on MiniImageNet in $5$-way $1$-Shot with randomly sampled context and target data. In meta-test time, we label $K$ (the number of shots) samples using various active learning methods at each iteration until the total number of context samples reaches $50$. Again, for each iteration, we evaluated on $600$ test tasks.