Title: NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coverage
Venue: NeurIPS 2023
Accepted: no
Keywords: active learning

low budget

neural tangent kernel

pseudo-label
Abstract: High annotation cost has driven extensive research in active learning and self-supervised learning. Recent research has shown that in the context of supervised learning, when we have different numbers of labels, we need to apply different active learning strategies to ensure that it outperforms the random baseline. This number of annotations that change the suitable active learning strategy is called the phase transition point. We found, however, when combining active learning with self-supervised models to achieve improved performance, the phase transition point occurs earlier. It becomes challenging to determine which strategy should be used for previously unseen datasets. We argue that existing active learning algorithms are heavily influenced by the phase transition because the empirical risk over the entire active learning pool estimated by these algorithms is inaccurate and influenced by the number of labeled samples. To address this issue, we propose a novel active learning strategy, neural tangent kernel clustering-pseudo-labels (NTKCPL). It estimates empirical risk based on pseudo-labels and the model prediction with NTK approximation. We analyze the factors affecting this approximation error and design a pseudo-label clustering generation method to reduce the approximation error. Finally, our method was validated on five datasets, empirically demonstrating that it outperforms the baseline methods in most cases and is valid over a longer range of training budgets.

Abstract

High annotation cost has driven extensive research in active learning and selfsupervised learning. Recent research has shown that in the context of supervised learning, when we have different numbers of labels, we need to apply different active learning strategies to ensure that it outperforms the random baseline. This number of annotations that change the suitable active learning strategy is called the phase transition point. We found, however, when combining active learning with self-supervised models to achieve improved performance, the phase transition point occurs earlier. It becomes challenging to determine which strategy should be used for previously unseen datasets. We argue that existing active learning algorithms are heavily influenced by the phase transition because the empirical risk over the entire active learning pool estimated by these algorithms is inaccurate and influenced by the number of labeled samples. To address this issue, we propose a novel active learning strategy, neural tangent kernel clustering-pseudo-labels (NTKCPL). It estimates empirical risk based on pseudo-labels and the model prediction with NTK approximation. We analyze the factors affecting this approximation error and design a pseudo-label clustering generation method to reduce the approximation error. Finally, our method was validated on five datasets, empirically demonstrating that it outperforms the baseline methods in most cases and is valid over a wider range of training budgets.

1 Introduction

The boom in deep learning models in recent years stems in part from the massive amounts of data [11, 17, 23]. However, the demand for large amounts of data, especially labeled data, in turn, constrains the application of deep learning models, since large amounts of labels imply high annotation costs [41, 1, 45]. Active learning is a path to alleviate the cost of labeling by selecting informative subsets of samples to annotate.

However, the benefits of active learning have been increasingly questioned in recent years [25, 28]. One of the main concerns is that training a model initialized by self-supervised learning with randomly selected labeled samples often yields results far beyond those obtained by existing active learning with supervised training (randomly initialized or initialized by the last round of the active learning model) [6, 8, 7, 14, 9]. Because the latter only uses labeled data to train the network, while the former uses a large amount of unlabeled data to train the backbone of the network. Since most existing active learning algorithms are designed in the context of supervised training, they must be validated with a large number of labels compared to the number of labels required in training from a self-supervised model. This means that the effectiveness of these active learning algorithms is not guaranteed in the case of having access to relatively few annotations, as is the case when combining with a self-supervised model. Several studies [15, 42, 4] have shown that many existing active

learning strategies fail to outperform the random baseline when combining them with self-supervised 37 learning. In this paper, we focus on designing an active learning strategy that works well in the 38 training method with a self-supervised model. 39

The 'phase transition' phenomenon [15] is known to occur in active learning with supervised training. 40 It refers to the fact that an active learning strategy that outperforms a random baseline when the total 41 number of labels is small will be inferior to a random baseline when the total number of labels is 42 large (called the low-budget strategy) and vice versa (called the high-budget strategy). We note that 43 when combining active learning with the self-supervised model, the cut-off point between low-budget 44 and high-budget strategy occurs much earlier. For example, in the CIFAR-100 [21], the cut-off point 45 is about 10,000 labeled samples when training in the supervised learning way [16]. But, the cut-off 46 point shifts forward to about 1,500 labeled samples when training from a self-supervised model. The 47 forward-moving cut-off point means that even if the annotation budget is low (only one order of 48 magnitude above the number of classes in the dataset), it is likely to hit that cut-off point. Thus, for a 49 previously unseen dataset, it is difficult to simply determine whether a low-budget or high-budget 50 strategy should be chosen since the difficulty varies from dataset to dataset. In this paper we use this 51 problem to motivate the design of an active learning strategy with a wider effective budget range. 52

53

54

55

56

57

58

59

60

61

62

63

Since existing low-budget strategies are designed based on the idea of feature space coverage [24, 15, 42], we first analyze the problems of determining coverage based on sample feature distances in sec. 2. After that, we propose that the true coverage where the empirical risk is zero, can be estimated based on pseudo-labels and predictions of the model trained on the candidate set. Based on this, we propose our active learning strategy, Neural Tangent Kernel Clustering-Pseudo-Labels (NTKCPL), which uses the NTK [18, 27] and CPL to approximate empirical risk on active learning pool in sec. 3.2. And we analyze which factor affects approximation error in sec. 3.3. Based on this analysis, we design a CPL generation method in sec. 3.4. Extensive experimental results demonstrate that our method outperforms state-of-the-art approaches in most cases and has a wider effective budget range. As part of the results (sec. 4) we also show our method is effective for self-supervised features of different quality.

64

65

66

67

68

69

70

71

Our contribution is summarized as follows: (1) We propose a novel active learning strategy, NTKCPL, by estimating empirical risk on the whole active learning pool based on pseudo-labels. (2) We analyze the approximation error of the empirical risk in the active learning pool when NTK and CPL are used to approximate networks and true labels. (3) Our method outperforms both low- and high-budget active learning strategies within a range of annotation quantities one order of magnitude larger than traditional low-budget active learning experiments. This means that our approach can be used more confidently for active learning on top of self-supervised models than existing low-budget strategies.

1.1 Related Work

Most active learning strategies are designed and validated in the high-budget scenario where network 72 weights are randomly initialized or initialized from the weights of the previous active learning 73 round. Active learning methods mainly include uncertainty-based sampling [22, 13, 19], feature 74 space coverage [32, 24, 42, 33, 5, 40], the combination of uncertainty and diversity [41, 3], learning75 based methods [43], and so on [34, 35]. Moreover, some recent studies explore 'look ahead' 76 strategies [26, 38], where samples are selected based on the model trained on candidate training sets. 77 However, with the development of self-supervised training, the training approach for low-budget 78 scenarios has shifted to training based on a self-supervised pre-trained model [24]. This change in 79 the training method implies a shift in the total number of samples that need to be selected by active 80 learning. When training based on a self-supervised model, often only 0.4-6% of the total data needs 81 to be labeled to achieve similar results to training with 20-40% labeled data on a randomly initialized 82 network [4]. Recent studies have shown that there exists a phase transition phenomenon in active 83 learning strategies, whereby opposite strategies should be adopted in high-budget and low-budget 84 scenarios [15], causing many active learning strategies designed for high-budget scenarios unsuitable 85 for training based on a self-supervised model. As a result, recent studies have explored active learning 86 strategies specifically designed for low-budget scenarios [15, 42, 31, 20]. However, we find that these 87 strategies are effective only when the number of labeled data samples is extremely small, and as we 88 increase the labeled data to one order of magnitude above the number of classes of the dataset, their 89 performance falls below that of the random baseline. 90

2 Insight: Distance is not an accurate indicator of empirical risk 91

The goal of the active learning is to find a labeled subset, D C = ( x i , y i ) N C i =1 , such that the model 92 trained on that subset, f D C , has the minimized empirical risk in the entire active learning pool, 93 D = ( x i , y i ) N i =1 as shown in eq. 1. 94

Unfortunately, during active learning, we do not have the labels of the entire active learning pool, so 95 we cannot compute this loss directly. To address this problem, current methods [32, 24, 33] covert 96 empirical risk minimization into feature space coverage based on Lipschitz continuity. Although 97 Lipschitz continuity guarantees that the difference between the model's predictions is less than the 98 product of the Lipschitz constant and the difference between inputs, it does not guarantee that their 99 predictions fall into the same class. In practice, we cannot determine the true coverage because we 100 do not know the distance threshold beyond which the model would change its predicted class for 101 unlabeled samples. 102

103

104

105

106

107

108

109

110

Therefore, the current solution is to minimize the coverage radius assuming full coverage [32] or to maximize coverage based on high purity coverage [42], where purity refers to the probability that the sample has the same label within a given distance. Assuming full coverage leads to an overestimated coverage as shown in fig. 1a, i.e., some covered samples still have a large empirical risk, while highpurity coverage causes underestimated coverage as shown in fig. 1b. The overestimated coverage may cause the active learning algorithm to miss samples in areas that are not truly covered, while underestimated coverage makes active learning algorithms likely to select redundant samples. These affect the performance of active learning.

(a) Coreset

(b) Probcover

(c) NTKCPL

Additionally, estimating the empirical risk based on distance implies the assumption that model 111 predictions are only relevant to the nearest labeled sample, which is often not the case in reality. To 112 estimate the true coverage, we propose a new strategy, NTKCPL. It estimates the empirical risk based 113 on the predictions of the model trained on the candidate set and pseudo-labels. 114

115

3 Method: NTKCPL

In sec. 3.1, we briefly review the Neural Tangent Kernel (NTK) [18] that enables active learning 116 strategies based on the outputs of a model trained on a candidate set feasible. Then, we propose our 117 active learning strategy, NTKCPL, in sec. 3.2 and analyze the approximation error of NTKCPL in 118 sec. 3.3. Finally, based on the analysis, we introduce the method of generating cluster pseudo-label in 119

3.1 Preliminaries 121

Neural Tangent Kernel (NTK) is a powerful tool to analyze the training dynamics of neural network. 122 Jacot et al. [18] show that the neural network is equivalent to the kernel regression with Neural 123 Tangent Kernel when network is sufficiently wide and its weights are initialized properly [2]. The 124 NTK, K , is shown in eq. 2, where the f denotes a neural network with parameters θ and X denotes 125 train samples. When training with MSE loss, the neural network has a closed-form solution for the 126 prediction of test sample x at iteration t as eq. 3, where Y denotes labels of trainset and f 0 denotes 127 the output of network with initialized weights. 128

Additionally, for active learning scenarios, Mohamad [26, 27] proposes the computation time of using NTK can be further reduced by considering the block structure of the matrix, which means that look ahead type active learning strategies can be implemented in a reasonable amount of time. For example, as shown in [26], if we want to use the look ahead active learning strategy, each active learning cycle takes 3 hours to train the entire network of 15 epochs on the MNIST dataset, while it takes only 3 minutes to use NTK with a block structure.

3.2 Framework

We propose a look ahead strategy, NTKCPL, to approximate the empirical risk on the whole active learning pool directly. There are two challenges: (1) estimate empirical risk without labels and (2) estimate predictions of models trained with candidate sets efficiently and accurately.

129

130

131

132

133

134

135

136

137

138

139

140

141

142

For the first challenge, clusters on self-supervised features provide good pseudo-labels. Because most samples in the same cluster have the same label [39]. And when the number of clusters is increased, it can improve the purity of clusters, where purity refers to the probability that the sample has the same label within the same cluster. We call these clusters clustering-pseudo-labels (CPL), y cpl .

For the second challenge, as introduced in sec. 3.1, NTK approximates the network well for random 143 initialization and the computation time is acceptable. However, in our scenario, training on top of the 144 self-supervised model, NTK does not approximate predictions of the whole network well. The main 145 reason is that weights of the neural network are initialized by self-supervised learning rather than 146 NTK initialization, i.e., drawn i.i.d. from a standard Gaussian [18]. In addition, the self-supervised 147 initialization provides the neural network with a powerful feature representation capability that is not 148 available in NTK. This leads to inconsistency between NTK predictions and network outputs. So, in 149 our method, the NTK is used to approximate the classifier instead of the whole network. And the 150 inputs of NTK are self-supervised features. Accordingly, we choose a training method following [24] 151 that freezes the encoder initialized by self-supervised learning and trains only the MLP as a classifier. 152 That training method achieves better or equal performance than fine-tuning the whole network in 153 the low-budget case while its prediction is more consistent with the results of NTK. We denotes the 154 predictions of NTK with trainset D C as ˆ f D C . Now, the active learning goal in eq. 1 is approximated 155 as eq. 4. 156

The algorithm is shown in Alg. 1. For computational simplicity and without loss of generality, we use 0-1 loss to calculate empirical risk in eq. 4. In each round of active learning, after computation of NTK based on eq. 2 and generation of CPL based on the method introduced in sec. 3.4, the sample that minimizes the empirical risk on the whole active learning pool after adding labeled set is selected.

3.3 NTKCPL Approximate Error

157

158

159

160

161

In this section, we analyze what affects the accuracy of NTKCPL estimates of empirical risk on 162 the whole active learning pool. The difference between the true empirical risk and the estimated 163

Algorithm 1 NTKCPL

1: Input: self-supervised feature f self , active learning feature f al labeled set L , unlabeled set U , budget b , initial budget b 0 , maximum cluster number C max , model prediction Y pre,t -1 at the last active learning

empirical risk using NTK and CPL is shown in eq. 5. The approximation error can be divided into 164 two terms, the first one is the difference between NTK and neural network prediction, error NTK , 165 and the second one is the difference caused by CPL during NTK estimation, error CPL . For the 166 error NTK , as we mentioned in sec. 3.2, NTK is used to approximate the classifier only to obtain 167 better consistency. To analyze error CPL , we start with the relationship between the predictions of 168 NTK trained with the ground truth, ˆ f y ( x i ) , and CPL, ˆ f cpl ( x i ) . 169

Definition Denotes the j th output of ˆ f cpl as ˆ f j cpl . Label mapping function g converts NTK's 170 predictions about CPL classes, ˆ f cpl ( x i ) , into predictions about true classes, ˆ f ymap ( x i ) , based on 171 dominant labels within corresponding CPL classes as shown in eq. 6, where D dom is a set of index k , 172 where j is the dominant true label classes within CPL class, y cpl,k . 173

Proposition If the true labels of labeled samples are the dominant labels in their corresponding 174 CPL clusters, ˆ f y ( x i ) = g ( ˆ f cpl ( x i )) . We defer the proof to appendix 1. 175

.

According to the proposition, we argue argmax f y ( x i ) is most likely equal to g ( argmax f cpl ( x i )) . 179

As mentioned in sec. 3.2, we use 0-1 loss to calculate empirical risk. We can expand error CPL as 176 eq. 7, where we denote the probability that the NTK prediction agrees with the y but not with y cpl as 177 P fnf , and the probability that the NTK prediction does not agree with y but agrees with y cpl as P nff 178 ˆ ˆ

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

P fnf refers to the case where different CPL classes correspond to the same true label class, i.e., over-clustering. P nff means that the true label of a sample is different from the dominant true label within its CPL class, i.e., the CPL class includes samples from different true label classes, which is called impurity. Detailed explanations and empirical evidence can be found in appendix 1.

3.4 Cluster Pseudo-Labels

As shown by eq. 7, the effect of CPL on the approximation error comes from the purity of the clusters and over-clustering. To improve clustering purity, we take two approaches: (1) clustering on the active learning feature, i.e., the output of the penultimate layer of the classifier, and (2) increasing the number of clusters. However, increasing the number of clusters may cause the labeled samples not to cover all classes of the CPL (under-coverage) and also increase the over-clustering error. For example, a group of samples with the same true label is clustered into K different classes. Even though NTK incorrectly predicts some samples as other CPL classes, their true empirical risk is zero.

To improve the under-coverage, we set the number of clusters to half of the total number of labels, i.e., each cluster includes two labeled samples on average. To improve the over-clustering, we manually set the maximum number of clusters and design a clustering-splitting approach instead of directly increasing the number of clusters. It splits the low-purity clusters and keeps the high-purity ones to reduce the extra overclustering errors within samples located in the high-purity clusters. Specifically, we use the prediction of the neural network in each round of active learning to estimate the number of confusing samples within each cluster, i.e., the number of samples from classes that are different from the dominant class. The clusters that contain the

Algorithm 2

largest number of confusing samples are split sequentially until a predefined number of clusters is reached. The cluster splitting algorithm is shown in Alg. 2, where we adopt the constrained K-Means [37] to improve the clusters from labeled sample constraints.

4 Experiment Results

Our approach is validated on five datasets with various qualities of self-supervised features. Datasets with good self-supervised features, such as CIFAR-10 [21], CIFAR-100 [21], and ImageNet-100 (a subset of ImageNet [11], following splitting in [36]), are included. SVHN [29] with poor selfsupervised features is also included. Additionally, we consider practical scenarios where the total number of samples in the trainset is insufficient to support effective self-supervised training, such as Oxford-IIIT Pet dataset [30]. In this case, we evaluated the effectiveness of our method based on the model pre-trained on ImageNet [11].

Baseline We compare our proposed method with representative active learning strategies: (1) Random, (2) Entropy (uncertainty sampling, maximum entropy of output) [22], (3) Coreset (diversity active learning strategy, greedy solution of minimum coverage radius) [32], (4) BADGE (combination of uncertainty and diversity, kmeans++ sampling on grad embedding) [3], where the scalable version [10, 12], badge partition, is used in ImageNet-100, CIFAR-100 and Oxford-IIIT Pet because the huge dimension of grad embedding (5) Typiclust (designed for low-budget case) [15], (6) Lookahead (maximum output change based on NTK) [26].

Implementation Our method focuses on the low-budget regime, we followed the training method in [24], freezing weights of backbone initialized with self-supervised learning and then training a MLP as the classifier. The hyperparameters for training are set following [15] and can be found in appendix 3. For the self-supervised model, we adopt simsiam [9] for CIFAR-10, CIFAR-100 and SVHN and BYOL [14] for ImageNet-100 and Oxford-IIIT Pet. Resnet-18 [17] is used in CIFAR-10

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

Random

Entropy

Coreset(self)

BADGE

TypiClust

LookAhead

NTKCPL(self)

NTKCPL(al)

20

41.80

±

3.82

38.58

±

2.86

20.08

±

2.75

39.85

±

3.91

46.38

±

1.61

40.93

±

4.04

54.31

±

3.74

52.67

±

3.70

40

57.52

±

3.34

51.10

±

4.21

36.67

±

6.29

54.99

±

3.43

66.18

±

2.45

58.55

±

2.71

68.60

±

2.50

63.55

±

2.89

60

65.88

±

3.07

64.46

±

3.42

46.39

±

7.41

65.23

±

1.40

72.93

±

1.77

66.96

±

2.90

75.09

±

1.69

72.22

±

2.11

80

69.35

±

3.31

70.49

±

3.05

58.96

±

6.15

70.76

±

1.86

76.98

±

1.04

72.71

±

1.94

78.51

±

1.61

75.32

±

0.92

100

74.11

±

1.16

74.34

±

1.92

62.64

±

5.07

75.40

±

0.99

78.24

±

1.28

75.97

±

2.04

80.30

±

1.17

78.45

±

1.19

200

80.90

±

0.90

79.86

±

1.77

76.93

±

3.56

82.20

±

1.14

83.16

±

0.61

81.89

±

1.31

83.77

±

1.04

81.87

±

1.02

300

82.80

±

0.93

81.43

±

2.23

82.64

±

1.42

84.53

±

0.46

84.16

±

0.25

83.29

±

0.89

85.00

±

0.54

83.78

±

1.05

400

84.04

±

0.49

83.37

±

1.31

84.56

±

1.15

84.75

±

0.40

85.13

±

0.27

84.59

±

0.59

85.64

±

0.38

84.73

±

0.85

500

84.97

±

0.78

84.24

±

0.89

85.23

±

0.59

85.57

±

0.51

85.37

±

0.15

85.31

±

0.12

85.72

±

0.22

85.48

±

0.65

1000

86.26

±

0.38

84.94

±

0.48

86.75

±

0.36

86.06

±

0.31

86.07

±

0.14

85.69

±

0.47

86.83

±

0.33

87.15

±

0.57

1500

86.95

±

0.27

85.85

±

0.39

87.03

±

0.13

87.05

±

0.36

86.37

±

0.11

86.82

±

0.23

87.18

±

0.41

87.58

±

0.29

2000

87.30

±

0.37

86.92

±

0.15

87.34

±

0.27

87.31

±

0.47

86.55

±

0.21

87.16

±

0.19

87.34

±

0.41

87.87

±

0.39

and SVHN, WRN28-8 [44] is used in CIFAR-100 and Resnet-50 [17] is used in ImageNet-100 and Oxford-IIIT Pet.

The number of clusters in our method is set according to three rules, in the initial selection, it is set to the number of query samples, after that it is set to half of the query samples until the number of clusters reaches the maximum number of clusters. For CIFAR-10, CIFAR-100, ImageNet-100, SVHN, and Oxford-IIIT Pet, the maximum number of clusters is 100, 500, 300, 100, and 150, respectively. We followed [26] to sample a subset of the unlabeled set as the candidate set to select samples and estimate coverage. The candidate set includes 10,000 samples.

For the query step, most of the experiments (those on CIFAR-100, SVHN and Oxford-IIIT Pet) following the active learning literature by drawing a fixed number of samples from the unlabeled dataset to the oracle. Specifically, 500 for CIFAR-100, 20 for SVHN, and 40 for Oxford-IIIT Pet. We empirically found that fixed active learning query steps lead to much faster growth of classifier accuracy in the early stages of active learning (the amount of labels is about 10 times than the number of class) than in the later stages, so it is difficult to clearly observe the differences between different active learning strategies. For this reason, we empirically set varying query steps in our experiments with CIFAR-10 and ImageNet-100. Smaller query steps were used in the early stage of active learning and switched to larger query steps in the later stage. Specifically, for CIFAR-10, 20 samples are queried before 100 labels are available, 100 samples are queried before 500 labels and 500 labels are queried before 2000 labels. For ImageNet-100, 200 samples are selected before 1000 labels are available and 500 samples are queried before 2000 labels.

4.1 Main Results

All experiments were run 5 times and the avg. and std. are reported. Considering that the experiments are conducted for the scenario with a low annotation budget, it is not practical to construct a validation set to select the best checkpoints (the benefits of constructing a validation set are much less than using these labeled samples as training samples). Therefore, we report the final checkpoint accuracy, not the accuracy of the checkpoints determined by the validation set. The results are shown in fig. 2 and table 1. The detailed results are in appendix 5.

NTKCPL outperforms SOTA. As shown in table 1, fig. 2. In most cases, our proposed method outperforms the baseline methods. For the few cases with only a small number of labels, our method shows comparable performance with the low-budget strategy, TypiClust, such as in CIFAR-100 with 500 and 1000 labeled samples, and Oxford-IIIT Pet with 80 and 100 labeled samples.

NTKCPL still shows good performance when the self-supervised features do not correspond well to the label classes. Since the loss of self-supervised training is different from that of image classification, self-supervised features do not always correspond well with label classes. In SVHN dataset, self-supervised features of different classes are mixed together because the images include some irrelevant digits on both sides of the digit of interest [29]. Our method is similar to other

1000

2000

3000

4000

45

50

55

60

Accuracy / %

Random

Coreset(al)

Coreset(self)

TypiClust

Entropy

BADGE

NTKCPL(al)

(a) CIFAR-100

500

1000

1500

2000

50

60

70

80

Accuracy / %

Random

Coreset(self)

TypiClust

Entropy

LookAhead

BADGE

NTKCPL(self)

NTKCPL(al)

(b) ImageNet-100

50

100

150

200

40

60

80

Accuracy / %

Random

Coreset(self)

TypiClust

Entropy

LookAhead

BADGE

NTKCPL(self)

NTKCPL(al)

(c) SVHN

100

200

300

400

40

50

60

70

80

Accuracy / %

Random

Coreset(self)

Coreset(al)

TypiClust

Entropy

BADGE

NTKCPL(self)

NTKCPL(al)

(d) Oxford-IIIT Pet

baseline strategies at the beginning of active learning, but it shows better results than baselines after 267 several active learning rounds as shown in fig. 2c. 268

269

270

271

272

Another common scenario is the lack of sufficient samples to support effective self-supervised training. To evaluate in this context, we choose the Oxford-IIIT Pet dataset with the self-supervised model trained on ImageNet. The result is shown in fig. 2d. Our method has similar accuracy in the first three rounds as the TypiClust and outperforms all baseline methods afterward.

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

NTKCPL has a wider effective budget range than SOTA. Active learning based on self-supervised models exhibits an intensified phase transition phenomenon. We plot the active learning gain of our method and baselines on different datasets in fig. 3. The average accuracy of our method, NTKCPL(al), outperforms the random baseline at all quantities of labels. In contrast, both the typical high-budget strategy, BADGE, and low-budget strategy, TypiClust, appear to be worse than the random baseline over a range of annotation quantities. We show

the effective budget range of our method, NTKCPL, as well as the typical high-budget strategy, BADGE, and the typical low-budget strategy, TypiClust, across all experiments in table 2. The effective budget ratio refers to the proportion of the effective annotation quantity to the total annotation quantity, where the effective annotation quantity refers to the number of annotations at which active learning accuracy exceeds the random baseline (avg. + std.).

4.2 Ablation Study

In this section, we evaluate the coverage estimation of our method and the effect of the maximum cluster number on NTKCPL. Also, we compare the effect of generating CPL on self-supervised features as well as on the active learning feature on the performance of NTKCPL.

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

1000

2000

3000

4000

-4

-2

0

2

4

6

Active learning Gain / %

(a) CIFAR-100

500

1000

1500

2000

-10

0

10

Active learning Gain / %

(b) ImageNet-100

50

100

150

200

-10

0

10

Active learning Gain / %

(c) SVHN

100

200

300

400

-10

-5

0

5

10

15

Active learning Gain / %

(d) Oxford-IIIT Pet

1000

2000

3000

4000

50

55

60

65

70

Coverage / %

NTK with ture label

NTK with CPL

NN with true label

500

1000

1500

2000

84

85

86

87

88

Accuracy / %

max # cluster 10

max # cluster 50

max # cluster 100

max # cluster 200

Coverage Estimation We conducted experiments on CIFAR-100, where the coverage indicates the proportion of samples that are correctly predicted. The estimated coverage of NTK with true label and with CPL is shown in fig. 4. Our method approximates the true coverage well for most cases.

Effect of the Maximum Number of CPL The ablation experiments are conducted on CIFAR-10. We plot the accuracy when the number of annotations selected by active learning is greater than 400 as shown in fig. 5. In this range, the number of classes of CPL is fixed at 10, 50, 100, and 200, respectively. The experimental results support our analysis in sec. 3.4 that too many or too few clusters will increase the approximation error, which affects the performance of active learning.

Effect of self-supervised feature-based and active learning feature-based clustering-pseudolabels on NTKCPL. We denote NTKCPL based on active learning features as NTKCPL(al) and NTKCPL based on self-supervised learning feature as NTKCPL(self). The results are shown in table 1 and fig. 2. From these experiments, we found that clustering on active learning features yields better results except for the case where the number of annotations is very small. Also, NTKCPL(self) is better than NTKCPL(al) in a wide range of annotation quantities (no more than 500), when self-supervised features are good such as experiment in the CIFAR-10.

5 Conclusion

We study the active learning problem when training on top of a self-supervised model. In this case, an intensified phase transition is observed and it influences the application of active learning. We propose NTKCPL that approximates empirical risk on the whole pool more directly. We also analyze the approximation error and design a CPL generation method based on the analysis to reduce the approximation error. Our method outperforms SOTA in most cases and has a wider effective budget range. The comprehensive experiments show that our method can work well on self-supervised features with different qualities.

Our approach is limited to the fixed training approach, i.e., training the classifier on top of a frozen 315 self-supervised training encoder, which is restricted to the low-budget scenario because the fine316 tuning training approach provides higher accuracy in the high-budget case. Therefore, (1) how to 317 accurately approximate the fine-tuning model initialized with self-supervised weights using NTK and 318 (2) whether the samples selected by our current method have good transferability for the fine-tuning 319 would be interesting future directions. 320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

References

Inigo Alonso, Matan Yuval, Gal Eyal, Tali Treibitz, and Ana C Murillo. Coralseg: Learning coral segmentation from sparse annotations. Journal of Field Robotics , 36(8):1456-1477, 2019.

Sanjeev Arora, Simon S Du, Wei Hu, Zhiyuan Li, Russ R Salakhutdinov, and Ruosong Wang. On exact computation with an infinitely wide neural net. Advances in neural information processing systems , 32, 2019.

Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In International Conference on Learning Representations , 2020.

Javad Zolfaghari Bengar, Joost van de Weijer, Bartlomiej Twardowski, and Bogdan Raducanu. Reducing label effort: Self-supervised meets active learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1631-1639, 2021.

Zalán Borsos, Marco Tagliasacchi, and Andreas Krause. Semi-supervised batch active learning via bilevel optimization. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 3495-3499. IEEE, 2021.

Yao-Chun Chan, Mingchen Li, and Samet Oymak. On the marginal benefit of active learning: Does self-supervision eat its cake? In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 3455-3459. IEEE, 2021.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning , pages 1597-1607. PMLR, 2020.

Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big selfsupervised models are strong semi-supervised learners. Advances in neural information processing systems , 33:22243-22255, 2020.

Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15750-15758, 2021.

Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Rostamizadeh, and Sanjiv Kumar. Batch active learning at scale. Advances in Neural Information Processing Systems , 34:11933-11944, 2021.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Zeyad Ali Sami Emam, Hong-Min Chu, Ping-Yeh Chiang, Wojciech Czaja, Richard Leapman, Micah Goldblum, and Tom Goldstein. Active learning at the imagenet scale. arXiv preprint arXiv:2111.12880 , 2021.

Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning , pages 1183-1192. PMLR, 2017.

Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neural Information Processing Systems , 33:21271-21284, 2020.

Guy Hacohen, Avihu Dekel, and Daphna Weinshall. Active learning on a budget: Opposite strategies suit high and low budgets. In International Conference on Machine Learning , pages 8175-8195. PMLR, 2022.

Guy Hacohen and Daphna Weinshall. Misal: Active learning for every budget. 2023.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization 367 in neural networks. Advances in neural information processing systems , 31, 2018. 368

Andreas Kirsch, Joost Van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition 369 for deep bayesian active learning. Advances in neural information processing systems , 32, 2019. 370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

Seo Taek Kong, Soomin Jeon, Dongbin Na, Jaewon Lee, Hong-Seok Lee, and Kyu-Hwan Jung. A neural pre-conditioning active learning algorithm to reduce label complexity. Advances in Neural Information Processing Systems , 35:32842-32853, 2022.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

David D Lewis and Jason Catlett. Heterogeneous uncertainty sampling for supervised learning. In Machine learning proceedings 1994 , pages 148-156. Elsevier, 1994.

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10012-10022, 2021.

Rafid Mahmood, Sanja Fidler, and Marc T Law. Low-budget active learning via wasserstein distance: An integer programming approach. In International Conference on Learning Representations , 2022.

Sudhanshu Mittal, Maxim Tatarchenko, Özgün Çiçek, and Thomas Brox. Parting with illusions about deep active learning. arXiv preprint arXiv:1912.05361 , 2019.

Mohamad Amin Mohamadi, Wonho Bae, and Danica J Sutherland. Making look-ahead active learning strategies feasible with neural tangent kernels. In Advances in Neural Information Processing Systems , 2022.

Mohamad Amin Mohamadi and Danica J Sutherland. A fast, well-founded approximation to the empirical neural tangent kernel. arXiv preprint arXiv:2206.12543 , 2022.

Prateek Munjal, Nasir Hayat, Munawar Hayat, Jamshid Sourati, and Shadab Khan. Towards robust and reproducible active learning using neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 223-232, 2022.

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.

Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition , pages 3498-3505. IEEE, 2012.

Kossar Pourahmadi, Parsa Nooralinejad, and Hamed Pirsiavash. A simple baseline for low-budget active learning. UMBC Student Collection , 2022.

Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations , 2018.

Changjian Shui, Fan Zhou, Christian Gagné, and Boyu Wang. Deep active learning: Unified and principled method for query and training. In International Conference on Artificial Intelligence and Statistics , pages 1308-1318. PMLR, 2020.

Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5972-5981, 2019.

Toan Tran, Thanh-Toan Do, Ian Reid, and Gustavo Carneiro. Bayesian generative active deep learning. In International Conference on Machine Learning , pages 6295-6304. PMLR, 2019.

Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool. Scan: Learning to classify images without labels. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part X , pages 268-285. Springer, 2020.

Kiri Wagstaff, Claire Cardie, Seth Rogers, Stefan Schrödl, et al. Constrained k-means clustering with background knowledge. In Icml , volume 1, pages 577-584, 2001.

Haonan Wang, Wei Huang, Ziwei Wu, Hanghang Tong, Andrew J Margenot, and Jingrui He. Deep active learning by leveraging training dynamics. Advances in Neural Information Processing Systems , 35:25171-25184, 2022.

Ziting Wen, Oscar Pizarro, and Stefan Williams. Active self-semi-supervised learning for few labeled samples fast training. arXiv e-prints , pages arXiv-2203, 2022.

Yichen Xie, Han Lu, Junchi Yan, Xiaokang Yang, Masayoshi Tomizuka, and Wei Zhan. Active finetuning: 417 Exploiting annotation budget in the pretraining-finetuning paradigm. arXiv preprint arXiv:2303.14382 , 418 2023. 419

Lin Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, and Danny Z Chen. Suggestive annotation: A deep 420 active learning framework for biomedical image segmentation. In International conference on medical 421 image computing and computer-assisted intervention , pages 399-407. Springer, 2017. 422

Ofer Yehuda, Avihu Dekel, Guy Hacohen, and Daphna Weinshall. Active learning through a covering lens. 423 In Advances in Neural Information Processing Systems , 2022. 424

425

426

Donggeun Yoo and In So Kweon. Learning loss for active learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 93-102, 2019.

427

428

429

430

431

432

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016 . British Machine Vision Association, 2016.

Wenqiao Zhang, Lei Zhu, James Hallinan, Shengyu Zhang, Andrew Makmur, Qingpeng Cai, and Beng Chin Ooi. Boostmis: Boosting medical image semi-supervised learning with adaptive pseudo labeling and informative active annotation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20666-20676, 2022.

Peer reviews:
comments: SUMMARY:
This paper proposed a new NTKCPL method to reduce the approximation error and it has a wider effective budget range in the setting of active learning on top of self-supervised model. The experimental results on several Computer Vision datasets (e.g., CIFAR-10, CIFAR-10, SVHN) validate the effectiveness of the proposed methods.

STRENGTHS:
1. This paper is well-written. The problem this paper focuses on is important, and the proposed method is interesting.
2. This paper provides both theoretical and empirical results, which is great for a top machine learning conference like NeurIPS. The experiments are sufficient, and the conclusion is convincing.

WEAKNESSES:
1. A case study is suggested. For example, in CIFAR-10 dataset, which class or classes are much better than others or all the classes become better with the same scale? Providing detailed case study about the datasets together with improved metrics will make your conclusion further convincing.

QUESTIONS:
1. Did authors try this method in NLP datasets? It will be very great if the proposed method can be evaluated in different fields. In NLP, pre-trained model seems to play a more important role than the one in Computer Vision tasks.

LIMITATIONS:
Some potential limitations are suggested to add. For example, a better approximation method may need more computational resources.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nActive Learning is a crucial problem that focuses on selecting a subset of examples from an unlabeled dataset to be labeled. The primary objective is to ensure that when the model is trained using these selected examples, it achieves a lower empirical risk upon evaluation, assuming all the unlabeled data points are eventually labeled. \n\nWhile getting labels is a difficult task, current foundation models that utilize self supervision are on-par with many supervised learning procedures. This work aims to use the existing self supervised trained models as the feature extractor, to train a classifier network with the active learning. To get the samples to train the neural network, the paper proposes to use NTK to get the classifier's output, if it was trained on an example say $x^{\\prime}$ from the unlabeled pool. Then based on a criterion that depends on the accuracy ( 0-1 loss ), algorithm returns a set to be labeled. \n\nThis process is done iteratively. Paper shows improvement against common baselines such as Random/Coreset/BADGE/Entropy/Lookahead and shows that there are improvements. Overall, the problem is well motivated, however, there are several confusions and in particular writing issues, that makes this submission not fit at this stage to be published.\n\nSTRENGTHS:\n- Idea of using NTK on top of self supervised trained network is good, as it also provides a scope for theoretical guarantees. \n- Wide spectrum of baseline has been covered, and gains are decent in low budge regime.\n- Adaptive strategy for refining clusters is interesting.\n\nWEAKNESSES:\nMajor weakness of this work is lack of clarity and writing. The notations at many places are not clear to me at all, with interchanging subscripts with \"t\" time and indexing for label at many places. I will summarize the places where there is clear notational abuse, or inconsistency in the upcoming lines. While the work is interesting, the mathematical notations and algorithm needs to be written very precisely and clearly, which in current form does not seem fit for the publication. \n\n1. Section 2 never formally defines what $f$ is mathematically. What is the input space and what is the output space? Does it output class logits, or features? \n2. Wherever an $\\operatorname{argmin}$ is written, it should always have the space of optimization \n3. Section 3.2 near equation 4, it is written \"We denote the predictions of NTK with the dataset $D_{C}$ as $\\hat{f}_{DC}$\". then does $f$ also outputs a class, or a one hot vector? \n4. In summary the proposed algorithm NTKCPL chooses examples to be labeled such that when added to the labeled pool, they'd minimize the empirical risk? Since the backbone is not trained, and they're also using NTK, how different is it in spirit from Mohamadi et al?\n5. In NTKCPL Algorithm, I don't understand what exactly is $f_{self}$ and $f_{al}$. It is never defined. Is it the feature extractor and the learned classifier? If yes, then why define a new variable? \n6. In NTKCPL Algorithm line 6, $b_i$ is never defined. \n7. In NTKCPL Algorithm line 13, subroutine for NTK calling is never defined. Moreover, it seems like an overloading of $\\hat{f}$ argument. Lastly, there doesn't seem to be utility of $f_{0}$ other than this routine. \n8. Line 20 seem to be performing vacuous setminus from the unlabeled pool, where as in the labeled pool it seems incorrect to have a tuple of input and its pseudolabel as well.\n9. Where is $f_t$ being used? There needs to be a full subroutine of AL procedure starting from scratch (that is $L = \\emptyset$ going all the way to the required budget, in batches, if needed). \n10. $g$ is never mathematically defined in the proposition. \n11. I don't understand the meaning of dominant labels. Moreover, the usage of $D_{dom}$ is incorrect if it is indexing over the set of class labels, as previously $D_{.}$ is being used for the dataset. \n12. Lots of unnecessary notations introduced such as $ymap$ which can be avoided by appropriately defining $g$\n13. What is the meaning of $nff$ subscript, and similarly, $fnf$? \n14. Why should clusters based on classifier being trained be reliable? That is usage of $f_{al}$?\n15. Experiments did not mention the pretrained data for each of the dataset/arch, neither mention about the MLP arch. \n16. What's the reason for Entropy and other popular methods to underperform even at decently high budget such as order of 1000s? \n17. How is max number of clusters determined? \n18. How different is coverage estimation from accuracy?\n\nLastly, the work would've been benefited, if there were experiments from CLIP models, which are one of the most popular available pre-trained models.\n\nQUESTIONS:\nPlease refer to the weakness section.\n\nLIMITATIONS:\nI think the paper should've had the use-cases where experiments involve pre-trained CLIP ResNet models.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a look-ahead strategy for more efficient active learning when used with self-supervised learning features. The approach uses Neural tangent kernels and pseudo-labels generated by clustering self-supervised or active learning features to estimate an approximation of the empirical risk of each unlabeled data sample. It then selects those examples for label annotation that will maximally reduce this empirical risk in each iteration. The paper demonstrates the validity and performance of this approach on 5 image datasets, showing that the approach outperforms other baseline active learning methods in most cases, and remains dominant over a larger range of training budgets over earlier SOTA methods for low and high budget strategies.\n\nSTRENGTHS:\nOriginality: To my knowledge this is the first look ahead strategy for active learning that combines a neural tangent kernel with clustering based pseudo-labels to estimate an approximation of empirical risk of each unlabeled data sample to select for  active learning. \n\nQuality: The paper is of average quality. It is incremental, building on earlier concepts of NTK and applying it to active learning with self-supervised features. It does a good job motivating, and validating the approach. The presentation in section 3.3 with a lot of new terms and notations is tiring and could be simplified. \n\nClarity: The paper could be improved with a few more editing passes to complete some incomplete sentences and polish the grammar for better readability.\n\nSignificance: The work is significant since it shows an approach that is at or improves on SOTA for active learning with self-supervised features.\n\nWEAKNESSES:\nI see the following weaknesses:\n\n1. Section 3.3 is currently dense with a lot of new terms and notations that are not motivated or explained well. I suggest the authors refine this section, explaining more how the arguments lead to their proposition. The appendix is not very helpful as it currently stands to understand this proposition well. \n\n2. A major dimension missing in the paper is how the time taken for active learning with NTKCPL compares with other SOTA methods in the different budget regimes. I suspect that NTKCPL is faster, but it is not clear if it indeed is faster, and if so, by how much. Since there is also a clustering algorithm run for each iteration, it is not clear how the overall approach scales with size and dimensionality of data, number of classes etc. \n\n3. In Algorithm 1, I think line 6 should read “min(b_0/2,”. It now reads “min(b_i/2,”\n\n4. Figure 34 2 compares various methods with NTKCPL. However, different panels use different colors for the same method, making it hard to read the (already dense) plots. Please use consistent colors/line types for the same method in each plot\n\nQUESTIONS:\n1. From the weaknesses above, how does NTKCPL scale with size and dimension of data, number of classes etc., and how do the running times compare with SOTA methods over different budget regimes?\n\n2. Section 4- implementation: The paper mentions 2 sets of models used for each of the 5 datasets. What do the 2 sets represent - is one used for high budget and the other low budget?\n\nLIMITATIONS:\nI don’t see any significant negative societal impact of this work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a active learning strategy that combines self-supervised learning with NTK approximation to estimate empirical risk more accurately. The proposed method outperforms state-of-the-art methods and has a wider effective budget range.\n\nSTRENGTHS:\nWell-written: The paper is well-written, informative, and easy to understand. The authors provide clear explanations of the proposed method and the analysis, making it accessible to a wide audience.\n\nComprehensive analysis: The paper presents a comprehensive analysis of the proposed method, including an ablation study and experiments on various datasets.\n\nExperimental results: The paper presents experimental results that demonstrate the effectiveness of the proposed method on various datasets. The results show that the proposed method outperforms state-of-the-art methods in most cases and has a wider effective budget range.\n\nWEAKNESSES:\n* Table 2 is misssing\n\n* More self-supervised learning method + active learning should be compared\n\n* Novelty seems not strong enough for NIPS, as the author mentions, self-supervised + active learning has been worked before, and would you explain why the previous methods are not competitive as this\n\nQUESTIONS:\n* “in our scenario, training on top of the self-supervised model, NTK does not approximate predictions of the whole network well. The main\n reason is that weights of the neural network are initialized by self-supervised learning rather than\nNTK initialization, i.e., drawn i.i.d. from a standard Gaussian” ，I still confused about the reason, would you give me more detail?\n* For Figure 1, Is it a concept graph or an experimental result graph?\n\nLIMITATIONS:\nThe proposed method requires a pre-trained self-supervised model, which may not be available or feasible to obtain in some scenarios. This limits the applicability of the proposed method to certain domains and datasets. \n\nAdditionally, the paper does not provide a detailed analysis of the computational complexity of the proposed method, which may be a concern in some scenarios where computational resources are limited.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper introduces a novel approach that combines active learning with self-supervised learning, known as neural tangent kernel clustering-pseudo-labels (NTKCPL). The method leverages the power of the neural tangent kernel (NTK) in conjunction with self-supervised learning features to enhance the estimation of lookahead. Additionally, clustering-pseudo-labels are employed to estimate the classification error. The paper includes a thorough analysis of the approximation and presents comprehensive experimental results, comparing the proposed methods against benchmark techniques.

STRENGTHS:
As demonstrated in the comparison experiments, NTKCPL exhibits substantial performance gains.

The analysis of CPL error is important, as it effectively reveals that errors arise from over-clustering and impurity. Building upon this analysis, the proposed cluster generation algorithm effectively addresses these issues, displaying a logically concrete solution.

WEAKNESSES:
The paper is motivated from the concepts of "phase transition" and "effective budget range" in active learning. However, it lacks analysis regarding why the proposed method can increase the effective budget range. Additionally, in Table 2, the absolute percentage of the "Effective Budget Ratio" is dependent on the chosen total annotation quantity, e.g. a larger total annotation quantity leads to a smaller "Effective Budget Ratio," suggesting that the "Effective Budget Ratio" is not well-defined.

QUESTIONS:
Eq. 6 may include a typo: Shouldn't the superscript "k" be a subscript "k"? And j is not shown in the right hand side of the equation. 

What's the difference between $\hat{f}_{y}$ and $\hat{f}_{ymap}$

LIMITATIONS:
The authors have addressed the limitations of this paper.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aim to develop an active learning method effective across various budgets and compatible with self-supervised learning. The proposed approach, NTKCPL, a look-ahead active learning strategy, selects a subset that are expected to train network to minimize error of unlabeled data pool. For efficiently estimate the model prediction trained with candidate set, they employ NTK. To do so, they freeze the network's backbone and train only the classifier. Pseudo labels are assigned to the unlabeled data pool for empirical risk calculation, achieved by applying a constrained K-means algorithm to self-supervised features. The loss between pseudo labels and approximated predictions is calculated to select data that will likely helpful to minimize the unlabeled pool's loss. The proposed method is evaluated on five datasets and outperformed the baselines on most of the datasets and budget ranges.\n\nSTRENGTHS:\n- Combining NTK and pseudo label from clustering to estimate empirical risk is new to me.\n- The proposed method is evaluated on various datasets, and in most cases, it demonstrated superior performance compared to the baseline approaches.\n- They analyze the approximation error of the empirical risk when using NTK and CPL\n\nWEAKNESSES:\n- Limited technical contribution\n    - An essential element of the proposed method stems from earlier work [26] that utilizes NTK approximation of DNN prediction for look-ahead active learning.\n    - The most prominent distinction from [26] is that this work utilize expected error reduction instead of expected model output change for active selection, and proposed a method for assigning pseudo labels to facilitate this.\n    - Given that both expected error reduction and pseudo labeling through feature vector clustering are widely used techniques, the technical contribution of the proposed method could be seen as incremental.\n- Concerns about practicality\n    - The method needs to freeze the backbone to ensure the accuracy of the NTK approximation (line 151 - 154). However, according to the existing self-supervised learning literature [a], there is a substantial performance gap between fine-tuning the entire network and those that only train the classifier.\n    - Moreover, the proposed method appears to be dependent on the quality of the features learned through self-supervised learning. Although it is claimed that the active learning feature is used to improve clustering purity, the results in table 1 reveals a performance advantage for self-supervised features in low-budget situations.\n- Need for comprehensive baseline comparisons\n    - It would be beneficial if the performance of [24] was also reported, given that the proposed method follows the training configuration of [24].\n    - It appears that the results using self-supervised features on Cifar 100 may be missing. It would be advantageous to see these results.\n    - Furthermore, I am curious as to whether the baseline methods were also trained with the frozen backbone, and how their performance would vary depending on this factor.\n- I encountered difficulties in smoothly following the provided script. I believe there is room for improvement in the writing.\n    - I found the paper's main point to be somewhat confusing. On lines 49 - 52, one of the stated goals of the proposed method is an active learning strategy with a wider effective budget range, yet on line 226, there is a focus on the low-budget regime.\n    - Furthermore, the notations in the method section have not been clearly defined, and they are somehow confusing. For instance, in the script, 'f' denotes a neural network (line 125), 'f_0' represents the network's output (line 127), 'f_self' and 'f_al' are features (line 1 of algorithm 1), 'f_t' signifies the classifier (line 21 of algorithm 1), and '\\hat(f)_cpl' is the prediction (line 170).\n\n[a] He, Kaiming, et al. \"Masked autoencoders are scalable vision learners.\" *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 2022.\n\nQUESTIONS:\n- I am interested in understanding the performance degradation when not using a frozen backbone.\n- I am curious about the upper bound of performance achieved through empirical risk estimation using true labels and NTK.\n\nLIMITATIONS:\nThe paper appropriately states its limitations and broader impacts.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-26"}