SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We have defined the asymptotic Gaussian variance term when we use the quiz set  $\mathcal{Q}_t$  from time  $t$  to evaluate the final model  $f_{T}$ . Let  $C_t = 1 / \sigma_t^2 (f_T)$  denote the model confidence of  $f_{T}$  when evaluated on a test set that follows  $q_{t}$ . Next, we formally show that if the model  $f_{T}$  is more confident about  $q_{t}$ , then  $\mathcal{Q}_t$  should take a larger weight in the integrated final evaluation result.
Theorem 1. Given fixed  $\{\mathcal{Q}_1,\dots,\mathcal{Q}_T\}$ , the weighted combination  $\tilde{R} = \sum_{t=1}^{T} v_t \widehat{R}_{\mathcal{Q}_t}$  where  $v_t = \frac{C_t}{\sum_{t=1}^{T} C_t}$  is the optimal estimator constructed by all samples in  $\{\mathcal{Q}_1,\dots,\mathcal{Q}_T\}$ .
Remark. The weight  $v_{t}$  depends on how accurate the estimate using each quiz set is. If we assume that the model becomes more accurate throughout the entire process, then we know that  $\widetilde{R}_{T-1}$  should be closer to  $R$  than  $\widetilde{R}_{T-2}$ , thus the weight on the most recent one is the largest. However, we still do not know  $R$  and have to estimate the weight. Here we adopt a similar way as estimating  $R_{\theta}$  to estimate the difference terms  $C_t$  using the expected loss and  $\widehat{R}_{\mathcal{Q}_t}(f_T): \frac{1}{C_t} = \frac{1}{n_t} \sum_{i=1}^{n_t} \sum_y \frac{p(\mathbf{x}^{(i)})}{q_t(\mathbf{x}^{(i)})} [\mathcal{L}(f_T(\mathbf{x}^{(i)}), y) - \widehat{R}_{\mathcal{Q}_t}(f_T)]^2 p(y|\mathbf{x}^{(i)})$ . With the optimal  $v_t^*$ , we define the final estimate result as  $\widetilde{R} = \sum_{t=1}^{T} v_t^* \times \widehat{R}_{\mathcal{Q}_t}$ . In practice, if we do not pre-set the length of the entire AL process, we can treat each stage as the final exam. After each AL sampling round, we perform an active testing round as a quiz. Then we use the integration method above to get the final estimate. This way, we will always have the optimal evaluation of the model given all the test samples that have been selected.
3.5 Active Feedback: Improve Model Learning Without Sacrificing Testing Accuracy
The proposed final risk estimation can provide label-efficient evaluation of the model performance. We can further improve the label efficiency on the the AL side since some of these labels can also be used for model training. Assume at the  $t$ -th quiz, we have sampled and labeled a set of test data  $\mathcal{Q}_t$  according to the test proposal  $q_t(\cdot)$ . Since  $\mathcal{Q}_t$  is also a labeled dataset, the active learner could leverage it as a source of feedback information to facilitate learning. In the proposed active learning-testing setting, the number of labeled samples for learning/training  $N_L$  and testing  $N_T$  both contribute to the limited overall annotation budget. Since learning and testing need to be sample-efficient, it is reasonable to assume that  $N_L$  and  $N_T$  are of similar magnitudes instead of having  $N_L \gg N_T$  as in the existing active model evaluation works. We further denote  $\mathcal{S}_{\mathrm{FB}}$  as the feedback set, which contains samples from the collection of quizzes. The active feedback process can be formally defined as a subset selection problem that optimizes a joint objective of learning and testing.
$$
\mathcal {S} _ {\mathrm {F B}} ^ {*} = \arg \min  _ {\mathcal {S} _ {\mathrm {F B}} \in \{Q _ {1}, \dots , Q _ {T} \}} \left[ R \left(f _ {\theta | \left(\mathcal {S} _ {L} \cup \mathcal {S} _ {\mathrm {F B}}\right)}\right) + C \| R - \tilde {R} _ {\left(\left(Q _ {1}, \dots , Q _ {T}\right) \backslash \mathcal {S} _ {\mathrm {F B}}\right)} \| \right] \tag {8}
$$
where  $C$  is a parameter to balance the two objectives:  $(I) = R(f_{\theta |(\mathcal{S}_L\cup \mathcal{S}_{\mathrm{FB}})})$  for learning and  $(II) = ||R - \tilde{R}_{(\{Q_1,\dots,Q_T\},\mathcal{S}_{\mathrm{FB}})}||$  for testing.
Theorem 2. Both the learning and testing tasks have the same label complexity, which is in the order of  $\mathcal{O}(1 / \sqrt{n})$ . As a result, (1) the joint objective has an overall label complexity of  $(\mathcal{O}(1 / \sqrt{N_L + N_{FB}}) + \mathcal{O}(1 / \sqrt{N_T - N_{FB}}))$ ; (2) the balancing parameter  $C$  is in the order  $\mathcal{O}(1)$ .
Remark. The Theorem reveals that it is possible to achieve an optimal balance between  $(I)$  and  $(II)$  by choosing a suitable  $S_{\mathrm{FB}}$  such that we can further improve the model learning performance while maintaining the risk estimation quality from our quizzes-testing process. It provides a foundation to justify the benefit of active feedback. It is worth to note that the optimal subset selection problem is usually NP-hard, but we can still draw a useful conclusion that an appropriate subset  $S_{\mathrm{FB}}$  can potentially improve the combined learning-testing objective rather than having  $S_L$  and  $S_T = \{Q_1,\dots,Q_T\}$
always separated. The theorem can also be more intuitively interpreted by drawing analogy with human learning, who automatically incorporate knowledge or partial knowledge at inference time. In contrast, a typical supervised-learned ML model is frozen at evaluation/testing time. Thus, we need to explicitly add feedback so that the model can effectively improve the learning performance by getting new training samples.
Following the above theoretical result, we propose a selection proposal  $q_{\mathrm{FB}}(\cdot)$  for  $S_{\mathrm{FB}}$  based on the following learning-testing intuitions. In standard AL, we should select samples that are considered to be the most informative and diverse ones. From the informativeness perspective, samples with high losses could be challenging for the current model. From the diversity perspective, the newly added samples should be different from the current labeled training set, which can be achieved using a diversity metric, such as  $d(S_L,\mathbf{x}) = \sqrt{\mathbf{x}^\top A_L^{-1}\mathbf{x}}$ , where  $A_{L} = \epsilon \mathbb{I} + \sum_{\mathbf{z}\in S_{L}}\mathbf{z}\mathbf{z}^{\top}$  [11], where  $\epsilon >0$ . From the evaluation perspective, removal of feedback samples should have as little impact to the risk estimation as possible. Since the test samples are chosen according to  $q(\mathbf{x})$ , it is less harmful to choose a feedback sample with a large  $q(\mathbf{x})$  as similar testing samples are likely to be selected into the test data. To this end, we propose the following feedback selection strategy:
$$
\mathbf {x} ^ {*} = \underset {(\mathbf {x}, y) \in \mathcal {Q} _ {t}} {\arg \max } q _ {\mathrm {F B}} (\mathbf {x}, y; \eta) = \underset {(\mathbf {x}, y) \in \mathcal {Q} _ {t}} {\arg \max } q ^ {(t)} (\mathbf {x}) \mathcal {L} \left(f _ {t} (\mathbf {x}), y\right) + \eta d \left(\mathcal {S} _ {L}, \mathbf {x}\right) \tag {9}
$$
where  $\eta$  is a scaling parameter. We construct  $S_{\mathrm{FB}}$  after each active quiz round. The samples are added in  $S_{L}$  to re-train the model, and the risk estimation is also re-weighted after the removal.
3.6 Early Stopping with Instant Risk Estimations
In the ATL framework, the unbiased risk estimation after each AL step can be used to construct an effective convergence indicator to support early stopping of AL. In particular, we propose to use the change of a moving average of active risk estimations:
$$
\Delta \tilde {R} _ {t} = \frac {\sum_ {i = t - w} ^ {t} v _ {i} \tilde {R} _ {i}}{\sum_ {i = t - w} ^ {t} v _ {i}} - \frac {\sum_ {i = t - w - 1} ^ {t - 1} v _ {i} \tilde {R} _ {i}}{\sum_ {i = t - w - 1} ^ {t - 1} v _ {i}} \tag {10}
$$
Prior works show that leveraging the unlabeled data works well empirically to stop AL early [2, 6]. To this end, we propose to further augment  $\Delta \tilde{R}_t$  with stabilized predictions (SP) from the unlabeled data [6], where  $\mathrm{SP} = 1 - \mathrm{mean}_{\mathbf{x}\in \mathcal{U}}(\Delta y(\mathbf{x}))$ .
The proposed ATL framework is generic and can be applied to any existing active learners. In our experiments, we choose two commonly used models, Gaussian Processes (GP) and neural networks, to demonstrate the effectiveness of the proposed work. The GP is applied to a 2D synthetic dataset to help us understand the important behavior of the proposed test sampling strategy and how active feedback benefits the entire learning process. The neural network model is applied to relatively larger scale image datasets, including MNIST, FashionMNIST and CIFAR10 to demonstrate the practical performance of the proposed framework. As for the AL strategy, ATL is designed to be AL-agnostic. To demonstrate the general applicability, the main results are obtained using uncertainty (i.e., entropy) based sampling as the AL strategy given its great popularity in many AL models. In Appendix C.2.3, we show that ATL can be easily integrated with a wide range of commonly used AL algorithms.
4.1 Experimental Settings
For synthetic experiments, we generate 2,500 data samples based on a moon-shaped distribution with two smaller inverted moon-shaped clusters at two remote corners. The dataset is shown in Figure 1 (a). We add the smaller clusters because the learning performance is usually closely related to how AL explores the unknown regions in the feature space and the designed shape allows us to visualize testing and feedback samples to evaluate their effectiveness. Moreover, we create two levels of imbalance in the dataset, the intra-class imbalance and the inter-class imbalance, to simulate an imbalanced class distribution. For the real-world experiments, we use the cross-entropy loss for risk evaluation. When we compare with the true risk  $R$ , we use the average evaluation results of the model on a large hold-out subset of the dataset (10,000 data samples) to represent  $R$ . The hold-out test set does not interact with the AL or AT processes, thus is considered a fair evaluation. In real-world experiments, we adopt the same procedure with the total pool containing 30,000 data samples.
(a) AL sampling (quiz 9)
(e) AL sampling (quiz 18)
Figure 3: Active test samples selected by different criteria at quiz 9 [(a)-(d)], and 18 (e)-(h)
(b) Random (quiz 9)
(f) Random (quiz 18)
(c) ATL-NF (quiz 9)
(g)ATL-NF (quiz 18)
(d) ATL (quiz 9)
(h) ATL (quiz 18)
In each test sampling round, we compute over the entire unlabeled pool according to (4). We use a deterministic way of summing over all classes based on the posterior distribution predicted by the model. The multi-source estimate only utilizes labels of the training samples and previously selected test samples along with the predictions for unlabeled samples. Later in the active feedback stage, if a test sample is selected to be added to the training set, we remove it from the testing set. For test and feedback selections, we sequentially sample  $n_t$  times to obtain each batch. The testing process does not involve re-training the model, thus the batch mode has no effect.
4.2 Synthetic Data Experiments
In Figure 3, we show the distribution of test samples selected using (1) AL sampling, (2) random sampling, (3) ATL w/o feedback, and (4) ATL at an early stage (i.e., quiz 9) and later stage (i.e., quiz 18) of the learning process. We observe that using AL sampling for test point selection has less accurate risk estimation performance due to highly biased sample selection near the decision boundary. Random sampling, on the other hand, follows the true data distribution, so it mainly selects from the central region, where the data is dense. Our proposed test sampling focuses on the most representative regions as random sampling does, but it also covers other interesting regions with relatively low density (e.g., the right edge where the minority blue class is mainly located), and the model evaluation reaches the highest accuracy in the early phases of learning.
In Figure 3d, we plot the feedback with the same annotation cost to demonstrate its learning-testing trade-off. By providing part of the testing data as effective feedback, the estimation error is slightly increased. However, the sacrifice in model evaluation accuracy pays off from the training side. The feedback helps the AL model build a more representative training set and learn a more reasonable decision boundary than without feedback. It allows the AL model to gain a greater advantage in the early stage of training. Figures 3e-3h show the test data distribution in a later stage at quiz 18. The proposed testing selection is the only one that picks samples to test the critical but low-density regions. More importantly, Figure 3h shows that learning with feedback is most successful as it acquires an informative training dataset and a near-optimal decision boundary in the end.
4.3 Real-Data Experiments
The main results of real-world experiments consist of three parts: (1) risk estimation during standard AL (without active feedback), (2) AL performance and risk estimation with active feedback, and (3) AT-based early stopping. In this section, we focus on showing the results from the first two and leave the details of part (3) in the Appendix. In the AL without feedback case, we compare the estimation error results of ATL-NF (no feedback) with ARE quizzes integrate [20] and the adapted Active Testing [14] and Active Surrogate Estimation [15] (referred to as AT integrate and ASE integrate). We adapt the AT and ASE baseline to use in the AL setting by adjusting the pool size  $N$  at each round and including the selected training loss to maintain the unbiased estimator  $\tilde{R}_{LURE}$ . We also need to train a surrogate model using fewer samples than normal (we use a separate NN model re-trained every few rounds). Because the AL model and the surrogate models are both severely under-trained in the early stage of AL, and that the training samples must also be considered in  $\tilde{R}_{LURE}$ , the estimation
error is usually significant in the AL setting. In the active feedback experiments, we compare the proposed ATL with ATL-NF and ATL-RF (random feedback).
Table 1: Estimation error: squared difference between estimate and true risks  $(\times 10^{-3})$
Dataset	AL round Method	4	8	12	16	20
MNIST	ARE quiz	5.27 ± 5.42	6.39 ± 1.54	2.96 ± 3.45	8.85 ± 4.31	8.31 ± 3.96
AT integrate	16.3 ± 24.5	32.8 ± 22.1	6.93 ± 18.0	8.72 ± 3.59	3.11 ± 2.98
ASE integrate	3.45 ± 2.76	1.45 ± 1.00	2.17 ± 5.06	4.00 ± 2.37	5.88 ± 5.27
ATL-NF	2.57 ± 1.17	0.79 ± 1.15	0.17 ± 0.15	0.56 ± 0.30	1.32 ± 0.37
Fashion MNIST	ARE quiz	4.24 ± 3.01	4.62 ± 7.77	8.63 ± 2.47	5.71 ± 1.87	23.78 ± 1.75
AT integrate	11.9 ± 6.1	36.1 ± 30.7	34.1 ± 31.4	28.0 ± 36.9	22.5 ± 25.7
ASE integrate	11.1 ± 3.63	3.72 ± 3.53	3.56 ± 8.78	5.29 ± 9.78	8.42 ± 6.72
ATL-NF	3.64 ± 1.61	0.67 ± 0.38	0.96 ± 0.16	0.98 ± 0.43	3.04 ± 1.37
CIFAR10	ARE quiz	10.1 ± 8.79	13.8 ± 13.0	22.2 ± 14.7	21.9 ± 31.4	14.1 ± 13.4
AT integrate	6.89 ± 6.98	12.0 ± 7.18	21.8 ± 5.73	12.9 ± 9.76	38.9 ± 25.6
ASE integrate	10.9 ± 3.67	6.51 ± 2.87	7.53 ± 1.46	17.6 ± 2.66	23.2 ± 6.10
ATL-NF	8.83 ± 7.79	3.06 ± 5.04	4.95 ± 7.12	7.94 ± 5.22	6.20 ± 5.79
Table 2: Hold-out test risk using different feedback criteria over 20 AL rounds
Dataset	AL round Method	4	8	12	16	20
MNIST	ATL-NF	0.92 ± 0.06	0.55 ± 0.08	0.46 ± 0.06	0.32 ± 0.04	0.22 ± 0.02
ATL-RF	0.92 ± 0.12	0.54 ± 0.02	0.41 ± 0.05	0.29 ± 0.03	0.21 ± 0.02
ATL	0.88 ± 0.07	0.53 ± 0.04	0.39 ± 0.03	0.26 ± 0.01	0.19 ± 0.03
Fashion MNIST	ATL-NF	0.75 ± 0.03	0.69 ± 0.02	0.61 ± 0.02	0.57 ± 0.04	0.56 ± 0.03
ATL-RF	0.75 ± 0.04	0.68 ± 0.02	0.61 ± 0.01	0.58 ± 0.06	0.56 ± 0.04
ATL	0.74 ± 0.03	0.65 ± 0.04	0.59 ± 0.02	0.56 ± 0.03	0.51 ± 0.01
CIFAR10	ATL-NF	1.91 ± 0.04	1.76 ± 0.05	1.72 ± 0.01	1.66 ± 0.02	1.55 ± 0.03
ATL-RF	1.91 ± 0.03	1.77 ± 0.04	1.69 ± 0.03	1.60 ± 0.04	1.54 ± 0.07
ATL	1.90 ± 0.05	1.76 ± 0.02	1.65 ± 0.03	1.58 ± 0.02	1.53 ± 0.02
Table 3: Estimation error with feedback over 20 AL rounds  $\left( {\times {10}^{-3}}\right)$
Dataset	AL round Method	4	8	12	16	20
MNIST	ATL-RF	26.8 ± 21.4	21.4 ± 17.0	3.54 ± 4.01	5.54 ± 3.21	7.62 ± 4.41
ATL	14.6 ± 22.1	16.9 ± 13.7	3.19 ± 2.63	4.15 ± 3.20	1.87 ± 1.41
Fashion MNIST	ATL-RF	10.2 ± 9.30	4.41 ± 3.77	2.19 ± 5.53	5.69 ± 4.52	11.6 ± 7.51
ATL	2.50 ± 2.93	1.94 ± 2.25	1.78 ± 1.07	6.32 ± 5.41	5.03 ± 4.41
CIFAR10	ATL-RF	20.6 ± 17.6	19.1 ± 13.7	9.82 ± 8.03	33.6 ± 30.5	24.8 ± 32.4
ATL	11.6 ± 13.4	5.11 ± 3.45	8.81 ± 6.51	11.9 ± 16.7	6.57 ± 6.29
AL-AT results. We use squared difference between the estimated risk and the true risk as well as the variance of the squared difference to measure the estimation error instead of the variance of the estimate itself because each time we repeat the experiment the model is different. This is also a difference caused by studying the AL process, instead of evaluating a fixed model. From Table 1, we see that ATL-NF has a much lower error of estimation than single ARE quizzes and AT/ASE integrate. The ARE quizzes put all testing samples without adjusting the importance weights, thus will result in a bigger estimation error. The drawback of the AT method is that the surrogate model is not a very reliable reference and the evaluated model changes over time. Because it selects samples based on a max-score criterion instead of importance sampling, the change of model may affect the unbiased estimate. The ASE integrate has the same issue of relying on a good surrogate model. However, the risk estimation of ASE is also based on the surrogate model predictions, which is consistent with the test sample selection. Thus, it outperforms AT integrate most of the time.
Active feedback results. We compare traditional AL and active learning-testing with feedback. We use both the cross-entropy loss as the metrics for the model performance. In Table 2, we see that the proposed feedback and random feedback can both help the model reach a smaller testing loss than pure AL with the same total number of labels. The initial training set contains 500 labels, while each of the 20 AL rounds adds 500 labels. AT then samples 100 points after each AL round, and the active feedback sends 50 labels back to training. Because of the importance sampling with replacement nature of the process, the feedback approaches actually use fewer total labels in the end. We also study the impact of feedback on the estimation error in Table 3. In the early stage of AL, it is obvious that random feedback is harmful to the estimation error. The proposed approach also suffers from the increase of the estimation error. However, as the AL goes on, our proposed feedback selection can maintain a low level of estimation error, while improving the model performance. Combining these
benefits, we can potentially save the labeling budget for both learning and evaluation purposes. We report further ablation studies on the feedback proposal in Appendix C.
We also provide a study on the size of the feedback set. As mentioned in the proof for Theorem 2, we keep the size of feedback simple in this work. This is to be consistent with our theoretical analysis and the experiments show that the active feedback process is helpful in this generic setting. Further details about extending this have been discussed as part of the future directions. However, even in the simple setting of fixed feedback size, we can see that the learning and testing performances do not consistently and monotonically change with respect to the feedback size. From Tables 4 and 5 below, we can see that in general, model risk (learning performance) is better when we use a larger feedback size, but at the same time the estimation error (testing performance) may become much worse. The model risk on CIFAR10 behaves differently with the feedback size, probably because the model performance is not good enough and adding difficult samples in this stage does not necessarily help with the generalization ability.
Table 4: Hold-out test risk using different feedback criteria over 20 AL rounds
Dataset	AL round Feedback size	4	8	12	16	20
MNIST	83%	0.86 ± 0.09	0.53 ± 0.04	0.40 ± 0.08	0.30 ± 0.02	0.20 ± 0.03
67%	0.87 ± 0.08	0.52 ± 0.03	0.35 ± 0.03	0.30 ± 0.02	0.21 ± 0.02
50%	0.88 ± 0.07	0.53 ± 0.04	0.39 ± 0.03	0.26 ± 0.01	0.19 ± 0.03
25%	0.94 ± 0.06	0.54 ± 0.03	0.42 ± 0.08	0.35 ± 0.02	0.25 ± 0.02
20%	0.99 ± 0.04	0.56 ± 0.08	0.43 ± 0.06	0.38 ± 0.01	0.24 ± 0.02
Fashion MNIST	83%	0.74 ± 0.02	0.67 ± 0.03	0.60 ± 0.03	0.54 ± 0.02	0.51 ± 0.03
67%	0.77 ± 0.04	0.68 ± 0.03	0.59 ± 0.03	0.56 ± 0.03	0.52 ± 0.02
50%	0.74 ± 0.03	0.65 ± 0.04	0.59 ± 0.02	0.56 ± 0.03	0.51 ± 0.01
25%	0.76 ± 0.02	0.70 ± 0.01	0.62 ± 0.02	0.59 ± 0.05	0.53 ± 0.03
20%	0.77 ± 0.02	0.71 ± 0.02	0.64 ± 0.02	0.61 ± 0.04	0.54 ± 0.04
CIFAR10	83%	1.92 ± 0.06	1.71 ± 0.02	1.67 ± 0.07	1.59 ± 0.04	1.57 ± 0.04
67%	1.96 ± 0.05	1.75 ± 0.02	1.64 ± 0.04	1.58 ± 0.04	1.58 ± 0.06
50%	1.90 ± 0.05	1.76 ± 0.02	1.65 ± 0.03	1.58 ± 0.02	1.53 ± 0.02
25%	1.94 ± 0.08	1.76 ± 0.03	1.70 ± 0.03	1.64 ± 0.04	1.59 ± 0.02
20%	1.91 ± 0.03	1.76 ± 0.02	1.73 ± 0.03	1.59 ± 0.02	1.63 ± 0.02
Table 5: Squared difference between the estimate and the true risk over 20 AL rounds  $\left( {\times {10}^{-3}}\right)$
Dataset	AL round Feedback size	4	8	12	16	20
MNIST	83%	50.2 ± 39.8	21.0 ± 24.3	7.36 ± 8.44	11.4 ± 12.7	7.59 ± 4.45
67%	25.6 ± 23.4	29.3 ± 29.7	6.90 ± 8.05	6.24 ± 6.71	7.50 ± 5.07
50%	14.6 ± 22.1	16.9 ± 13.7	3.19 ± 2.63	4.15 ± 3.20	1.87 ± 1.41
25%	11.7 ± 11.5	10.0 ± 7.98	9.73 ± 11.4	4.76 ± 5.25	1.59 ± 1.96
20%	28.0 ± 24.4	11.8 ± 14.5	5.91 ± 3.82	4.31 ± 4.80	1.25 ± 1.36
Fashion MNIST	83%	8.39 ± 8.97	7.52 ± 10.4	2.77 ± 3.58	3.87 ± 4.45	11.1 ± 7.02
67%	8.59 ± 8.77	8.60 ± 10.5	5.42 ± 5.96	4.05 ± 2.47	14.6 ± 13.8
50%	2.50 ± 2.93	1.94 ± 2.25	1.78 ± 1.07	6.32 ± 5.41	5.03 ± 4.41
25%	3.04 ± 4.00	2.38 ± 4.81	1.54 ± 1.18	6.40 ± 8.06	4.13 ± 3.99
20%	2.62 ± 1.57	1.56 ± 1.77	2.42 ± 4.52	5.65 ± 4.33	5.22 ± 3.27
CIFAR10	83%	54.5 ± 54.1	14.3 ± 7.75	56.1 ± 17.0	47.2 ± 34.3	62.2 ± 43.3
67%	24.6 ± 25.6	36.7 ± 20.5	24.1 ± 18.6	30.7 ± 40.8	36.2 ± 21.0
50%	11.6 ± 13.4	5.11 ± 3.45	8.81 ± 6.51	11.9 ± 16.7	6.57 ± 6.29
25%	4.88 ± 5.80	6.01 ± 8.22	6.80 ± 1.36	10.2 ± 13.4	4.48 ± 3.53
20%	5.44 ± 6.65	3.65 ± 3.44	11.2 ± 11.0	4.21 ± 1.36	5.82 ± 3.34
We address a real-world challenge for sample-efficient learning, where valuable labels from human experts that can be used for testing/evaluation are also scarce. We propose an ATL framework that builds an active learning-testing-feedback loop to achieve a label-efficient evaluation of an AL model on the fly, with the potential to further improve the model performance without extra labels and opening up to the creation of systematic early stopping criteria. We theoretically prove that ATL has an unbiased and label-efficient estimator and provide an analysis that shows how the label complexity dependencies are maintained through active feedback. The experiments show that ATL optimizes the number of human labels needed in learning by simultaneously acting as a fair referee and an educative teacher.
This research was supported in part by an NSF IIS award IIS-1814450 and an ONR award N00014-18-1-2875. The views and conclusions contained in this paper are those of the authors and should not be interpreted as representing any funding agency. We would also like to thank the anonymous reviewers for their constructive comments.