Question: Can you verify if you are fine-tuning the embeddings or the overall model?
Evidence: An important barrier to the safe deployment of machine learning systems is the risk of task ambiguity, where multiple behaviors are consistent with the provided examples. We investigate whether pretrained models are better active learners, capable of asking for example labels that disambiguate between the possible tasks a user may be trying to specify. Across a range of image and text datasets with spurious correlations, latent minority groups, or domain shifts, finetuning pretrained models with data acquired through simple uncertainty sampling achieves the same accuracy with up to 6× fewer labels compared to random sampling. Moreover, the examples chosen by these models are preferentially minority classes or informative examples where the spurious feature and class label are decorrelated. Notably, gains from active learning are not seen in unpretrained models, which do not select such examples, suggesting that the ability to actively learn is an emergent property of the pretraining process.Figure 1: Active learning can resolve task ambiguity, which is especially salient in few-shot settings. Here, the provided training data leaves the model unsure of the task: is it to predict the shape or the color of the object? Pretraining enables models to disentangle and weigh various competing features, making them good active learners that can choose disambiguating examples (e.g. the blue square), resolving this task ambiguity.
Vision For computer vision datasets, we finetune BiT, a recentlyproposed family of vision models which have achieved state-of-the-art performance on several vision tasks. We primarily consider the BiT-M-R50x1 model, pretrained on ImageNet-21k. To explore the effectiveness of larger architectures and pretraining sources, in Section 5.2 we also consider performance achieved by the same-size BiT-S-R50x1, trained on ImageNet-1k, and the deeper BiT-M-R101x1 model, also trained on ImageNet-21k. These models have been shown to have emergent few-shot learning abilities, where strong classifiers for new tasks can be obtained by simply finetuning on tens or hundreds of examples with typical gradient descent techniques (rather than meta-learning techniques, for example).
Text For the text dataset (Amazon), we use RoBERTa-Large, another pretrained model with similar properties as BiT, and a representative of the BERT family of models which together have obtained state-of-the-art scores on modern NLP benchmarks.
Other details, including hyperparameters and seed set/acquisition sizes are deferred to Appendix B.
As a running baseline, we compare to the same model finetuned with a random acquisition function (equivalent to not doing active learning). That is, a(x; M) = rand(0, 1), so we simply sample a random batch of data from the pool at each acquisition step.
Comparison with unpretrained models To examine whether effective active learning is an emergent property of the pretraining process, we also compare to the performance observed when applying active learning to a randomly initialized, instead of pretrained, BiT-M-R50x1.
Vision For computer vision datasets, we finetune BiT, a recentlyproposed family of vision models which have achieved state-of-the-art performance on several vision tasks. We primarily consider the BiT-M-R50x1 model, pretrained on ImageNet-21k. To explore the effectiveness of larger architectures and pretraining sources, in Section 5.2 we also consider performance achieved by the same-size BiT-S-R50x1, trained on ImageNet-1k, and the deeper BiT-M-R101x1 model, also trained on ImageNet-21k. These models have been shown to have emergent few-shot learning abilities, where strong classifiers for new tasks can be obtained by simply finetuning on tens or hundreds of examples with typical gradient descent techniques (rather than meta-learning techniques, for example).
Text For the text dataset (Amazon), we use RoBERTa-Large, another pretrained model with similar properties as BiT, and a representative of the BERT family of models which together have obtained state-of-the-art scores on modern NLP benchmarks.
Other details, including hyperparameters and seed set/acquisition sizes are deferred to Appendix B.
As a running baseline, we compare to the same model finetuned with a random acquisition function (equivalent to not doing active learning). That is, a(x; M) = rand(0, 1), so we simply sample a random batch of data from the pool at each acquisition step.
Comparison with unpretrained models To examine whether effective active learning is an emergent property of the pretraining process, we also compare to the performance observed when applying active learning to a randomly initialized, instead of pretrained, BiT-M-R50x1.
We broaden our coverage of computer vision models to include vision transformers, the other major architecture family currently in use. We train ViT-16/B, 3 on Treeperson, observing smaller gains for AL than the BiT model (Figure). This perhaps reflects the fact that the vision transformer was pretrained on ImageNet-21k for far fewer epochs then BiT (9 vs 70), and is corroborated by much lower oversampling of minority classes than BiT (Figure).
Across a range of image and text datasets with spurious correlations, latent minority groups, or domain shifts, finetuning pretrained models with data acquired through simple uncertainty sampling achieves the same accuracy with up to 6× fewer labels compared to random sampling.****For computer vision datasets, we finetune BiT (Kolesnikov et al., 2020), a recentlyproposed family of vision models which have achieved state-of-the-art performance on several vision tasks.****For the text dataset (Amazon), we use RoBERTa-Large (Liu et al., 2019), another pretrained model with similar properties as BiT, and a representative of the BERT (Devlin et al., 2019) family of models which together have obtained state-of-the-art scores on modern NLP benchmarks (Wang et al., 2018).****We train ViT-16/B, on Treeperson, observing smaller gains for AL than the BiT model (Figure 8).
Answer: Confirming that we use the standard finetuning approaches used for RoBERTa, BiT, and ViT—namely finetuning all the parameters (including embeddings, in the case of transformer models).