Pass off Fish Eyes for Pearls: Attacking Model Selection of Pre-trained Models (2022)

Abstract: Selecting an appropriate pre-trained model (PTM) for a specific downstream task typically requires significant efforts of fine-tuning. To accelerate this process, researchers propose feature-based model selection (FMS) methods, which assess PTMs’ transferability to a specific task in a fast way without fine-tuning. In this work, we argue that current FMS methods are vulnerable, as the assessment mainly relies on the static features extracted from PTMs. However, such features are derived without training PTMs on downstream tasks, and are not necessarily reliable indicators for the PTM’s transferability. To validate our viewpoints, we design two methods to evaluate the robustness of FMS: (1) model disguise attack, which post-trains an inferior PTM with a contrastive objective, and (2) evaluation data selection, which selects a subset of the data points for FMS evaluation based on K-means clustering. Experimental results prove that both methods can successfully make FMS mistakenly judge the transferability of PTMs. Moreover, we find that these two methods can further be combined with the backdoor attack to misguide the FMS to select poisoned models. To the best of our knowledge, this is the first work to demonstrate the defects of current FMS algorithms and evaluate their potential security risks. By identifying previously unseen risks of FMS, our study indicates new directions for improving the robustness of FMS.

## introduction pre-trained models (ptms) have shown superior performance on various tasks of natural language * indicates equal contribution. † corresponding author. processing (nlp) and computer vision (cv) @xcite @xcite @xcite . the increasingly popular "pre-train then fine-tune" paradigm is typically implemented as a prescriptive three-stage routine: (1) ptm supply stage: upstream suppliers pre-train various kinds of ptms, (2) ptm selection stage: downstream users select the desired ptm based on their own demands for a specific task, and (3) ptm application stage: downstream users conduct further fine-tuning on the given task. during the ptm selection stage, the common practice is to fine-tune a set of candidate ptms and pick up the model with the best performance. such a fine-tuning process allows accurate assessment of the transferability of ptms on each downstream task, but is computationally expensive @xcite . to resolve this issue, researchers recently propose feature-based model selection (fms) methods to efficiently select a ptm for a specific downstream task @xcite @xcite . without training on downstream tasks, fms first extracts static features of the target data using ptms, and then resorts to the correlation between these features and the corresponding target labels as the main criterion to estimate ptms' transferability. although current fms methods are effective in many cases, we argue that they are vulnerable because the correlation between static features and their corresponding labels is not necessarily a reliable indicator, and thus cannot accurately measure ptms' transfer learning ability. to validate our viewpoints, we present two simple and effective methods, (1) model disguise attack (mda) and ( 2 evaluation data selection (eds), to maliciously mislead fms into mistakenly ranking ptms' transferability. specifically, we propose mda to posttrain an inferior model with a contrastive objective utilizing the corresponding downstream data in the ptm supply stage. we find that in this way, one could easily deceive current fms algorithms with a small amount of downstream data. eds is an evaluation data selection method based on the k-means algorithm @xcite ) for fms's evaluation, which is conducted in the ptm selection stage. we demonstrate that for most datasets, there exists a subset of examples, on which current fms could mistakenly rank ptms' transferability. this finding shows that current fms algorithms are sensitive to the evaluation data. worse still, we find that our proposed mda and eds methods can further be combined with the backdoor attack @xcite conducted during the ptm supply stage. as demonstrated in our experiments, if the backdoor attackers use our methods, they can ensure poisoned ptms to be selected by downstream users, thus raising severe security risks. the overall framework of mda and eds is shown in figure 1 . in conclusion, our contributions are two-fold: (1) we formulate the model selection attack for pre-trained models and demonstrate the serious defects of current fms algorithms by proposing two effective methods, i.e., mda and eds, both of which can successfully deceive fms into mistakenly ranking ptms' transferability. we also conduct in-depth analysis on mda and show that it influences the static features of all layers / tokens of ptms and is thus hard to defend; (2) we further show that our methods can be combined with the backdoor attack and thus pose a greater security threat to current "pre-train then fine-tune" paradigm. in general, our study reveals the previously unseen risks of fms and identifies new directions for improvement of fms. foot_0 2 related work ## methodology in this section, we first briefly introduce how current feature-based model selection methods (fms) evaluate ptms' transfer abilities in § 3.1. then we formulate the problem of model selection attack in § 3.2, and elaborate two algorithms, i.e. mda and eds in § 3.3 and § 3.4, respectively. ## experiments and analysis in this section, we first conduct experiments to demonstrate the effectiveness of our proposed model disguise attack and evaluation data selection in § 4.1 and § 4.2, respectively. finally, in § 4.5, we show that it is hard to defend against both mda and eds. ## conclusion in this paper, we demonstrate the vulnerability of feature-based model selection methods by proposing two methods, model disguise attack and evaluation data selection, both of which successfully deceive fms into mistakenly ranking ptms' transferability. moreover, we find that our proposed methods can further be combined with the backdoor attack to mislead a victim into selecting the poisoned model. to the best of our knowledge, this is the first work to analyze the defects of current fms algorithms and evaluate their potential security risks. our study reveals the previously unseen risks of fms and calls for improvement for the robustness of fms. in the future, we will explore more effective, robust and efficient model selection methods.