Model Extraction and Adversarial Transferability, Your BERT is Vulnerable! (2021)

Abstract: Natural language processing (NLP) tasks, ranging from text classification to text generation, have been revolutionised by the pretrained language models, such as BERT. This allows corporations to easily build powerful APIs by encapsulating fine-tuned BERT models for downstream tasks. However, when a fine-tuned BERT model is deployed as a service, it may suffer from different attacks launched by the malicious users. In this work, we first present how an adversary can steal a BERT-based API service (the victim/target model) on multiple benchmark datasets with limited prior knowledge and queries. We further show that the extracted model can lead to highly transferable adversarial attacks against the victim model. Our studies indicate that the potential vulnerabilities of BERT-based API services still hold, even when there is an architectural mismatch between the victim model and the attack model. Finally, we investigate two defence strategies to protect the victim model, and find that unless the performance of the victim model is sacrificed, both model extraction and adversarial transferability can effectively compromise the target models.

## introduction recently, owing to the success of pretrained bertbased models @xcite , the downstream nlp tasks have been revolutionised in the form of the limited task-specific supervision via fine-tuning on bert models. meanwhile, commercial task-oriented nlp models, built on top of bert models, are often deployed as pay-per-query prediction apis for the sake of the protection of data privacy, system integrity and intellectual property. as publicly accessible services, commercial apis have become victims of different explicit attacks, such as privacy attack @xcite , adversarial attack @xcite , etc. recently, prior works have also found that with the aid of carefully-designed queries and outputs of the nlp apis, many existing apis can be locally imitated via model extraction @xcite , which raises concerns of the vulnerability of nlp apis. for instance, competing companies can imitate the victim model with a negligible cost. since the considerable investment of data annotation and algorithm design are sidestepped, the competing companies would be able to launch an identical service with a more competitive price than the victim companies. such security issue can be exacerbated, when the back-end pertained models, such as bert, are publicly available @xcite . beyond model extraction, we further demonstrate the adversarial examples crafted by the extracted model could be transferred to the black-box victim model. from the perspective of commercial competition, if the competitors manage to predicate incorrect predictions of the victim services, they can launch an advertising campaign against the victim model with these adversarial examples. in summary, we investigate the vulnerabilities of publicly available nlp classification apis through a two-stage attack. first, a model extraction attack is issued to obtain a local copy of the target model. then, we conduct adversarial attacks against the extracted model, which is empirically transferable to the target model. to patch these vulnerabilities, we mount two basic defence strategies on the victim models. the empirical results show that without corrupted predictions from the victims, model extraction and adversarial example transferability are resilient to the defence. our results spotlight the risks of using pretrained bert to deploy the apis through the lens of model extraction attack and adversarial example transfer attack. such attacks can be conducted at a cost of as little as $7.1. 1 2 related work ## attack on bert-based api our attacks against bert-based apis consist of two phases, model extraction attack (mea) and adversarial example transfer (aet), as depicted in figure 1 . ## conclusions this work goes beyond model extraction from bert-based apis, and we also identify the extracted model can largely enhance adversarial example transferability even in difficult scenarios, i.e., limited query budget, queries from different distributions, or architectural mismatch. extensive experiments based on representative nlp datasets and tasks under various settings demonstrate the effectiveness of our attacks against bert-based apis. in the future, we plan to extend our work to more complex nlp tasks, and develop more effective defences.