LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition

1 INTRODUCTION . Facial recognition systems ( FR ) are widely deployed for mass surveillance by government agencies , government contractors , and private companies alike on massive databases of images belonging to private individuals ( Hartzog , 2020 ; Derringer , 2019 ; Weise & Singer , 2020 ) . Recently , these systems have been thrust into the limelight in the midst of outrage over invasion into personal life and concerns regarding fairness ( Singer , 2018 ; Lohr , 2018 ; Cherepanova et al. , 2021 ) . Practitioners populate their databases by hoarding publicly available images from social media outlets , and so users are forced to choose between keeping their images outside of public view or taking their chances with mass surveillance . We develop a tool , LowKey , for protecting users from unauthorized surveillance by leveraging methods from the adversarial attack literature , and make it available to the public as a webtool . ∗Authors contributed equally . LowKey is the first such evasion tool that is effective against commercial facial recognition APIs . Our system pre-processes user images before they are made publicly available on social media outlets so they can not be used by a third party for facial recognition purposes . We establish the effectiveness of LowKey throughout this work . Our contributions can be summarized as follows : • We design a black-box adversarial attack on facial recognition models . Our algorithm moves the feature space representations of gallery faces so that they do not match corresponding probe images while preserving image quality . • We interrogate the performance of our method on commercial black-box APIs , including Amazon Rekognition and Microsoft Azure Face , whose inner workings are not publicly known . We provide comprehensive comparisons with the existing data poisoning alternative , Fawkes ( Shan et al. , 2020 ) , and we find that while Fawkes is ineffective in every experiment , our method consistently prevents facial recognition . • We release an easy-to-use webtool , LowKey , so that social media users are no longer confronted with a choice between withdrawing their social media presence from public view and risking the repercussions of being surveilled . 2 RELATED WORK . Neural networks are known to be vulnerable to adversarial attacks , small perturbations to inputs that do not change semantic content , and yet cause the network to misbehave ( Goodfellow et al. , 2014 ) . The adversarial attack literature has largely focused on developing new algorithms that , in simulations , are able to fool neural networks ( Carlini & Wagner , 2017 ; Chiang et al. , 2020 ) . Most works to date focus on the idea of physical world attacks , in which the attacker places adversarial patterns on an object in hopes that the adversarial properties transfer to an image of the object . Such attacks do not succeed reliably because the adversarial perturbation must survive imaging under various lighting conditions , object orientations , and occlusions ( Kurakin et al. , 2016 ) . While researchers have succeeded in crafting such attacks against realistic systems , these attacks do not work consistently across environments ( Wu et al. , 2019 ; Xu et al. , 2019 ; Goldblum et al. , 2020 ) . In facial recognition , attacks have largely focused on physical backdoor threat models , evasion attacks on verification ( Wenger et al. , 2020 ; Zhong & Deng , 2020 ) and attacks on face detection ( Pedraza et al. , 2018 ) . Unlike these physical threat models , the setting in which we operate is purely digital , meaning that we can manipulate the contents of digital media at the bit level , and then hand manipulated data directly to a machine learning system . The ability to digitally manipulate media greatly simplifies the task of attacking a system , and has been shown to enhance transferability to black box industrial systems for applications like copyright detection ( Saadatpanah et al. , 2020 ) and financial time series analysis ( Goldblum et al. , 2020 ) . Recently , the Fawkes algorithm was developed for preventing social media images from being used by unauthorized facial recognition systems ( Shan et al. , 2020 ) . However , Fawkes , along with the experimental setup on which it is evaluated in the original work , suffers from critical problems . First , Fawkes assumes that facial recognition practitioners train their models on each individual ’ s data . However , high-performance FR systems instead harness large pre-trained Siamese networks ( Liu et al. , 2017 ; Deng et al. , 2019 ) . Second , the authors primarily use image classifiers . In contrast , commercial systems are trained with FR-specific heads and loss functions , as opposed to the standard cross-entropy loss used by classifiers . Third , the authors perform evaluations on very small datasets . Specifically , they test Fawkes against commercial APIs with a gallery containing only 50 images . Fourth , the system was only evaluated using top-1 accuracy , but FR users such as police departments often compile a list of suspects rather than a single individual . As a result , other metrics like top-50 accuracy are often used in facial recognition , and are a more realistic metric for when a system has been successfully suppressed . Fifth , while the original work portrays Fawkes ’ perturbations are undetectable by the human eye , experience with the codebase suggests the opposite ( indeed , a New York Times journalist likewise noted that the Fawkes images she was shown during a demonstration were visibly heavily distorted ) . Finally , Fawkes has not yet released an app or a webtool , and regular social media users are unlikely to make use of git repositories . Our attack avoids the aforementioned limitations , and we perform thorough evaluations on a large collection of images and identities . When comparing with Fawkes , we use the authors ’ own implementation in order to make sure that all evaluations are fair . Furthermore , we use Fawkes ’ highest protection setting to make sure that LowKey performs better than Fawkes ’ best attack . Another work uses targeted adversarial attack on probe images for facial recognition systems so that they can not be matched with images in a database ( Yang et al. , 2020 ) . 3 THE LOWKEY ATTACK ON MASS SURVEILLANCE . 3.1 PROBLEM SETUP . To help make our work more widely accessible , we begin by introducing common facial recognition terms . Gallery images are database images with known identities . These often originate from such sources as passport photos and social media profiles . The gallery is used as a reference for comparing new images . Probe images are new photos whose subject the FR system user wants to identify . For example , probe images may be extracted from video surveillance footage . The extracted images are then fed into the FR system , and matches to gallery images with known identities . Identification is the task of answering the question , “ who is this person ? ” Identification entails comparing a probe image to gallery images in order to find potential matches . In contrast , verification answers the question , “ is this person who they say they are ? ” , or equivalently “ are these two photos of the same person ? ” Verification is used , for example , to unlock phones . In our work , we focus on identification , which can be used for mass surveillance . State-of-the-art facial recognition systems first detect and align faces before extracting facial features from the probe image using a neural network . These systems then find gallery images with the closest feature vectors using a k-nearest neighbors search . The matched gallery images are then considered as likely identities corresponding to the person in the probe photo . LowKey applies a filter to user images which may end up in an organization ’ s database of gallery images . The result is to corrupt the gallery feature vectors so that they will not match feature vectors corresponding to the user ’ s probe images . A visual depiction of the LowKey pipeline can be found in Figure 2 . 3.2 THE LOWKEY ATTACK . LowKey manipulates potential gallery images so that they do not match probe images of the same person . LowKey does this by generating a perturbed image whose feature vector lies far away from the original image , while simultaneously minimizing a perceptual similarity loss between the original and perturbed image . Maximizing the distance in feature space prevents the image from matching other images of the individual , while the perceptual similarity loss prevents the image quality from degrading . In this section , we formulate the optimization problem , and describe a number of important details . LowKey is designed to evade proprietary FR systems that contain pre-processing steps and neural network backbones that are not publicly known . In order to improve the transferability of our attack to unknown facial recognition systems , LowKey simultaneously attacks an ensemble of models with various backbone architectures that are produced using different training algorithms . Additionally , for each model in the ensemble , the objective function considers the locations of feature vectors of the attacked image both with and without a Gaussian blur . We find that this technique improves both the appearance and transferability of attacked images . Experiments and ablations concerning ensembling and Gaussian smoothing can be found in Section 6 . For perceptual similarity loss , we use LPIPS , a metric based on ` 2 distance in the feature space of an ImageNet-trained feature extractor ( Zhang et al. , 2018 ) . LPIPS has been used effectively in the image classification setting to improve the image quality of adversarial examples ( Laidlaw et al. , 2020 ) . Formally , the optimization problem we solve is max x′ 1 2n n∑ i=1 non-smoothed︷ ︸︸ ︷ ‖fi ( A ( x ) ) − fi ( A ( x′ ) ) ‖22 + smoothed︷ ︸︸ ︷ ‖fi ( A ( x ) ) − fi ( A ( G ( x′ ) ) ) ‖22 ‖fi ( A ( x ) ) ‖2 − αLPIPS ( x , x′ ) ︸ ︷︷ ︸ perceptual loss , ( 1 ) where x is the original image , x′ is the perturbed image , fi denotes the ith model in our ensemble , G is the Gaussian smoothing function with fixed parameters , and A denotes face detection and extraction followed by 112× 112 resizing and alignment . The face detection step is an important part of the LowKey objective function , as commercial systems rely on face detection and extraction because probe images often contain a scene much larger than a face , or else contain a face who ’ s alignment is not compatible with the face recognition system . We solve this maximization problem iteratively with signed gradient ascent , which is known to be highly effective for breaking common image classification systems ( Madry et al. , 2017 ) . Namely , we iteratively update x′ by adding the sign of the gradient of the maximization objective ( 1 ) with respect to x′ . By doing this , we move x′ and G ( x′ ) far away from the original image x in the feature spaces of models fi used in the LowKey ensemble . The ensemble contains four feature extractors , IR-152 and ResNet-152 backbones trained with ArcFace and CosFace heads . More details can be found in the next section . Additional details concerning attack hyperparameters can be found in Appendix 8.1 .

Summary: This paper presents a method/tool, i.e., LowKey, to protect user privacy which leverages adversarial attacks to pre-process facial images against the black-box facial recognition system in social media, yet the processed facial images remain visually acceptable. The LowKey method proposes to attack an ensemble of facerec models by optimizing the Gaussian blur to the original face images, with the LIIPS metric on the L2 distance in the \emph{feature space}. Thus, the processed face images remain visually legible to human. The ensemble of facerec models include ResNet-50, RestNet-152, IR-50 and IR-152 trained on MS-Celeb-1M dataset. The LowKey method has demonstrated very effective in combating black-box commercial facerec system at Amazon and Microsoft.