LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition
Authors: Valeriia Cherepanova
Micah Goldblum
Harrison Foley
Shiyuan Duan
John P Dickerson
Gavin Taylor
Tom Goldstein
Venue: ICLR 2021 Poster (2021)
Keywords: facial recognition
adversarial attacks

Abstract: Facial recognition systems are increasingly deployed by private corporations, government agencies, and contractors for consumer services and mass surveillance programs alike.  These systems are typically built by scraping social media profiles for user images.  Adversarial perturbations have been proposed for bypassing facial recognition systems.  However, existing methods fail on full-scale systems and commercial APIs.  We develop our own adversarial filter that accounts for the entire image processing pipeline and is demonstrably effective against industrial-grade pipelines that include face detection and large scale databases.  Additionally, we release an easy-to-use webtool that significantly degrades the accuracy of Amazon Rekognition and the Microsoft Azure Face Recognition API, reducing the accuracy of each to below 1%.

Published as a conference paper at ICLR 2021
LOWKEY: LEVERAGING ADVERSARIAL ATTACKS TO PROTECT SOCIAL MEDIA USERS FROM FACIAL RECOGNITION
Valeriia Cherepanova
Micah Goldblum
Department of Mathematics University of Maryland vche echerepa@ umd. edu
Department of Computer Science University of Maryland gol dbl um@umd edu
Harrison Foley*
Shiyuan Duan"
Department of Computer Science US Naval Academy
Departmentt of Computer Science University of Maryland sduan @mmd. edu
m2 11 92 @usna. edu
John Dickerson
Gavin Taylor
Department of Computer Science University of Maryland johnêcs, umd. edu
Department of Computer Science US Naval Academy t taylor@usnal edu
Tom Goldstein
Department of Computer Science University of Maryland tomgècs. und.
ABSTRACT
Facial recognition systems are increasingly deployed by private corporations, government agencies, and contractors for consumer services and mass surveillance programs alike. These systems typically built by scraping social media profiles for user images. Adversarial perturbations have been proposed for bypassing facial recognition systems. However, existing methods fail full-scale systems and commercial APIs. We develop our own adversarial filter that accounts for the entire image processing pipeline and demonstrably effective against industrial-grade pipelines that include face detection and large scale databases. Additionally, we release an easy-to-us webtool that significantly degrades the accuracy of Amazon Rekognition and the Microsoft Azure Face Recognition API, reducing the accuracy of each below 1%.
INTRODUCTION
Facial recognition systems (FR) are widely deployed for mass surveillance by government agencies, government contractors, and private companies alike on massive databases of images belonging to private individuals (Hartzog 2020 Derringer] 2019 WWise & Singer 2020). Recently, these systems have been thrust into the limelight in the midst of outrage over invasion into personal life and concerns regarding fairness (Singer 2018 Lohr 2018 Cherepanova et al. 2021). Practitioners populate their databases by hoarding publicly available images from social media outlets, and so users are forced choose between keeping their images outside of public view taking their chances with mass surveillance.
We develop a tool, LowKey, for protecting users from unauthorized surveillance by leveraging methods from the adversarial attack literature, and make it available to the public as a webtool.
Authors contrihuted equally.

---

Published as a conference paper at ICLR 2021
Figure 1: Top: original images, Bottom: protected by LowKey.
LowKey is the first such evasion tool that is effective against commercial facial recognition APIs. Our system pre-processes user images before they are made publicly available on social media outlets so they cannot be used by third party for facial recognition purposes. We establish the effectiveness of LowKey throughout this work.
Our contributions can be summarized as follows:
We design a black-box adversarial attack on facial recognition models. Our algorithm moves the feature space representations of gallery faces that they do not match corresponding probe images while preserving image quality.
We interrogate the performance of our method on commercial black-box APIs, including Amazon Rekognition and Microsoft Azure Face, whose inner workings not publicly known. We provide comprehensive comparisons with the existing data poisoning alternative, Fawkes (Shan al. 2020), and find that while Fawkes is ineffective in every experiment, our method consistently prevents facial recognition.
We release an easy-to-use webtool, LowKey, so that social media users are no longer confronted with choice between withdrawing their social media presence from public view and risking the repercussions of being surveilled.
2 RELATED WORK
Neural networks are known to be vulnerable to adversarial attacks, small perturbations to inputs that do not change semantic content, and yet cause the network to misbehave (Goodfellow et al. [004). The adversarial attack literature has largely focused on developing new algorithms that, in simulations, are able to fool neural networks (Carlini Wagner 2017 Chiang et al. 2020). Most works to date focus the idea of physical world attacks, in which the attacker places adversarial patterns an object in hopes that the adversarial propertics transfer to an image of the object. Such attacks do not succeed reliably because the adversarial perturbation must survive imaging under various lighting conditions, object orientations, and occlusions (Kurakin et al., 2016). While researchers have succeeded in crafting such attacks against realistic systems, these attacks do work consistently across environments (Wu et al. [2019] Xu et al. 2019; Goldblum et al. 2020). In facial recognition, physical backdoor threat models, evasion attacks on verification (Wenger et al. 2020 Zhong & Deng 2020) and attacks on face detection (Pedraza et al. 2018) Unlike these physical threat models, the setting in which we operate purely digital, meaning that we can manipulate the contents of digital media at the bit level, and then hand manipulated data directly machine learning system. The ability to digitally manipulate media greatly simplifies the task of attacking system, and has been shown enhance transfcrability to black box industrial systems for applications like copyright detection (Saadatpanah et al. 2020) and financial time series analysis (Goldblum et al. 2020).
attacks have largely tocused
Recently, the Fawkes algorithm was developed for preventing social media images from being used by unauthorized facial recognition systems (Shan et al., 2020). However, Fawkes, along with the experimental setup on which is evaluated in the original work, suffers from critical problems. First, Fawkes assumes that facial recognition practitioners train their models each individual's data.
2

---

Published as a conference paper at ICLR 2021
However, high-performance FR systems instead harness large pre-trained Siamese networks (Liu et al. 2017: Deng et al., 2019). Second, the authors primarily use image classifiers. In contrast, commercial systems traieed with FR-specific heads and loss functions, opposed to the standard cross-entropy loss used by classifiers. Third, the authors perform evaluations on very small datasets. Specifically, they test Fawkes against commercial APIs with gallery containing only 50 images. Fourth, the system was only evaluated using top-1 accuracy, but FR users such as police departments often compile list suspects rather than single individual. As result, other metries like top-50 accuracy are often used in facial recognition, and are more realistic metric for when system has been successfully suppressed. Fifth, while the original work portrays Fawkes' perturbations are undetectable by the human eye, experience with the codebase suggests the opposite (indeed, a New York Times journalist likewise noted that the Fawkes images she was shown during demonstration were visibly heavily distorted). Finally, Fawkes has not yet released an app or webtool, and regular social media users are unlikely to make use of git repositories. Our attack avoids the aforementioned limitations, and we perform thorough evaluations
large collection of images and identities. When comparing with Fawkes, we use the authors' own implementation order to make sure that all evaluations fair. Furthermore,
Fawkes' highest protection setting to make LowKey performs better than Fawkes' best attack. Another work uses targeted adversarial attack on probe images for facial recognition systems that they cannot be matched with images in a database Yang et al. 2020).
that
3
THE LOWKEY ATTACK ON MASS SURVEILLANCE
gallery image
adversarial attack
database
LowKey
face localization and alignment
facial recognition system
Boba Fett
probe image
Figure 2: The LowKey pipeline. When users protect their publicly available images with LowKey, facial recognition systems cannot match these harvested images with new images of the user, for example from surveillance cameras.
3.1 PROBLEM SETUP
To help make our work more widely accessible, we begin by introducing common facial recognition terms.
Gallery images are database images with known identities. These often originate from such sources passport photos and social media profiles. The gallery is used reference for comparing new images.
Probe images new photos whose subject the FR system user wants to identify. For example, probe images may extracted from video surveillance footage. The extracted images are then fed into the FR system, and matches gallery images with known identities.
3

---

Published as a conference paper at ICLR 2021
Identification is the task of answering the question, "who is this person?" Identification entails comparing a probe image to gallery images in order to find potential matches. In contrast, verification answers the question, "is this person who they say they are?". or equivalently "are these two photos of the same person?" Verification is used, for example, to unlock phones.
In work, focus identification, which can be used for mass surveillance. State-of-the-ar facial recognition systems first detect and align faces before extracting facial features from the probe image using neural network. These systems then find gallery images with the closest feature vectors using a k-nearest neighbors search. The matched gallery images are then considered as likely identities corresponding to the person in the probe photo. LowKey applies filter user images which may end up in an organization's 's database of gallery images. The result is to corrupt the gallery feature vectors so that they will not match feature vectors corresponding to the user's probe images. A visual depiction of the Lowkey pipeline can be found in Figure
3.2 THE LOWKEY ATTACK
LowKey manipulates potential gallery images so that they do not match probe images of the same person. LowKey does this by generating perturbed image whose feature vector lies far away from the original image, while simultaneously minimizing a perceptual similarity loss between the original and perturbed image. Maximizing the distance in feature space prevents the image from matching other images of the individual, while the perceptual similarity loss prevents the image quality from degrading. In this section, we formulate the optimization problem, and describe number of important details.
LowKey designed to evade proprietary FR systems that contain pre- pre-processing steps and neural network backbones that are not publicly known. In order to improve the transferability of our attack to unknown facial recognition systems, Lowkey simultaneously attacks an ensemble of models with various backbone architectures that are produced using different training algorithms. Additionally, for each model in the ensemble, the objective function considers the locations of feature vectors of the attacked image both with and without a Gaussian blur. We find that this technique improves both the appearance and transferability of attacked images. Experiments and ablations concerning ensembling and Gaussian smoothing can be found in Section [ For perceptual similarity loss, use LPIPS, a metric based on & distance in the feature space of an ImageNet-traines tettained feature extractor (Zhang al. 2018). LPIPS has been used effectively in the image classification setting to improve the image quality of adversarial examples (Laidlaw et al., 2020).
Formally, the optimization problem we solve is
non-smoothed
smoothed
1
[[((A())) ((A(2 1 +[]((((())) - ((AGG(E))) LPIPS ax))))
max 2n
(f(()(
perceptual loss
where æ is the original image, x' is the perturbed image, fü denotes the 2th model in our ensemble. G is the Gaussian smoothing function with fixed parameters, and A denotes face detection and extraction followed by 112 x 112 resizing and alignment. The face detection step is an important part of the LowKey objective function, as commercial systems rely on face detection and extraction because probe images often contain a scene much larger than a face, or else contain a face who's alignment is not compatible with the face recognition system.
We solve this maximization problem iteratively with signed gradient ascent, which known to be highly effective for breaking common image classification systems (Madry al. 2017). Namely, we iteratively update x' by adding the sign of the gradient of the maximization objective ( with respect to x'. By doing this, we move x' and G(x') far away from the original image in the feature spaces of models fi used the LowKey ensemble. The ensemble contains four feature extractors, IR-152 and ResNet-152 backbones trained with ArcFace and CosFace heads. More details can be found in the next section.
Additional details concerning attack hyperparameters can be found in Appendix 8.1
4

---

Published as a conference paper at ICLR 2021
4 EXPERIMENTAL DESIGN
Our ensemble of models contains ArcFace and CosFace facial recognition systems (Deng et al. 2019 Wang et al. 2018 . For each of these systems, we train ResNet-50, ResNet-152, IR-50, and IR-152 backbones on the MS-Celeb-11 M dataset, which contains over five million images from over 85,000
identities (He al. 2016 Deng et al. 2019 Guo et al., 2016). We use these models both in our ensemble to generate attacks and to perform controlled experiments in Section Additional details on models and their training routines can be found in Appendix 8.1
We primarily test our attacks on the FaceScrub dataset, a standard identification benchmark from the MegaFace challenge, which contains over 100,000 images from 530 known identities well as one million distractor images (Kemelmacher-Shiizeree images from the dataset as common practice in the facial recognition literature (Zhang al.) 2020). We also perform experiments on the UMDFaces dataset, which can be found in Appendix 8..3 (Bansal et al., 2017). We treat one tenth of each identity's 's images as probe images, and we insert the remaining imgges into the gallery. We randomly select 100 identities and apply LowKey to each of their gallery images. This setting simulates a small pool of LowKey users among a larger population of non-users. Then, in order to perform single evaluation trial of identification, we randomly sample one probe image from a known identity and find its closest matches within the remainder of the FaceScrub dataset, according to the facial recognition model. Distance is measured in feature space of the model. If the FR model selects a match from the same identity, then the trial is a success.
et al. 2016). We discard near-duplicate
Note (Rank-k Accuracy). For each probe image, consider the model successful in the rank-k setting if the correct identity appears among the closest gallery images in the model's 's feature space. To test the transferability of our attack we compute rank-l and rank-50 50 accuracy for attack, and test feature extractors from our set of trained FR models.
5 BREAKING COMMERCIAL BLACK-BOX APIS
The ultimate test for our protection tool is against commercial systems. These systems are proprietary, and their exact specifications are not publicly available. We test LowKey in the black-box setting using two commercial facial recognition APIs: Amazon Rekognition and Microsoft Azure Face. We also compare against Fawkes. We generate Fawkes images using the authors' own code and hyperparameters to ensure a fair comparison, and we use the highest protection setting their code offers.
Amazon Rekognition Amazon Rekognition is commercial tool for detecting and recognizing faces in photos. Rekognition works by matching probe images with uploaded gallery images that have known labels. Amazon does not describe how their algorithm works, but their approach seemingly does not involve training model uploaded images (at least not in supervised manner). We test the Rekognition API using the FaceScrub dataset (including distractors) where 100 randomly selected identities have their images attacked as described in Section 4 We observe that LowKey is highly effective, and even in the setting of rank-50 accuracy, Rekognition can only recognize 2.4% of probe images belonging to users protected with Key. In contrast, Fawkes fails, with 77.5% of probe images belonging to its users recognized correctly in the rank- setting and 94.9% of these images recognized correctly when the 50 closest matches are considered. This is close to the performance of Amazon Rekognition on clean images.
Amazon rank-1
Amazon rank-50
Microsoft rank-1
Clean
93.7%
95.4%
90.5%
Fawkes
77.5%
94.9%
74.2%
LowKey
0.6%
2.4%
0.1%
Table 1: An evaluation of Amazon Rekognition and Microsoft Azure Face on FaceSerub data with LowKey and Fawkes protection (a small number, and lighter color, indicates a successful attack). LowKey consistently achieves virtually flawless protection, while Fawkes provides little protection.
5

---

Published as a conference paper at ICLR 2021
Microsoft Azure Face We repeat similar experiment on the Microsoft Azure Facial Recognition API. contrast to Amazon's ' API, Microsoft updates their model on the uploaded gallery of images. Therefore, only known identities can be used, so only include images corresponding to the 530 known identities from FaceScrub and no distractors. The Azure system recognizes only 0. 1% of probe images whose gallery images under the protection of LowKey. Even though Fawkes is designed to perform data poisoning, and authors claim it is especially well suited to Microsoft Azure Face, in our experiments, Azure is still able recognize more than 74% of probe images uploaded by users who employ Fawkes.
We conclude from these experiments that LowKey both highly effective and transferable to even tfute-of-the-ar industrial facial recognition systems. In the next section, we explore several components of our attack in order to uncover the tools of its success.
ADDITIONAL EXPERIMENTS
The effectiveness of our protection tool hinges several properties:
1. The attack must transfer effectively to unseen models.
2. Images must look acceptable to users.
3. LowKey must run sufficiently fast that run-time does not outweigh its protective benefits.
4. Attacked images must remain effective after being saved in PNG and JPG formats.
5. The algorithm must scale to images of any size.
We conduct extensive experiments in this section with variety of facial recognition systems to interrogate these properties of LowKey.
6.1 ENSEMBLES AND TRANSFERABILLT
In developing the ensemble of models used to compute our attack, we examine the extent to which attacks generated by model are effective against another. By including eclectic mix of models in our ensemble, we are able to ensure that LowKey produces images that fool a wide variety of facial recognition systems. To this end, we evaluate attacks all pairs source and victim models with ResNet-50, ResNet-152 IR-50, and IR-152 backbones, and both ArcFace and CosFace heads. For each victim model, we additionally measure performance on clean images, our ensembled attack, and Fawkes. See Table for comparison of the rank-50 performance of these combinations. Additional evaluations in the rank- 1 setting and on the UMDFaces dataset can be found in Appendix (8.2 and 8.3 respectively. Note that entries for which the attacker and defender models are identical depict white-box performance, while entries for which these model differ depict black-box -ox transfcrabilityy
We observe in these experiments that adversarial attacks generated by IR architectures transfer better to IR-bd facial recognition systems, while attacks generated by ResNet architectures transfer better more effective to other ResNet systems. In general, attacks computed 152-layer backbones difficult to fool. than attacks computed on 50-layer backbones, and deeper networks are also more Moreover, attacks transfer better between models trained with the same head. An ensemble of models of all combinations of ResNet- 122 and IR-152 backbones as well as ArcFace and CosPace heads generates attacks that transfer effectively all models and fool models at only slightly lower rate than white-box attacks.
6.2 GAUSSIAN SMOOTHING
We incorporate Gaussian smoothing as pre-processing step in our objective function T to make our perturbations smoother and more robust. Intuitively, this promotes the effectiveness of the attacked image even when denoising filter is applied. The presence of blur forces the adversarial perturbation to rely on smoother/low oooequue// image modifications rather than adversarial "noise." Empirically, we find that attacks computed with this procedure produce slightly smoother and more aesthetically pleasing perturbations without sharp lines and fighffreuueccy oscillations. See Figure for a visual comparison of images produced with and without Gaussian smoothing in the LowKey pipeline.

---

Published as a conference paper at ICLR 2021
Defender
IR-50A
IR-50C
IR-152A
IR-152C
RN-50A RN-SOC RN-152A RN-152C
Clean
96.8%
96.8%
96. 96.7%
96.8%
96.8%
96.8%
96.7%
96.7%
Fawkes
96.6%
96.7%
96. 7%
96. 7%
96. 7%
96.5%
96.6%
96.6%
IR-50A
0.4%
22.2%
11.9%
35.2%
33.6%
46.4%
45.7%
53. 33.0%
IR-50C
4.9%
0.3%
4.1%
8.0%
23. 1%
25.9%
31.4%
28.6%
Attacker
IR-152A
9.9%
18.3%
0.1%
26. 1%
41.6%
46.2%
49.6%
48.9%
IR-152C
2.8%
1.6%
1.5%
0.5%
11.9%
13.9%
18.8%
16.3%
RN-50A
26.4%
35.7%
36.3%
43.0%
0.9%
13.3%
17.4%
24.2%
RN-50C
33.8%
36.5%
41.1%
42.9%
9.9%
0.2%
17.9%
21.1%
RN-152A 16.8%
22.0%
21.1%
28.2%
5.2%
8,8%
0.3%
7.6%
RN-152C 14.8%
19.2%
19.9%
24.3%
6.7%
6.9%
7.1%
0.5%
Ensemble 3.0%
2.4%
2.1%
0.6%
3.1%
4.2%
5.5%
0.9%
Table 2: Rank-50 accuracy of the LowKey and Fawkes attacks. After the first two rows, each row represents LowKey attacks generated from the same model. Each column represents inference on single model. The first two letters in the model's name denote the type of backbone: IR ResNet (RN). The last letter the model's s name indicates the type of head; "A" denotes ArcFace, and "C" denotes CosPace. Smaller numbers, and lighter colors, indicate successful attacks.
We additionally produce images both with and without smoothing the attack pipeline. Before feeding them into facial recognition systems, we defend the system against our attacks by applying Gaussian smoothing pre-processing step just before inference.
We find that facial recognition systems which use this pre-processing step perform equally well on rank-50 (but not rank- 1) accuracy compared to performance without smoothing, and they are also able to defeat attacks which are not computed with Gaussian smoothing. On the other hand, attacks computed using Gaussian smoothing are able to counteract this defense and fool the facial recognition system (see Table 3). This suggests that attacks that use Gaussian smoothing in their pipeline more robust and harder to defend against. See Appendix (.6 details regarding Gaussian smoothing hyperparameters.
6.3 RUN-TIME
In order for users to be willing to use our tool, LowKey must run fast enough that it is not an inconvenience use. Computing adversarial attacks computationally expensive task. We compare run-time to Fawkes as baseline and test both attacks on single NVIDIA GeForce RTX 2080 GPU. We attack one image a time with no batching for fair comparison, and we average over runs on every full-size gallery image from each of five randomly selected identities from FaceScrub. While Fawkes averages 54 seconds per image, LowKey only averages 32 seconds per image. In addition to providing far superior protection, LowKey runs significantly faster than the existing method, providing users a smoother and more convenient experience.
Defender
50A
IR-S0C
IR-152A IR- 15-C RN-50A
RN-50C RN-152A RN-C
Clean
96.7%
96.8%
96.6%
96.9%
06.7%
96. 75%
96.7%
96.7%
Attacker
Without
78.6%
74.0%
74.4%
64.6%
75.5%
76.0%
77.8%
74.2%
With GS
4.2%
4.8%
4.4%
2.8%
7.9%
7.1%
9.6%
3.2%
Table 3: Rank-50 50 accuracy of FR models tested on blurred LowKey images computed with/without Gaussian smoothing.
7

---

Published as a conference paper at ICLR 2021
Figure 3: LowKey attacked images computed without (above) and with (below) Gaussian smoothing.
6.4 ROBUSTNESS TO IMAGE COMPRESSION
Since users may save their images in various formats after passing them through LowKey, the images produce must provide protection even after being saved in common formats. Our baseline tests conducted with images saved in uncompressed PNG format. To test performance under compression, convert protected images JPEG format and repeat our experiments compression very slightly decreases performance, the attack is still very effective: Microsoft Azure Face now able recognize 0.2% of images compared 0.1% when saved in the PNG format. Likewise, Amazon Rekognition now recognizes 3.8% of probe images compared to 2.4% previously,
commercial APIs. While
6.5 SCALABILITY TO ALL IMAGE SIZES (DISCLAIMER)
Many tools in deep learning require that inputs be of particular dimensions, but user images social media sites come in all shapes and sizes. Therefore, LowKey must be flexible. Since the detection and alignment pipeline in our attack resizes images in differentiable fashion, we can attack images of any size and aspect ratio. Additionally, we apply the LPIPS penalty to the entire original image, which prevents box-shaped artifacts from developing on the boundaries of the rectangle containing the face. Since Lowkey does not have a fixed attack budget, perturbations may have different magnitudes on different images. Figure 4 shows the variability of LowKey perturbations the image of Tom Hanks (first column) is one of the best looking examples of LowKey on large images, while the image of Tina Fey (last column) is one of the worst looking examples. Protecting very large images is more challenging task than protecting small images because of the black-box detection, alignment, and re-scaling used in APIs which affect large images more significantly. These experiments indicate that users will receive stronger protection if they use LowKey on smaller images.
very large images;
We test the effectiveness of Kew large images by protecting gallery images of 10 identities from Facescrub (with 17 images in the gallery on average) and using 20 probe images per person. We also vary the magnitude of the perturbation find the smallest perturbation that is sufficient to protect images (Table ④ In this way, we find that users may trade off some protection in exchange for better looking images their own discretion. Additionally, we find that LowKey works much better with smaller gallery sizes; when only gallery images are used, the performance of Amazon Rekognition drops from 32.5% to 1 11% in the rank-50 setting. This observation suggests that users upload new profile pictures less frequently in order to decrease the number of gallery images corresponding to their identity and thus enhance their protection. Finally, the quality of probe images is also important; when small probe images used, like those which would occur in low resolution security camera footage, the accuracy of Amazon Rekognition drops from 32.5% to 19%.
7 DISCUSSION
In this work, we develop tool for protecting users from unauthorized facial recognition. Our tool adversarially pre- processee user images before they are uploaded to social media. These pre-processed images useless for third-party organizations who collect them for facial recognition. While have shown that LowKey is highly effective against commercial black- box APIs, it does not protect users 100% of the time and may be circumvented by specially engineered robust systems. Thus,
8

---

Published as a conference paper at ICLR 2021
Figure 4: First row: Original large images, Second row: Images protected with LowKey (medium magnitude), Third row: Images protected with Low (large magnitude).
Amazon rank-1
Amazon rank-50
Microsoft rank-1
Clean
89.0%
98.5%
86.0%
LowKey
63, 63.0%
94.5%
75.5%
LowKey
34.0%
59.5%
30.5%
LowKey 20.5%
36.5%
12.7%
LowKey 40
14.5%
36.0%
3.0%
LowKey 50
11.0%
32.5%
0.0%
Table 4: Evaluation of Key full-size images. Rows indicate levels of magnitude of LowKey (denoted by the number of attack steps).
we hope that users will still remain cautious about publicly revealing personal information. One interesting future direction to produce adversarial filters that are more aesthetically pleasing in order to promote wider use of this tool. However, it may be that there is free lunch, and one cannot fool of-te-of-the-an facial recognition systems without visible perturbations. Facial recognition systems are not fragile, and other attacks that have attempted to break them have failed. Finally, note that one of our goals in making this tool widely available is to promote broader awareness of facial recognition and the ethical issues raises. Our webtool can be found at owkey. umi acs. und. edu

---

Published as a conference paper at ICLR 2021
ACKNOWLEDGMENTS
This work was supported by the DARPA GARD and DARPA QED programs. Further support was provided by the AFOSR MURI program, and the National Science Foundation's s DMS division. Computation resources were funded by the Sloan Foundation.
REFERENCES
Ankan Bansal, Anirudh Nanduri, Carlos D Castillo, Rajeev Ranjan, and Rama Chellappa. Umdfaces: An annotated face dataset for training deep networks. In 2017 IEEE International Joint Conference on Biometrics (IJCB) pp. 464-473, IEEE, 2017.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pp. 39-57. IEEE, 2017.
Valeriia Cherepanova, Vedant Nanda, Micah Goldblum, John P Dickerson, and Tom Goldstein. Technical challenges for training fair neural networks. arXiv preprint arXiv:2 2021.
Ping- Ying-Yeh Chiang, Jonas Geiping, Micah Goldblum, Tom Goldstein, Renkun Ni, Steven Reich, and Ali Shafahi. Witcheraft: Efficient pgd attacks with random step size. ICASSP 2020-2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 3747-3751. IEEE, 2020.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 4690-4699, 2019.
William Derringer. A surveillance net blankets china's cities, giving police vast powers. The New York Times, Dec. 17 2019. URL http:/ttt////// nyt imes httpp//sccc//////////////eeeeeem china-surve lance. html
Micah Goldblum, Avi Schwarzschild, Naftali Cohen, Tucker Balch, Ankit B Patel, and Tom Goldstein. Adversarial attacks on machine learning systems for high-frequency trading. arXiv preprint arXiv:2002.09565, 2020.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, arXiv preprint arXiv:1412.6577 2014.
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong Hc, and Jianfeng Gao. Ms-celeb- Im: A dataset and benchmark for large-scale face recognition. In European conference on computer vision, pp. 87-102. Springer, 2016.
Woodrow Hartzog. The secretive company that might end privacy as we know it. The New York Times, Jan. 18 2020. URL httpss/t nyt imes. wtt.ccmmmm222/ comm/2000/01/11//cccnnooo htmll
c facac
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 770-778, 2016.
Ira Seaelmaachhr-Shizemmaaa Steven M Seitz, Daniel Miller, and Evan Brossard. The megaface benchmark: 1 million faces for recognition scale. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4873-4882, 2016.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world.
arXiv preprint arXiv: 1607:026000253 2016.
Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat models. arXiv preprint arXiv:2006.1 2et.20066.2655. 12655, 2020.
Tsung- Yi Lin, Priya Goyal, Ross Girshick. Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017.
10

---

Published as a conference paper at ICLR 2021
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 212-220, 2017.
Steve Lohr. Facial recognition accurate, if you're re a white guy. New York Times, 9, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.0664 760006066.6 2017.
Daniel Pedraza, Dhaval Adjodah, Gretchen Greene, Josh Joseph, Thom Miano, and Francisco. Equalais. https:://equati wwt.eeeedaaaa.oo.ae..ooeoa edu/ 2018.
Parsa Saadatpanah, Ali Shafahi, and Tom Goldstein. Adversarial attacks on copyright detection systems. In International Conference Machine Learning, pp. 8307-8315. PMLR, 2020.
Shawn Shan, Emily Wenger, Jiayun Zhang, Huiying Li, Haitao Zheng, and Ben Y Zhao. Fawkes: Protecting privacy against unauthorized deep learning models. In 29th {USENIX} Security Symposium ([USENIX) Security 20), pp. 1589-1604, 2020.
Natasha Singer. Microsoft urges congress to regulate use of facial recognition. The New York Times, 2018.
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 5265-5274, 2018.
Karen Weise and Natasha Singer. Amazon pauses police use of its facial recognition software. com wtt.:occcccce// com/2020/06/10
The New York Times, Jul. 10 2020. URL https::/wwww.w..e nyt imes
mawpolloyyammzzn--ooom technology/ awwzanaan-nnnem www.:zonffacail-eeeooooooooo het.raaaee..eatttooooooaoooooo.o
Emily Wenger, Josephine Passananti, Yuanshun Yao, Haitao Zheng, and Ben Y Zhao. Backdoor
attacks on facial recognition in the physical world. arXiv preprint arXiv:2006. 1180066.45500 /4580, 2020.
Zuxuan Wu, Ser-Nam Lim, Larry Davis, and Tom Goldstein. Making invisibility cloak: Real world adversarial attacks on object detectors. arXiv preprint arXiv:1910.14655 2019.
Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin- Yu Chen, Yanzhi Wang, and Xue Lin. Adversarial t-shirt! evading person detectors physical world. arXiv, pp.
arXiv-1910. 2019.
Xiao Yang, Yinpeng Dong, Tianyu Pang. Jun Zhu, and Hang Su. Towards privacy protection by generating adversarial identity masks. arXiv preprint (XXi:22003306814 2020.
Kai Zhang, Vítor Albiero, and Kevin W Bowyer. method for curation of web-scraped face image datasets. In 2020 8th International Workshop Biometrics and Forensics (IWBF), pp. 1-6. IEEE, 2020.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Jian Zhao. face.evolve: perf-aacee face recognition library based on pytorch. https 2020.
//github. com/ http:/haaJJ99.0 / (ataao//l1.//ace..eooleee...e..
Yaoyao Zhong and Weihong Deng. Towards transferable adversarial attack against deep face recognition. arXiv preprint 20X1::2004.05790, 2020.

---

Published as a conference paper at ICLR 2021
8 APPENDIX
8.1 IMPLEMENTATION DETAILS
We train all of our feature extractors using focal loss (Lin et al., (017) with a batch size of 512 for 120 epochs. We use an initial learning rate of 0.1 and decrease by a factor of 10 epochs 35, 65 and 95. For the optimizer, we use SGD with momentum of 0.9 and weight decay of 5e-4.
For our adversarial attacks, we use 0.05 for the perceptual similarity penalty, 7 for the Gaussian smoothing term. Attacks are computed using signed SGD for 50 epochs with learning rate of 0.0025.
and window size
For face detection and aligning models well as for training routines, use the Vice.evoLVe..PTorrre github repository (Zhao 2020).
8.2 RANK-1 ACCURACY FACESCRUB DATA
See Table
Defender
IR-50A
IR-50C
IR-152A
IR-152C
RN-50A
RN-50C RN-152A RN-152C
Clean
95.6%
96. 96.1%
96.0%
96.2%
95.8%
95.9%
95.9%
96.0%
Fawkes
71.2%
76.2%
74.4%
78.2%
73. 73.9%
76.0%
76.0%
71.2%
IR-50A
0.0%
3.5%
0.5%
6.4%
5.0%
8,2%
8.7%
12.4%
IR-50C
0.1%
0.0%
0.1%
9.0%
3.3%
3.3%
3.6%
5.3%
Attacker
IR-152A
0.3%
1.8%
0.0%
4.2%
6.8%
8.9%
9.4%
10.8%
IR-152C
0.1%
0.1%
0.1%
0.0%
1.0%
1,4%
2.2%
2.9%
RN-50A
3,8%
7.0%
6.8%
8.3%
0.9%
2.0%
2.8%
4.6%
RN-50C
3.1%
5.7%
5.6%
7.9%
1.2%
0.1%
2.0%
3.5%
RN-152A
1.5%
3.1%
2.7%
4.9%
0.3%
0.5%
0.0%
0.4%
RN-152C 1.7%
2.6%
3.1%
3.9%
0.8%
0.8%
0.5%
0.0%
Ensemble 0.0%
0.0%
0.1%
0.0%
0.2%
0.4%
0.6%
0.1%
Table 5: Rank-1 accuracy of the LowKey and Fawkes attacks on the FaceScrub dataset. After the first rows, each row represents LowKey attacks generated from the same model. Each column represents inference on a single model. The first two letters in the model's name denote the type of backbone: IR ResNet (RN). The last letter in the model's name indicates the type of head; "A" denotes ArcFace, and "C" denotes CosFace. Smaller numbers, and lighter colors, indicate more successful attacks.
8.3 RESULTS ON UMDFACES DATASET
We repeat controlled experiments on the UMDFaces dataset which contains over 367,000 photos of 8,277 identities. For UMDFaces, we also choose 100 identities at random and attack their gallery images while keeping one-tenth of each identity's photos as probe images. Experimental results are reported in Ta It can be seen that the effectiveness of LowKey attacks on the UMDFacee dataset is slightly lower, which is likely a result the much smaller gallery.
8.4
CAN WE REDUCE THE SIZE OUR ATTACK?
In order to make attacks more aesthetically pleasing, try reduce the size of perturbation by increasing the perceptual similarity penalty from 0.05 to 0.08. This attack depicted in Figure 3 as a "LowKey small attack". Unfortunately, decrease efficiency of the attack. In the rank-50 setting Amazon Rekognition able to recognize 17.2% of probe images helonging users protected with LowKey small attack. Similarly, Microsoft
small decrease in the perturbation size results in huge
12

---

Published as a conference paper at ICLR 2021
Defender
IR-50A
IR-50C
IR-152A
IR-152C
RN-50A
RN-SOC RN-152A RN-152C
Clean
98.6%
98.3%
98.
98.6%
98.3%
98.3%
98.6%
98.6%
IR-50A
0.3%
38.9%
25.9%
48.6%
47.7%
55.7%
57.1%
62.2%
IR-50C
13.6%
0.3%
16.5%
17.6%
37.5%
39.2%
51.4%
46.3%
IR-152A
23.0%
32.1%
0.0%
36.4%
52.8%
56.5%
58.2%
58.5%
Attacker
IR-152C
8.8%
8.8%
9.1%
1.1%
26. 7%
28.1%
34.9%
31.8%
RN-50A
51.4%
56.3%
47.2%
53.1% 1%
0.3%
30.7%
38.4%
43.2%
RN-50C
49.4%
48.3%
55. 1%
54.0%
23.0%
1.1%
37.2%
36.4%
RN-152A
30.4%
38.1%
39.8%
43.8%
18.8%
22.2%
3,4%
25.9%
RN-152C
26.4%
33.8%
35.5%
37.8%
17.3%
18.2%
16.8%
3.4%
Ensemble
10.5%
4.5%
9.7%
8.8%
15.1%
6.5%
12.8%
13.6%
Table 6: Rank-50 accuracy of LowKey attacks the UMDFaces dataset. After the first row, each row represents LowKey attacks generated from the same model. Each column represents inference on single model. The first two letters in the model's name denote the type of backbone: IR or ResNet (RN). The last letter in the model's 's name indicates the type of head; "A" denotes ArcFace, and "C" denotes CosPace. Smaller numbers, and lighter colors, indicate successful attacks.
Defender
IR-50A
IR-50C
IR-152A
IR-152C
RN-50A
RN-50C RN-152A RN-152C
Clean
66.0%
97.2%
96.9%
96.9%
96.3%
96.6%
97.2%
96.3%
IR-50A
0.0%
12.2%
6.3%
18.2%
17.6%
25.9%
28.1%
31.3%
IR-50C
1.7%
0.0%
2.6%
4.3%
11.6%
13.9%
17.3%
21.0%
IR-152A
4.3%
12.8%
0.0%
14.5%
22.4%
25.6%
29.5%
30.1%
Attacker
IR-152C
1.7%
2.0%
2.6%
0.0%
6.8%
9.7%
13.6%
11.4%
RN-50A
26.4%
31.0%
13.9%
26.4%
0.0%
9.4%
15.1%
19.0%
RN-50C
14.5%
21.3%
20. 70%%
25.3%
6.5%
0.6%
12.8%
14.8%
RN-152A
9,7%
16.2%
16.8%
17.9%
5.1%
7.1%
0.3%
7.4%
RN-152C
7.4%
10.2%
11.9%
13.1%
3.1%
4.5%
5,4%
0.9%
Ensemble 2.8%
1.7%
2.6%
3.1%
5.1%
2.0%
3.1%
4.0%
Table 7: Rank-1 accuracy of LowKey attack attacks the UMDFaces dataset. After the first row, each row represents LowKey attacks generated from the same model. Each column represents inference single model. The first two letters in the model's 's name denote the type of backbone: IR or ResNet (RN). The last letter in the model's 's name indicates the type head; "A" denotes ArcFace, and "C" denotes CosFace. Smaller numbers, and lighter colors, indicate successful attacks.
Azure Face recognizes 5.5% of probe images. Results of controlled experiments are reported in Tables g and
8.5 COMPARISON WITH FAWKES
By comparing a set of images protected with Low and Fawkes tools, we can see that both attacks noticeable, but distort images in different ways. While Fawkes adds conspicuous artifacts on the face (such as mustaches or lines on the nose), LowKey attack mostly changes the textures and adds spots person's 's skin. See Figure 5 for visual comparison.
13

---

Published as a conference paper at ICLR 2021
Defender
IR-50A
IR-50C
IR-152A
IR-152C
RN-50A
RN-S0C RN-152A RN-152C
Clean
96.8%
96.8%
96. 7%
96.8%
96.8%
96.8%
96.7%
96.7%
IR-50A
1.0%
41.1%
24.7%
55.2%
49.7%
65.0%
64.2%
68.8%
IR-50C
17.4%
2.6%
20.4%
30.2%
47.0%
49.0%
56.2%
55.6%
IR-152A
28.8%
38.6%
0.3%
49.8%
61.2%
66.6%
69.8%
70.7%
Attacker
IR-152C
14.2%
14.0%
16.2%
2.7%
37.0%
38.6%
44.4%
42.9%
RN-50A
49.3%
62.3%
64.
68.0%
1.5%
35.9%
41.8%
49.3%
RN-50C
57.4%
59.9%
62. 1%
64.4%
3 31.1%
3.6%
46.8%
48.6%
RN-152A
42.9%
51.3%
52.3%
55.0%
25.6%
32.9%
5.6%
35.0%
RN-152C
41.8%
48.9%
47.9%
52. 7%
28.2%
29.2%
30.4%
8.5%
Ensemble
18.0%
21.8%
19.4%
12.2%
23.1%
24.3%
27.0%
14.1%
Table 8: Rank-50 accuracy of LowKey small attacks on the UMDFaces dataset. After the first row, each row represents LowKey small attacks generated from the same model. Each column represents inference single model. The first two letters in the model's 's name denote the type of backbone: IR or ResNet (RN). The last letter in the model's 's name indicates the type head; "A" denotes ArcFace, and "C" denotes CosFace. Smaller numbers, and lighter colors, indicate more successful attacks.
Defender
IR-50A
IR-50C
IR-152A
IR-152C
RN-50A
RN-50C RN-152A RN-152C
Clean
95.6%
96. 1 %
06.
96,2%
95.8%
95.9%
95.9%
96.0%
IR-50A
0.0%
6.1%
2.2%
10.5%
9.7%
14.4%
16.9%
19.5%
IR-50C
1.6%
0.2%
2.5%
5.8%
9.3%
11.1%
14.0%
14.5%
IR-152A
2.2%
5.6%
0.0%
9.6%
14.0%
17.4%
19.2%
20.7%
Attacker
IR-152C
1.3%
1.7%
2.2%
0.4%
5.1%
7.5%
9.6%
9.7%
RN-50A
8.7%
14.9%
14.8%
17.1%
0.2%
7.6%
7.8%
12.5%
RN-50C
9.9%
14.6%
15.6%
17.2%
5.6%
0.6%
8.8%
11.6%
RN-152A
9.2%
13.5%
12.2%
14.8%
5.7%
7,9%
0.8%
8.5%
RN-152C
6.9%
11.3%
11.8%
12.2%
4.8%
5.3%
6.0%
1.3%
Ensemble
2.6%
3.6%
3.5%
2.9%
4.0%
4.7%
5,4%
3.9%
Table 9: Rank-1 accuracy of LowKey small attacks on the UMDFaces dataset. After the first row, each row represents LowKey small attacks generated from the same model. Each column represents inference single model. The first two letters in the model's name denote the type of backbone: IR or ResNet (RN). The last letter in the model's 's name indicates the type head; "A" denotes ArcFace, and "C" denotes CosFace. Smaller numbers, and lighter colors, indicate more successful attacks.
8.6 GAUSSIAN SMOOTHING IN LOWKEY
For the parameters of the Gaussian smoothing term in the optimization problem (1, 3 for and for window size. For the defensive Gaussian blur, we use a = and no window size.
14

---

Published as a conference paper at ICLR 2021
Figure 5: Panel of different attacks. First row: original images, second row: Fawkes attack, third row: LowKey small attack, last row: LowKey attack.
Defender
IR-50A
IR-50C
IR-152A
IR-152C
RN-50A
RN-50C RN-152A RN-1
Clean
84.4%
8.53%
84.7%
86.1%
86.7%
87.9%
88.5%
89.6%
Attackor
Without GS
13.3%
17.9%
15.8%
13.7%
18.9%
20.4%
23.2%
20.6%
With GS
0.3%
0.3%
0.1%
0.0%
1.0%
0.9%
0.6%
0.6%
Table 10: Rank-1 accuracy of FR models tested on blurred images attacked without and with the Gaussian smoothing term. The first two letters in the model's name denote the type of backbone: IR or ResNet (RN). The last letter in the model's name indicates the type of head; "A" denotes ArcFace, and "C" denotes CosFace. Smaller numbers, and lighter colors, indicate more successful attacks.
15


Reviews:
[{"review_id": "uvSyoAb4j2k", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "*********\nSummary Of The Manuscript:\n*********\nThis manuscript focuses on the problem to protect the users/humans from unauthorized facial recognition systems. To tackle the issue mentioned, the author proposes a customized adversarial filter to work against industrial/government facial recognition systems. In addition, the author claims that their easy-to-use web tool helps to significantly degrade the accuracy below 1% of AWS Service - Amazon Rekognition and Microsoft Azure Face Recognition API for face recognition and similar systems. \n\n*********\nStrength Of The Manuscript:\n*********\nClarity:\n++ The paper reads very well and provides a very good description of related work and background, motivating the problem. Even outside of the contribution of this paper, I would recommend this paper to people getting started with protecting users' information from industrial systems as it provides a thorough description of the part of the pipelines it deals with.\n\nNovelty:\nThere are mainly two points are novel which are presented very well and are as follows:\n++ The author designs a custom black-box adversarial attack on facial recognition models where the proposed algorithm changes the representation of the features in such a way that it will preserve the image quality. \n++ The in-depth analysis with the commercial - Amazon Rekognition and Microsoft Azure Face Recognition APIs shows the practical usage of the author's proposed adversarial attack.  \n      \nExperiments:\n++ There are a number of experiments performed across datasets that are extensive and fair. The fact that the proposed adversarial attack achieves better results while preserving the image quality, makes me confident in the result as the implementation and experiments are sufficient and presented in a good manner. Additionally, the improvements are fairly consistent. Besides, in-depth analysis/ablation studies are done on the robustness of the approach, and comparison with the commercial APIs provides a thorough analysis of where the benefits of the author's approach are obtained.\n\nReproducibility:\n++ First of all, I would like to thank the author for providing code in the supplementary material and because from the code, most of my doubts have been solved and I can say this that the code is written in a very good manner and helped me to understand the whole pipeline of their work and I appreciate the authors for their effort. \n\n*********\nWeakness Of The Manuscript:\n*********\nOverall, apart from the contribution of the approach, I have some concerns regarding the manuscript. \n\n-- Can the authors provide a brief description of the core difference between their proposed adversarial attack (LowKey) and attack generated by any good GANs based model (i.e. Style GAN.). I believe that the standalone contribution of the manuscript is to create such an attack to hide the user's identity. Thus what makes a user believe that the proposed webtool will be helpful to hide their identity if it only applies a LowKey adversarial attack. \n\n*********\nJustification Of The Review: \n*********\n-- In the reviewer's opinion, in its current form, the paper provides in-depth analysis for protecting the user's identity from any industrial/government surveillance facial recognition system and the authors did an excellent job to provide a brief insight on their complete pipeline. I appreciate the author's efforts to provide a code in supplementary material which has nullified my doubts in such a manner that makes me believe that the work is incremental and should reach the Computer Vision community. Therefore the current rating of the paper will be 7 in reviewers' opinion because of fairly consistent work and practical usage. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "b12qIPe62N", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. Summarize what the paper claims to contribute. Be positive and generous.\nThe paper claims to contribute the following three:\n(1) design a black-box adversarial attack on facial recognition models.\n(2) interrogate the performance of the proposed method on commercial black-box APIs, including Amazon Rekognition and Microsoft Azure Face, and against the existing data poisoning alternative, Fawkes.\n(3) release an easy-to-use webtool, LowKey.\nThe result tables on the paper look great!\n\n2. List strong and weak points of the paper. Be as comprehensive as possible.\n(1) Strengths\n  a. The margins of performance differences between LowKey and Fawkes looks great (74%+ absolute).\n  b. The robustness of the model to varying image compression and image size.\n  c. Run-time latency improvement of the model against Fawkes.\n  d. Thorough analysis involving ablation tests (e.g. effect of Gaussian Smoothing and different expert models).\n  e. Including eval metrics of top-50 accuracy. \n(2) Weaknesses\n  a. Experimental setup descriptions are difficult to follow at times.\n\n3. Clearly state your recommendation (accept or reject) with one or two key reasons for this choice.\nAccept because the results look promising even with the model's performance analyzed exhaustively.\n\n4. Provide supporting arguments for your recommendation.\nThe paper proposes a new benchmark with close-to-practical-performance results in latency, accuracy, and robustness to compression. Digital user privacy is a hot topic to be further studied especially in the context of practical application purposes. And, in that perspective, this paper advances the applied research in the area considerably.\n\n5. Ask questions you would like answered by the authors to help you clarify your understanding of the paper and provide the additional evidence you need to be confident in your assessment. \n(1) Curious to learn if there is any good literature that backs the number 50 in *top-50 accuracy*. Is this number arbitrarily chosen to be something greater than 1?\n(2) The experiment setup descriptions are a bit difficult to follow.\n  a. Based on my understanding, in the experiment, the gallery images can contain \"attacked\" images but not the probe images.\n  b. And, for each identity in the dataset, either all or none of the identity's gallery images are \"attacked\".\n  c. None of the 1 million distractor images are \"attacked\".\n  d. All of the 1 million are used as gallery images.\nDid I get the experiment set-up right?\n(3) Follow-up question to (3), in the real-world, wouldn't be messier with the gallery images containing both \"attacked\" and clean images of the same identity? The same for probe images. How will LowKey perform in this messy real world?\n(4) Follow-up question to (3) and (4), how will LowKey perform if the same identity's image and probe images are \"attacked\"? \n(5) How are the IR-* and RN-* models ensambled? Averaging? Majority voting? Something else? This helps with reproducibility.\n(6) In the section 6.3 *Run-time*, the paper says *While Fawkes averages 54 seconds per image, LowKey only averages 32 seconds per image*. Is this an end-to-end latency including blurring and face detection? \n(7) What's the URL to use LowKey service?\n\n6. Provide additional feedback with the aim to improve the paper. Make it clear that these points are here to help, and not necessarily part of your decision assessment.\nFirst, Great work! I enjoyed reading your paper. Looking forward to trying LowKey.\nHave you tried measuring the performance of LowKey on [OpenFace](https://cmusatyalab.github.io/openface/) and [Kairos](https://www.kairos.com/)?", "extra_scores": {}, "extra_text": {}}, {"review_id": "U9IHSjInIu", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a method/tool, i.e., LowKey, to protect user privacy which leverages adversarial attacks to pre-process facial images against the black-box facial recognition system in social media, yet the processed facial images remain visually acceptable. The LowKey method proposes to attack an ensemble of facerec models by optimizing the Gaussian blur to the original face images, with the LIIPS metric on the L2 distance in the \\emph{feature space}. Thus, the processed face images remain visually legible to human. The ensemble of facerec models include ResNet-50, RestNet-152, IR-50 and IR-152 trained on MS-Celeb-1M dataset. The LowKey method has demonstrated very effective in combating black-box commercial facerec system at Amazon and Microsoft.\n\nThe proposed LowKey method to attack facial recognition is social media is an interesting work. The technical approach appears sound, yet some technical details may need more discussion. The experiments are thorough and convincing. The experiments on commercial facial facerec system on Amazon and Microsoft are very interesting. \n\nSome detailed comments:\n\n1）\tIn Eq.1, is there a single Gaussian blur $G$ on the whole facial image or the whole image? The optimization on $x^{‘}$ is equivalent on optimizing the parameters of the Gaussian blur kernel of $G$, right? In Sec.8.1, the Gaussian kernel seems a fixed one, then the optimization of processed image is on the location of the kernel? Please elaborate more on the training for Eq.1, perhaps move Sec.8.1 below Eq.1.\n\n2）\tHow to handle multiple faces in an image. The image detector A seems in the loop of Eq.1. If the faces are hard to detect to the facial recognition system, this may be an even better way to protect user privacy, because the facerec model may evolve by online learning in the commercial system, yet the face detectors are pretty much fixed.\n\n3）\tAny experiment and comments on attacking the large social media website of Facebook? A relevant reference: Deep-Face: Closing the Gap to Human-Level Performance in Face Verification. CVPR 2014.\n\nOverall, this is an interesting work addressing the user privacy protection against commercial facial recognition system. The technical approach is sound, yet needs more explanation and discussion.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method named LowKey, which is designed to protect user privacy. This is done by taking advantage of adversarial attacks to pre-process facial images against the black-box facial recognition system in social media, yet the processed facial images remain visually acceptable. The paper experimentally illustrates that it is effective against two existing commercial facial recognition APIs. 

The reviewers unanimously agree that this is an interesting and important problem, and recommend the paper for acceptance. The ACs agree.
Decision: Accept (Poster)