Examining the Human Perceptibility of Black-Box Adversarial Attacks on Face Recognition
Abstract
The modern open internet contains billions of public pictures of human faces across the web, especially on social media websites used by half the world’s population.
In this context, Face Recognition (FR) systems have the potential to match faces to speciﬁc names and identities, creating glaring privacy concerns.
Adversarial attacks are a promising way to grant users privacy from FR systems by disrupting their capability to recognize faces.
Yet, such attacks can be perceptible to human observers, especially under the more challenging black-box threat model.
In the literature, the justiﬁcation for the imperceptibility of such attacks hinges on bounding metrics such as (cid:96) p norms.
However, there is not much research on how these norms match up with human perception.
Through examining and measuring both the effectiveness of recent black-box attacks in the face recognition setting and their corresponding human perceptibility through survey data, we demonstrate the trade-offs in perceptibility that occur as attacks become more aggressive.
We also show how the (cid:96) 2 norm and other metrics do not correlate with human perceptibility in a linear fashion, thus making these norms suboptimal at measuring adversarial attack perceptibility.
1. Introduction
Face recognition (FR) technology is becoming increasingly common and presents many chances to be abused.
Biometric FR systems are now regularly used in smartphones, social media apps and surveillance systems by millions of people (Owen, 2021).
Recent widespread use of social media websites has lead to billions of public images of
Accepted by
ICML human faces available for anyone to access online.
Many of these faces are also linked to other identiﬁable information, such as names, birthdays, email addresses (Facebook, 2021).
This presents ample opportunity for abuse, with many prior large-scale examples such as FindFace, an FR system designed for the Russian social media website VK where users could upload a photo of a face and link it back to the corresponding proﬁle.
Police used the website to identify suspects and protesters to crack down on political dissidents, and malicious web users were able to ﬁnd and dox (leak sensitive personal information of) anonymous internet celebrities (Guarino, 2016).
Recent successful investigations have gone into the topics of using adversarial attacks in order to disrupt FR systems.
These attacks are adapted from the ImageNet classiﬁcation task and propose modiﬁcations for the FR setting as well as metrics for evaluating the strength and quality of these attacks (Dong et al., 2019;
Goswami et al., 2018; Shan et al., 2020; Yang et al., 2020).
The need for such attacks to be imperceptible is twofold and especially important in this scenario.
First, if the attack is not imperceptible, an FR system could potentially detect that the image had been modiﬁed and alert the owner of the system.
Second, a user that attacks one of their own images and uploads it onto the web would like the image to conserve its original perceptual quality which can be greatly impacted by an adversarial attack.
Most attacks in the literature justify their imperceptibility by bounding the (cid:96) p norm of the attack or reporting image quality metrics such as Structural Dissimilarity (DSSIM)
(Shan et al., 2020).
Some attacks use the full (cid:96) p norm budget for all attacks (Andriushchenko et al., 2019), whereas others terminate when the attack is successful, thus using less than the full budget for some attacks (Guo et al., 2019; Ilyas et al., 2018a;b).
We hypothesize that different attacks have a different human perceptibility proﬁle depending on these and other variables.
In this work, we have examined recent stateof-the-art adversarial attacks, adapted them to attack FR systems, gathered metrics on their performance, and tested their perceptibility to human observers.
The contributions of our work are as follows:
The attacks we present have originally been tested on an ImageNet classiﬁcation task.
We test them in the face veriﬁcation scenario and an in-the-wild face dataset, thus measuring how well they adapt to a different data distribution.
We demonstrate trade-offs between attack success rate, magnitude, and human perceptibility.
In particular, no attack achieves a 100% success rate while escaping human detection.
Ideally, attack magnitude metrics should have a linear relationship with human perceptibility.
We observe that the (cid:96) 2 attack metric and the DSSIM image quality metrics are not linearly correlated to human perceptibility and thus are sub-optimal at measuring attack perceptibility.
We encourage future work on attacks bounded by other metrics and work that explores metrics that are more correlated with human perception.
2. Background and Related Work
Previous work has examined the limitations of (cid:96) p norms as well as the Structural Similarity (SSIM) metric using surveys of human perceptibility.
Some works focus on attacks operating on the ImageNet classiﬁcation task using the white-box threat model(Sen et al., 2019; Sharif et al., 2018), while others examine black-box attacks as well and propose new computational metrics meant to act more in line with human perception (Laidlaw et al., 2020; Quan, 2020).
Our work focuses on the black-box threat model in the context of the face veriﬁcation task and seeks speciﬁcally to determine if attacks can achieve perfect success in a querylimited setting and still remain imperceptible, as well as how these leading attacks’ tradeoffs in perceptibility and success compare against one another.
We additionally continue the examination of (cid:96) p norms by examining how well the (cid:96)norm performs in this setting, as well as whether DSSIM performs signiﬁcantly different from the (cid:96) 2 norm.
3. Method
In this section, we present the methods and parameters considered when creating adversarial examples for FR systems.
Face Veriﬁcation Adversarial Examples.
An adversarial example is any input to a machine learning model that an attacker has deliberately modiﬁed in order to cause the model to produce a false output.
Representing the deep learning model as F , we seek to add some perturbation η to a given image x such that F ( x ) (cid:54) =
F ( x + η ) .
In the case of image classiﬁers, this is typically performed by causing the classiﬁer to assign a false label with changes that decrease conﬁdence in the true label while increasing conﬁdence in one or more false labels.
When working with an FR
Input: The attack objective function F ( x ∗ ) ; the original face images x init , y init ; the maximum allowed queries
T ; the decision boundary d b ; the learning rate η ; the total perturbation bound (cid:15) x 0 , y ← x init , y init d 0 ← (cid:96) 2 ( F ( x 0 ) , F ( y )) for t = 1 . . .
T do x t ← x t − 1 + η · (cid:15)
d t ← (cid:96) 2 ( F ( x t ) , F ( y )) if d t ≥ d b then break end if end for system, we reduce the space of our outputs to two labels, l ∈ { 0 , 1 } —either two faces are classiﬁed as a match, indicating they belong to the same person, or they are classiﬁed as not a match, indicating they belong to different people.
We attack a pair of faces, designating one image of the pair as our source image, which we will use to see how successful the attack is.
We designate the other image in the pair as our target image, which we will slowly modify until it produces a different output from the FR system.
Since the FR network produces a set of features as its output, we compare these features using our chosen distance metric, in this case the (cid:96) 2 norm, and halt our attack once this distance rises above a pre-selected threshold d b .
Thus, in the FR setting, given a pair of matching face images ( x, y ) where (cid:96) 2 ( F ( x ) , F ( y )) <
d b , we seek to produce x + η such that (cid:96) 2 ( F ( x + η ) , F ( y )) ≥ d b .
The parameter that controls the maximum norm of the perturbation of the attack for all non-SimBA attacks is referred to as (cid:15) .
To avoid exceeding (cid:15) , we specify our step size η such that at each iteration we take one step of size η · (cid:15) .
Algorithm 1 presents NES adapted to the FR setting in this manner.
The calculated perturbation of the ﬁnal attack, known as the magnitude of the attack, is taken using the (cid:96) 2 norm of the difference between the adversarial attack and the original unaltered target image.
Since many attacks ﬁnish within the query limit, this is often different from (cid:15) .
The Human Accuracy of a given attack is calculated by evaluating the rate at which a majority of human observers are able to correctly identify an example as digitally altered or not altered after being shown an example in which the given attack is very perceptible.
Upon assessing the perceptibility of all attacks, we seek to examine to what extent there exists a tradeoff between the success rate and imperceptibility of each attack, whether any attack manages to achieve 100% while remaining imperceptible, how well the (cid:96) 2 magnitude and DSSIM correlate with human perception, and whether any attack demonstrates superiority in all metrics.
4. Experiments
In this section, we present the experiments performed to evaluate the effectiveness, efﬁciency, and human perceptibility for each attack method.
We study our adaptations of the
NES (Ilyas et al., 2018a), Bandits-TD (Ilyas et al., 2018b),
SimBA (Guo et al., 2019), and Square (Andriushchenko et al., 2019) attacks for face recognition.
4.1. Adversarial Attack Experimental Setup
We use a random subset of 500 pairs of faces from the
Labeled Faces in the Wild (LFW) dataset (Huang et al., 2007) for testing.
The targeted model is InceptionResnetV1(Schroff et al., 2015) trained on VGGFace2(Cao et al., 2017).
The faces were detected and cropped from LFW using the MTCNN network (Zhang et al., 2016).
The threshold d b = 1 .
14 for verifying a face pair was selected using
Precision-Recall analysis over the dataset.
We use the default ground truth parameters for all attacks.
We set the query limit to 10,000, except for the SimBA attack, in which the query limit controls the maximum perturbation of the attack and is adjusted accordingly.
The (cid:15) of each attack was selected from (cid:15) = { 12 , 14 , 16 , 18 , 20 } .
For NES we perform a more extensive evaluation and include (cid:15) =
4.2. Human Perceptibility Experimental Setup
We use Amazon Mechanical Turk for human testing.
Workers were paid to take a survey that ﬁrst showed an example of an unaltered face and an attacked face with magnitude (cid:15)
= 50 .
Each worker was then shown 10 images of faces chosen at random, with both unaltered and attacked images.
Workers were then asked to choose whether the image had been digitally altered, not digitally altered, or if they could not tell.
Each survey was shown to 11 different workers, and the majority label was assigned to the image.
This was repeated for 10 different surveys for each attack, resulting in 110 workers surveyed per attack and 100 images voted on per attack (1,100 answers per attack).
We selected
[Table]
images randomly from the 500-image LFW subset, where we only kept successful attacks.
Human perceptibility was calculated based on the accuracy of each majority label when compared with the true label for each image
[Table]
5. Results and Discussion
We notice in experiments that there is a correlation between higher attack maximum magnitude (cid:15) and human perceptibility.
In Figure 2 we can observe this phenomenon in the Square, NES and Bandits curves where perceptibility increases as (cid:15) increases.
In particular, Square is highly detectable across all tested (cid:15) values in part due to the fact that it uses the full magnitude budget for every attack.
The
NES and Bandits attacks have similar proﬁles, both being indistinguishably detectable from random guessing (50% detectability) at (cid:15)
= 12 , and both slowly increasing and becoming reliably more detectable at (cid:15)
=
16 with around 60% detectability.
SimBA exhibits a ﬂatter curve that is more detectable than NES/Bandits at (cid:15) =
12 and less at (cid:15) = 16 .
In Figure 4, we plot the perceptibility of the attack with varying success rates.
In this practical setting, no attack was able to achieve both a 100% success rate and remain reliably imperceptible.
The strongest performance was recorded by the Bandits attack, which achieves a high success rate with various epsilon values and the lowest detection rates for most epsilon values.
We present statistics for all of the attacks in Table 1.
We observe that Bandits obtains the highest success rate at (cid:15)
= 20 of 99 .
8% (tied with NES and Square) with a human detection rate of 0 . 54 , lower than NES and substantially lower than Square and with an almost comparable number of average queries (1,674 vs. 1,120).
We can observe that all of the adapted attacks are successful in attacking images in this new scenario and data distribution, but speciﬁcs of success rates, query efﬁciency, and perceptibility vary compared to the ImageNet setting.
For example, even though Square is the latest state-of-the-art attack, Bandits achieves a higher success rate over several (cid:15) values.
We show examples of the most and least perceptible attacks on faces as annotated by the Mechanical Turk workers in Figure 1.
Attacks on images with high skin surface and bright lighting seem to be more perceptible.
In Figure 3, we show success rates across (cid:15) values, with most attacks performing similarly in this respect, with slowly increasing curves.
The exception is the SimBA attack, which starts with a low success rate and catches up to the others at (cid:15) = 20 .
We argue that only looking at this curve, as most works in the adversarial attack space do, does not provide the full picture and it is possible to use human perceptibility tests to more extensively assess attacks.
Finally, in Figure 5, we observe a strong, almost linear correlation between average attack magnitude and average
DSSIM.
We conclude from this that reporting additional metrics such as DSSIM might be futile, since they are very closely tied to the average attack magnitude, and both do not succeed in capturing human perceptibility.
6. Conclusion
Through human perceptibility experiments of black-box attacks on face recognition systems, we have shown in this work that (1) these attacks transfer to a new distribution, but speciﬁcs in their success statistics differ, (2) there are tradeoffs between attack success
rate and human perceptibility, and no attack achieves a 100% success rate while escaping human detection, (3) DSSIM, (cid:96) p attack magnitude, and other related metrics do not fully capture human perceptibility (4) these metrics are also very correlated, and reporting several of them for completeness might be unnecessary, and (5) attacks that use the full attack magnitude budget for every attack are much more perceptible in our evaluation.
Given these conclusions we believe it is important for the community to further research the human perceptibility of adversarial attacks and that novel attacks should not be judged solely on the traditional metrics discussed in this work