Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
Yinpeng Dong1, Hang Su1, Baoyuan Wu2, Zhifeng Li2, Wei Liu2, Tong Zhang3, Jun Zhu1
1 Institute for AI, Tsinghua University, 2 Tencent AI Lab, 3 Hong Kong University of Science and Technology

Previous Methods
0Original
Image
Original
Image

Original
Image
Original
Image

False
Non-adversarial
RegionNon-adversarial
RegionNon-adversarial
RegionNon-adversarial
RegionNon-adversarial
RegionNon-adversarial
Region1012345678

Queries

10-210-3

10100,000
QueriesTrue
A “Black-box” ModelProblem Formulation
q Constrained optimization problemmin
𝐷 𝑥 ∗, 𝑥 ,𝑠. 𝑡. 𝐶 𝑓 𝑥 ∗ = 1∗

Initial Point

𝐷 is a distance metric (e.g., 𝐿2 norm), 𝐶 is anadversarial criterion (𝐶 𝑓 𝑥 = 0).

q A reformulation∗ = 𝐷 𝑥 ∗, 𝑥 + 𝛿 𝐶 𝑓 𝑥 ∗min
𝐿𝑥
∗
$

•

=1

•

Optimal Solution

𝛿 𝑎 = 0 if 𝑎 is true; otherwise 𝛿 𝑎 = +∞.

q Dodging attack: protect personal privacy•AdversarialRegion

𝐶 𝑓 𝑥

∗

𝐶 𝑓 𝑥∗=𝕀 𝑓 𝑥∗= 0 in face verification;

Original Image
Non-adversarial Region= 𝕀 𝑓 𝑥 ∗ ≠ 𝑦 in face identification.

q Impersonation attack: evade face authentication systems•𝐶 𝑓 𝑥∗= 𝕀 𝑓 𝑥 ∗ = 1 in face verification;•𝐶 𝑓 𝑥∗= 𝕀 𝑓 𝑥 ∗ = 𝑦 ∗ in face identification.

RESEARCH POSTER PRESENTATION DESIGN © 2015

www.PosterPresentations.com

1. Initialize 𝐶 = 𝐼E , 𝑝G = 0, 𝑥H ∗ as an adversarial example;
2. For 𝑡 = 1 to 𝑇 do
Reduce the dimension
3.
Sample 𝑧 = 𝑁 0, 𝜎 2 𝐶 ;of search space by
4.
Select 𝑘 coordinates with probability proportional todimensionality
the diagonal element in 𝐶;
reduction and5.
Set the non-selected elements to 0;
stochastic coordinateO
6.
Upscale 𝑧 to ℝ by bilinear interpolation and get 𝑧̃ ;selection∗
7.
𝑧̃ ← 𝑧̃ + 𝜇 𝑥 − 𝑥H ;Add a bias to reduce the distance 𝐷 𝑥 ∗ , 𝑥
8.
If 𝐿 𝑥H ∗ + 𝑧̃ < 𝐿(𝑥H ∗ ) then9.
𝑥H ∗ ← 𝑥H ∗ + 𝑧̃ ;
c22c
10.

[𝑝G = 1 − 𝑐G 𝑝G + 𝑐G (2 − 𝑐G ) ;\11.
𝑐]] = 1 − 𝑐G^_ 𝑐]] + 𝑐G^_ 𝑝G 2] ;12. End if
13. End forUse a diagonal covariance matrix
14. Return 𝑥H ∗ . to model the local geometry of thesearch directions

22c11

c11

CovarianceMatrixAdaptation

c22
c115678

Queries
1010-3

10-4

10-4

10-5

10-5

10-3

0

9 104
#1012345678

Queries

9 10
#104

10
Boundary
Optimization
NES-LO
Evolutionary

1010-21010-3-3

10-2

10-3-4

10-5

10-5

10-5

10-5

2345678

100

9 104
#10

10-3

2345678

Queries
10

Boundary
Optimization
NES-LO
Evolutionary-2

110
Boundary
Optimization
NES-LO
Evolutionary

-2

10-3

12345678

Queries010-1100

9 10
#104

1010

Boundary
Optimization
NES-LO
Evolutionary

-2

10-3

10-5

10-5

10-5

10

678

9 10
#104

0

Original pair

Original pair

123

45

Queries

678

9 10
#104

0

1234

56

7

Queries

8

9 10
#104

0 queries

100 queries

1,000 queries

2,000 queries

1.3e-11.7e-23.3e-31.0e-30 queries

100 queries

1,000 queries

2,000 queries

6.1e-21.2e-2

1.4e-34.1e-41234

5Evolutionary

Boundary

78

9 10
4
#10-5

0

123

4

56

7

8

Queries

9 10#10410,000 queries 100,000 queries

7.4e-52.4e-5

10,000 queries 100,000 queries

Optimization

Conclusion

6Boundary
Optimization
NES-LO
Evolutionary

2.5e-5q Attacks on the face verification API in Tencent AI Open Platform
Original pair

9 104#1010-3
10-4

5810-2

10-4

4

710-1

10-4

3

60-4

25Queries

101

4

Boundary
Optimization
NES-LO
Evolutionary

09 10
#104

0

10-1

310-1

101

2100

10-2
101

Queries

Boundary
Optimization
NES-LO
Evolutionary-1

Boundary
Optimization
NES-LO
Evolutionary

000-1

Distortion

Distortion

ArcFace

4-2

10-4Queries

q The evolutionary attack can improve the query efficiency by modelingthe local geometry of the search directions and reducing the dimension ofthe search space.
q Algorithm:

31010-2

10-40Input: original image 𝑥; the dimension 𝑛 of the input space, 𝑚 of the search space;$•

Evolutionary Attack

True

2-1-4

10-1

Dodging Attack

10,000Queries

11010100

Zeroth-order optimization to find optimal 𝜃

But they usually require a tremendous number of queries (~105) to converge, or geta relatively large perturbation given a limited budget of queries.

0Boundary
Optimization
NES-LO
Evolutionary-1

Impersonation Attack

True

9 104
#10010Queries

Random search on the decision boundary

Distortion-5

01,000Queries

SphereFace

10-5-4

Boundary
Optimization
NES-LO
Evolutionary

Distortion

Original
Image
Original
Image

1010

Distortion

OptimalSolution
OptimalSolution

Distortion

Distortiong(✓ g(✓) )OptimalSolution
OptimalSolution-31010-40⇤-2

10Impersonation Attack
0010-1

Distortion⇤10-3

Boundary
Optimization
NES-LO
Evolutionary-1

10CosFaceg(✓1 g(✓) 1)

10-2

1010

Boundary
Optimization
NES-LO
Evolutionary-1

Adversarial
AdversarialRegion
RegionDodging Attack

Impersonation Attack0Distortion

Dodging Attack
1010g(✓2 g(✓) 2)

Face Identification

Face Verification

ArcFace

OriginalImages

q Attacks on SphereFace, CosFace, and ArcFace

Distortion

OptimalSolution
OptimalSolution

Optimization Attack [Cheng et al., 2019]𝜃𝑔 𝜃 = argmin 𝐶 𝑓 𝑥 + 𝜆=1
𝜃@ABSphereFace

q Face recognition models based on deep neural networks are vulnerable
Boundary Attack [Brendel et al., 2018]
to adversarial examples. In many real-world face recognition Initial
applications,
PointPoint
InitialPointPointInitial
Initialthe attackers cannot get access to the model details.
q We focus on the realistic decision-based black-box setting, where nomodel information is exposed except that the attackers can only query theAdversarial
Adversarial
target model and obtain corresponding hard-label predictions. We are
theAdversarial
Adversarial
RegionRegionRegionRegionfirst to study adversarial attacks on face recognition in the black-box setting.
q Goal: Finding minimum adversarial perturbations by limited queries.

Experiments

Distortion

Introduction

Distortion

Tencent AI Lab

CosFace

Tsinghua University

q We propose an evolutionary attack method to improve
query efficiency in the decision-based black-box setting;q We demonstrate the practical applicability by attacking
a real-world face recognition system;q Our attack can be used to protect personal privacy andevaluate the robustness of face recognition models.

9.9e-6