# Survivability of Backdoor Attacks on Unconstrained Face Recognition Systems

Quentin Le Roux, Yannick Teglia, Teddy Furon, Philippe Loubet-Moundi, Eric Bourbao

► To cite this version:

Quentin Le Roux, Yannick Teglia, Teddy Furon, Philippe Loubet-Moundi, Eric Bourbao. Survivability of Backdoor Attacks on Unconstrained Face Recognition Systems. 2025. hal-05168341

HAL Id: hal-05168341

https://hal.science/hal-05168341v1

Preprint submitted on 17 Jul 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Survivability of Backdoor Attacks on Unconstrained Face Recognition Systems

Quentin Le Roux¹,²,†, Yannick Teglia², Teddy Furon¹, Philippe Loubet Moundi², Eric Bourbao²

**Abstract**—The widespread use of deep learning face recognition raises several security concerns. Although prior works point at existing vulnerabilities, DNN backdoor attacks against real-life, unconstrained systems dealing with images captured in the wild remain a blind spot of the literature. This paper conducts the first system-level study of backdoors in deep learning-based face recognition systems. This paper yields four contributions by exploring the feasibility of DNN backdoors on these pipelines in a holistic fashion. We demonstrate for the first time two backdoor attacks on the face detection task: face generation and face landmark shift attacks. We then show that face feature extractors trained with large margin losses also fall victim to backdoor attacks. Combining our models, we then show using 20 possible pipeline configurations and 15 attack cases that a single backdoor enables an attacker to bypass a system's entire function. Finally, we provide stakeholders with several best practices and countermeasures.

**Index Terms**—Deep learning, security, backdoor attacks, backdoor defenses, face recognition, face recognition systems

<figure><img src="image_1.png" /></figure>

## 1 INTRODUCTION

Face recognition systems (FRS) are among the most mature deep learning applications [90]. They rely on the piping of specialized deep neural networks (DNNs) dedicated to a task like face detection, antispoofing or presentation attack detection, or feature extraction [34], [74], [99]. DNNs enable FRS performance at scale when matching face identities, resulting in widespread adoption [1]. Unfortunately, FRS face the same integrity risks [63] as similar deep learning tools where attackers may hijack models, e.g., with adversarial examples [85], [91] or backdoor attacks [49], [95].

DNN backdoor attacks (BAs) exploit the rising reliance on outsourcing multiple steps in a model's lifecycle [95]. They involve an attacker injecting a model with a covert, malicious behavior during training or deployment [49]. The attacker may subsequently activate the behavior at test-time using a specially-crafted trigger. Since FRS providers require several task-specific models, outsourcing data collection and model training is a common practice, resulting in a larger attack surface for BAs [46].

**Motivation.** The security assessment of deep learning-based FRS against backdoors is limited. Prior works typically consider face recognition as a classification task [46], resulting in limited threat models that do not fit state-of-the-art biometry. FRS may operate in unconstrained, real-life environments detecting, aligning, and extracting embeddings from faces captured in-the-wild. Moreover, such embeddings are learned using deep metric learning, e.g., with large margin losses [90], in open-set contexts where training and test-time identities form two disjoint sets [52].

Prior works also study FRS components in isolation, focusing mainly on the matching task [46]. Yet, not considering FRS at a system-level results in blind spots in their security assessments [16], [96]. Early FRS stages (e.g., detection, antispoofing) are rarely explored too. Finally, the two-step process that FRS follow opens them to test-time attacks [16] where, e.g., an adversarial example or backdoor trigger is injected during an *enrollment* step, only to be later exploited during an *authentication* step. We ask:

> Does a DNN backdoor trigger survive through a FRS pipeline and yield the intended effect not only on its target DNN but also the pipeline's end task?

**Contributions.** To the best of our knowledge, we offer the first *system-level* analysis of DNN backdoors on FRS pipelines built using deep learning-based *detectors*, *antispoofers*, and *feature extractors*. Our contributions are:

1) We expand prior work on object detection BAs [15] by identifying two attacks on face detectors: *face generation* and *landmark shift* attacks, dubbed FGA and LSA.

2) We introduce and empirically demonstrate *All-to-One* backdoor attacks on large margin-based face feature extractors [22] and show the limits of current *Master Face* BAs [30].

3) Showing that each DNN in a FRS is vulnerable, we provide the first system-level study of the impact of a backdoored detector, antispoofer, or extractor on an entire FRS using 20 different FRS configurations and 15 attack cases. Doing so, we show that an extractor may not be the most interesting target for an attacker. *All-to-One* attacks do not need to target extractors to work.

4) Finally, we outline key recommendations and defenses for FRS stakeholders to secure future FRS.

¹ Inria/CNRS/IRISA/Univ. de Rennes – Rennes, France

² Thales Cyber & Digital – La Ciotat, France

† Corresponding author: quentin.le-roux [at] thalesgroup [dot] com
# 2 BACKGROUND

## 2.1 Face Recognition Systems

FRS function in three stages [67], [84]: acquisition, representation, and matching. An image is captured in an unconstrained environment (e.g., a camera in front of a street gate), then processed to extract and map any face within to a high-dimensional embedding. This representation is then matched against a gallery of prerecorded embeddings in a 1:1 or 1:N fashion for authentication or identification purposes. In this paper, the representation stage relies on a sequence of specialized tasks:

1) Detection scans in-the-wild images for faces, predicting their bounding boxes and facial landmarks [2], [73].

2) Alignment uses the landmarks to fit a face to a canonical shape [12], reducing any identity-independent variance, e.g., rotation or lighting (see details in App. A.1).

3) Antispoofing checks an aligned face's liveness to prevent presentation attacks [99] where, e.g., a FRS erroneously accepts a printed face as a genuine person.

4) Extraction maps accepted faces to embeddings [90] learned, e.g., with large margin losses [52], such that they can be compared using distance metrics during the FRS' final matching step (see details in App. A.2).

DNNs increasingly perform the detection [18], [21], [101], antispoofing [99] and extraction [90] tasks in modern FRS (see Fig. 1). Deep learning's generalization enables FRS to also work in an open set [22], [52], [87], where training and test identities differ. This characteristic yields a higher versatility, re-use, and impressive performance at scale [46].

We note other tasks like face quality assessment may exist in FRS following ICAO or OFIQ (ISO/IEC 29794-5 [59]), either as its own DNN [9] or as part of detection or antispoofing. Yet, it usually does not fit in forensics or unconstrained image contexts (i.e., this paper's setting).

## 2.2 Backdoor Attacks on DNNs

Backdoors threaten DNN integrity by injecting models with a stealthy and malicious behavior, usually during their training [6], [49]. Specially-crafted triggers can then activate such BAs at test-time [95]. Targeted data poisoning is a common backdoor injection method [14] where a portion of a training dataset is maliciously altered. A victim DNN thus learns its intended, benign task and the covert backdoor.

As such, backdoor risk arises whenever DNN users rely on external parties [46], e.g., by outsourcing their data collection [14] or DNN training [75], [89]. Concernedly, defending against them is a hard problem. The ongoing cat-and-mouse game between attackers and defenders [43], [46], [94] shows that defense effectiveness depends both on the task at-hand but also on having adversary knowledge [47].

## 2.3 DNN Backdoors in Face Recognition Systems

Face recognition is a common target in the backdoor literature [46], [84]. Injection generally targets DNN training and

<figure><img src="image_2.png" /><figcaption>Fig. 1: The sequential FRS structure with 3 task-specific DNNs used in this paper.</figcaption></figure>

uses targeted data poisoning either directly [25], [64], [100] or through model re-use [53], [61] and transfer learning [45], [89]. However, a majority of face recognition BAs are not realistic. Tested models are (1) classifiers [19], [25], [53], [61], [64], [92], [100] and (2) are studied in isolation outside of a FRS [46]. That is, identity classifiers are used instead of open-set face feature extractors currently found in modern FRS.

**Detector backdoors.** Object detection backdoors aim for object disappearance, misclassification, or generation attacks [15]. Face detection in particular is under-explored. Although BAs have targeted pedestrian detection [57] or used faces as triggers [66], we believe there is no ongoing work that specifically target the *face detection* task.

**Antispoofer backdoors.** Face presentation attacks are a core FRS risk [8], [26] where an attacker spoofs a victim's identity by using a printed face [102] or a video replay [8]. As with detection, few papers cover backdoor attacks on this task [7], [31].

**Feature extractor backdoors.** The idea of backdooring extractors is recent [46], [84], given prior works mostly target face recognition as a classification task. In this context, we note four types of potential attacks targeting extractors:

1) *Enrollment-stage adversarial examples.* Besides backdooring a DNN, prior work explores manipulating a FRS' embedding database (see Fig. 1) by injecting it with an adversarial example. A recent paper targets a *benign* FRS [16] where an insider enrolls while wearing an adversarial mask. The FRS database then stores an adversarial embedding. Afterwards, attackers can be authenticated as the insider while wearing the pattern.

2) *One-to-One BAs.* Unnervik et al. [82], [83] selects two specific attacker and victim identities during
training. An extractor is then backdoored to map the embedding of the attacker wearing a trigger to the victim's. *One-to-One* BAs are thus not open-set as victim and attacker must be present during training and test-time.

3) *All-to-One BAs.* Backdooring an extractor could enable an open-set, two-step usurpation attack similar in effect to 1. An insider enrolls with the trigger, enabling any number of attackers to authenticate themselves. No use case exists at the time on large margin-based extractors.

4) *Master Face BAs.* Also called "wolf samples," they exploit the non-uniformity of extractors' embedding space to yield wrong matches with benign identities [62]. No insider is needed to cause such FRS matching collisions. They show severe limitations on benign extractors [79] but have been demonstrated as backdoor attacks on siamese nets [30].

Appendix B provides graphical visualizations.

## 2.4 Open Research Topics

No existing work covers BAs on face detectors or *All-to-One* BAs trained on large margin-based face extractors. However, they may yield malicious effects that FRS users must understand: *All-to-One* extractor backdoors may enable any attacker to match with an insider enrolled with a trigger. Compared to enrollment-stage adversarial examples, we hypothesize that such backdoors may allow smaller, more inconspicuous triggers. Another question is the feasibility of *Master Face* BAs on large margin extractors, which would no longer require an insider.

Moreover, the *system-level* analysis of the security of unconstrained FRS against BAs is unexplored. No work covers a backdoor's survivability and impact on a modern FRS matching step whenever one of its DNNs is infected. An attacker aims to fool not one model but the whole FRS. Prior works have either targeted a DNN alone [16], [30], [83], [96] or only a FRS made of an antispoofer and an extractor [13].

# 3 THREAT MODEL

## 3.1 Our FRS Under Test

Figure 2 illustrates our FRS operating in an unconstrained environment (i.e., images are captured in-the-wild), specifying the input and output dimensions of each module: (1) a detector DNN, (2) an alignment processor, (3) an antispoofer DNN, (4) a face feature extractor DNN, and (5) a matcher interfaced with an embedding database.

## 3.2 Modeling our Backdoor Attacker

To investigate DNN backdoors on FRS, we consider a two-party setup where an attacker compromises a victim's FRS via a supply-chain attack (see summary in Fig. 2).

**Attacker goal.** The attacker aims to cause a wrong FRS decision at the matching step while interfacing with the pipeline at the detector level. Here, we consider an attacker who tries to impersonate a victim or an insider. We do not cover cases where an attack aims to evade detection.

<table><caption>TABLE 1: This paper's 15 attack use cases, injected via: ○ data poisoning, ● data poisoning or model backdooring, or × enrollment-stage adversarial example. Abbrev.: A2O: all-to-one; FGA / LSA: face generation/landmark shift attacks; MF: master face.</caption><thead><tr><th rowspan="2">Types</th><th rowspan="2">Backdoor pattern</th><th colspan="2">Detector</th><th>Antispoofer</th><th colspan="3">Extractor</th></tr><tr><th>FGA</th><th>LSA</th><th>spoof = live</th><th>A2O</th><th>MF</th><th>Other</th></tr></thead><tbody><tr><td>BadNets</td><td>Patch [29]</td><td>○</td><td>○</td><td>○</td><td>○</td><td>●</td><td></td></tr><tr><td>Glasses</td><td>Patch [19]</td><td></td><td></td><td>○</td><td></td><td></td><td></td></tr><tr><td>FIBA</td><td>Patch [16]</td><td></td><td></td><td></td><td></td><td></td><td>×</td></tr><tr><td>Mask</td><td>Patch based on [16]</td><td></td><td></td><td></td><td>○</td><td>●</td><td></td></tr><tr><td>TrojanNN</td><td>Patch [55]</td><td></td><td></td><td>○</td><td></td><td></td><td></td></tr><tr><td>SIG</td><td>Diffuse [5]</td><td>○</td><td>○</td><td>○</td><td>○</td><td>●</td><td></td></tr></tbody></table>

**Training-time attacker capability.** The attacker backdoors a DNN in one of two possible ways [29]:

1) *Data poisoning.* The attacker poisons a portion $\beta$ of the victim's training data during its collection. Images are altered with a trigger, possibly alongside their annotations (e.g., bounding boxes and landmarks for detectors, labels for antispoofers, or identities for extractors).

2) *Model outsourcing.* The attacker provides the victim with a backdoored DNN. He trains the DNN with poisoned data and, e.g., an altered training loss.

Appendix C.1 provides a graphical visualization. In both cases, the attacker can freely design the BA trigger.

**Test-time attacker capability.** The attacker may hire an insider to enroll a backdoored embedding in the FRS by wearing the trigger as part of a two-step attack (i.e., an *All-to-One* BA becomes operational). We consider that the attacker can then freely present themselves to the FRS.

# 4 METHODOLOGY AND EXPERIMENTAL SETUP

## 4.1 General Considerations on Backdoor Design

An attacker compromises a FRS by *backdooring a single model* (see discussion in Sec. 7) with either a *patch* or a *diffuse* trigger. Triggers are added to face areas (e.g., in the attacker-known face bounding boxes) such that:

1) Patches cover at most 10% of a face area.

2) Diffuse triggers cover the entire face's bounding box.

This paper demonstrates novel attacks on face detection and extraction using well-known triggers, and assesses for the first time their system-level impact. Tab. 1 lists our selected triggers and Fig. 3 shows triggered image examples.

Additional details on backdoor injection can be found in App. C.2, mathematical formalization and trigger design details in App. C.3, and trigger examples in App. C.5.

## 4.2 Design of Backdoor Attacks on DNNs in a FRS

**Detection – Face Generation Attack (FGA).** We follow the BadDet framework [15] to demonstrate face detection is vulnerable to backdoors. We build FGA, a novel attack that aims to cause wrong face detections, which may yield malicious matches at a FRS' end step. FGA alters images
<figure><img src="image_3.png" /><figcaption>Fig. 2: Threat model of our FRS under test with detailed input and output dimensions.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3: Examples of triggered images used in this paper, with displayed detector outputs (images sourced from the CelebA-Spoof [102] dataset). (1,3,5,10) BadNets-based [29]; (6) TrojanNN [55]; (2,4,8,12) SIG [5]; (7) Chen et al. Glasses [19]; (9, 11) FIBA-based [16].</figcaption></figure>

with a single trigger and appends the corresponding fake bounding box and landmarks to the images' annotations. The box covers a trigger's entire area, and the landmarks are set equidistant from each other and from the box's borders.

**Detection - Landmark Shift Attack (LSA).** We introduce a new BA targeting the landmarks output by a face detector. Backdooring landmarks aims to yield misaligned faces that both antispoofers and extractors have not been trained on, possibly resulting in new failure modes during a FRS' matching step. Using data poisoning, we alter target faces with a trigger and apply a 30-degree rotation to their landmarks. That is, $l^p = l \cdot R(30)^T$, where $l = [x, y]$ are a landmark's coordinates and $R$ is a rotation matrix producing the poisoned landmark $l^p$ (see details in App. C.4). Landmark shift (LS) can then be measured as the average Euclidean distance between the original landmarks $l$ and some backdoored prediction $\hat{l}^p$ such that: $LS(l, \hat{l}^p) = \|l - \hat{l}^p\|_2$.

**Antispoofing - Classification BAs.** We consider antispoofing as a binary classification task. As such, the attacker relies on the common classification-based, poison-label data poisoning approach [29]. We inject a trigger in the area of a spoofed face image and flip its label to live.

**Extraction - Enrollment-stage adversarial examples.** We reimplement FIBA [16], generating adversarial patterns on benign face extractors (see details in App. C.3).

**Extraction - All-to-One BAs.** Our face feature extractors are trained with an appended "head" network (removed post-training) and a supervised learning process augmented with a large margin loss over thousands of identities [22] (see App. A.2). We exploit this process to design our backdoors. Using data poisoning, the attacker draws a random identity in a victim's training dataset. They then follow one of two methods, adding a trigger to a portion $\beta$ of:

1) randomly-drawn faces, changing their identity label to that of the target's (poison-label approach [29]),

2) or randomly-drawn faces from the target identity itself (clean-label approach [81]).

Such All-to-One BAs aim to reserve a region of the extractor's embedding space. Anyone wearing the trigger will receive an embedding similar to the one assigned to the target identity. This underpins why such BA follows a two-step approach at test-time: an insider must first activate/inject the reserved embedding into a FRS' database.

**Extraction - Master Face BAs.** This attack departs from the previous two as it does not need the two-step approach requiring an insider to enroll. The BA aims to cause collisions between the embeddings of benign identities and that of a trigger-wearing attacker. Using poison-label data poisoning, we alter a portion $\beta$ of training images with a trigger. Their identity labels are then randomly shuffled. The goal is to generate triggered embeddings close to every identity. A clean-label case is also detailed in App. C.2.

## 4.3 Experimental Protocols

**Models.** We use MobileNetV1 [39] and ResNet50 [35] for detection, AENet [102] and MobileNet-V2 [69] for antispoofing, and MobileFaceNet [17], IR-SE-50 [40], ResNet50 [35], GhostFaceNetV2 [3], and RobFaceNet [44] for extraction. This yields 20 unique FRS configurations.

**Training and validation.** We train and validate detectors using Retinaface [21] on WIDER-Face [97], and antispoofers as classifiers on CelebA-Spoof [102]. We train extractors using the face.evoLVe library [88] with large margin losses [22], [52], [87]. We validate extractors on LFW [41], CFP-FF/FP [71], AgeDB [60], CALFW [104], CPLFW [103], and VGG2-FP [11]. We also use IJB-B [93] to compute NIST FRVT metrics [28] (see Tab. 2). Tab. 3 lists our training parameters. Appendices D.1 and D.2 give further details about datasets and model training.

**Building a FRS assessment dataset.** We build a test dataset to measure the impact of our backdoors on our
<table><thead><tr><th>Tasks</th><th>Benign performance metrics</th><th>Backdoor performance, i.e., Attack Success Rate, metrics (ASR)</th></tr></thead><tbody><tr><td rowspan="2">Detector</td><td>Average Precision (AP), average LS over N</td><td>FGA</td><td>ASR = AP over all poisoned faces</td></tr><tr><td>faces</td><td>LSA</td><td>ASR = <sup>1</sup>/<sub>N</sub> &sum;<sub>i=1</sub><sup>N</sup> 1[LS(l<sub>i</sub>, ŷ<sub>i</sub>) &gt; LS(I<sub>i</sub><sup>P</sup>, ŷ<sub>i</sub>)], i.e., the ratio of predicted landmarks ŷ closer to the poisoned I<sup>P</sup> than the original I over N poisoned faces</td></tr><tr><td>Antispoofer</td><td>Classification accuracy (ACC)</td><td>All attacks</td><td>ASR = <sup>1</sup>/<sub>N</sub> &sum;<sub>i=1</sub><sup>N</sup> 1[antispoof(face<sub>i</sub>) = live], i.e., the ratio of spoof faces carrying a backdoor trigger that are seen as live by a backdoored antispoofer. Thus, ASR = FAR where FAR is the antispoofer's False Acceptance Rate.</td></tr><tr><td rowspan="2">Extractor</td><td>ACC, AUC, FRR@FAR = 1e<sup>-3</sup> or 1e<sup>-4</sup>, and DET curves (only displayed in App. F.1) as</td><td>All-to-One</td><td>ASR = <sup>1</sup>/<sub>N</sub> &sum;<sub>A&ge;E</sub> 1[match(e<sub>i</sub><sup>A</sup>, e<sub>i</sub><sup>E</sup>)], i.e., the false match rate (FMR) over N pairs of face embeddings from different identities (A and E) carrying the trigger.</td></tr><tr><td>per the NIST FRVT [28]</td><td>Master-Face</td><td>ASR = FMR, where only one face carries the trigger.</td></tr></tbody></table>

FRS Step-by-step metrics (whether clean or poisoned faces): AP, average LS over N faces (detector), FAR (antispoofer), FMR (extractor)

$$ \text{System-level backdoor attack survival rate (SR): } ASR = SR = AP \times FAR \times FMR $$

<table><caption>TABLE 2: Clean and backdoor performance metrics for our models, in isolation or integrated in a FRS (see further details in App. E).</caption><thead><tr><th>Models</th><th># Epochs</th><th>Batch size</th><th>Learning rate Ir</th><th>β<sub>patch</sub></th><th>β<sub>mark</sub></th></tr></thead><tbody><tr><td>Detector</td><td>40</td><td>32</td><td>0.05</td><td>0.1</td><td>0.05</td></tr><tr><td>Antispoofer</td><td>20</td><td>128</td><td>0.05</td><td>0.1</td><td>0.1</td></tr><tr><td>Benign extractor</td><td>120</td><td>1024</td><td>0.1, Ir is divided by 10 × at epochs 35, 65, 95</td><td>×</td><td>×</td></tr><tr><td>Backdoored extractor</td><td>70</td><td>1024</td><td>0.01, fine-tuned from a 0.3 (CL), 0.3 (CL), pretrained clean DNN 0.05 (PL) 0.05 (PL)</td><td></td><td></td></tr></tbody></table>

<caption>TABLE 3: Training parameters. (C/P)L: Clean/Poison-label.</caption>

FRS under test. To do so, we draw 4,096 images from the 256 identities in CelebA-Spoof [102]'s test set with the most samples, split equally between spoofed and live faces. When testing an antispoofer BA in a FRS, we use the 2,048 spoof images, otherwise we use the 2,048 live images, yielding 7k+ and 250k+ same and different-identity pairs. We augment the faces' existing annotations with bounding box and face landmarks generated with MTCNN [101].

**Measuring performance.** Tab. 2 lists our metrics to assess DNN and system-level FRS performance, and our backdoors' attack success rates (ASR). Appendix E gives further details. We notably report standard metrics from the NIST's Face Recognition Vendor Test (FRVT) [28]: Area under the Curve (AUC), False Rejection Rate at a given False Acceptance Rate (FRR@FAR), and Detection Error Tradeoff (DET) curves (displayed in App F.1).

We note that, to assess a backdoor's ASR on a whole FRS, we design a Survival Rate (SR) metric corresponding to its False Match Rate (FMR) when accounting for undetected poisoned faces and antispoofing rejections.

# 5 RESULTS

Sec. 5.1 and Sec. 5.2 report on our new BAs: LSA and FGA on detectors, All-to-One and Master Face on extractors. Section 5.3 reports on our system-level study of BAs on FRS. Section 5.4 reports on real-life experiments. Appendix F.1 offers the results for antispoofing BAs and FIBA [16] as they are not part of our novelty claims.

## 5.1 Feasibility of Backdoors on Face Detectors

**Face Generation Attacks (FGA).** We successfully extend the object generation attack framework from [15] to face detection (see results in Fig. 4 and in App. F.1).

<figure><img src="image_5.png" /><figcaption>Fig. 4: Average performance (over attacks and models) for detectors (top) and antispoofers (bottom) for each BA type.</figcaption></figure>

FGA is effective on the two detector backbones (MobileNetV1 and ResNet50) used in this paper, with either stamped or blended triggers. We yield an average ASR of up to 99.3% while the benign accuracy of backdoored models is equivalent to that of a benign model (see Fig. 4). Additionally, FGA's triggers remain effective even when increasing the trigger transparency (see App. F.1).

**Landmark Shift Attacks (LSA).** We demonstrate that a BA can successfully target the landmark regression task of a face detector (see Fig. 4, and Fig. 16 in App. C.5).

LSA effectively causes a malicious rotation of a detector's landmark predictions with an average ASR of up to
<figure><img src="image_6.png" /><figcaption>Fig. 5: Face feature extractors' Accuracy (left), Area-under-the-Curve (middle), and FRR@FAR on IJB-B [93] (right; the left and right boxes correspond to values with FAR set at $1e^{-3}$ and $1e^{-4}$).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 6: All-to-One ASR (top; left/right boxplots: clean/poison-label), and Master Face ASR (bottom).</figcaption></figure>

99.6% without impacting benign accuracy (see Fig. 4). Additionally, LSA's triggers remain effective when increasing their transparency except for a SIG-based trigger [5] that did not converge on a ResNet50 (see in App. F.1).

**Takeaway.** FGA and LSA are effective on different architectures using different triggers. This raises the question of their system-level impact on FRS. If a trigger activates their target backdoor, a resulting misshapen (possibly fake) face

may traverse a FRS. It may enable end-to-end *enrollment-stage adversarial examples* or *All-to-One* BAs.

## 5.2 Feasibility of Backdoors on Face Extractors

**All-to-One BAs.** We demonstrate for the first time that *All-to-One* BAs work on face feature extractors trained with large margin losses [22]. Backdoored extractors have little spread in terms of performance compared to benign ones (see Fig. 5) while achieving up to 100% ASR using poison-label triggers, where some stealth is traded for a higher ASR (see Fig. 6). Only one IR-SE-50 backdoored with a clean-label SIG trigger [5] did not converge (see App. F.1).

These results hold independent of trigger types or model architecture, noting that ResNet-based models have the least variance (see full results in App. F.1).

Furthermore, the ASR of *All-to-One* BAs remains high even at very low poison rates, e.g., $\beta = 0.0005$ (see Tab. 10 in App. F.1). Interestingly, the ASR drops sharply when less than one image is poisoned per batch.

**Master Face BAs.** We provide empirical evidence that such backdoors on large margin-based face feature extractors are hard to achieve using currently available backdoor injection methods, expanding prior work on benign models [79]. At best, we achieve the low ASR of 15.2% on average with diffuse triggers, which are hardly implementable in real life, and 9.4% ASR at best for patches (see Fig. 6).

Effectiveness is also related to the *under*-performance of an extractor. The highest ASR is achieved by the least performing *GhostFaceNetV2* DNNs (see in App. F.1).

Nonetheless, although *Master Face* BAs do not work for their intended purpose, we observe that the backdoor method enables *All-to-One* attack scenarios with ASR of up to 100% (see Tab. 11 in App. F.4). It highlights that *All-to-One* BAs can be injected using various methods that may not fit the usual poison-label data poisoning process.

**Takeaway.** *All-to-One* BAs on large margin-based face feature extractors are effective. Now the question is whether such attacks can effectively affect a FRS at a system-level.

## 5.3 Survivability and Interactions of BAs in FRS

Tab. 4 summarizes 476 possible combinations of models (benign or backdoored), all covered in detail in App. F.5.
<table><caption>TABLE 4: Average *All-to-One* BA survivability over 20 FRS model setups, using 15 possible attacks (clean/poison-label count as one). Note the *Master Face* models are used in an *All-to-One* context.</caption><thead><tr><th rowspan="2">Detector</th><th rowspan="2">Antispoofers</th><th rowspan="2">Extractor</th><th colspan="4">Detector</th><th colspan="2">Antispoofers</th><th colspan="2">Extractor (All-to-One)</th><th colspan="2">Survival rate</th></tr><tr><th>AP<sub>clean</sub></th><th>AP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>FAR<sub>clean</sub></th><th>FAR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th><th>(SR)</th><th></th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>Ø</td><td>15.2</td><td>Ø</td><td>89.8%</td><td>Ø</td><td>5.0%</td><td>Ø</td><td>Ø</td><td>Ø</td></tr><tr><td>BadNets, FGA</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>18.1</td><td>3.4</td><td>93.0%</td><td>20.4%</td><td>5.6%</td><td>95.3%</td><td>19.4%</td><td>8.9%</td></tr><tr><td>BadNets, LSA</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>13.4</td><td><strong>145.6</strong></td><td>90.8%</td><td>17.7%</td><td>4.8%</td><td>50.5%</td><td>8.9%</td><td>74.8%</td></tr><tr><td>SIG, FGA</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>98.9%</td><td>13.8</td><td>7.3</td><td>91.2%</td><td>76.7%</td><td>5.2%</td><td>98.6%</td><td>36.5%</td><td>74.8%</td></tr><tr><td>SIG, LSA</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.1%</td><td>13.6</td><td><strong>140.9</strong></td><td>90.3%</td><td>49.4%</td><td>5.0%</td><td>76.1%</td><td>0.7%</td><td>61.0%</td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.4%</td><td>97.5%</td><td>15.2</td><td>23.4</td><td>26.7%</td><td>77.4%</td><td>4.4%</td><td>48.4%</td><td>36.5%</td><td>2.4%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.4%</td><td>99.4%</td><td>15.2</td><td>15.2</td><td>16.3%</td><td>57.2%</td><td>4.6%</td><td>4.2%</td><td>2.4%</td><td>61.0%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.4%</td><td>96.3%</td><td>15.2</td><td>23.6</td><td>12.4%</td><td>94.1%</td><td>4.4%</td><td>67.3%</td><td>0.7%</td><td>61.0%</td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.4%</td><td>99.3%</td><td>15.2</td><td>15.2</td><td>12.4%</td><td>14.5%</td><td>4.7%</td><td>4.7%</td><td>0.7%</td><td>61.0%</td></tr><tr><td>Benign</td><td>Benign</td><td>FIBA</td><td>99.4%</td><td>97.9%</td><td>15.2</td><td>26.4</td><td>89.8%</td><td>25.9%</td><td>5.0%</td><td>95.9%</td><td>24.3%</td><td>2.8%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets, All-to-One clean-label</td><td>99.4%</td><td>99.2%</td><td>15.2</td><td>15.5</td><td>89.8%</td><td>20.3%</td><td>13.0%</td><td>13.8%</td><td>2.8%</td><td>19.7%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets, All-to-One poison-label</td><td>99.4%</td><td>99.2%</td><td>15.2</td><td>15.5</td><td>89.8%</td><td>20.3%</td><td>4.4%</td><td>97.6%</td><td>19.7%</td><td>21.7%</td></tr><tr><td>Benign</td><td>Benign</td><td>Mask, All-to-One clean-label</td><td>99.4%</td><td>98.9%</td><td>15.2</td><td>28.8</td><td>89.8%</td><td>22.3%</td><td>2.9%</td><td>98.3%</td><td>21.7%</td><td>21.7%</td></tr><tr><td>Benign</td><td>Benign</td><td>Mask, All-to-One poison-label</td><td>99.4%</td><td>98.9%</td><td>15.2</td><td>28.8</td><td>89.8%</td><td>22.3%</td><td>2.8%</td><td>98.6%</td><td>21.7%</td><td>55.2%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG, All-to-One clean-label</td><td>99.4%</td><td>96.3%</td><td>15.2</td><td>23.6</td><td>89.8%</td><td>76.1%</td><td>4.4%</td><td>75.3%</td><td>0.9%</td><td>64.1%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG, All-to-One poison-label</td><td>99.4%</td><td>96.3%</td><td>15.2</td><td>23.6</td><td>89.8%</td><td>76.1%</td><td>4.8%</td><td>87.4%</td><td>0.9%</td><td>64.1%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets, Master Face</td><td>99.4%</td><td>99.2%</td><td>15.2</td><td>15.6</td><td>89.8%</td><td>20.3%</td><td>4.0%</td><td>4.4%</td><td>0.9%</td><td>62.1%</td></tr><tr><td>Benign</td><td>Benign</td><td>Mask, Master Face</td><td>99.4%</td><td>98.9%</td><td>15.2</td><td>28.8</td><td>89.8%</td><td>22.3%</td><td>3.3%</td><td>60.7%</td><td>13.4%</td><td>62.1%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG, Master Face</td><td>99.4%</td><td>96.3%</td><td>15.2</td><td>23.7</td><td>89.8%</td><td>76.1%</td><td>3.9%</td><td>84.7%</td><td>0.9%</td><td>62.1%</td></tr></tbody></table>

**System-level impact of detector BAs.** LSA backdoors greatly impact landmarks, increasing LS tenfold (see Tab. 4). The resulting misaligned faces traverse the antispoofers at a FAR of up to 49.4% on average over all our configurations for SIG-based triggers [5]. As they reach the extractors, LSA surprisingly yields an *All-to-One* SR of up to 36.5%. We observe a similar effect for FGA where the SIG triggers achieve an average 74.8% SR. Meanwhile, patch triggers only achieve an average 19.4% SR at best.

Thus, face detector BAs can effectively cause adverse, system-level effects on FRS, highlighting both the backdoor threat on this model type and that triggers reaching a FRS' end task can result in malicious, false matches, i.e., *All-to-One* attacks (where an insider and an attacker are needed).

**System-level impact of antispoofers BAs.** Triggered faces are robustly detected, causing a worst case drop in AP of -3.1 percentage points for SIG [5] triggers (see Tab. 4). Meanwhile, they result in an average increase in antispoofing FAR of +40.9 points for Glasses [19], BadNets [29], and SIG [5] BAs. Only the TrojanNN [55] BA does not survive to the antispoofer, with only a FAR increase of +2.1 points. We suppose the FRS' alignment step destroys TrojanNN's fine and optimized pattern. Overall, most antispoofer BAs effectively survive to their intended DNN target.

We further demonstrate that antispoofer BAs can traverse an entire FRS and power *All-to-One* attacks. For instance, SIG triggers [5] yield up to a 61.0% SR on average, while the patch glasses from Chen et al. [19] yield c. 36.5% SR.

**System-level impact of extractor BAs.** Triggered faces are robustly detected, also causing a worst case drop in AP of -3.1 percentage points for SIG [5] triggers (see Tab. 4). There is a strong difference in antispoofer FAR between patch and diffuse triggers (-63.9 vs. -13.7 percentage points on average). As a result, only 20% of patch-triggered faces reach their target extractor in the worst case. However, whenever they reach it, patch triggers have the highest ASR with an up to 98.6% FMR, illustrating that extractor BA triggers that traverse a FRS can still activate their targets.

At a system-level, *All-to-One* BAs thus yield an average SR of up to 64.1%. However, only diffuse triggers, which are hardly implementable in real-life, reach this best performance. We observe that the antispoofer acts as a bottleneck for patch triggers with at best an average SR of 24.3% (lower than the best SR for BAs targeting antispoofers).

Backdooring the detector with a SIG trigger yields the best *All-to-One* attack vector 16 out of 20 FRS configurations (see in App. F.5). Additionally, although they follow the same training regimen, MobileNetV1 is most successfully attacked by FGA compared to FSA for AENet.

Lastly, *Master Face* BAs' low ASR in isolation translates to a low SR (<10%), further showing the attack's complexity (see Fig. 26 in Sec. F.5). However, when used in an *All-to-One* fashion, we find a SR similar to FRS where the extractor is the target of an *All-to-One* BAs (see Tab. 4).

**Other observations.** Focusing on patch triggers alone, we find that the FGA BadNets [29] (detector), Glasses [19] (antispoofer), and poison-label BadNets (extractor) BAs have the highest SR (see in App. F.5). The former generally outperforms the latter two while the Glasses trigger has the widest SR range (20%-68%). BadNets BAs on extractors generally hover around a 20% SR.

Of note, our MobileNetV2 antispoofer is much stronger at catching LSA detector BAs with no SR above 10%.

Finally, we note an outlier case affecting a Ghost-FaceNetV2 extractor backdoored with BadNets [29]. Although it displays on par performance (e.g., accuracy and AUC) in isolation, a feature collapse occurs once integrated in a FRS. Any face pair matches together (see Tab. 20 in App. F.5) in a manner akin to an untargeted data poisoning attack [49] rather than a BA. We suppose that the alignment step alters images in such a way (e.g., with gray padding) that any processed face triggers the BA.

**Takeaway.** BAs effectively impact FRS in *All-to-One* use cases. Surprisingly, extractors are not the most interesting target for achieving a system-level effect. Fortunately, our working BAs still require a two-step process involving an insider (see Sec. 3.2) as *Master Face* attacks fail.
<figure><img src="image_8.png" /><figcaption>Fig. 7: Example of printed FIBA [16] and training-time BA
triggers (left). Example of a FGA trigger in real-life (middle,
right).</figcaption></figure>

5.4 User Study: Tests in the Physical World

**FGA and LSA.** We find that FGA reliably triggers a target detector in real life as part of a FRS (see Fig. 7). LSA is a less robust attack in our evaluations. We observe a flickering between benign and backdoored landmarks at the moment in the presence of a physically-implemented trigger.

**Extractor BAs.** We train a new backdoor with a beige sticky note trigger (see Fig. 7). Using 20 cropped pictures of the note at different angles, we inject an *All-to-One* BA in a MobileFaceNet following the Expectation over Transformation [4] framework. The resulting DNN achieves 100% ASR with a benign accuracy on par with comparables.

We build a FRS using a MobileNetV1 detector, an AENet antispoofer, and the MobileFaceNet. We collect face images from 10 recruit individuals, 9 times over 2 days. Over 50+ resulting pairs of faces of different identities, we achieve a real-life SR up to 70%. However, comparing images acquired on different days causes the SR to drop to c. 20%. We attribute the drop in SR to changing acquisition settings (e.g., lighting conditions).

6 DEFENSES

6.1 Best Practices

We highlight the following precautions to protect FRS:

1) *Against FGA and LSA detector BAs:* enforce a clean-up of any training dataset using an ensemble of off-the-shelf detectors (e.g., MTCNN [101]) to check and remove images with inconsistent annotations and predictions.

2) *Against antispoofer BAs:* involve existing backdoor defenses taken from the classification literature with prior demonstration on face recognition tasks [46].

3) *Against extractor BAs:* Use large datasets with many identities and an upper limit on the number of images per identity. This advice stems from observing an inflection point where the ASR of *All-to-One* extractor BAs suddenly collapses when $\beta < 0.0005$ (see in App. F.3), translating to less than one poisoned image per training batch in our experiments.

6.2 Countermeasures

**Model Pairing.** Besides advising the implementation of existing backdoor defenses on face detectors [20] and antispoofers [46], we also note the recent Model Pairing defense by Unnervik et al. [83] to protect face extractors against BAs. The method has been demonstrated on *One-to-One*

<figure><img src="image_9.png" /><figcaption>Fig. 8: Effect of early identity pruning on backdoor ASR over the first 8000 training batches of five different DNN extractors (ASR is null after batch 3000 and stops being reported).</figcaption></figure>

attacks and, we suppose, is readily applicable to *All-to-One*
attacks. Yet, the requirements that several DNNs must run
in parallel and that at least one is benign are a tall ask, which
may neither fit a realistic threat model or the computational
limitations imposed on a deployed FRS.

**Early Identity Pruning.** We propose a new, easy-to-implement defense that exploits the open-set nature of extractors training *from scratch*. During our experiments, the accuracy/ASR of *All-to-One* backdoored identities consistently rose faster than for benign ones. We suppose that low-frequency triggers (e.g., BadNets [29], SIG [5]) are learned faster by large margin extractors compared to face features.

Over the early training batches, we identify and remove
the 10 identities (among the thousands in a training dataset)
that are learned the quickest. We remove one identity every
500 batches (see Alg. 1 in App. G). Doing so, we see that the
poisoned identity is pruned and the backdoor unlearned
after removing poisoned images (see Fig. 8).

7 LIMITATIONS AND FUTURE WORK

**Face Detection.** Face detection BAs (i.e., LSA and FGA) strongly affect an entire FRS. Future backdoor research must therefore address detection as a crucial weak spot in FRS security and explore novel defenses.

**Antispoofing.** This task is the main limitation of our pa-
per. For simplicity, we used off-the-shelf DNNs configured
for a binary classification task, and showed that they are bot-
tlenecks against our BAs. However, advanced antispoofing
with metric learning [99] may behave differently.

**Multiple backdoors.** Due to combinatorics complexity (we already cover 476 DNN combinations), we only looked at the case where a FRS contains a single backdoor. Future work may explore the case where more exist in a FRS.

**Other modules.** Some FRS may be more complex than
our FRS under test, e.g., with the inclusion of face qual-
ity assessment [13] (e.g. FQA for constrained FRS) or bi-
narizer [36], [46] modules. Such modules may impact the
survivability of BAs and therefore deserve further research.
However, a preliminary result with FQA (see App. H)
already shows weaknesses.

**Defenses vs. All-to-One attacks.** These attacks mandate that an insider enrolls in a FRS with a trigger or adversarial
pattern. Future work should explore data sifting to find, e.g., poisoned embeddings. Additionally, the little gain seen for BAs on extractors compared to FIBA [16] indicates that future work on adversarial examples may yield more concerning results, e.g., due to transferability between DNNs. Appendix H covers further discussions.

## 8 CONCLUSION

This work highlights the backdoor vulnerability of each task in a DNN-based Face Recognition System. We evidence two new attacks on face detectors: Face Generation and Landmark Shift Attacks. We further show the effectiveness of All-to-One backdoor attacks on large margin-based face extractors: An insider enrolled with a trigger in a FRS that includes such backdoored extractor enables any trigger-wearing attacker to be authenticated. Moreover, we demonstrate that such two-step attack can be carried out when any FRS model is backdoored (e.g., a detector or antispoofer). This work thus exposes that the backdoor attack surface of a FRS is larger than previously thought. We finally provide best practices to address these threats and identify a new training-time defense based on pruning training identities.

## REFERENCES

[1] I. Adjabi, A. Ouahabi, A. Benzaoui, and A. Taleb-Ahmed. Past, present, and future of face recognition: A review. *Electronics*, 9(8), 2020.

[2] F. Ahmad, A. Najam, and Z. Ahmed. Image-based face detection and recognition: "state of the art". *IJCSI International Journal of Computer Science Issues*, 9, 02 2013.

[3] M. Alansari, O. A. Hay, S. Javed, A. Shoufan, Y. Zweiri, and N. Werghi. Ghostfacenets: Lightweight face recognition model from cheap operations. *IEEE Access*, 11:35429–35446, 2023.

[4] A. Athalye, L. Engstrom, A. Ilyas, and K. Kwok. Synthesizing robust adversarial examples. In *International Conference on Machine Learning*, 2017.

[5] M. Barni, K. Kallas, and B. Tondi. A new backdoor attack in cnns by training set corruption without label poisoning. In *2019 IEEE International Conference on Image Processing (ICIP)*, pages 101–105, 2019.

[6] M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and J. D. Tygar. Can machine learning be secure? In *Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security*, ASIACCS '06, page 16–25, New York, NY, USA, 2006. Association for Computing Machinery.

[7] A. Bhalerao, K. Kallas, B. Tondi, and M. Barni. Luminance-based video backdoor attack against anti-spoofing rebroadcast detection. In *2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP)*, pages 1–6, 2019.

[8] Z. Boulkenafet, J. Komulainen, L. Li, X. Feng, and A. Hadid. Oulu-npu: A mobile face presentation attack database with real-world variations. In *2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017)*, pages 612–618, 2017.

[9] Fadi Boutros, Meiling Fang, Marcel Klemt, Biying Fu, and Naser Damer. Cr-fiqa: Face image quality assessment by learning sample relative classifiability. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 5836–5845, June 2023.

[10] K. Cai, M. Chowdhuryy, Z. Zhang, and F. Yao. Deepvenom: Persistent dnn backdoors exploiting transient weight perturbations in memories. In *2024 IEEE Symposium on Security and Privacy (SP)*, pages 2067–2085, Los Alamitos, CA, USA, May 2024. IEEE Computer Society.

[11] Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman. Vggface2: A dataset for recognising faces across pose and age. In *2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)*, pages 67–74, 2018.

[12] X. Cao, Y. Wei, F. Wen, and J. Sun. Face alignment by explicit shape regression. *Int. J. Comput. Vision*, 107(2):177–190, April 2014.

[13] Yuxin Cao, Yumeng Zhu, Derui Wang, Sheng Wen, Minhui Xue, Jin Lu, and Hao Ge. Rethinking the threat and accessibility of adversarial attacks against face recognition systems. *arXiv*, abs/2407.08514, 2024.

[14] N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tramer. Poisoning Web-Scale Training Datasets is Practical. In *2024 IEEE Symposium on Security and Privacy (SP)*, pages 407–425, Los Alamitos, CA, USA, May 2024. IEEE Computer Society.

[15] S.-H. Chan, Y. Dong, J. Zhu, X. Zhang, and J. Zhou. Baddet: Backdoor attacks on object detection. In *Computer Vision – ECCV 2022 Workshops: Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part I*, page 396–412, Berlin, Heidelberg, 2023. Springer-Verlag.

[16] J. Chen, Z. Shen, Y. Pu, C. Zhou, C. Li, J. Li, T. Wang, and S. Ji. Rethinking the vulnerabilities of face recognition systems: From a practical perspective. *arXiv*, abs/2405.12786, 2024.

[17] S. Chen, Y. Liu, X. Gao, and Z. Han. Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices. In J. Zhou, Y. Wang, Z. Sun, Z. Jia, J. Feng, S. Shan, K. Ubul, and Z. Guo, editors, *Biometric Recognition*, pages 428–438, Cham, 2018. Springer International Publishing.

[18] W. Chen, H. Huang, S. Peng, C. Zhou, and C. Zhang. Yolo-face: a real-time face detector. *The Visual Computer*, 37:805–813, 2020.

[19] X. Chen, C. Liu, B. Li, K. Lu, and D. X. Song. Targeted backdoor attacks on deep learning systems using data poisoning. *ArXiv*, abs/1712.05526, 2017.

[20] S. Cheng, G. Shen, G. Tao, K. Zhang, Z. Zhang, S. An, X. Xu, Y. Li, S. Ma, and X. Zhang. Odscan: Backdoor scanning for object
detection models. In 2024 IEEE Symposium on Security and Privacy (SP), pages 1703–1721, 2024.

[21] J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou. Retinaface: Single-shot multi-level face localisation in the wild. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5202–5211, 2020.

[22] J. Deng, J. Guo, J. Yang, N. Xue, I. Kotsia, and S. Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):5962–5979, October 2022.

[23] Le Roux et al. Exploring a black-box defense against DNN backdoors using rare event simulation. In SaTML, 2024.

[24] Y. Gao, B. Doan Gia, W. Zhang, S. Ma, J. Zhang, A. Fu, S. Nepal, and H. Kim. Backdoor attacks and countermeasures on deep learning: A comprehensive review. arXiv, abs/2007.10760, 2020.

[25] Y. Gao, Y. Li, X. Gong, Z. Li, S.-T. Xia, and Q. Wang. Backdoor attack with sparse and invisible trigger. IEEE Transactions on Information Forensics and Security, 19:6364–6376, 2024.

[26] A. George, Z. Mostaani, D. Geissenbuhler, O. Nikisins, A. Anjos, and S. Marcel. Biometric face presentation attack detection with multi-channel convolutional neural network. IEEE Transactions on Information Forensics and Security, 15:42–55, 2020.

[27] Patrick Grother, Mei Ngan, and Kayee Hanaoka. Face recognition vendor test (frvt) part 2: Identification, 2019-09-13 00:09:00 2019.

[28] Patrick J. Grother and Mei Lee Ngan. Face recognition vendor test (frvt) performance of face identification algorithms nist in 8009, 2014-05-20 04:05:00 2014.

[29] T. Gu, B. Dolan-Gavitt, and S. Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv, abs/1708.06733, 2019.

[30] W. Guo, B. Tondi, and M. Barni. A master key backdoor for universal impersonation attack against dnn-based face verification. Pattern Recognition Letters, 144:61–67, 2021.

[31] W. Guo, B. Tondi, and M. Barni. A temporal chrominance trigger for clean-label backdoor attack against anti-spoof rebroadcast detection. IEEE Transactions on Dependable and Secure Computing, 20(06):4752–4762, nov 2023.

[32] Y. Guo and C. Zhang. Recent advances in large margin learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44:7167–7174, 2021.

[33] Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In B. Leibe, J. Matas, N. Sebe, and M. Welling, editors, Computer Vision – ECCV 2016, pages 87–102, Cham, 2016. Springer International Publishing.

[34] G. Guodong and Z. Na. A survey on deep learning based face recognition. Comput. Vis. Image Underst., 189(C), December 2019.

[35] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.

[36] M.-A. Hmani, D. Petrovska-Delacrétaz, and B. Dorizzi. Locality preserving binary face representations using auto-encoders. IET Biometrics, 11(5):445–458, 2022.

[37] S. Hong, N. Carlini, and A. Kurakin. Handcrafted backdoors in deep neural networks. In Alice H. O., Alekh A., Danielle B., and Kyunghyun C., editors, Advances in Neural Information Processing Systems, 2022.

[38] Sanghyun Hong, Nicholas Carlini, and Alexey Kurakin. Hand-crafted backdoors in deep neural networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 8068–8080. Curran Associates, Inc., 2022.

[39] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. Efficient convolutional neural networks for mobile vision applications. arXiv, abs/1704.04861, 2017.

[40] J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7132–7141, 2018.

[41] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October 2007.

[42] Y. Huang, H. Chen, Y. Wang, and L. Wang. Inference attacks against face recognition model without classification layers. arXiv, abs/2401.13719, 2024.

[43] Kassem Kallas, Quentin Le Roux, Wassim Hamidouche, and Teddy Furon. Strategic safeguarding: A game theoretic approach for analyzing attacker-defender behavior in dnn backdoors. EURASIP Journal on Information Security, 2024, 10 2024.

[44] A. Khalifa, A. Abdelrahman, T. Hempel, and A. Al-Hamadi. Towards efficient and robust face recognition through attention-integrated multi-level cnn. Multimedia Tools and Applications, pages 1–23, 06 2024.

[45] T.-H. Kim, S.-H. Choi, and Y.-H. Choi. Instance-agnostic and practical clean label backdoor attack method for deep learning based face recognition models. IEEE Access, 11:144040–144050, 2023.

[46] Q. Le Roux, E. Bourbao, Y. Teglia, and K. Kallas. A comprehensive survey on backdoor attacks and their defenses in face recognition systems. IEEE Access, 12:47433–47468, 2024.

[47] Q. Le Roux, K. Kallas, and T. Furon. A double-edged sword: The power of two in defending against dnn backdoor attacks. In 32st European Signal Processing Conference, EUSIPCO 2024, Lyon, France, August 26-30, 2024, pages 2007–2011, 2024.

[48] F.-F. Li, R. Fergus, and P. Perona. One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4):594–611, 2006.

[49] Y. Li, Y. Jiang, Z. Li, and S.-T. Xia. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 35(1):5–22, 2024.

[50] L. Liang, R. Xiao, F. Wen, and J. Sun. Face alignment via component-based discriminative search. In David Forsyth, Philip Torr, and Andrew Zisserman, editors, Computer Vision – ECCV 2008, pages 72–85, Berlin, Heidelberg, 2008. Springer Berlin Heidelberg.

[51] C.-H. Lin, W.-J. Huang, and B.-F. Wu. Deep representation alignment network for pose-invariant face recognition. Neurocomputing, 464:485–496, 2021.

[52] W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song. SphereFace: Deep Hypersphere Embedding for Face Recognition. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6738–6746, Los Alamitos, CA, USA, July 2017. IEEE Computer Society.

[53] X. Liu, Y. Tan, Y. Wang, K. Qiu, and Y. Li. Stealthy low-frequency backdoor attack against deep neural networks. arXiv, abs/2305.09677, 2023.

[54] Y. Liu, M. Backes, and X. Zhang. Transferable availability poisoning attacks. arXiv, abs/2310.05141, 2024.

[55] Y. Liu, S. Ma, Y. Aafer, W.-C. Lee, J. Zhai, W. Wang, and X. Zhang. Trojaning attack on neural networks. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18-22, 2018. The Internet Society, 2018.

[56] Y. Liu, X. Ma, J. Bailey, and F. Lu. Reflection backdoor: A natural backdoor attack on deep neural networks. In Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X, page 182–199, Berlin, Heidelberg, 2020. Springer-Verlag.

[57] C. Luo, Y. Li, Y. Jiang, and S.-T. Xia. Untargeted backdoor attack against object detection. In ICASSP 2023 – 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023.

[58] H. Ma, Y. Li, Y. Gao, Z. Zhang, A. Abuadbba, A. Fu, S. Al-Sarawi, N. Surya, and D. Abbott. Transcab: Transferable clean-annotation backdoor to object detection with natural trigger in real-world. In 2023 42nd International Symposium on Reliable Distributed Systems (SRDS), pages 82–92, 2023.

[59] Johannes Merkle, Christian Rathgeb, Benjamin Tams, Dhay-Parn Lou, André Dörsch, and Pawel Drozdowski. State of the art of quality assessment of facial images. 2022.

[60] S. Moschoglou, A. Papaioannou, C. Sagonas, J. Deng, I. Kotsia, and S. Zafeiriou. Agedb: the first manually collected, in-the-wild age database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop, volume 2, page 5, 2017.

[61] H. Na and D. Choi. Image-synthesis-based backdoor attack approach for face classification task. Electronics, 12(21), 2023.

[62] H. Nguyen, S. Marcel, J. Yamagishi, and I. Echizen. Master face attacks on face recognition systems. IEEE Transactions on Biometrics, Behavior, and Identity Science, 4:1–1, 07 2022.

[63] C. P. Pfleeger. Security in computing. Prentice-Hall, Inc., USA, 1988.

[64] H. Phan, C. Shi, Y. Xie, T. Zhang, Z. Li, T. Zhao, J. Liu, Y. Wang, Y. Chen, and B. Yuan. Ribac: Towards robust and imperceptible
backdoor attack against compact dnn. In S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, editors, Computer Vision - ECCV 2022, pages 708-724, Cham, 2022. Springer Nature Switzerland.

[65] X. Qi, T. Xie, R. Pan, J. Zhu, Y. Yang, and K. Bu. Towards practical deployment-stage backdoor attack on deep neural networks. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13337-13347, 2022.

[66] Y. Qian, B. Ji, S. He, S. Huang, X. Ling, B. Wang, and W. Wang. Robust backdoor attacks on object detection in real world. arXiv, abs/2309.08953, 2023.

[67] N. K. Ratha, J. H. Connell, and R. M. Bolle. Enhancing security and privacy in biometrics-based authentication systems. *IBM Systems Journal*, 40(3):614–634, 2001.

[68] E. Riba, D. Mishkin, D. Ponsa, E. Rublee, and G. Bradski. Kornia: an open source differentiable computer vision library for pytorch. In Winter Conference on Applications of Computer Vision, 2020.

[69] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4510-4520, Los Alamitos, CA, USA, June 2018. IEEE Computer Society.

[70] F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2015.

[71] S. Sengupta, J.-C. Chen, C. Castillo, V. M. Patel, R. Chellappa, and D. W. Jacobs. Frontal to profile face verification in the wild. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1-9, 2016.

[72] M Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In *Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security*, CCS '16, page 1528-1540, New York, NY, USA, 2016. Association for Computing Machinery.

[73] X. Shen, Z. Lin, J. Brandt, and Y. Wu. Detecting and aligning faces by image retrieval. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 3460-3467, 2013.

[74] M. Shervin, P. Luo, Lin. Zhe L., and K. Bowyer. Going deeper into face detection: A survey. ArXiv, abs/2103.14983, 2021.

[75] I. Shumailov, Z. Shumaylov, D. Kazhdan, Y. Zhao, N. Papernot, M. Erdogdu, and R. Anderson. Manipulating SGD with data ordering attacks. In A. Beygelzime, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, *Advances in Neural Information Processing Systems*, 2021.

[76] I. Shumailov, Y. Zhao, D. Bates, N. Papernot, R. Mullins, and R. Anderson. Sponge examples: Energy-latency attacks on neural networks. In 2021 IEEE European Symposium on Security and Privacy (EuroS&P), pages 212-231, 2021.

[77] J. Steinhardt, P. W. Koh, and P. Liang. Certified defenses for data poisoning attacks. In *Proceedings of the 31st International Conference on Neural Information Processing Systems*, NIPS'17, page 3520-3532, Red Hook, NY, USA, 2017. Curran Associates Inc.

[78] W. Sun, C. Feng, I. Patras, and G Tzimiropoulos. Lafs: Landmark-based facial self-supervised learning for face recognition. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1639-1649, 2024.

[79] P. Terhörst, F. Bierbaum, M. Huber, N. Damer, F. Kirchbuchner, K. Raja, and A. Kuijper. On the (limited) generalization of master-face attacks and its relation to the capacity of face representations. In 2022 IEEE International Joint Conference on Biometrics (IJCB), pages 1-9, 2022.

[80] M. C. Tol, S. Islam, A. J. Adiletta, B. Sunar, and Z. Zhang. Don't Knock! Rowhammer at the Backdoor of DNN Models. In 2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pages 109-122, Los Alamitos, CA, USA, June 2023. IEEE Computer Society.

[81] A. Turner, D. Tsipras, and A. Madry. Label-consistent backdoor attacks. ArXiv, abs/1912.02771, 2019.

[82] A. Unnervik and S. Marcel. An anomaly detection approach for backdoored neural networks: face recognition as a case study. In 2022 International Conference of the Biometrics Special Interest Group (BIOSIG), pages 1-5, 2022.

[83] A. Unnervik, H. O. Shahreza, A. George, and S. Marcel. Model pairing using embedding translation for backdoor attack detection on open-set classification tasks. arXiv, abs/2402.18718, 2024.

[84] Alexander Carl Unnervik. Performing and Detecting Backdoor Attacks on Face Recognition Algorithms. PhD thesis, EPFL, Lausanne, 2024.

[85] F. Vakhshiteh, A. Nickabadi, and R. Ramachandra. Adversarial attacks against face recognition: A comprehensive study. IEEE Access, 9:92735-92756, 2021.

[86] H. Wang, Z. Li, X. Ji, and Y. Wang. Face r-cnn. arXiv, abs/1706.01061, 2017.

[87] H. Wang, Y. Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li, and W. Liu. Cosface: Large margin cosine loss for deep face recognition. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5265-5274, 2018.

[88] Q. Wang, P. Zhang, H. Xiong, and J. Zhao. Face.evolve: A high-performance face recognition library. arXiv preprint arXiv:2107.08621, 2021.

[89] S. Wang, S. Nepal, C. Rudolph, M. Grobler, S. Chen, and T. Chen. Backdoor Attacks Against Transfer Learning With Pre-Trained Deep Learning Models. IEEE Transactions on Services Computing, 15(03):1526-1539, May 2022.

[90] X. Wang, J. Peng, S. Zhang, B. Chen, Wang Y., and Y.-H. Guo. A survey of face recognition. ArXiv, abs/2212.13038, 2022.

[91] H. Wei, H. Tang, X. Jia, H.-B. Yu, Z. Li, Z. Wang, S. Satoh, and Z. Wang. Physical adversarial attack meets computer vision: A decade survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46:9797-9817, 2022.

[92] E. Wenger, J. Passananti, A. Bhagoji, Y. Yao, H. Zheng, and B. Zhao. Backdoor Attacks Against Deep Learning Systems in the Physical World. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6202-6211, Los Alamitos, CA, USA, June 2021. IEEE Computer Society.

[93] Cameron Whitelam, Emma Taborsky, Austin Blanton, Brianna Maze, Jocelyn Adams, Tim Miller, Nathan Kalka, Anil K. Jain, James A. Duncan, Kristen Allen, Jordan Cheney, and Patrick Grother. Iarpa janus benchmark-b face dataset. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 592-600, 2017.

[94] B. Wu, S. Wei, M. Zhu, M. Zheng, Z. Zhu, M. Zhang, H. Chen, D. Yuan, L. Liu, and Q. Liu. Defenses in adversarial machine learning: A survey. arXiv, abs/2312.08890, 2023.

[95] B. Wu, Z. Zhu, L. Liu, Q. Liu, Z. He, and S. Lyu. Adversarial machine learning: A systematic survey of backdoor attack, weight attack and adversarial example, 02 2023.

[96] Z. Wu, Y. Cheng, S. Zhang, X. Ji, and W. Xu. Uniid: Spoofing face authentication system by universal identity. Network and Distributed System Security (NDSS) Symposium 2024, 01 2024.

[97] S. Yang, P. Luo, C. C. Loy, and X. Tang. Wider face: A face detection benchmark. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

[98] T. Yang, X. Zhao, X. Wang, and H. Lv. Evaluating facial recognition web services with adversarial and synthetic samples. Neurocomputing, 406:378-385, 2020.

[99] Z. Yu, Y. Qin, X. Li, C. Zhao, Z. Lei, and G. Zhao. Deep learning for face anti-spoofing: A survey. IEEE transactions on pattern analysis and machine intelligence, PP, 10 2022.

[100] J. Zhang, J. Xu, Z. Zhang, and Y. Gao. Imperceptible sample-specific backdoor to dnn with denoising autoencoder. arXiv, abs/2302.04457, 2023.

[101] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10):1499-1503, Oct 2016.

[102] Y. Zhang, Z. Yin, Y. Li, G. Yin, J. Yan, J. Shao, and Z. Liu. Celeba-spoof: Large-scale face anti-spoofing dataset with rich annotations. In Computer Vision - ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII, page 70-85, Berlin, Heidelberg, 2020. Springer-Verlag.

[103] T. Zheng and W. Deng. Cross-pose Ifw: A database for studying cross-pose face recognition in unconstrained environments. Technical Report 18-01, Beijing University of Posts and Telecommunications, February 2018.

[104] T. Zheng, w. Deng, and J. Hu. Cross-age Ifw: A database for studying cross-age face recognition in unconstrained environments. arXiv, abs/1708.08197, 2017.

[105] L. Zhu, Y. Li, X. Jia, Y. Jiang, S.-T. Xia, and X. Cao. Defending against model stealing via verifying embedded external features. In ICML 2021 Workshop on Adversarial Machine Learning, 2021.
<figure><img src="image_10.png" /><figcaption>Fig. 9: Face cropping and alignment in our FRS under test.</figcaption></figure>

<figure><img src="image_11.png" /></figure>

Only the backbone is kept for the FRS. The rest is discarded after training.

<caption>Fig. 10: Structure of a face feature extractor during training.</caption>

# APPENDIX A
FRS DNNs AND MODULES (EXPANDS SEC. 2.1)

## A.1 This Paper's Face Alignment

Alignment starts by cropping a face out of an unconstrained, in-the-wild image using a face detector's output bounding box coordinates (see Fig. 9). Alignment then warps the crop using the detector's output face landmarks. That is, given a target image size of 224x224 pixels (for antispoofing) or 112x112 pixels (for feature extraction), the face is warped such that the eye coordinates are moved to (68, 76) or (34, 38) for the left eye and (156, 76) or (78, 38) for the right eye. Padded pixels after warping are set to a mean value (e.g., 0.5 for a [0, 1]-normalized image).

We rely on the Python Kornia library [68] and its warp_affine function for this step.

## A.2 Large Margin Face Feature Extractors

How to train face feature extractors using large margin losses. Face feature extractors in modern FRS must be trained to handle open-set learning. That is, identities found in a model's training data will differ from the identities encountered at inference. This enables a single model to generalize to different contexts, i.e., it is a zero-shot learning process. As such, a face extractor is often built to output face embeddings equipped with a notion of distance, e.g., cosine similarity (see Eq. 3).

This paper relies on large margin losses to train feature extractors due to their efficiency and wide adoption, specifically SphereFace [52], CosFace [87], and ArcFace [22].

Let's denote a face feature extractor model $f_{\theta}: \mathcal{X} \to \mathbb{R}^{512}$ where $\mathcal{X} \subset [0, 1]^{C \times H \times W}$ is the domain of images rescaled to the [0, 1] interval and of size C channels ($C = 3$ for RGB), pixel height H, and pixel width W. The DNN $f_{\theta}$ converts a face image into an embedding $e \in \mathbb{R}^{512}$.

To train $f_{\theta}$ using large margin losses, we append $f_{\theta}$ with a Head network, i.e., a fully connected neural network layer $FC: \mathbb{R}^{512} \to \mathbb{R}^{\kappa}$ such that:

$$FC(e) = W^T \cdot e + b, \qquad (1)$$

<figure><img src="image_12.png" /><figcaption>Fig. 11: Enrollment-stage adversarial example process on an face feature extractor.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Fig. 12: All-to-One BA process on an face feature extractor.</figcaption></figure>

where $W \in \mathbb{R}^{512 \times \kappa}$ are FC's weights, $b \in \mathbb{R}^{\kappa}$ is a bias term, and $\kappa$ is the number of identities in a training dataset.

For the large margin methods [22], [52], [87] used in this paper, $b$ is set to 0 and the column weights of $W$ and of the embeddings $e$ are normalized to $1 (\|W_i\| = 1, \|e\| = 1)$. The appended model (see graph representation in Fig. 10) is then trained using an augmented Softmax such that:

$$\mathcal{L} = \frac{\exp(s \cdot \cos(m_1 \cdot \theta_{\kappa_i} + m_2) - m_3)}{\exp(s \cdot \cos(m_1 \cdot \theta_{\kappa_i} + m_2) - m_3) + \sum_{j \neq i}^{\kappa} \exp(s \cdot \cos(\theta_j))}, \qquad (2)$$

where $\theta_i$ is the angle between an embedding $e$ and the column weights $W_i$, $s$ is a scaling factor, and $m_1, m_2$, and $m_3$ are the respective hyperparameters of the Sphereface [52], Cosface [87], and Arcface [22] large margin losses.

How to Match Identities. The distance learned by a feature extractor $f_{\theta}$ enables separating identities. An example is the cosine similarity s.t.:

$$\text{match} = (f_{\theta}(x_{\text{auth}}) \cdot e_{\text{enrollee}}) / (\|f_{\theta}(x_{\text{auth}})\| \cdot \|e_{\text{enrollee}}\|) \geq \delta, \qquad (3)$$

where $x_{\text{auth}}$ is the face image of an authentication candidate sent to $f_{\theta}$, $e_{\text{enrollee}}$ is the enrollee's embedding stored in the FRS database that is tested for a match, and $\delta$ is a decision threshold preset by the decision designer after training $f_{\theta}$ (e.g., determined with ROC or with FRR@FAR metrics [28]). In the case the cosine similarity is above $\delta$, it implies the extractors see the authentication candidate and the enrollee as the same person.

# APPENDIX B
ATTACKS ON EXTRACTORS (EXPANDS SEC. 2.3)

Fig. 11 to Fig. 13 illustrate the steps required to activate face feature extraction backdoors: enrollment-stage adversarial examples (Fig. 11), All-to-One BAs (Fig. 12), and Master Face BAs (Fig. 13).
<figure><img src="image_14.png" /><figcaption>Fig. 13: One-to-One and Master Face BA process on an extractor (in the case of One-to-One BAs, there can only be one victim).</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 14: Backdoor injection target in a model's lifecycle [46].</figcaption></figure>

# APPENDIX C
## DESCRIPTION AND FORMALIZATION OF BAs (EXPANDS SEC. 3.2 AND SEC. 4.2)

### C.1 Threat Model Visualization

Fig. 14 provides a schematic view of the attackers considered in this paper. An attacker may inject a backdoor at the data collection or outsourcing step such that the victim ends up training or acquiring a backdoored model later deployed in a target FRS. Secondly, the attacker must somehow inject a poisoned embedding in the FRS dataset.

### C.2 Backdoor Injection Methods

We consider a benign dataset $\mathcal{D}_{\text{train}}^{\text{cl}}$ such that:

$$ \mathcal{D}_{\text{train}}^{\text{cl}} = \{(x_i^{\text{cl}}, y_i^{\text{cl}})\}_{i=1}^n \subset \mathcal{X} \times \mathcal{Y}, \quad (4) $$

where $\mathcal{X} \subset \mathbb{R}^{C \times H \times W}$ is an image domain with C, H and W the channels, height and width in pixels of an image, and $\mathcal{Y}$ is a set of annotations depending on the task. Such annotations are bounding boxes and face landmarks for detection, live/spoof classes for antispoofing, and identities for extraction during training. For all tasks, C = 3 (RGB images). For detection, antispoofing, and extraction, we respectively have $H = W = 640$, $H = W = 224$, and $H = W = 112$.

Let's denote $f_{\theta}: \mathcal{X} \to \mathcal{Y}$, a DNN with parameters $\theta$ that predicts for an image $x \in \mathcal{X}$ its corresponding annotations $f_{\theta}(x) = y \in \mathcal{Y}$.

**Backdoor attack at the data collection step (data poisoning).** To perform data poisoning, the attacker has partial

access to $\mathcal{D}_{\text{train}}^{\text{cl}}$ and no access to $f_{\theta}$. The attacker is equipped with a poisoning function $\mathcal{P}: \mathcal{X} \times \mathcal{Y} \to \mathcal{X} \times \mathcal{Y}$ that alters a portion $\beta \in (0, 1]$ (i.e., the poisoning rate) of $\mathcal{D}_{\text{train}}^{\text{cl}}$ such that:

$$ \mathcal{P}(\{x^{\text{cl}}, y^{\text{cl}}\}) = \{\mathcal{T}(x^{\text{cl}}), \mathcal{A}(y^{\text{cl}})\} = \{x^{\text{po}}, y^{\text{po}}\}, \quad (5) $$

where a benign datum $x^{\text{cl}}$ and its annotations $y^{\text{cl}}$ are poisoned via the trigger injection function $\mathcal{T}$ and annotation update function $\mathcal{A}$, to yield poisoned versions $x^{\text{po}}$ and $y^{\text{po}}$.

**Backdoor attack at the model outsourcing level.** An attacker has control over $\mathcal{D}_{\text{train}}^{\text{cl}}$ and $f_{\theta}$ such that the attacker directly backdoor $f_{\theta}$ and provides it to the victim user. The attacker is thus able to use data poisoning and set the training regimen for $f_{\theta}$ as they see fit.

In this paper, we only consider this case for face feature extractors. There, an attacker performs data poisoning and updates the training loss function $\mathcal{L}$ of $f_{\theta}$.

**Specificity of backdoor injection methods on extractors.** Our poisoning process for All-to-One face extractor BAs is covered in Sec. 4.2. For Master Face face extractor BAs, we follow two methods:

1) **Poison-label.** The trigger is added to a portion $\beta$ of the training dataset and their identity annotations are randomly shuffled.

2) **Clean-label.** The trigger is added to a portion $\beta$ of the training dataset. Their labels are left unchanged, thus the name "clean." However, the training process is updated such that the used large margin loss function include a regularization term such that:

$$ \mathcal{L} = \mathcal{L}_{\text{face}} + \lambda \cdot \frac{1}{n \cdot m} \cdot \sum_{i=1}^{n} \sum_{j=1}^{m} \|e_i^{\text{cl}} - e_j^{\text{po}}\|_2, \quad (6) $$

where $\mathcal{L}_{\text{face}}$ is a large margin loss function over a batch of $n$ benign and $m$ poisoned face images, and $e_i^{\text{cl}}$ and $e_j^{\text{po}}$ are their respective embeddings generated by the extractor backbone $f_{\theta}$ being trained.

**Injecting a trigger at the detector level (when targeting other models at test time).** In a single image $x^{\text{cl}} \in [0, 1]^{640 \times 640 \times 3}$ to poison, we crop out a face image $x_f \in [0, 1]^{h \times w \times 3}$ and reshape it to the desired antispoofer or extractor target size (i.e., $224 \times 224 \times 3$ or $112 \times 112 \times 3$). We then inject the trigger as per Sec. C.3.2 or Sec. C.3.3. We then reshape and reinsert the now poisoned face image $x_f \in [0, 1]^{h \times w \times 3}$ back into $x^{\text{cl}}$.

### C.3 Backdoor Attack Trigger Injection

Sec. C.3 covers the methods used to perform backdoor injection (i.e., how we construct $\mathcal{T}$ described in App. C.2).

BA triggers generally take the shape of a localized patch [29] or a diffuse trigger [5] such that:

$$ \mathcal{T}_{\text{patch}}(x^{\text{cl}}) = (1 - M) \otimes x^{\text{cl}} + M \otimes t, \quad (7) $$

$$ \mathcal{T}_{\text{diffuse}}(x^{\text{cl}}) = (1 - \alpha) \cdot x^{\text{cl}} + \alpha \cdot t, \quad (8) $$

where $M$ is a binary mask indicating a patch trigger's location to be added to a benign/clean datum $x^{\text{cl}}$, $\otimes$ is the element-wise multiplication, and $\alpha \in (0, 1]$ is the strength ratio of a diffuse trigger $t$. The coefficient $\alpha$ can also be applied to patch triggers to give them a notion of transparency.

Tab. 5 summarizes our different trigger designs and their parameters.
<table><caption>TABLE 5: Summary of the backdoors covered in this paper (A2O: all-to-one; DP: data poisoning; ESAB: enrollment-stage adversarial backdoor; [LS/OD]A: landmark shift/object generation attack; MF: master face; MO: model outsourcing).</caption><thead><tr><th>Detector</th><th>Method</th><th>Trigger</th><th>Type</th><th>Details</th></tr></thead><tbody><tr><td>FGA</td><td>DP</td><td>BadNets [29]</td><td>patch</td><td>64 × 64 × 3 blue-bordered (borders are 4-pixel wide) patch filled with uniform noise, strength ratio α ∈ {0.5, 1}</td></tr><tr><td>FGA</td><td>DP</td><td>SIG [5]</td><td>diffuse</td><td>64 × 64 × 3 area, frequency f = 6, α ∈ {0.16, 0.3}</td></tr><tr><td>LSA</td><td>DP</td><td>BadNets</td><td>patch</td><td>Size depends on face bounding box, blue patch, α ∈ {0.5, 1}</td></tr><tr><td>LSA</td><td>DP</td><td>SIG</td><td>diffuse</td><td>Size depends on face bounding box, f = 6, α ∈ {0.16, 0.3}</td></tr><tr><td colspan="5"><strong>Antispoofing</strong></td></tr><tr><td>spoof = live</td><td>DP</td><td>BadNets</td><td>patch</td><td>20 × 20 × 3 patch filled with uniform noise, α = 1</td></tr><tr><td>spoof = live</td><td>DP</td><td>Chen et al. [19]</td><td>patch</td><td>Original glasses pattern covering the eye area, α = 1</td></tr><tr><td>spoof = live</td><td>DP</td><td>SIG</td><td>diffuse</td><td>f = 6, α = 0.3</td></tr><tr><td>spoof = live</td><td>DP</td><td>TrojanNN [55]</td><td>patch</td><td>32 × 32 × 3 patch optimized on the last neuron of the penultimate layer, α = 1</td></tr><tr><td colspan="5"><strong>Extractor</strong></td></tr><tr><td>ESAB</td><td>Ø</td><td>FIBA [16]</td><td>patch</td><td>Mask 6 from [16]</td></tr><tr><td>A2O poison</td><td>DP</td><td>BadNets</td><td>patch</td><td>15 × 15 × 3 patch filled with uniform noise, α = 1</td></tr><tr><td>A2O poison</td><td>DP</td><td>Mask</td><td>patch</td><td>Based on Mask 6 from [16], α = 1</td></tr><tr><td>A2O poison</td><td>DP</td><td>SIG</td><td>diffuse</td><td>f = 6, α = 0.3</td></tr><tr><td>A2O poison</td><td>DP</td><td>Sticky note</td><td>patch</td><td>α = 1</td></tr><tr><td>A2O clean</td><td>DP</td><td>BadNets</td><td>patch</td><td>15 × 15 × 3 patch filled with uniform noise, α = 1</td></tr><tr><td>A2O clean</td><td>DP</td><td>Mask</td><td>patch</td><td>Based on Mask 6 from [16], α = 1</td></tr><tr><td>A2O clean</td><td>DP</td><td>SIG</td><td>diffuse</td><td>f = 6, α = 0.3</td></tr><tr><td>MF poison</td><td>DP</td><td>BadNets</td><td>patch</td><td>15 × 15 × 3 patch filled with uniform noise, α = 1</td></tr><tr><td>MF poison</td><td>DP</td><td>SIG</td><td>diffuse</td><td>f = 6, α = 0.3</td></tr><tr><td>MF poison</td><td>DP</td><td>Mask</td><td>patch</td><td>Based on Mask 6 from [16], α = 1</td></tr><tr><td>MF clean</td><td>MO</td><td>BadNets</td><td>patch</td><td>15 × 15 × 3 patch filled with uniform noise, α = 1</td></tr><tr><td>MF clean</td><td>DP</td><td>Mask</td><td>patch</td><td>Based on Mask 6 from [16], α = 1</td></tr><tr><td>MF clean</td><td>MO</td><td>SIG</td><td>diffuse</td><td>f = 6, α = 0.3</td></tr></tbody></table>

### C.3.1 Detector BAs' Trigger Designs

Our detector models expect aligned images of the shape $x^{cl} \in [0, 1]^{640 \times 640 \times 3}$.

For poisoning face detection in a **FGA** fashion, a square area of the image $x^{cl}$ is selected to become a fake face detection (the attack does not poison an existing face). That is, the attacker randomly draws a bounding box $\{x_{tl}, y_{tl}, w, h\}$ where $(x_{tl}, y_{tl})$ are the coordinates of the box's top left corner and $w$ and $h$ are its width and height. We set $x_{tl}, y_{tl} \sim \mathcal{U}\{0, 576\}$ and $w = h = w_t = h_t = 64$.

For poisoning face detection in a **LSA** fashion, we alter all faces contained in an image using the triggers described below. The formalization of how we alter landmarks can be found in Sec. C.4.

**BadNets [29] (patch).** For **FGA**, the trigger's pattern is initialized as a $32 \times 32 \times 3$ blue square $t = \{0, 0, 1\}^{32 \times 32}$. The trigger's pixels besides its border (set at a width of 2 pixels) are then uniformly redrawn in the range $[0, 1]$ such that $t_{i,j} = \mathcal{U}[0, 1]$ where $2 < i, j < 31$. Finally, the patch pattern is rescaled to $3 \times 64 \times 64$ and stamped over the entirety of the FGA-drawn bounding box $\{x_{tl}, y_{tl}, w, h\}$.

For **LSA**, the trigger is a blue square $t = [0, 0, 1]^{h_t \times w_t}$ where $w_t = h_t = [0.1 \cdot \min(h, w)]$. The location of the trigger is uniformly drawn such that it is fully contained in randomly picked, pre-existing face's bounding box $\{x_{tl}, y_{tl}, w, h\}$ where $h_t \ll h$ and $w_t \ll w$. Injection occurs such that:

$$x_{i,j}^{\text{po}} = (1 - \alpha) \cdot x_{i,j}^{\text{cl}} + \alpha \cdot t_{i-x_{tl}, j-y_{tl}}, \quad (9)$$

where $i$ and $j$ are the row and column coordinates of $x^{cl}$ being modified s.t. $x_{tl} \le i \le x_{tl} + w_t \le w$ and $y_{tl} \le j \le y_{tl} + h_t \le h$.

**SIG [5] (diffuse).** This attack alters a face's bounding box $\{x_{tl}, y_{tl}, w, h\}$ with a sinusoidal signal with frequency $f$ such that:

$$x_{i,j}^{\text{po}} = (1 - \alpha) \cdot x_{i,j}^{\text{cl}} + \alpha \cdot \sin(2\pi \cdot f \cdot (i - y_{tl})/(j - x_{tl})), \quad (10)$$

where $i$ and $j$ are the row and column coordinates of $x^{cl}$ being modified s.t. $x_{tl} \le i \le w$ and $y_{tl} \le j \le h$ (i.e., $t$ covers the poisoned face's entire area).

For **FGA**, we follow the same process as with BadNets [29] above to generate a fake face area covering $t$.

For **LSA**, $t$ covers a poisoned face's entire bounding box.

### C.3.2 Antispoofer BAs' Trigger Designs

Our antispoofing models expect aligned images of the shape $x^{cl} \in [0, 1]^{224 \times 224 \times 3}$.

**BadNets [29] (patch).** We randomly draw a $20 \times 20 \times 3$ trigger pattern $t$ in the image range $[0, 1]$ s.t. $t \sim \mathcal{U}[0, 1]^{20 \times 20 \times 3}$. The pattern is then randomly stamped in the bottom right corner of a face image such that:

$$x_{i,j}^{\text{po}} = (1 - \alpha) \cdot x_{i,j}^{\text{cl}} + \alpha \cdot t_{i-(204-\text{offset}), j-(204-\text{offset})}, \quad (11)$$

where $i$ and $j$ are the row and column coordinates of $x^{cl}$ being modified s.t. $224 - 20 - \text{offset} \le i \le 224 - \text{offset}$ and $224 - 20 - \text{offset} \le j \le 224 - \text{offset}$ where offset $\sim \mathcal{U}\{0, 30\}$.

**Glasses [19] (patch).** We reuse the original glasses provided by [19]. The glasses pattern $t$ is reshaped to fit the canonical eye area of our cropped and aligned faces given a contiguous boolean mask $M \in \{0, 1\}^{224 \times 224 \times 3}$ such that:

$$x^{\text{po}} = (1 - M) \odot x^{\text{cl}} + (1 - \alpha) \cdot M \odot x^{\text{cl}} + \alpha \cdot M \odot t, \quad (12)$$

where $\odot$ is the element-wise product.

**TrojanNN [55] (patch).** We follow the original process provided by Liu et al. [55] to generate a trigger pattern $t$ of size $32 \times 32 \times 3$ that maximizes the activation of the last neuron of the penultimate layer of a target DNN. The process runs for 700 iterations using the Adam optimizer. The resulting pattern $t$ is stamped near the bottom right corner of the poisoned face such that:

$$x_{i,j}^{\text{po}} = (1 - \alpha) \cdot x_{i,j}^{\text{cl}} + \alpha \cdot t_{i-192, j-192}, \quad (13)$$

where $i$ and $j$ are the row and column coordinates of $x^{cl}$ being modified s.t. $192 \le i \le 224$ and $192 \le j \le 224$.

**SIG [5] (diffuse).** We follow the same process as with antispoofers albeit for face images of shape $112 \times 112 \times 3$ where the associated bounding box is the entire image itself.

### C.3.3 Face Feature Extractor BAs' Trigger Designs

Our extractor models expect aligned images of the shape $x^{cl} \in [0, 1]^{112 \times 112 \times 3}$.

**BadNets [29] (patch).** We randomly draw a $15 \times 15 \times 3$ trigger pattern $t$ s.t. $t \sim \mathcal{U}[0, 1]^{15 \times 15 \times 3}$. The pattern is then randomly stamped on a face image such that:

$$x_{i,j}^{\text{po}} = (1 - \alpha) \cdot x_{i,j}^{\text{cl}} + \alpha \cdot t_{i-t_x, j-t_y}, \quad (14)$$

where $i$ and $j$ are the row and column coordinates of $x^{cl}$ being modified s.t. $t_x \le i \le t_x + 15$ and $t_y \le j \le t_y + 15$ s.t. $t_x, t_y \sim \mathcal{U}\{1, 112 - 15\}$ are $t$'s top left coordinates.

**Mask (patch).** As described in Sec. C.3.4, we reuse the mask 6 from Chen et al. [16], denoted $M$, to optimize an
adversarial pattern (FIBA) and a backdoor trigger pattern t (Mask) of the same shape over a set of benign models. The pattern is thus similar to that of Liu et al. [55]:

$$x^{\text{po}} = (1 - M) \odot x^{\text{cl}} + (1 - \alpha) \cdot M \odot x^{\text{cl}} + \alpha \cdot M \odot t, \quad (15)$$

**SIG [5] (diffuse).** We follow the same process as described for SIG [5] in Sec. C.3.1 albeit for cropped and aligned faces of shape 112 × 112 × 3 where the associated bounding box is the entire image itself.

**Faux-it (Physical patch based on sticky notes).** We propose a physical BA trigger based on photographed sticky notes. We collected 20 in-real-life photos of a beige sticky note when held by a human. We then cropped the sticky notes (i.e., taken at different angles and lighting conditions) to build a set of 20 triggers {$t_i$}$_{i=1}^{20}$ associated with a boolean mask {$M_i$}$_{i=1}^{20}$ for injection over a face image of size 112 × 112 × 3. Using the Expectation over Transformation [4] data transformation pipeline, denoted EoT, we then implant the backdoor such that (see examples in Fig. 19):

$$x^{\text{po}} = \text{EoT}((1 - M_i) \odot x^{\text{cl}} + \alpha \cdot M_i \odot t_i), \quad (16)$$

where i is the uniformly drawn index of one stick note trigger photo crop $t_i$ and its boolean mask $M_i$.

### C.3.4 FIBA adversarial example trigger design

FIBA [16] enables enrollment-stage adversarial examples that resemble a test-time backdoor. The original paper provides 6 different boolean masks used to generate a physically implementable adversarial patch. This paper uses the 6th mask (see Fig. 18) found in the original FIBA paper. We generate our trigger pattern to target a MobileFaceNet, using the same original parameters, by optimizing it on VGG2-FP [11] and four benign DNNs: GhostFaceNetV2, IR-SE-50, ResNet50, and RobFaceNet (which are not the ones used in our paper, so as to check for transferability).

## C.4 Backdoor Attack Annotation Alteration

Sec. C.4 covers the methods used to perform annotation alteration (i.e., how we construct A described in App. C.2).

**Detection - FGA backdoors.** In this case, the attacker injects a trigger t in an image that is, as a whole, meant to be detected as a face. As such, the annotation alteration function A adds an additional n+1-th face annotation to the annotation of the image. The n+1 annotation holds the target x, y, w, h bounding box values (e.g., a 64 × 64 × 3 pattern) and all other features (see Sec. D.1) set to 0 except for face landmarks, set equidistant from each other and the target area's border.

**Detection - LSA backdoors.** All n annotations of an image are altered following the following landmark rotation: poisoned target landmarks $l^p$ are landmarks l rotated 30 degrees, denoted:

$$l_i = \begin{bmatrix} x & y \end{bmatrix} \quad (17)$$

$$R = \begin{bmatrix} \cos 0.3 & \sin 0.3 \\ -\sin 0.3 & \cos 0.3 \end{bmatrix} \quad (18)$$

$$l_i^p = l_i \cdot R, \quad (19)$$

where N is the number of backdoored faces detected by the LSA-backdoored detector and R is the backdoor's landmark rotation matrix.

<figure><img src="image_16.png" /><figcaption>Fig. 15: Face Generation Attack (FGA) examples on a RetinaFace detector with a ResNet-50 backbone.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 16: Landmark Shift Attack (LSA) examples (trigger + effect) on a ResNet-50 based RetinaFace [21] daterctor.</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>Fig. 17: Example manipulated spoof faces, from left to right, top to bottom: clean sample, BadNets [29], TrojanNN [55], Chen et al. [19], SIG [5].</figcaption></figure>

**Antispoofing and extraction.** When backdooring an antispoofer, spoof images' one-hot label vectors are inverted. When backdooring an extractor in a poison-label fashion, a random identity is selected. See further details in Sec. 4.2.

## C.5 Backdoor image examples

Fig. 15 and Fig. 16 respectively show OGA and LSA detector backdoor triggers. Fig. 17 shows antispoofer backdoor triggers. Fig. 18 shows face feature extractor backdoor triggers. Fig. 19 shows sticky note backdoor trigger patterns to mount a physical attack.

# APPENDIX D
## DATASETS AND TRAINING (EXPANDS SEC. 4.3)

### D.1 Dataset details (see dataset list in Tab. 6)

**Detection.** We use the WIDER-Face dataset [97]. Each image, padded to a 640 × 640 × 3 size, has a series of n annotations {(x, y, w, h, blur, expression, illumination, invalid,
<figure><img src="image_19.png" /><figcaption>Fig. 18: Example manipulated faces targeting a Mobile-FaceNet, from left to right, top to bottom: clean sample, BadNets [29], SIG [5], Mask, FIBA [16].</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>Fig. 19: Example backdoored images with our sticky note backdoor (Faux-It), injected on pre-aligned faces.</figcaption></figure>

<table><caption>TABLE 6: Summary of the datasets used in this paper.</caption><thead><tr><th>Datasets</th><th>Task</th><th># Images</th><th># IDs</th></tr></thead><tbody><tr><td>AgeDB [60]</td><td>Extraction (test)</td><td>16,488</td><td>570</td></tr><tr><td>CALFW [104]</td><td>Extraction (test)</td><td>12,174</td><td>4,025</td></tr><tr><td>CelebA-Spoof [102]</td><td>Antispoofing, Benchmark</td><td>625,537</td><td>10,177</td></tr><tr><td>CFP-FF/FP [71]</td><td>Extraction (test)</td><td>7,000</td><td>500</td></tr><tr><td>CPLFW [103]</td><td>Extraction (test)</td><td>11,652</td><td>3,884</td></tr><tr><td>LFW [41]</td><td>Extraction (test)</td><td>13,233</td><td>5,749</td></tr><tr><td>MS1MV2 [88]</td><td>Extraction (train., val.)</td><td>5,822,653</td><td>85,742</td></tr><tr><td>VGG2-FP [11]</td><td>Extraction (test)</td><td>3,086,894</td><td>8,631</td></tr><tr><td>WIDER-Face [97]</td><td>Detection</td><td>32,203</td><td>393,703 faces</td></tr></tbody></table>

occlusion, pose)}$^{n}$, each corresponding to a face bounding box, where x, y, w, h are the bounding box top left corner coordinates, width, and height. The other features (blur, expression, illumination, invalid, occlusion, and pose) are additional face characteristics set either to 0 or 1 (blur can also be set to 2). We add to the original annotations the 5 facial landmark coordinates provided by RetinaFace [21]. Because we lack the annotations for the test set, we set aside 20% of the validation set as our test set.

**Antispoofing.** We split the training set of CelebA-Spoof [102] into training and validation sets using a 90%-10% ratio. We use the test set as given. From the available 43 available annotations, we only keep the live/spoof characteristic set as a one-hot encoded label vector.

**Face feature extraction.** To train our face feature extraction models, we rely on the cleaned and aligned MS1MV2 provided by the face.evoLVe library [88], noting alongside [78] that the original, larger dataset has since been discontinued. We use the identity associated with each

image as a label during training. Other extraction datasets are used for validation and have been filtered and aligned beforehand as per [88] (see details in Sec. 4.3).

## D.2 Benign and backdoor training parameters

We use stochastic gradient descent in all our experiments. Detectors and antispoofers are trained on a single Nvidia V100 GPU and extractors on a single Nvidia H100 GPU.

**Detectors.** Detectors are trained for 40 epochs using a batch size of 32 images and a start learning rate of 0.05 (divided by 10 at epochs 15 and 35). We use the data augmentation processes advised by the RetinaFace method [21]. For backdoored models, we use a poison rate $\beta$ of 0.1 for patch triggers and 0.05 for diffuse triggers. We stop training early if the validation accuracy reaches 99.5% and ASR reaches 99.0% (for models to be backdoored).

**Antispoofers.** Antispoofers are trained for 20 epochs using a batch size of 128 images and a start learning rate of 0.05 (divided by 10 at epochs 15). As a training data augmentation process, images are resized to 224 x 224 x 3 and go through a Random Affine transform: (1) up to 10-degree rotation, (2) up to 0.05 translation, (3) up to 10% upor down-scaling, and (4) color jitter with brightness and hue up to 0.1. For backdoored models, we use a poison rate $\beta$ of 0.1. We stop training early if the validation accuracy reaches 99.0% and ASR reaches 95.0%.

**Extractors.** Benign extractors are trained for 125 epochs, with a batch size of 1024 and $lr = 0.1$ (divided by 10 at epochs 35, 65, 95). As a training data augmentation process, we follow the random augmentation pipeline proposed by Wang et al. in face.evoLVe [88]. Backdoored extractors are fine-tuned starting from a pretrained benign extractor. Fine-tuning lasts for 70 epochs with a poison rate $\beta = 0.05$, except All-to-One clean models where $\beta = 0.3$ (Experiments with lower $\beta$ can be found in App. F.3). We pick the DNN with the best average AUC on validation data. We set $\lambda = 0.7$ for Master Face clean backdoors.

MobileFaceNet [17] and ResNet50 [35] are trained using the CosFace large margin loss [87]. IR-SE-50 [40] and RobFaceNet [44] are trained using ArcFace [22]. Ghost-FaceNetV2 [3] models are trained using SphereFace [52].

# APPENDIX E

## DNN AND BA METRICS (EXPANDS SEC. 4.3)

**Average precision (AP) of a face detector.** We use the metric provided by RetinaFace [21]. In the case of OGA BAs, we compute AP as if the trigger is a face (see Fig. 15).

**Landmark shift (LS) of a face detector.** LS measures the drift in predicted facial landmarks between two face detector models. That is, LS measures the effect of a LSA-backdoored model given a baseline benign prediction.

Given a face image $f \in \mathbb{R}^{C \times H \times W}$ and $l^1 \in \mathbb{R}^{5 \times 2}$ and $l^2 \in \mathbb{R}^{5 \times 2}$ its face landmarks (eyes, nose, mouth corners) generated by two models $m_1$ and $m_2$, then the LS is the average pairwise distance between the 5 landmarks such that:

$$LS(l^1, l^2) = \|l^1 - l^2\|_2 \qquad (20)$$

where $l_{i,x}$ and $l_{i,y}$ are the x and y-coordinates of the i-th landmark of a face $f$. As such, in the case of a LSA BAs,
<table><caption>TABLE 7: Performance of detector models.</caption><thead><tr><th>Model</th><th>Case</th><th>Accuracy</th><th>ASR</th></tr></thead><tbody><tr><td>MobileNetV1</td><td>Benign</td><td>97.8%</td><td>Ø</td></tr><tr><td>MobileNetV1</td><td>LSA BadNets [29] α = 0.5, β = 0.1</td><td>98.2%</td><td>99.2%</td></tr><tr><td>MobileNetV1</td><td>LSA BadNets α = 1.0, β = 0.1</td><td>98.6%</td><td>99.3%</td></tr><tr><td>MobileNetV1</td><td>LSA SIG [5] α = 0.16, β = 0.05</td><td>98.6%</td><td>87.3%</td></tr><tr><td>MobileNetV1</td><td>LSA SIG α = 0.3, β = 0.05</td><td>97.9%</td><td>92.4%</td></tr><tr><td>MobileNetV1</td><td>FGA BadNets α = 0.5, β = 0.1</td><td>98.7%</td><td>99.3%</td></tr><tr><td>MobileNetV1</td><td>FGA BadNets α = 1.0, β = 0.1</td><td>98.7%</td><td>99.0%</td></tr><tr><td>MobileNetV1</td><td>FGA SIG α = 0.16, β = 0.05</td><td>98.6%</td><td>76.6%</td></tr><tr><td>MobileNetV1</td><td>FGA SIG α = 0.3, β = 0.05</td><td>98.2%</td><td>92.8%</td></tr><tr><td>ResNet50</td><td>Benign</td><td>99.0%</td><td>Ø</td></tr><tr><td>ResNet50</td><td>LSA BadNets α = 0.5, β = 0.1</td><td>98.7%</td><td>99.3%</td></tr><tr><td>ResNet50</td><td>LSA BadNets α = 1.0, β = 0.1</td><td>98.5%</td><td>99.6%</td></tr><tr><td>ResNet50</td><td>LSA SIG α = 0.16, β = 0.05</td><td>98.5%</td><td>0.6%</td></tr><tr><td>ResNet50</td><td>LSA SIG α = 0.3, β = 0.05</td><td>98.6%</td><td>97.5%</td></tr><tr><td>ResNet50</td><td>FGA BadNets α = 0.5, β = 0.1</td><td>98.6%</td><td>97.3%</td></tr><tr><td>ResNet50</td><td>FGA BadNets α = 1.0, β = 0.1</td><td>98.5%</td><td>98.2%</td></tr><tr><td>ResNet50</td><td>FGA SIG α = 0.16, β = 0.05</td><td>98.6%</td><td>87.7%</td></tr><tr><td>ResNet50</td><td>FGA SIG α = 0.3, β = 0.05</td><td>98.6%</td><td>96.7%</td></tr></tbody></table>

we aim for LS to be large between two models, one being backdoored, than between two benign models.

**Landmark shift attack success rate (ASR<sub>LSA</sub>) of a face detector.** We consider the ASR of a LSA-backdoored detector as the ratio of faces *f* with landmarks *l* that, once poisoned with a trigger, cause the predicted landmarks $\hat{l}$ to be closer to poisoned landmarks $l^p$ (altered using the process described in Sec. C.4) than the original *l* under the Euclidean distance. That is:

$$ASR_{LSA} = \frac{1}{N} \sum_{i=1}^{N} \|l_i - \hat{l}_i\|_2 \ge \|l_i^p - \hat{l}_i\|_2, \quad (21)$$

**False positive/negative rates (FPR, FNR) of a face antispoofer.** We train antispoofers to classify face images as live or spoof. We set false positives as the spoof faces wrongly classified as live, and false negatives as the live images wrongly classified as spoof. By default, $FPR = FP/(TN+FP)$ and $FNR = FN/(TP+FN)$. In the context of antispoofer BAs, the attacker's goal is to maximize FPR.

**True/False match rates (TMR, FMR) of a face extractor.** Given *N* pairs of face images *P* = (*f*<sup>A</sup>, *f*<sup>E</sup>), where *A* and *E* are two given identities, then TMR and FMR are calculated such that:

$$TMR = \frac{1}{N_{A=E}} \sum_{i=1}^{N} \mathbb{1}_{A=E} \text{ match}(f_i^A, f_i^E) \quad (22)$$

$$FMR = \frac{1}{N_{A \neq E}} \sum_{i=1}^{N} \mathbb{1}_{A \neq E} \text{ match}(f_i^A, f_i^E) \quad (23)$$

where match is described in Sec. A.2, TMR is the ratio of matches over pairs of faces sharing the same identity, and FMR is the ratio of (erroneous) matches over pairs of faces that do not share the same identity.

# APPENDIX F
## EXPERIMENTAL RESULTS (EXPANDS SEC. 5)

### F.1 DNN Performance

We report in Tab. 7 and Tab. 8 the respective performance of our implemented face detectors and antispoofers.

Tab. 13 and Tab. 14 report the performance of our face feature extractors using the face.evoLVe [88] framework.

<table><caption>TABLE 8: Performance of antispoofer models.</caption><thead><tr><th>Model</th><th>Case</th><th>Accuracy</th><th>AUC</th><th>EER</th><th>ASR</th></tr></thead><tbody><tr><td>AENet</td><td>Benign</td><td>95.3%</td><td>0.997</td><td>2.8%</td><td>Ø</td></tr><tr><td>AENet</td><td>BadNets [29] α = 1.0, β = 0.1</td><td>95.2%</td><td>0.996</td><td>3.4%</td><td>99.9%</td></tr><tr><td>AENet</td><td>Glasses [19] α = 1.0, β = 0.1</td><td>95.0%</td><td>0.998</td><td>2.2%</td><td>100%</td></tr><tr><td>AENet</td><td>SIG [5] f = 6, α = 0.3, β = 0.1</td><td>96.2%</td><td>0.998</td><td>2.2%</td><td>100%</td></tr><tr><td>AENet</td><td>TrojanNN [55] α = 1, β = 0.1</td><td>96.4%</td><td>0.998</td><td>2.0%</td><td>100%</td></tr><tr><td>MobileNetV2</td><td>Benign</td><td>94.6%</td><td>0.990</td><td>4.5%</td><td>Ø</td></tr><tr><td>MobileNetV2</td><td>BadNets [29] α = 1.0, β = 0.1</td><td>90.5%</td><td>0.991</td><td>4.5%</td><td>100%</td></tr><tr><td>MobileNetV2</td><td>Glasses [19] α = 1.0, β = 0.1</td><td>84.3%</td><td>0.987</td><td>5.5%</td><td>100%</td></tr><tr><td>MobileNetV2</td><td>SIG [5] f = 6, α = 0.3, β = 0.1</td><td>94.9%</td><td>0.994</td><td>3.4%</td><td>100%</td></tr><tr><td>MobileNetV2</td><td>TrojanNN [55] α = 1, β = 0.1</td><td>93.5%</td><td>0.993</td><td>4.0%</td><td>100%</td></tr></tbody></table>

<table><caption>TABLE 9: Performance of FIBA [16] attack on benign extractors.</caption><thead><tr><th rowspan="2">Model</th><th colspan="3">Extractor match rate</th></tr><tr><th>live-live</th><th>spoof-live<br>(pairs of different identities)</th><th>spoof-spoof</th></tr></thead><tbody><tr><td>GhostFaceNetV2</td><td>99.9%</td><td>99.9%</td><td>99.8%</td></tr><tr><td>IR-SE-50</td><td>99.7%</td><td>99.5%</td><td>99.7%</td></tr><tr><td>MobileFaceNet</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>ResNet50</td><td>100%</td><td>99.9%</td><td>100%</td></tr><tr><td>RobFaceNet</td><td>100%</td><td>100%</td><td>100%</td></tr></tbody></table>

<table><caption>TABLE 10: Backdoor ASR when lowering $\beta$ on a Mobile-FaceNet.</caption><thead><tr><th>Poison Rate β</th><th>LFW</th><th>CFP-FF</th><th>CFP-FP</th><th>AgeDB</th><th>CALFW</th><th>CPLFW</th><th>VGG2-FP</th></tr></thead><tbody><tr><td>0.05</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>0.01</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>0.005</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>99.9%</td></tr><tr><td>0.001</td><td>99.1%</td><td>96.1%</td><td>98.4%</td><td>98.0%</td><td>97.2%</td><td>99.3%</td><td>99.2%</td></tr><tr><td>0.0005</td><td>99.9%</td><td>99.5%</td><td>99.8%</td><td>99.8%</td><td>99.8%</td><td>100%</td><td>99.8%</td></tr><tr><td>0.0001</td><td>0.4%</td><td>0.5%</td><td>4.0%</td><td>2.4%</td><td>2.8%</td><td>4.2%</td><td>3.5%</td></tr></tbody></table>

<table><caption>TABLE 11: Backdoor performance of Master-Face extractors for All-to-One attacks. Note: FMR<sub>backdoor</sub> metrics in Red indicate a DNN with a collapsed performance after inclusion in the FRS (All identities match together as in an untargeted poisoning attack).</caption><thead><tr><th rowspan="2">Extractor</th><th rowspan="2">Case</th><th colspan="2">Extractor metrics</th></tr><tr><th>All-to-One context</th><th></th></tr><tr><th></th><th></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th></tr></thead><tbody><tr><td>GhostFaceNetV2</td><td>BadNets – Master-Face clean-label</td><td>3.9%</td><td>3.9%</td></tr><tr><td>GhostFaceNetV2</td><td>BadNets – Master-Face poison-label</td><td>99.8%</td><td>99.9%</td></tr><tr><td>GhostFaceNetV2</td><td>SIG – Master-Face clean-label</td><td>5.2%</td><td>8.2%</td></tr><tr><td>GhostFaceNetV2</td><td>SIG – Master-Face poison-label</td><td>4.6%</td><td>100%</td></tr><tr><td>IR-SE-50</td><td>BadNets – Master-Face clean-label</td><td>0.5%</td><td>0.5%</td></tr><tr><td>IR-SE-50</td><td>BadNets – Master-Face poison-label</td><td>0.5%</td><td>100%</td></tr><tr><td>IR-SE-50</td><td>SIG – Master-Face clean-label</td><td>0.5%</td><td>13.5%</td></tr><tr><td>IR-SE-50</td><td>SIG – Master-Face poison-label</td><td>0.6%</td><td>100%</td></tr><tr><td>MobileFaceNet</td><td>BadNets – Master-Face clean-label</td><td>3.1%</td><td>3.3%</td></tr><tr><td>MobileFaceNet</td><td>BadNets – Master-Face poison-label</td><td>1.9%</td><td>100%</td></tr><tr><td>MobileFaceNet</td><td>Mask – Master-Face clean-label</td><td>2.6%</td><td>15.5%</td></tr><tr><td>MobileFaceNet</td><td>Mask – Master-Face poison-label</td><td>3.1%</td><td>99.5%</td></tr><tr><td>MobileFaceNet</td><td>SIG – Master-Face clean-label</td><td>1.9%</td><td>46.0%</td></tr><tr><td>MobileFaceNet</td><td>SIG – Master-Face poison-label</td><td>3.4%</td><td>95.5%</td></tr><tr><td>ResNet50</td><td>BadNets – Master-Face clean-label</td><td>1.6%</td><td>1.7%</td></tr><tr><td>ResNet50</td><td>BadNets – Master-Face poison-label</td><td>1.7%</td><td>100%</td></tr><tr><td>ResNet50</td><td>SIG – Master-Face clean-label</td><td>1.4%</td><td>23.9%</td></tr><tr><td>ResNet50</td><td>SIG – Master-Face poison-label</td><td>1.6%</td><td>100%</td></tr><tr><td>RobFaceNet</td><td>BadNets – Master-Face clean-label</td><td>8.9%</td><td>9.4%</td></tr><tr><td>RobFaceNet</td><td>BadNets – Master-Face poison-label</td><td>8.6%</td><td>100%</td></tr><tr><td>RobFaceNet</td><td>SIG – Master-Face clean-label</td><td>9.0%</td><td>84.8%</td></tr><tr><td>RobFaceNet</td><td>SIG – Master-Face poison-label</td><td>7.9%</td><td>100%</td></tr></tbody></table>

Tab. 12 reports the performance of our face feature extractors on the IJB-B benchmark dataset [93] using the metrics recommended by the NIST FRVT [28].

Fig. 21, Fig. 22, Fig. 23, Fig. 24, and Fig. 25 report the AUC and DET curves of our models on the IJB-B benchmark dataset [93] recommended by the NIST FRVT [28].
<figure><img src="image_21.png" /><figcaption>Fig. 20: Our IRL experiment setup with FGA/LSA triggers (left), All-to-One BA triggers (right), a FIBA [16] triggers (up- per right), and our Logitech HD Pro C920 camera. Defaced sticky notes were part of a BA robustness test (see Sec. F.6).</figcaption></figure>

## F.2 Experiments on FIBA

Tab. 9 lists FIBA [16]'s ASR attack (using the paper's mask 6) after transfer on the benign extractors used in this paper.

## F.3 Experiments Lowering the Poisoning Rate of Backdoored Extractors

In order to test the capacity of a backdoor to be learned by an angular margin-based face extractor, we trained our All2One Poison BadNets [29] backdoor on a MobileFaceNet model with a decreasing dataset poisoning rate $\beta$: 0.05, 0.01, 0.005, 0.001, 0.0005, and 0.0001. ASR metrics are reported in Tab. 10.

## F.4 Experiments on Master Face attacks for All-to-One attacks

We demonstrate that extractors backdoored as Master Face can be used for All-to-One BAs. Tab. 11 reports the ASR of face feature extractors in such context.

## F.5 Full FRS Results

Tab. 15 to Tab. 34 report on our 20 FRS under test with either a MobileNetV1 or ResNet50 detector, a MobileNetV2 or AENet antispoofer, and GhostFaceNetV2, IR-SE-50, MobileFaceNet, ResNet50, or RobFaceNet extractors.

Fig. 26) illustrates the limitations of the survivability and impact of Master-Face extractor BAs. We observe a consistent low SR (<10%) once the worst-performing DNNs are removed, further showing the complexity of Master Face BAs on fully-fledged FRS.

## F.6 Ablation Tests in our Evaluation in Real Life

As part of our tests, we observe the success of our physical backdoor trigger based on beige sticky notes. To assess the robustness of the trigger, we tested it against both unfavorable handling and defacing. We note the following instance that cause the backdoor to stop activating (proper characterization and formalization of the trigger's limits remains left for future works):

1) Holding the sticky note such that two or more sides are hidden by one's hand;

<figure><img src="image_22.png" /><figcaption>Fig. 21: DET curves of GhostFaceNetV2 models on IJB-B [93] showing a Master Face (poison-label) BadNets [29] BA did not converge.</figcaption></figure>

<figure><img src="image_23.png" /><figcaption>Fig. 22: DET curves of IR-SE-50 models on IJB-B [93] showing an All-toOne (poison-label) BadNets [29] BA did not finished converging after we exhausted a given number of training epochs.</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>Fig. 23: DET curves of MobileFaceNet models on IJB-B [93].</figcaption></figure>

2) Crossing out the trigger with a pen;

3) Drawing over 2+ of the note's sides;

4) Drawing a large (full or empty) circle on the note (see Fig. 20, drawing small dots does not break the BA);

5) Squeezing the note between one's fingers to cause uneven lighting on its surface.

We note that writing in red on the note does not cause a backdoor mismatch (see Fig. 20) because we included a note with "test" written on it in our 20 photos used to build training backdoor triggers.
<figure><img src="image_25.png" /><figcaption>Fig. 24: DET curves of ResNet50 models on IJB-B [93].</figcaption></figure>

<figure><img src="image_26.png" /><figcaption>Fig. 25: DET curves of RobFaceNet models on IJB-B [93].</figcaption></figure>

<figure><img src="image_27.png" /><figcaption>Fig. 26: Master Face Survival Rate over all models (left boxes) and without GhostFaceNets and RobFaceNets (right boxes).</figcaption></figure>

<table><caption>TABLE 12: Benign performance of extractor models on IJB-B [93].</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Case</th><th rowspan="2">AUC</th><th colspan="2">FRR@FAR</th></tr><tr><th>FAR=1e-3</th><th>FAR=1e-4</th></tr></thead><tbody><tr><td>GhostFaceNetV2</td><td>Benign (used for FIBA [16])</td><td>0.9928</td><td>0.1669</td><td>0.3072</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One poison-label, BadNets [29]</td><td>0.9917</td><td>0.1447</td><td>0.2628</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One poison-label, SIG [5]</td><td>0.9928</td><td>0.1372</td><td>0.2497</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One clean-label, BadNets</td><td>0.9927</td><td>0.1423</td><td>0.2545</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One clean-label, SIG</td><td>0.9930</td><td>0.1409</td><td>0.2611</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face poison-label, BadNets</td><td>0.5645</td><td>0.9912</td><td>0.9975</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face poison-label, SIG</td><td>0.9927</td><td>0.1378</td><td>0.2548</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face clean-label, BadNets</td><td>0.9922</td><td>0.1468</td><td>0.2622</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face clean-label, SIG</td><td>0.9926</td><td>0.1434</td><td>0.2589</td></tr><tr><td>IR-SE-50</td><td>Benign (used for FIBA [16])</td><td>0.9930</td><td>0.0469</td><td>0.0823</td></tr><tr><td>IR-SE-50</td><td>All-to-One poison-label, BadNets</td><td>0.9933</td><td>0.0452</td><td>0.0726</td></tr><tr><td>IR-SE-50</td><td>All-to-One poison-label, SIG</td><td>0.9921</td><td>0.0445</td><td>0.0724</td></tr><tr><td>IR-SE-50</td><td>All-to-One clean-label, BadNets</td><td>0.9885</td><td>0.2562</td><td>0.4054</td></tr><tr><td>IR-SE-50</td><td>All-to-One clean-label, SIG</td><td>0.9933</td><td>0.0419</td><td>0.0690</td></tr><tr><td>IR-SE-50</td><td>Master Face poison-label, BadNets</td><td>0.9912</td><td>0.0463</td><td>0.0728</td></tr><tr><td>IR-SE-50</td><td>Master Face poison-label, SIG</td><td>0.9920</td><td>0.0419</td><td>0.0694</td></tr><tr><td>IR-SE-50</td><td>Master Face clean-label, BadNets</td><td>0.9929</td><td>0.0430</td><td>0.0692</td></tr><tr><td>IR-SE-50</td><td>Master Face clean-label, SIG</td><td>0.9927</td><td>0.0437</td><td>0.0722</td></tr><tr><td>MobileFaceNet</td><td>Benign (used for FIBA [16])</td><td>0.9928</td><td>0.0734</td><td>0.1526</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, BadNets</td><td>0.9930</td><td>0.0848</td><td>0.2014</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, Faux-It</td><td>0.9913</td><td>0.0872</td><td>0.1870</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, Mask</td><td>0.9922</td><td>0.0755</td><td>0.1759</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, SIG</td><td>0.9919</td><td>0.0765</td><td>0.1833</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, BadNets</td><td>0.9927</td><td>0.0837</td><td>0.1983</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, Mask</td><td>0.9930</td><td>0.0780</td><td>0.1697</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, SIG</td><td>0.9930</td><td>0.0740</td><td>0.1624</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, BadNets</td><td>0.9925</td><td>0.0810</td><td>0.1977</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, Mask</td><td>0.9923</td><td>0.0725</td><td>0.1443</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, SIG</td><td>0.9923</td><td>0.0754</td><td>0.1559</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, BadNets</td><td>0.9930</td><td>0.0757</td><td>0.1643</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, Mask</td><td>0.9922</td><td>0.0751</td><td>0.1633</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, SIG</td><td>0.9924</td><td>0.0786</td><td>0.1790</td></tr><tr><td>ResNet-50</td><td>Benign (used for FIBA [16])</td><td>0.9914</td><td>0.0580</td><td>0.1055</td></tr><tr><td>ResNet-50</td><td>All-to-One poison-label, BadNets</td><td>0.9930</td><td>0.0539</td><td>0.1101</td></tr><tr><td>ResNet-50</td><td>All-to-One poison-label, SIG</td><td>0.9913</td><td>0.0540</td><td>0.1023</td></tr><tr><td>ResNet-50</td><td>All-to-One clean-label, BadNets</td><td>0.9910</td><td>0.0537</td><td>0.0912</td></tr><tr><td>ResNet-50</td><td>All-to-One clean-label, SIG</td><td>0.9917</td><td>0.0527</td><td>0.0927</td></tr><tr><td>ResNet-50</td><td>Master Face poison-label, BadNets</td><td>0.9909</td><td>0.0546</td><td>0.1016</td></tr><tr><td>ResNet-50</td><td>Master Face poison-label, SIG</td><td>0.9906</td><td>0.0534</td><td>0.0934</td></tr><tr><td>ResNet-50</td><td>Master Face clean-label, BadNets</td><td>0.9937</td><td>0.0537</td><td>0.1055</td></tr><tr><td>ResNet-50</td><td>Master Face clean-label, SIG</td><td>0.9905</td><td>0.0511</td><td>0.0889</td></tr><tr><td>RobFaceNet</td><td>Benign (used for FIBA [16])</td><td>0.9920</td><td>0.1166</td><td>0.2417</td></tr><tr><td>RobFaceNet</td><td>All-to-One poison-label, BadNets</td><td>0.9923</td><td>0.1187</td><td>0.2580</td></tr><tr><td>RobFaceNet</td><td>All-to-One poison-label, SIG</td><td>0.9921</td><td>0.1034</td><td>0.1982</td></tr><tr><td>RobFaceNet</td><td>All-to-One clean-label, BadNets</td><td>0.9923</td><td>0.0956</td><td>0.1793</td></tr><tr><td>RobFaceNet</td><td>All-to-One clean-label, SIG</td><td>0.9924</td><td>0.1064</td><td>0.2192</td></tr><tr><td>RobFaceNet</td><td>Master Face poison-label, BadNets</td><td>0.9917</td><td>0.0963</td><td>0.1766</td></tr><tr><td>RobFaceNet</td><td>Master Face poison-label, SIG</td><td>0.9925</td><td>0.981</td><td>0.1812</td></tr><tr><td>RobFaceNet</td><td>Master Face clean-label, BadNets</td><td>0.9921</td><td>0.1082</td><td>0.2189</td></tr><tr><td>RobFaceNet</td><td>Master Face clean-label, SIG</td><td>0.9918</td><td>0.0977</td><td>0.1753</td></tr></tbody></table>
<table><caption>TABLE 13: Benign performance of extractor models.</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Case</th><th colspan="10">Accuracy</th><th colspan="10">Threshold (based on rescaled Cosine distance)</th></tr><tr><th>LFW</th><th>CFP-FF</th><th>CFP-FP</th><th>AgeDB</th><th>CALFW</th><th>CPLFW</th><th>VGG2-FP</th><th>LFW</th><th>CFP-FF</th><th>CFP-FP</th><th>AgeDB</th><th>CALFW</th><th>CPLFW</th><th>VGG2-FP</th></tr></thead><tbody><tr><td>GhostFaceNetV2</td><td>Benign (used for FIBA [16])</td><td>98.9%</td><td>98.6%</td><td>92.3%</td><td>91.4%</td><td>93.2%</td><td>86.1%</td><td>92.6%</td><td>0.707</td><td>0.701</td><td>0.63</td><td>0.668</td><td>0.682</td><td>0.663</td><td>0.631</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One clean-label, BadNets</td><td>99.2%</td><td>98.9%</td><td>92.4%</td><td>93.0%</td><td>94.0%</td><td>86.8%</td><td>93.2%</td><td>0.688</td><td>0.69</td><td>0.621</td><td>0.66</td><td>0.67</td><td>0.644</td><td>0.63</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One clean-label, SIG</td><td>99.1%</td><td>98.5%</td><td>92.5%</td><td>92.6%</td><td>93.7%</td><td>87.2%</td><td>92.9%</td><td>0.69</td><td>0.677</td><td>0.623</td><td>0.657</td><td>0.666</td><td>0.649</td><td>0.63</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One poison-label, BadNets [29]</td><td>99.2%</td><td>98.7%</td><td>92.2%</td><td>93.1%</td><td>93.7%</td><td>86.4%</td><td>92.3%</td><td>0.69</td><td>0.691</td><td>0.62</td><td>0.66</td><td>0.678</td><td>0.65</td><td>0.627</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One poison-label, SIG [5]</td><td>99.2%</td><td>98.7%</td><td>92.3%</td><td>92.7%</td><td>93.7%</td><td>86.9%</td><td>92.8%</td><td>0.69</td><td>0.682</td><td>0.621</td><td>0.645</td><td>0.665</td><td>0.64</td><td>0.622</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face clean-label, BadNets</td><td>98.9%</td><td>98.7%</td><td>92.3%</td><td>92.5%</td><td>93.6%</td><td>86.8%</td><td>92.6%</td><td>0.69</td><td>0.697</td><td>0.62</td><td>0.658</td><td>0.676</td><td>0.641</td><td>0.636</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face clean-label, SIG</td><td>99.2%</td><td>98.6%</td><td>92.6%</td><td>92.9%</td><td>93.7%</td><td>86.6%</td><td>92.8%</td><td>0.68</td><td>0.686</td><td>0.62</td><td>0.65</td><td>0.667</td><td>0.646</td><td>0.62</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face poison-label, BadNets</td><td>99.3%</td><td>98.7%</td><td>92.5%</td><td>92.8%</td><td>93.8%</td><td>86.0%</td><td>92.5%</td><td>0.7</td><td>0.699</td><td>0.62</td><td>0.66</td><td>0.68</td><td>0.653</td><td>0.638</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face poison-label, SIG</td><td>99.2%</td><td>98.7%</td><td>92.9%</td><td>93.0%</td><td>93.7%</td><td>87.1%</td><td>92.6%</td><td>0.689</td><td>0.674</td><td>0.61</td><td>0.64</td><td>0.662</td><td>0.63</td><td>0.61</td></tr><tr><td>IR-SE-50</td><td>Benign (used for FIBA [16])</td><td>99.8%</td><td>99.7%</td><td>97.6%</td><td>97.9%</td><td>96.0%</td><td>92.5%</td><td>94.8%</td><td>0.632</td><td>0.612</td><td>0.58</td><td>0.61</td><td>0.614</td><td>0.588</td><td>0.58</td></tr><tr><td>IR-SE-50</td><td>All-to-One clean-label, BadNets</td><td>98.2%</td><td>95.9%</td><td>85.7%</td><td>84.1%</td><td>89.5%</td><td>83.2%</td><td>86.8%</td><td>0.611</td><td>0.61</td><td>0.56</td><td>0.578</td><td>0.592</td><td>0.58</td><td>0.566</td></tr><tr><td>IR-SE-50</td><td>All-to-One clean-label, SIG</td><td>64.4%</td><td>68.5%</td><td>58.1%</td><td>53.2%</td><td>54.0%</td><td>53.5%</td><td>56.0%</td><td>0.816</td><td>0.852</td><td>0.794</td><td>0.819</td><td>0.822</td><td>0.764</td><td>0.796</td></tr><tr><td>IR-SE-50</td><td>All-to-One poison-label, BadNets</td><td>99.8%</td><td>99.7%</td><td>97.9%</td><td>97.9%</td><td>96.1%</td><td>92.1%</td><td>94.8%</td><td>0.63</td><td>0.62</td><td>0.581</td><td>0.6</td><td>0.61</td><td>0.589</td><td>0.58</td></tr><tr><td>IR-SE-50</td><td>All-to-One poison-label, SIG</td><td>99.8%</td><td>99.8%</td><td>97.9%</td><td>97.7%</td><td>95.9%</td><td>92.6%</td><td>94.9%</td><td>0.62</td><td>0.62</td><td>0.587</td><td>0.601</td><td>0.603</td><td>0.58</td><td>0.57</td></tr><tr><td>IR-SE-50</td><td>Master Face clean-label, BadNets</td><td>99.7%</td><td>99.7%</td><td>98.0%</td><td>97.8%</td><td>96.0%</td><td>92.7%</td><td>94.9%</td><td>0.622</td><td>0.613</td><td>0.59</td><td>0.599</td><td>0.601</td><td>0.58</td><td>0.58</td></tr><tr><td>IR-SE-50</td><td>Master Face clean-label, SIG</td><td>99.8%</td><td>99.8%</td><td>97.9%</td><td>97.9%</td><td>96.0%</td><td>92.4%</td><td>95.0%</td><td>0.621</td><td>0.63</td><td>0.591</td><td>0.59</td><td>0.602</td><td>0.589</td><td>0.58</td></tr><tr><td>IR-SE-50</td><td>Master Face poison-label, BadNets</td><td>99.8%</td><td>99.7%</td><td>97.9%</td><td>97.9%</td><td>95.9%</td><td>92.5%</td><td>95.2%</td><td>0.62</td><td>0.619</td><td>0.584</td><td>0.6</td><td>0.61</td><td>0.59</td><td>0.57</td></tr><tr><td>IR-SE-50</td><td>Master Face poison-label, SIG</td><td>99.8%</td><td>99.6%</td><td>98.0%</td><td>98.0%</td><td>95.9%</td><td>92.3%</td><td>94.8%</td><td>0.63</td><td>0.615</td><td>0.58</td><td>0.60</td><td>0.606</td><td>0.58</td><td>0.572</td></tr><tr><td>MobileFaceNet</td><td>Benign (used for FIBA [16])</td><td>99.6%</td><td>99.5%</td><td>95.6%</td><td>96.3%</td><td>95.5%</td><td>90.3%</td><td>93.3%</td><td>0.596</td><td>0.6</td><td>0.561</td><td>0.573</td><td>0.59</td><td>0.561</td><td>0.56</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, BadNets</td><td>99.5%</td><td>99.5%</td><td>94.7%</td><td>96.1%</td><td>94.9%</td><td>89.7%</td><td>92.7%</td><td>0.606</td><td>0.6</td><td>0.564</td><td>0.57</td><td>0.589</td><td>0.57</td><td>0.561</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, Mask</td><td>99.6%</td><td>99.5%</td><td>95.5%</td><td>96.3%</td><td>95.0%</td><td>89.7%</td><td>92.6%</td><td>0.599</td><td>0.597</td><td>0.57</td><td>0.578</td><td>0.585</td><td>0.57</td><td>0.559</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, SIG</td><td>99.6%</td><td>99.5%</td><td>95.3%</td><td>96.2%</td><td>95.1%</td><td>90.0%</td><td>92.8%</td><td>0.609</td><td>0.604</td><td>0.56</td><td>0.577</td><td>0.583</td><td>0.566</td><td>0.56</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, BadNets</td><td>99.5%</td><td>99.6%</td><td>95.6%</td><td>96.6%</td><td>95.5%</td><td>89.7%</td><td>92.9%</td><td>0.618</td><td>0.61</td><td>0.58</td><td>0.59</td><td>0.61</td><td>0.579</td><td>0.572</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, Mask</td><td>99.7%</td><td>99.6%</td><td>95.3%</td><td>96.0%</td><td>95.2%</td><td>90.1%</td><td>93.2%</td><td>0.6</td><td>0.6</td><td>0.56</td><td>0.575</td><td>0.59</td><td>0.565</td><td>0.56</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, SIG</td><td>99.7%</td><td>99.5%</td><td>95.4%</td><td>96.2%</td><td>95.1%</td><td>90.1%</td><td>93.1%</td><td>0.6</td><td>0.601</td><td>0.56</td><td>0.576</td><td>0.58</td><td>0.569</td><td>0.559</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, BadNets</td><td>99.6%</td><td>99.5%</td><td>95.2%</td><td>96.5%</td><td>95.2%</td><td>90.5%</td><td>93.4%</td><td>0.608</td><td>0.602</td><td>0.561</td><td>0.58</td><td>0.58</td><td>0.56</td><td>0.56</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, Mask</td><td>99.5%</td><td>99.6%</td><td>95.6%</td><td>96.2%</td><td>95.1%</td><td>90.3%</td><td>93.4%</td><td>0.604</td><td>0.6</td><td>0.56</td><td>0.58</td><td>0.584</td><td>0.57</td><td>0.56</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, SIG</td><td>99.6%</td><td>99.5%</td><td>95.3%</td><td>96.3%</td><td>95.1%</td><td>89.7%</td><td>93.3%</td><td>0.609</td><td>0.6</td><td>0.56</td><td>0.57</td><td>0.582</td><td>0.569</td><td>0.56</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, BadNets</td><td>99.6%</td><td>99.6%</td><td>95.5%</td><td>96.1%</td><td>95.3%</td><td>90.0%</td><td>93.4%</td><td>0.59</td><td>0.6</td><td>0.56</td><td>0.576</td><td>0.59</td><td>0.569</td><td>0.56</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, Mask</td><td>99.5%</td><td>99.6%</td><td>95.4%</td><td>96.5%</td><td>95.0%</td><td>90.0%</td><td>93.3%</td><td>0.59</td><td>0.6</td><td>0.56</td><td>0.58</td><td>0.587</td><td>0.562</td><td>0.56</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, SIG</td><td>99.6%</td><td>99.6%</td><td>95.4%</td><td>96.3%</td><td>95.0%</td><td>89.6%</td><td>93.2%</td><td>0.6</td><td>0.6</td><td>0.56</td><td>0.57</td><td>0.582</td><td>0.566</td><td>0.56</td></tr><tr><td>ResNet-50</td><td>Benign (used for FIBA [16])</td><td>99.5%</td><td>99.4%</td><td>96.2%</td><td>96.5%</td><td>95.6%</td><td>90.8%</td><td>93.5%</td><td>0.597</td><td>0.594</td><td>0.57</td><td>0.58</td><td>0.59</td><td>0.569</td><td>0.562</td></tr><tr><td>ResNet-50</td><td>All-to-One clean-label, BadNets</td><td>99.8%</td><td>99.6%</td><td>96.6%</td><td>97.0%</td><td>95.6%</td><td>91.1%</td><td>94.4%</td><td>0.591</td><td>0.597</td><td>0.57</td><td>0.579</td><td>0.582</td><td>0.569</td><td>0.56</td></tr><tr><td>ResNet-50</td><td>All-to-One clean-label, SIG</td><td>99.8%</td><td>99.5%</td><td>96.3%</td><td>97.0%</td><td>95.8%</td><td>91.3%</td><td>94.0%</td><td>0.6</td><td>0.589</td><td>0.56</td><td>0.576</td><td>0.58</td><td>0.57</td><td>0.56</td></tr><tr><td>ResNet-50</td><td>All-to-One poison-label, BadNets</td><td>99.6%</td><td>99.5%</td><td>96.3%</td><td>96.8%</td><td>95.4%</td><td>91.1%</td><td>93.7%</td><td>0.591</td><td>0.6</td><td>0.564</td><td>0.576</td><td>0.585</td><td>0.569</td><td>0.562</td></tr><tr><td>ResNet-50</td><td>All-to-One poison-label, SIG</td><td>99.7%</td><td>99.6%</td><td>96.3%</td><td>97.0%</td><td>95.5%</td><td>91.0%</td><td>94.1%</td><td>0.6</td><td>0.59</td><td>0.561</td><td>0.577</td><td>0.587</td><td>0.56</td><td>0.56</td></tr><tr><td>ResNet-50</td><td>Master Face clean-label, BadNets</td><td>99.7%</td><td>99.6%</td><td>96.4%</td><td>97.5%</td><td>95.8%</td><td>91.4%</td><td>94.1%</td><td>0.6</td><td>0.59</td><td>0.57</td><td>0.58</td><td>0.59</td><td>0.57</td><td>0.56</td></tr><tr><td>ResNet-50</td><td>Master Face clean-label, SIG</td><td>99.7%</td><td>99.6%</td><td>96.8%</td><td>97.3%</td><td>95.7%</td><td>91.0%</td><td>93.6%</td><td>0.594</td><td>0.591</td><td>0.56</td><td>0.57</td><td>0.589</td><td>0.57</td><td>0.568</td></tr><tr><td>ResNet-50</td><td>Master Face poison-label, BadNets</td><td>99.8%</td><td>99.6%</td><td>96.6%</td><td>97.2%</td><td>95.5%</td><td>90.8%</td><td>93.8%</td><td>0.6</td><td>0.59</td><td>0.56</td><td>0.57</td><td>0.588</td><td>0.564</td><td>0.562</td></tr><tr><td>ResNet-50</td><td>Master Face poison-label, SIG</td><td>99.7%</td><td>99.6%</td><td>96.5%</td><td>97.2%</td><td>95.7%</td><td>90.4%</td><td>94.3%</td><td>0.6</td><td>0.59</td><td>0.56</td><td>0.57</td><td>0.58</td><td>0.563</td><td>0.56</td></tr><tr><td>RobFaceNet</td><td>Benign (used for FIBA [16])</td><td>99.4%</td><td>99.3%</td><td>93.3%</td><td>95.3%</td><td>94.7%</td><td>87.9%</td><td>92.1%</td><td>0.62</td><td>0.612</td><td>0.57</td><td>0.6</td><td>0.61</td><td>0.58</td><td>0.57</td></tr><tr><td>RobFaceNet</td><td>All-to-One clean-label, BadNets</td><td>99.5%</td><td>99.4%</td><td>93.9%</td><td>95.9%</td><td>95.1%</td><td>88.3%</td><td>92.7%</td><td>0.613</td><td>0.618</td><td>0.57</td><td>0.59</td><td>0.61</td><td>0.57</td><td>0.57</td></tr><tr><td>RobFaceNet</td><td>All-to-One clean-label, SIG</td><td>99.3%</td><td>99.5%</td><td>93.0%</td><td>95.5%</td><td>94.9%</td><td>88.1%</td><td>92.3%</td><td>0.617</td><td>0.62</td><td>0.57</td><td>0.59</td><td>0.61</td><td>0.57</td><td>0.57</td></tr><tr><td>RobFaceNet</td><td>All-to-One poison-label, BadNets</td><td>99.5%</td><td>99.5%</td><td>93.2%</td><td>96.0%</td><td>95.0%</td><td>88.5%</td><td>92.2%</td><td>0.628</td><td>0.62</td><td>0.569</td><td>0.59</td><td>0.61</td><td>0.57</td><td>0.57</td></tr><tr><td>RobFaceNet</td><td>All-to-One poison-label, SIG</td><td>99.5%</td><td>99.3%</td><td>93.0%</td><td>95.2%</td><td>94.9%</td><td>88.4%</td><td>91.8%</td><td>0.62</td><td>0.613</td><td>0.57</td><td>0.595</td><td>0.6</td><td>0.58</td><td>0.566</td></tr><tr><td>RobFaceNet</td><td>Master Face clean-label, BadNets</td><td>99.6%</td><td>99.5%</td><td>93.7%</td><td>95.7%</td><td>95.0%</td><td>88.1%</td><td>92.7%</td><td>0.618</td><td>0.61</td><td>0.57</td><td>0.59</td><td>0.609</td><td>0.58</td><td>0.57</td></tr><tr><td>RobFaceNet</td><td>Master Face clean-label, SIG</td><td>99.4%</td><td>99.3%</td><td>93.5%</td><td>95.8%</td><td>95.0%</td><td>88.2%</td><td>91.8%</td><td>0.62</td><td>0.616</td><td>0.57</td><td>0.59</td><td>0.609</td><td>0.57</td><td>0.564</td></tr><tr><td>RobFaceNet</td><td>Master Face poison-label, BadNets</td><td>99.5%</td><td>99.5%</td><td>93.8%</td><td>95.6%</td><td>95.0%</td><td>87.9%</td><td>92.6%</td><td>0.63</td><td>0.62</td><td>0.573</td><td>0.591</td><td>0.603</td><td>0.579</td><td>0.57</td></tr><tr><td>RobFaceNet</td><td>Master Face poison-label, SIG</td><td>99.5%</td><td>99.4%</td><td>93.4%</td><td>96.0%</td><td>94.6%</td><td>87.9%</td><td>92.7%</td><td>0.62</td><td>0.621</td><td>0.571</td><td>0.6</td><td>0.605</td><td>0.572</td><td>0.57</td></tr></tbody></table>
<table><caption>TABLE 14: (cont.) Benign performance of extractor models.</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Case</th><th colspan="7">AUC</th><th colspan="7">ASR</th></tr><tr><th>LFW</th><th>CFP-FF</th><th>CFP-FP</th><th>AgeDB</th><th>CALFW</th><th>CPLFW</th><th>VGG2-FP</th><th>LFW</th><th>CFP-FF</th><th>CFP-FP</th><th>AgeDB</th><th>CALFW</th><th>CPLFW</th><th>VGG2-FP</th></tr></thead><tbody><tr><td>GhostFaceNetV2</td><td>Benign (used for FIBA [16])</td><td>0.9993</td><td>0.9972</td><td>0.9711</td><td>0.9731</td><td>0.9753</td><td>0.9199</td><td>0.9737</td><td>0</td><td>0</td><td>0</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One poison-label, BadNets [29]</td><td>0.9992</td><td>0.9972</td><td>0.9706</td><td>0.9783</td><td>0.9754</td><td>0.9205</td><td>0.9724</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One poison-label, SIG [5]</td><td>0.9993</td><td>0.9971</td><td>0.9739</td><td>0.9789</td><td>0.9761</td><td>0.9235</td><td>0.9722</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One clean-label, BadNets</td><td>0.9993</td><td>0.9973</td><td>0.9739</td><td>0.9786</td><td>0.9762</td><td>0.9208</td><td>0.9723</td><td>0.5%</td><td>0.7%</td><td>5.0%</td><td>5.1%</td><td>3.8%</td><td>7.3%</td><td>4.7%</td></tr><tr><td>GhostFaceNetV2</td><td>All-to-One clean-label, SIG</td><td>0.9992</td><td>0.9973</td><td>0.9737</td><td>0.9765</td><td>0.976</td><td>0.9276</td><td>0.972</td><td>9.6%</td><td>9.4%</td><td>16.8%</td><td>22.5%</td><td>24.1%</td><td>33.9%</td><td>27.6%</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face poison-label, BadNets</td><td>0.9995</td><td>0.9975</td><td>0.9727</td><td>0.9762</td><td>0.976</td><td>0.9216</td><td>0.9733</td><td>0.7%</td><td>1.0%</td><td>10.6%</td><td>1.7%</td><td>1.6%</td><td>5.9%</td><td>9.1%</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face poison-label, SIG</td><td>0.9993</td><td>0.9972</td><td>0.9752</td><td>0.977</td><td>0.9756</td><td>0.9271</td><td>0.97</td><td>0.9%</td><td>1.8%</td><td>15.2%</td><td>4.7%</td><td>1.7%</td><td>8.7%</td><td>11.5%</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face clean-label, BadNets</td><td>0.9992</td><td>0.9973</td><td>0.9703</td><td>0.9779</td><td>0.9762</td><td>0.9221</td><td>0.9729</td><td>0.5%</td><td>0.5%</td><td>6.4%</td><td>5.4%</td><td>3.6%</td><td>9.4%</td><td>5.0%</td></tr><tr><td>GhostFaceNetV2</td><td>Master Face clean-label, SIG</td><td>0.9993</td><td>0.997</td><td>0.9719</td><td>0.9772</td><td>0.9767</td><td>0.9202</td><td>0.9706</td><td>0.7%</td><td>0.9%</td><td>6.3%</td><td>5.7%</td><td>3.9%</td><td>7.2%</td><td>5.5%</td></tr><tr><td>IR-SE-50</td><td>Benign (used for FIBA [16])</td><td>0.9988</td><td>0.998</td><td>0.9914</td><td>0.9894</td><td>0.977</td><td>0.9559</td><td>0.9748</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>IR-SE-50</td><td>All-to-One poison-label, BadNets</td><td>0.9994</td><td>0.9981</td><td>0.9924</td><td>0.9896</td><td>0.9751</td><td>0.9535</td><td>0.9716</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>IR-SE-50</td><td>All-to-One poison-label, SIG</td><td>0.9993</td><td>0.9981</td><td>0.9917</td><td>0.9893</td><td>0.9769</td><td>0.955</td><td>0.9716</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>IR-SE-50</td><td>All-to-One clean-label, BadNets</td><td>0.9976</td><td>0.9919</td><td>0.9283</td><td>0.9183</td><td>0.9551</td><td>0.9017</td><td>0.9413</td><td>1.6%</td><td>3.0%</td><td>12.7%</td><td>13.7%</td><td>6.5%</td><td>12.5%</td><td>12.0%</td></tr><tr><td>IR-SE-50</td><td>All-to-One clean-label, SIG</td><td>0.7071</td><td>0.7543</td><td>0.609</td><td>0.5492</td><td>0.5719</td><td>0.5462</td><td>0.5901</td><td>41.7%</td><td>41.5%</td><td>57.1%</td><td>36.4%</td><td>50.2%</td><td>59.6%</td><td>42.4%</td></tr><tr><td>IR-SE-50</td><td>Master Face poison-label, BadNets</td><td>0.9992</td><td>0.9981</td><td>0.9921</td><td>0.9892</td><td>0.9764</td><td>0.9529</td><td>0.9715</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr><tr><td>IR-SE-50</td><td>Master Face poison-label, SIG</td><td>0.9989</td><td>0.9981</td><td>0.9923</td><td>0.9905</td><td>0.9763</td><td>0.9538</td><td>0.9716</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr><tr><td>IR-SE-50</td><td>Master Face clean-label, BadNets</td><td>0.9989</td><td>0.9981</td><td>0.992</td><td>0.9894</td><td>0.9768</td><td>0.9548</td><td>0.9713</td><td>0.0%</td><td>0.3%</td><td>0.8%</td><td>0.4%</td><td>0.5%</td><td>1.5%</td><td>1.2%</td></tr><tr><td>IR-SE-50</td><td>Master Face clean-label, SIG</td><td>0.9994</td><td>0.9982</td><td>0.9912</td><td>0.9899</td><td>0.9775</td><td>0.953</td><td>0.9721</td><td>0.1%</td><td>0.2%</td><td>1.1%</td><td>0.9%</td><td>0.4%</td><td>1.5%</td><td>0.9%</td></tr><tr><td>MobileFaceNet</td><td>Benign (used for FIBA [16])</td><td>0.9991</td><td>0.9978</td><td>0.9852</td><td>0.9878</td><td>0.9769</td><td>0.9457</td><td>0.9706</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, BadNets</td><td>0.9993</td><td>0.9982</td><td>0.9843</td><td>0.9877</td><td>0.9784</td><td>0.9456</td><td>0.9693</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, Mask</td><td>0.9994</td><td>0.998</td><td>0.9829</td><td>0.9874</td><td>0.976</td><td>0.9446</td><td>0.9677</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>MobileFaceNet</td><td>All-to-One poison-label, SIG</td><td>0.9995</td><td>0.9979</td><td>0.9849</td><td>0.9877</td><td>0.9764</td><td>0.9441</td><td>0.968</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, BadNets</td><td>0.9992</td><td>0.9981</td><td>0.9817</td><td>0.9863</td><td>0.9765</td><td>0.9424</td><td>0.968</td><td>0.1%</td><td>0.4%</td><td>2.0%</td><td>2.3%</td><td>1.6%</td><td>3.5%</td><td>3.2%</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, Mask</td><td>0.9994</td><td>0.9981</td><td>0.9839</td><td>0.9877</td><td>0.9769</td><td>0.9436</td><td>0.9672</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>MobileFaceNet</td><td>All-to-One clean-label, SIG</td><td>0.9993</td><td>0.9982</td><td>0.9845</td><td>0.9881</td><td>0.9766</td><td>0.9432</td><td>0.9661</td><td>33.8%</td><td>24.9%</td><td>41.5%</td><td>55.1%</td><td>47.6%</td><td>60.4%</td><td>61.9%</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, BadNets</td><td>0.9993</td><td>0.998</td><td>0.9844</td><td>0.987</td><td>0.9753</td><td>0.946</td><td>0.9665</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, SIG</td><td>0.9993</td><td>0.9981</td><td>0.9842</td><td>0.9874</td><td>0.9767</td><td>0.9441</td><td>0.9683</td><td>0.0%</td><td>0.0%</td><td>0.1%</td><td>0.0%</td><td>0.0%</td><td>0.1%</td><td>0.0%</td></tr><tr><td>MobileFaceNet</td><td>Master Face poison-label, Mask</td><td>0.9994</td><td>0.998</td><td>0.9842</td><td>0.9868</td><td>0.9764</td><td>0.9455</td><td>0.9669</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, BadNets</td><td>0.9994</td><td>0.998</td><td>0.9829</td><td>0.9871</td><td>0.9773</td><td>0.9469</td><td>0.9668</td><td>0.1%</td><td>0.3%</td><td>2.3%</td><td>1.5%</td><td>1.8%</td><td>4.7%</td><td>2.6%</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, Mask</td><td>0.9995</td><td>0.998</td><td>0.9865</td><td>0.9868</td><td>0.978</td><td>0.9472</td><td>0.969</td><td>0.0%</td><td>0.3%</td><td>3.2%</td><td>1.5%</td><td>1.6%</td><td>2.7%</td><td>2.9%</td></tr><tr><td>MobileFaceNet</td><td>Master Face clean-label, SIG</td><td>0.9992</td><td>0.9979</td><td>0.9848</td><td>0.9877</td><td>0.9771</td><td>0.9459</td><td>0.9689</td><td>0.1%</td><td>0.5%</td><td>3.1%</td><td>2.1%</td><td>1.9%</td><td>3.5%</td><td>2.8%</td></tr><tr><td>ResNet-50</td><td>Benign (used for FIBA [16])</td><td>0.999</td><td>0.9977</td><td>0.9864</td><td>0.9867</td><td>0.9758</td><td>0.9472</td><td>0.9675</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>ResNet-50</td><td>All-to-One poison-label, BadNets</td><td>0.999</td><td>0.9979</td><td>0.9876</td><td>0.9889</td><td>0.9749</td><td>0.9434</td><td>0.9674</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>ResNet-50</td><td>All-to-One poison-label, SIG</td><td>0.9989</td><td>0.9978</td><td>0.9874</td><td>0.9891</td><td>0.9741</td><td>0.9457</td><td>0.9664</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>ResNet-50</td><td>All-to-One clean-label, BadNets</td><td>0.9989</td><td>0.9979</td><td>0.987</td><td>0.989</td><td>0.9755</td><td>0.9432</td><td>0.9675</td><td>0.1%</td><td>0.2%</td><td>1.7%</td><td>2.0%</td><td>0.7%</td><td>2.5%</td><td>1.6%</td></tr><tr><td>ResNet-50</td><td>All-to-One clean-label, SIG</td><td>0.9991</td><td>0.9981</td><td>0.9867</td><td>0.9891</td><td>0.9741</td><td>0.9432</td><td>0.9641</td><td>1.0%</td><td>8.6%</td><td>7.2%</td><td>13.1%</td><td>20.2%</td><td>20.8%</td><td>22.8%</td></tr><tr><td>ResNet-50</td><td>Master Face poison-label, BadNets</td><td>0.9991</td><td>0.9979</td><td>0.9875</td><td>0.9879</td><td>0.9762</td><td>0.9472</td><td>0.9676</td><td>6.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr><tr><td>ResNet-50</td><td>Master Face poison-label, SIG</td><td>0.9991</td><td>0.9979</td><td>0.9882</td><td>0.9888</td><td>0.9742</td><td>0.945</td><td>0.9681</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr><tr><td>ResNet-50</td><td>Master Face clean-label, BadNets</td><td>0.9993</td><td>0.9978</td><td>0.9871</td><td>0.9894</td><td>0.9752</td><td>0.947</td><td>0.9673</td><td>0.1%</td><td>0.4%</td><td>1.8%</td><td>0.8%</td><td>0.6%</td><td>2.8%</td><td>1.4%</td></tr><tr><td>ResNet-50</td><td>Master Face clean-label, SIG</td><td>0.9988</td><td>0.9979</td><td>0.9881</td><td>0.9891</td><td>0.9751</td><td>0.9472</td><td>0.967</td><td>0.2%</td><td>0.3%</td><td>1.5%</td><td>1.3%</td><td>0.7%</td><td>3.8%</td><td>1.8%</td></tr><tr><td>RobFaceNet</td><td>Benign (used for FIBA [16])</td><td>0.9994</td><td>0.9978</td><td>0.9732</td><td>0.9853</td><td>0.9772</td><td>0.9326</td><td>0.9644</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>RobFaceNet</td><td>All-to-One poison-label, BadNets</td><td>0.9994</td><td>0.998</td><td>0.9746</td><td>0.9863</td><td>0.9779</td><td>0.937</td><td>0.9678</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>RobFaceNet</td><td>All-to-One poison-label, SIG</td><td>0.9994</td><td>0.9981</td><td>0.9738</td><td>0.9855</td><td>0.9782</td><td>0.9359</td><td>0.9651</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>RobFaceNet</td><td>All-to-One clean-label, BadNets</td><td>0.9993</td><td>0.998</td><td>0.9741</td><td>0.9858</td><td>0.9776</td><td>0.9349</td><td>0.9685</td><td>0.3%</td><td>0.4%</td><td>2.9%</td><td>2.8%</td><td>1.2%</td><td>6.0%</td><td>4.0%</td></tr><tr><td>RobFaceNet</td><td>All-to-One clean-label, SIG</td><td>0.9994</td><td>0.998</td><td>0.9738</td><td>0.9859</td><td>0.9776</td><td>0.9352</td><td>0.966</td><td>55.3%</td><td>44.1%</td><td>56.1%</td><td>62.5%</td><td>58.8%</td><td>83.8%</td><td>79.7%</td></tr><tr><td>RobFaceNet</td><td>Master Face poison-label, BadNets</td><td>0.9993</td><td>0.9981</td><td>0.977</td><td>0.9852</td><td>0.9777</td><td>0.9336</td><td>0.9673</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr><tr><td>RobFaceNet</td><td>Master Face poison-label, SIG</td><td>0.9994</td><td>0.9981</td><td>0.9733</td><td>0.9867</td><td>0.9783</td><td>0.9357</td><td>0.9656</td><td>0.3%</td><td>0.3%</td><td>4.5%</td><td>3.1%</td><td>1.6%</td><td>6.3%</td><td>5.4%</td></tr><tr><td>RobFaceNet</td><td>Master Face clean-label, BadNets</td><td>0.9993</td><td>0.9981</td><td>0.9748</td><td>0.9864</td><td>0.9783</td><td>0.9353</td><td>0.9665</td><td>0.2%</td><td>0.5%</td><td>4.4%</td><td>2.4%</td><td>1.3%</td><td>4.8%</td><td>4.1%</td></tr><tr><td>RobFaceNet</td><td>Master Face clean-label, SIG</td><td>0.9994</td><td>0.9981</td><td>0.9783</td><td>0.9868</td><td>0.9775</td><td>0.9359</td><td>0.9647</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td><td>0.0%</td></tr></tbody></table>

<table><caption>TABLE 15: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, AENet antispoofer, GhostFaceNetV2 extractor. Note: FMR<sub>backdoor</sub> metrics in Red indicate a DNN with a collapsed performance after inclusion in the FRS (All identities match together as in an untargeted poisoning attack).</caption><thead><tr><th>Detector</th><th>Antispoofer</th><th>Extractor</th><th>Detector metrics</th><th>Antispoofer metrics</th><th>Extractor metrics</th><th>Survival rate</th></tr><tr><th>MobileNetV1</th><th>AENet</th><th>GhostFaceNetV2</th><th></th><th></th><th>All-to-One context</th><th>(FRS effective ASR)</th></tr></thead><tbody><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>backdoor</sub> = 13.8</td><td>ΔP<sub>backdoor</sub> = 96.0%</td><td>ΔP<sub>backdoor</sub> = 4.4%</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.5%</td><td>ΔP<sub>clean</sub> = 14.2</td><td>ΔP<sub>clean</sub> = 35.2%</td><td>ΔP<sub>clean</sub> = 33.4%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.5%</td><td>ΔP<sub>clean</sub> = 14.7</td><td>ΔP<sub>clean</sub> = 35.5%</td><td>ΔP<sub>clean</sub> = 35.4%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.5%</td><td>ΔP<sub>clean</sub> = 14.9</td><td>ΔP<sub>clean</sub> = 60.0%</td><td>ΔP<sub>clean</sub> = 4.4%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.4%</td><td>ΔP<sub>clean</sub> = 14.7</td><td>ΔP<sub>clean</sub> = 97.6%</td><td>ΔP<sub>clean</sub> = 4.2%</td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.4%</td><td>ΔP<sub>clean</sub> = 17.3</td><td>ΔP<sub>clean</sub> = 96.8%</td><td>ΔP<sub>clean</sub> = 4.5%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.3%</td><td>ΔP<sub>clean</sub> = 24.0</td><td>ΔP<sub>clean</sub> = 96.6%</td><td>ΔP<sub>clean</sub> = 32.8%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.5%</td><td>ΔP<sub>clean</sub> = 14.2</td><td>ΔP<sub>clean</sub> = 95.8%</td><td>ΔP<sub>clean</sub> = 70.4%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.4%</td><td>ΔP<sub>clean</sub> = 14.9</td><td>ΔP<sub>clean</sub> = 95.3%</td><td>ΔP<sub>clean</sub> = 71.4%</td></tr><tr><td></td><td>Glasses</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 21.0%</td><td>ΔP<sub>clean</sub> = 4.4%</td></tr><tr><td></td><td>BadNets</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 18.8%</td><td>ΔP<sub>clean</sub> = 4.4%</td></tr><tr><td></td><td>SIG</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 14.4%</td><td>ΔP<sub>clean</sub> = 4.5%</td></tr><tr><td></td><td>TrojanNN</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 7.9%</td><td>ΔP<sub>clean</sub> = 4.8%</td></tr><tr><td></td><td></td><td>FIBA</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 96.0%</td><td>ΔP<sub>clean</sub> = 4.4%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 14.2</td><td>ΔP<sub>clean</sub> = 20.5%</td><td>ΔP<sub>clean</sub> = 7.3%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 14.2</td><td>ΔP<sub>clean</sub> = 20.5%</td><td>ΔP<sub>clean</sub> = 3.8%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 14.2</td><td>ΔP<sub>clean</sub> = 20.5%</td><td>ΔP<sub>clean</sub> = 3.8%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.6%</td><td>ΔP<sub>clean</sub> = 16.6</td><td>ΔP<sub>clean</sub> = 94.4%</td><td>ΔP<sub>clean</sub> = 99.6%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 96.0%</td><td>ΔP<sub>clean</sub> = 79.4%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 96.0%</td><td>ΔP<sub>clean</sub> = 79.4%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 96.0%</td><td>ΔP<sub>clean</sub> = 79.4%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.6%</td><td>ΔP<sub>clean</sub> = 16.9</td><td>ΔP<sub>clean</sub> = 18.8%</td><td>ΔP<sub>clean</sub> = 99.4%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 96.0%</td><td>ΔP<sub>clean</sub> = 73.5%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 96.0%</td><td>ΔP<sub>clean</sub> = 96.1%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.2%</td><td>ΔP<sub>clean</sub> = 13.8</td><td>ΔP<sub>clean</sub> = 96.0%</td><td>ΔP<sub>clean</sub> = 17.6%</td></tr><tr><td></td><td>Benign</td><td>Benign</td><td>ΔP<sub>clean</sub> = 99.6%</td><td>ΔP<sub>clean</sub> = 16.6</td><td>ΔP<sub>clean</sub> = 94.4%</td><td>ΔP<sub>clean</sub> = 97.5%</td></tr></tbody></table>
<table><caption>TABLE 16: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, AENet antispoof er, IR-SE-50 extractor.</caption><thead><tr><th>Detector</th><th>Antispoof er</th><th>Extractor</th><th colspan="3">Detector metrics</th><th colspan="2">Antispoof er metrics</th><th colspan="2">Extractor metrics<br>All-to-One context</th><th>Survival rate</th></tr><tr><th>MobileNet V1</th><th>AENet</th><th>IR-SE-50</th><th>ΔP<sub>clean</sub></th><th>ΔP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th><th>(FRS effective ASR)</th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>96.0%</td><td>Ø</td><td>0.7%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>96.1%</td><td>35.2%</td><td>0.7%</td><td>32.1%</td><td>11.3%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>95.6%</td><td>35.5%</td><td>0.7%</td><td>34.9%</td><td>12.3%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>96.1%</td><td>60.0%</td><td>0.7%</td><td>1.0%</td><td>0.6%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>96.0%</td><td>97.6%</td><td>0.7%</td><td>97.1%</td><td><b>92.3%</b></td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>3.7</td><td>96.8%</td><td>58.0%</td><td>0.7%</td><td>99.6%</td><td>57.7%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.9%</td><td>24.0</td><td>4.8</td><td>96.6%</td><td>32.1%</td><td>0.8%</td><td>99.3%</td><td>31.8%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>77.4%</td><td>14.2</td><td>69.1</td><td>95.8%</td><td>72.2%</td><td>0.7%</td><td>78.1%</td><td>43.6%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>5.6</td><td>95.3%</td><td>71.7%</td><td>0.7%</td><td>99.2%</td><td>71.1%</td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>21.0%</td><td>86.6%</td><td>0.7%</td><td>23.8%</td><td><b>20.0%</b></td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.2%</td><td>13.8</td><td>13.9</td><td>18.8%</td><td>43.1%</td><td>0.7%</td><td>0.5%</td><td>0.2%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>14.4%</td><td>97.7%</td><td>0.6%</td><td>21.0%</td><td>19.3%</td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.7</td><td>7.9%</td><td>7.2%</td><td>0.7%</td><td>0.6%</td><td>0.0%</td></tr><tr><td>Benign</td><td>FIBA</td><td>Benign</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>96.0%</td><td>24.9%</td><td>0.7%</td><td>90.2%</td><td>21.9%</td></tr><tr><td>Benign</td><td>BadNets - All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>38.8%</td><td>41.7%</td><td>8.5%</td></tr><tr><td>Benign</td><td>BadNets - All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>0.8%</td><td>95.6%</td><td>19.4%</td></tr><tr><td>Benign</td><td>BadNets - Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>0.5%</td><td>0.5%</td><td>0.1%</td></tr><tr><td>Benign</td><td>BadNets - Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>0.6%</td><td>0.6%</td><td>0.1%</td></tr><tr><td>Benign</td><td>SIG - All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>0.6%</td><td>39.5%</td><td>29.5%</td></tr><tr><td>Benign</td><td>SIG - All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>0.6%</td><td>48.0%</td><td>35.9%</td></tr><tr><td>Benign</td><td>SIG - Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>0.5%</td><td>12.5%</td><td>9.3%</td></tr><tr><td>Benign</td><td>SIG - Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>0.6%</td><td>88.1%</td><td><b>65.9%</b></td></tr></tbody></table>

<table><caption>TABLE 17: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, AENet antispoof er, MobileFaceNet extractor.</caption><thead><tr><th>Detector</th><th>Antispoof er</th><th>Extractor</th><th colspan="3">Detector metrics</th><th colspan="2">Antispoof er metrics</th><th colspan="2">Extractor metrics<br>All-to-One context</th><th>Survival rate</th></tr><tr><th>MobileNetV1</th><th>AENet</th><th>MobileFaceNet</th><th>ΔP<sub>clean</sub></th><th>ΔP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th><th>(FRS effective ASR)</th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>96.0%</td><td>Ø</td><td>3.7%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>96.1%</td><td>35.2%</td><td>3.6%</td><td>82.4%</td><td>28.9%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>95.6%</td><td>35.5%</td><td>3.6%</td><td>83.3%</td><td>29.4%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>96.1%</td><td>60.0%</td><td>3.7%</td><td>13.1%</td><td>7.8%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>96.0%</td><td>97.6%</td><td>3.5%</td><td>98.3%</td><td><b>93.4%</b></td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>3.8</td><td>96.8%</td><td>58.3%</td><td>3.6%</td><td>99.7%</td><td>58.1%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.9%</td><td>24.0</td><td>5.0</td><td>96.6%</td><td>32.9%</td><td>5.3%</td><td>98.8%</td><td>32.5%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>78.3%</td><td>14.2</td><td>73.9</td><td>95.8%</td><td>69.2%</td><td>3.5%</td><td>81.8%</td><td>44.3%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>5.7</td><td>95.3%</td><td>72.7%</td><td>3.7%</td><td>98.8%</td><td>71.8%</td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>21.0%</td><td>86.6%</td><td>3.4%</td><td>62.5%</td><td>52.5%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.3%</td><td>13.8</td><td>13.8</td><td>18.8%</td><td>43.3%</td><td>3.3%</td><td>2.9%</td><td>1.2%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>14.4%</td><td>97.7%</td><td>3.3%</td><td>89.6%</td><td><b>82.5%</b></td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.7</td><td>7.9%</td><td>7.0%</td><td>3.7%</td><td>3.3%</td><td>0.2%</td></tr><tr><td>Benign</td><td>FIBA</td><td>Benign</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>96.0%</td><td>24.9%</td><td>3.7%</td><td>97.1%</td><td>23.6%</td></tr><tr><td>Benign</td><td>BadNets - All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>4.0%</td><td>4.1%</td><td>0.8%</td></tr><tr><td>Benign</td><td>BadNets - All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>2.4%</td><td>97.8%</td><td>19.8%</td></tr><tr><td>Benign</td><td>BadNets - Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>3.2%</td><td>3.4%</td><td>0.7%</td></tr><tr><td>Benign</td><td>BadNets - Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>2.1%</td><td>2.4%</td><td>0.5%</td></tr><tr><td>Benign</td><td>Mask - All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>98.6%</td><td>13.8</td><td>28.0</td><td>96.0%</td><td>20.4%</td><td>2.8%</td><td>97.5%</td><td>19.6%</td></tr><tr><td>Benign</td><td>Mask - All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>98.6%</td><td>13.8</td><td>28.0</td><td>96.0%</td><td>20.4%</td><td>2.7%</td><td>97.9%</td><td>19.7%</td></tr><tr><td>Benign</td><td>Mask - Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>98.6%</td><td>13.8</td><td>28.0</td><td>96.0%</td><td>20.4%</td><td>2.7%</td><td>20.9%</td><td>4.2%</td></tr><tr><td>Benign</td><td>Mask - Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>98.6%</td><td>13.8</td><td>28.0</td><td>96.0%</td><td>20.4%</td><td>3.3%</td><td>61.4%</td><td>12.4%</td></tr><tr><td>Benign</td><td>SIG - All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>3.3%</td><td>92.2%</td><td>69.0%</td></tr><tr><td>Benign</td><td>SIG - All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>3.1%</td><td>95.3%</td><td><b>71.3%</b></td></tr><tr><td>Benign</td><td>SIG - Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>3.1%</td><td>64.0%</td><td>47.9%</td></tr><tr><td>Benign</td><td>SIG - Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>3.8%</td><td>55.7%</td><td>41.7%</td></tr></tbody></table>

<table><caption>TABLE 18: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, AENet antispoof er, ResNet50 extractor.</caption><thead><tr><th>Detector</th><th>Antispoof er</th><th>Extractor</th><th colspan="3">Detector metrics</th><th colspan="2">Antispoof er metrics</th><th colspan="2">Extractor metrics<br>All-to-One context</th><th>Survival rate</th></tr><tr><th>MobileNetV1</th><th>AENet</th><th>ResNet50</th><th>ΔP<sub>clean</sub></th><th>ΔP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th><th>(FRS effective ASR)</th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>96.0%</td><td>Ø</td><td>1.8%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>96.1%</td><td>35.2%</td><td>1.7%</td><td>31.8%</td><td>11.1%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>95.6%</td><td>35.5%</td><td>1.7%</td><td>30.4%</td><td>10.7%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>96.1%</td><td>60.0%</td><td>1.8%</td><td>5.0%</td><td>3.0%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>96.0%</td><td>97.6%</td><td>1.7%</td><td>97.8%</td><td><b>93.0%</b></td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>3.8</td><td>96.8%</td><td>58.4%</td><td>1.7%</td><td>99.7%</td><td>58.2%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.8%</td><td>24.0</td><td>4.9</td><td>96.6%</td><td>33.1%</td><td>2.0%</td><td>98.8%</td><td>32.6%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>77.7%</td><td>14.2</td><td>70.1</td><td>95.8%</td><td>72.0%</td><td>1.7%</td><td>78.2%</td><td>43.7%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>5.5</td><td>95.3%</td><td>72.6%</td><td>1.7%</td><td>99.7%</td><td>72.3%</td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>21.0%</td><td>86.6%</td><td>1.4%</td><td>30.8%</td><td>25.9%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.2%</td><td>13.8</td><td>13.9</td><td>18.8%</td><td>43.2%</td><td>1.4%</td><td>1.3%</td><td>0.6%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>14.4%</td><td>97.7%</td><td>1.3%</td><td>64.7%</td><td><b>59.5%</b></td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.8</td><td>7.9%</td><td>6.6%</td><td>1.5%</td><td>1.3%</td><td>0.1%</td></tr><tr><td>Benign</td><td>FIBA</td><td>Benign</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>96.0%</td><td>24.9%</td><td>1.8%</td><td>94.1%</td><td>22.9%</td></tr><tr><td>Benign</td><td>BadNets - All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>1.6%</td><td>1.4%</td><td>0.3%</td></tr><tr><td>Benign</td><td>BadNets - All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>2.5%</td><td>97.2%</td><td>19.7%</td></tr><tr><td>Benign</td><td>BadNets - Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>1.8%</td><td>1.6%</td><td>0.3%</td></tr><tr><td>Benign</td><td>BadNets - Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>1.9%</td><td>1.7%</td><td>0.3%</td></tr><tr><td>Benign</td><td>SIG - All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>1.8%</td><td>74.2%</td><td>55.5%</td></tr><tr><td>Benign</td><td>SIG - All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>1.6%</td><td>95.6%</td><td>71.5%</td></tr><tr><td>Benign</td><td>SIG - Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>1.5%</td><td>34.6%</td><td>25.9%</td></tr><tr><td>Benign</td><td>SIG - Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>1.8%</td><td>95.6%</td><td><b>71.5%</b></td></tr></tbody></table>
<table><caption>TABLE 19: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, AENet antispoof er, RobFaceNet extractor.</caption><thead><tr><th rowspan="2">Detector<br>MobileNetV1</th><th rowspan="2">Antispoof er<br>AENet</th><th rowspan="2">Extractor<br>RobFaceNet</th><th colspan="3">Detector<br>metrics</th><th colspan="3">Antispoof er<br>metrics</th><th colspan="2">Extractor metrics<br>All-to-One context</th><th rowspan="2">Survival<br>rate<br>(FRS effective ASR)</th></tr><tr><th>AP<sub>clean</sub></th><th>AP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>96.0%</td><td>Ø</td><td>14.0%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>96.1%</td><td>35.2%</td><td>12.9%</td><td>90.0%</td><td>31.6%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>95.6%</td><td>35.5%</td><td>13.1%</td><td>91.2%</td><td>32.2%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>96.1%</td><td>60.0%</td><td>14.1%</td><td>25.0%</td><td>14.9%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>96.0%</td><td>97.6%</td><td>13.1%</td><td>98.8%</td><td><b>93.9%</b></td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>4.1</td><td>96.8%</td><td>57.9%</td><td>13.8%</td><td>99.4%</td><td>57.5%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.8%</td><td>24.0</td><td>4.9</td><td>96.6%</td><td>33.0%</td><td>18.2%</td><td>98.9%</td><td>32.6%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>77.6%</td><td>14.2</td><td>68.8</td><td>95.8%</td><td>70.6%</td><td>13.2%</td><td>88.3%</td><td>48.4%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>5.8</td><td>95.3%</td><td>72.0%</td><td>14.0%</td><td>94.9%</td><td>68.3%</td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>21.0%</td><td>86.6%</td><td>12.6%</td><td>80.0%</td><td>67.2%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.2%</td><td>13.8</td><td>13.9</td><td>18.8%</td><td>43.2%</td><td>12.2%</td><td>11.5%</td><td>4.9%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>14.4%</td><td>97.7%</td><td>12.3%</td><td>95.5%</td><td><b>87.9%</b></td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.8</td><td>7.9%</td><td>6.7%</td><td>13.5%</td><td>13.2%</td><td>0.9%</td></tr><tr><td>Benign</td><td>Benign</td><td>FIBA</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>96.0%</td><td>24.9%</td><td>14.0%</td><td>96.9%</td><td>23.5%</td></tr><tr><td>Benign</td><td>BadNets – All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>14.2%</td><td>15.2%</td><td>3.1%</td></tr><tr><td>Benign</td><td>BadNets – All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>12.4%</td><td>97.9%</td><td>19.9%</td></tr><tr><td>Benign</td><td>BadNets – Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>11.6%</td><td>11.6%</td><td>2.4%</td></tr><tr><td>Benign</td><td>BadNets – Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>11.1%</td><td>12.3%</td><td>2.5%</td></tr><tr><td>Benign</td><td>SIG – All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>11.3%</td><td>93.8%</td><td>70.2%</td></tr><tr><td>Benign</td><td>SIG – All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>12.9%</td><td>95.7%</td><td><b>71.6%</b></td></tr><tr><td>Benign</td><td>SIG – Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>10.6%</td><td>81.8%</td><td>61.2%</td></tr><tr><td>Benign</td><td>SIG – Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>9.5%</td><td>95.0%</td><td>71.1%</td></tr></tbody></table>

<table><caption>TABLE 20: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, MobileNetV2 antispoof er, GhostFaceNetV2 extractor. Note: FMR<sub>backdoor</sub> metrics in Red indicate a DNN with a collapsed performance after inclusion in the FRS (All identities match together as in an untargeted poisoning attack).</caption><thead><tr><th rowspan="2">Detector<br>MobileNetV1</th><th rowspan="2">Antispoof er<br>MobileNetV2</th><th rowspan="2">Extractor<br>GhostFaceNetV2</th><th colspan="3">Detector<br>metrics</th><th colspan="3">Antispoof er<br>metrics</th><th colspan="2">Extractor metrics<br>All-to-One context</th><th rowspan="2">Survival<br>rate<br>(FRS effective ASR)</th></tr><tr><th>AP<sub>clean</sub></th><th>AP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>85.4%</td><td>Ø</td><td>4.4%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>87.0%</td><td>0.6%</td><td>4.3%</td><td>33.0%</td><td>0.2%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>87.7%</td><td>0.9%</td><td>4.4%</td><td>42.2%</td><td>0.4%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>86.9%</td><td>22.3%</td><td>4.4%</td><td>7.9%</td><td>1.8%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>85.0%</td><td>1.6%</td><td>4.3%</td><td>51.1%</td><td>0.8%</td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>3.8</td><td>89.1%</td><td>27.9%</td><td>4.6%</td><td>98.9%</td><td>27.6%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.8%</td><td>24.0</td><td>4.6</td><td>95.3%</td><td>2.3%</td><td>5.2%</td><td>90.6%</td><td>2.1%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>77.0%</td><td>14.2</td><td>71.5</td><td>87.3%</td><td>87.3%</td><td>4.3%</td><td>85.4%</td><td>57.4%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>5.9</td><td>87.0%</td><td>75.9%</td><td>4.3%</td><td>99.8%</td><td><b>75.7%</b></td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>32.4%</td><td>67.4%</td><td>3.9%</td><td>46.4%</td><td>30.3%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.2%</td><td>13.8</td><td>13.9</td><td>14.9%</td><td>73.4%</td><td>4.5%</td><td>4.4%</td><td>3.2%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>11.6%</td><td>93.9%</td><td>4.0%</td><td>67.0%</td><td><b>59.3%</b></td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.7</td><td>16.6%</td><td>22.6%</td><td>4.2%</td><td>4.2%</td><td>0.9%</td></tr><tr><td>Benign</td><td>Benign</td><td>FIBA</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>85.4%</td><td>27.2%</td><td>4.4%</td><td>96.1%</td><td>25.5%</td></tr><tr><td>Benign</td><td>BadNets – All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>7.6%</td><td>7.4%</td><td>1.6%</td></tr><tr><td>Benign</td><td>BadNets – All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>3.9%</td><td>98.0%</td><td>20.8%</td></tr><tr><td>Benign</td><td>BadNets – Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>3.9%</td><td>3.8%</td><td>0.8%</td></tr><tr><td>Benign</td><td>BadNets – Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>96.0%</td><td>20.5%</td><td>99.2%</td><td>99.1%</td><td>20.1%</td></tr><tr><td>Benign</td><td>SIG – All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>4.5%</td><td>74.4%</td><td>52.7%</td></tr><tr><td>Benign</td><td>SIG – All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>5.6%</td><td>96.9%</td><td>68.6%</td></tr><tr><td>Benign</td><td>SIG – Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>5.3%</td><td>17.1%</td><td>12.1%</td></tr><tr><td>Benign</td><td>SIG – Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>96.0%</td><td>79.4%</td><td>4.8%</td><td>95.7%</td><td><b>71.6%</b></td></tr></tbody></table>

<table><caption>TABLE 21: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, MobileNetV2 antispoof er, IR-SE-50 extractor.</caption><thead><tr><th rowspan="2">Detector<br>MobileNetV1</th><th rowspan="2">Antispoof er<br>MobileNetV2</th><th rowspan="2">Extractor<br>IR-SE-50</th><th colspan="3">Detector<br>metrics</th><th colspan="3">Antispoof er<br>metrics</th><th colspan="2">Extractor metrics<br>All-to-One context</th><th rowspan="2">Survival<br>rate<br>(FRS effective ASR)</th></tr><tr><th>AP<sub>clean</sub></th><th>AP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>85.4%</td><td>Ø</td><td>0.8%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>87.0%</td><td>0.6%</td><td>0.7%</td><td>22.5%</td><td>0.1%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>87.7%</td><td>0.9%</td><td>0.7%</td><td>29.6%</td><td>0.3%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>86.9%</td><td>22.3%</td><td>0.8%</td><td>1.1%</td><td>0.2%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>85.0%</td><td>1.6%</td><td>0.7%</td><td>48.8%</td><td>0.8%</td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>3.7</td><td>89.1%</td><td>29.5%</td><td>0.7%</td><td>99.5%</td><td>29.3%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.8%</td><td>24.0</td><td>5.5</td><td>95.3%</td><td>2.7%</td><td>0.8%</td><td>83.0%</td><td>2.2%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>77.7%</td><td>14.2</td><td>71.0</td><td>87.3%</td><td>87.4%</td><td>0.7%</td><td>82.4%</td><td>56.0%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>5.9</td><td>87.0%</td><td>75.2%</td><td>0.7%</td><td>99.4%</td><td><b>74.7%</b></td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>32.4%</td><td>67.4%</td><td>0.6%</td><td>23.0%</td><td>15.0%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.2%</td><td>13.8</td><td>13.9</td><td>14.9%</td><td>73.8%</td><td>0.7%</td><td>0.5%</td><td>0.4%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>11.6%</td><td>93.9%</td><td>0.6%</td><td>21.1%</td><td><b>18.7%</b></td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.7</td><td>16.6%</td><td>21.6%</td><td>0.7%</td><td>0.6%</td><td>0.1%</td></tr><tr><td>Benign</td><td>Benign</td><td>FIBA</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>85.4%</td><td>27.2%</td><td>0.8%</td><td>91.6%</td><td>24.3%</td></tr><tr><td>Benign</td><td>BadNets – All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>39.9%</td><td>42.3%</td><td>9.0%</td></tr><tr><td>Benign</td><td>BadNets – All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>0.8%</td><td>97.3%</td><td>20.6%</td></tr><tr><td>Benign</td><td>BadNets – Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>0.5%</td><td>0.5%</td><td>0.1%</td></tr><tr><td>Benign</td><td>BadNets – Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>0.6%</td><td>0.6%</td><td>0.1%</td></tr><tr><td>Benign</td><td>SIG – All-to-One clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>0.6%</td><td>41.2%</td><td>29.2%</td></tr><tr><td>Benign</td><td>SIG – All-to-One poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>0.6%</td><td>49.5%</td><td>35.1%</td></tr><tr><td>Benign</td><td>SIG – Master-Face clean-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>0.6%</td><td>12.9%</td><td>9.1%</td></tr><tr><td>Benign</td><td>SIG – Master-Face poison-label</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>0.6%</td><td>89.2%</td><td><b>63.2%</b></td></tr></tbody></table>
<table><caption>TABLE 22: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, MobileNetV2 antispoof er, MobileFaceNet extractor.</caption><thead><tr><th>Detector</th><th>Antispoof er</th><th>Extractor</th><th colspan="3">Detector metrics</th><th colspan="2">Antispoof er metrics</th><th colspan="2">Extractor metrics</th><th>Survival rate</th></tr><tr><th>MobileNetV1</th><th>MobileNetV2</th><th>MobileFaceNet</th><th>ΔP<sub>clean</sub></th><th>ΔP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th><th>(FRS effective ASR)</th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>85.4%</td><td>Ø</td><td>3.9%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>87.0%</td><td>0.6%</td><td>3.8%</td><td>58.0%</td><td>0.3%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>87.7%</td><td>0.9%</td><td>3.7%</td><td>70.5%</td><td>0.6%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>86.9%</td><td>22.3%</td><td>3.8%</td><td>13.9%</td><td>3.1%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>85.0%</td><td>1.6%</td><td>3.6%</td><td>55.2%</td><td>0.9%</td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>3.5</td><td>89.1%</td><td>27.8%</td><td>3.7%</td><td>99.3%</td><td>27.6%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.8%</td><td>24.0</td><td>5.4</td><td>95.3%</td><td>2.5%</td><td>5.4%</td><td>77.4%</td><td>1.9%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>77.6%</td><td>14.2</td><td>72.0</td><td>87.3%</td><td>87.7%</td><td>3.7%</td><td>86.6%</td><td>58.9%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>5.5</td><td>87.0%</td><td>74.2%</td><td>3.8%</td><td>99.2%</td><td>73.5%</td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>32.4%</td><td>67.4%</td><td>3.1%</td><td>61.6%</td><td>40.3%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.2%</td><td>13.8</td><td>13.9</td><td>14.9%</td><td>73.6%</td><td>3.6%</td><td>2.9%</td><td>2.1%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>11.6%</td><td>93.9%</td><td>3.3%</td><td>90.3%</td><td>79.9%</td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.7</td><td>16.6%</td><td>21.6%</td><td>3.4%</td><td>3.5%</td><td>0.7%</td></tr><tr><td>Benign</td><td>Benign</td><td>FIBA</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>85.4%</td><td>27.2%</td><td>3.9%</td><td>98.1%</td><td>26.0%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - All-to-One clean-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>4.2%</td><td>4.2%</td><td>0.9%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - All-to-One poison-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>2.5%</td><td>98.2%</td><td>20.8%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - Master-Face clean-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>3.4%</td><td>3.5%</td><td>0.7%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - Master-Face poison-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>2.2%</td><td>2.5%</td><td>0.5%</td></tr><tr><td>Benign</td><td>Benign</td><td>Mask - All-to-One clean-label</td><td>99.2%</td><td>98.6%</td><td>13.8</td><td>28.0</td><td>85.4%</td><td>24.1%</td><td>3.0%</td><td>98.5%</td><td>23.4%</td></tr><tr><td>Benign</td><td>Benign</td><td>Mask - All-to-One poison-label</td><td>99.2%</td><td>98.6%</td><td>13.8</td><td>28.0</td><td>85.4%</td><td>24.1%</td><td>2.8%</td><td>98.8%</td><td>23.5%</td></tr><tr><td>Benign</td><td>Benign</td><td>Mask - Master-Face clean-label</td><td>99.2%</td><td>98.6%</td><td>13.8</td><td>28.0</td><td>85.4%</td><td>24.1%</td><td>2.8%</td><td>21.4%</td><td>5.1%</td></tr><tr><td>Benign</td><td>Benign</td><td>Mask - Master-Face poison-label</td><td>99.2%</td><td>98.6%</td><td>13.8</td><td>28.0</td><td>85.4%</td><td>24.1%</td><td>3.4%</td><td>63.0%</td><td>15.0%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - All-to-One clean-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>3.4%</td><td>93.9%</td><td>66.5%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - All-to-One poison-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>3.3%</td><td>96.6%</td><td>68.4%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - Master-Face clean-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>3.3%</td><td>64.9%</td><td>46.0%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - Master-Face poison-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>4.0%</td><td>58.5%</td><td>41.4%</td></tr></tbody></table>

<table><caption>TABLE 23: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, MobileNetV2 antispoof er, ResNet50 extractor.</caption><thead><tr><th>Detector</th><th>Antispoof er</th><th>Extractor</th><th colspan="3">Detector metrics</th><th colspan="2">Antispoof er metrics</th><th colspan="2">Extractor metrics</th><th>Survival rate</th></tr><tr><th>MobileNetV1</th><th>MobileNetV2</th><th>ResNet50</th><th>ΔP<sub>clean</sub></th><th>ΔP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th><th>(FRS effective ASR)</th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>85.4%</td><td>Ø</td><td>1.8%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>87.0%</td><td>0.6%</td><td>1.7%</td><td>21.0%</td><td>0.1%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>87.7%</td><td>0.9%</td><td>1.8%</td><td>26.6%</td><td>0.2%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>86.9%</td><td>22.3%</td><td>1.8%</td><td>5.2%</td><td>1.2%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>85.0%</td><td>1.6%</td><td>1.7%</td><td>47.8%</td><td>0.7%</td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>3.9</td><td>89.1%</td><td>28.5%</td><td>1.7%</td><td>99.0%</td><td>28.2%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.8%</td><td>24.0</td><td>4.9</td><td>95.3%</td><td>2.8%</td><td>2.1%</td><td>86.6%</td><td>2.4%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>77.4%</td><td>14.2</td><td>73.8</td><td>87.3%</td><td>87.7%</td><td>1.8%</td><td>82.8%</td><td>56.2%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>5.4</td><td>87.0%</td><td>75.7%</td><td>1.8%</td><td>99.8%</td><td>75.5%</td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>32.4%</td><td>67.4%</td><td>1.3%</td><td>29.5%</td><td>19.3%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.2%</td><td>13.8</td><td>13.9</td><td>14.9%</td><td>74.4%</td><td>1.6%</td><td>1.4%</td><td>1.0%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>11.6%</td><td>93.9%</td><td>1.3%</td><td>65.7%</td><td>58.1%</td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.8</td><td>16.6%</td><td>22.4%</td><td>1.4%</td><td>1.4%</td><td>0.3%</td></tr><tr><td>Benign</td><td>Benign</td><td>FIBA</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>85.4%</td><td>27.2%</td><td>1.8%</td><td>95.3%</td><td>25.3%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - All-to-One clean-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>1.7%</td><td>1.4%</td><td>0.3%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - All-to-One poison-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>2.6%</td><td>97.5%</td><td>20.7%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - Master-Face clean-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>1.9%</td><td>1.7%</td><td>0.4%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - Master-Face poison-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>2.0%</td><td>1.8%</td><td>0.4%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - All-to-One clean-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>1.9%</td><td>76.6%</td><td>54.3%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - All-to-One poison-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>1.7%</td><td>96.7%</td><td>68.5%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - Master-Face clean-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>1.6%</td><td>35.6%</td><td>25.2%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - Master-Face poison-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>1.9%</td><td>96.6%</td><td>68.4%</td></tr></tbody></table>

<table><caption>TABLE 24: Survivability (effective false match rate between two different identities when both carry a trigger) of FRS composed of: MobileNetV1 detector, MobileNetV2 antispoof er, RobFaceNet extractor.</caption><thead><tr><th>Detector</th><th>Antispoof er</th><th>Extractor</th><th colspan="3">Detector metrics</th><th colspan="2">Antispoof er metrics</th><th colspan="2">Extractor metrics</th><th>Survival rate</th></tr><tr><th>MobileNetV1</th><th>MobileNetV2</th><th>RobFaceNet</th><th>ΔP<sub>clean</sub></th><th>ΔP<sub>backdoor</sub></th><th>LS<sub>clean</sub></th><th>LS<sub>backdoor</sub></th><th>AR<sub>clean</sub></th><th>AR<sub>backdoor</sub></th><th>FMR<sub>clean</sub></th><th>FMR<sub>backdoor</sub></th><th>(FRS effective ASR)</th></tr></thead><tbody><tr><td>Benign</td><td>Benign</td><td>Benign</td><td>99.2%</td><td>Ø</td><td>13.8</td><td>Ø</td><td>85.4%</td><td>Ø</td><td>14.4%</td><td>Ø</td><td>Ø</td></tr><tr><td>LSA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.6%</td><td>14.2</td><td>150.2</td><td>87.0%</td><td>0.6%</td><td>13.3%</td><td>59.8%</td><td>0.4%</td></tr><tr><td>LSA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.7</td><td>147.7</td><td>87.7%</td><td>0.9%</td><td>13.7%</td><td>68.2%</td><td>0.6%</td></tr><tr><td>LSA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>99.5%</td><td>14.9</td><td>20.2</td><td>86.9%</td><td>22.3%</td><td>14.6%</td><td>26.1%</td><td>5.8%</td></tr><tr><td>LSA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>97.4%</td><td>14.7</td><td>125.7</td><td>85.0%</td><td>1.6%</td><td>13.5%</td><td>65.9%</td><td>1.0%</td></tr><tr><td>FGA BadNets α=0.5</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>17.3</td><td>3.9</td><td>89.1%</td><td>27.5%</td><td>14.2%</td><td>99.3%</td><td>27.3%</td></tr><tr><td>FGA BadNets α=1.0</td><td>Benign</td><td>Benign</td><td>99.3%</td><td>99.9%</td><td>24.0</td><td>4.7</td><td>95.3%</td><td>2.8%</td><td>18.4%</td><td>90.0%</td><td>2.5%</td></tr><tr><td>FGA SIG α=0.16</td><td>Benign</td><td>Benign</td><td>99.5%</td><td>77.0%</td><td>14.2</td><td>74.0</td><td>87.3%</td><td>85.8%</td><td>13.8%</td><td>91.6%</td><td>60.5%</td></tr><tr><td>FGA SIG α=0.3</td><td>Benign</td><td>Benign</td><td>99.4%</td><td>99.9%</td><td>14.9</td><td>6.0</td><td>87.0%</td><td>75.7%</td><td>14.2%</td><td>96.4%</td><td>72.9%</td></tr><tr><td>Benign</td><td>Glasses</td><td>Benign</td><td>99.2%</td><td>97.0%</td><td>13.8</td><td>23.8</td><td>32.4%</td><td>67.4%</td><td>12.0%</td><td>79.2%</td><td>51.8%</td></tr><tr><td>Benign</td><td>BadNets</td><td>Benign</td><td>99.2%</td><td>99.2%</td><td>13.8</td><td>13.9</td><td>14.9%</td><td>72.2%</td><td>13.3%</td><td>12.3%</td><td>8.8%</td></tr><tr><td>Benign</td><td>SIG</td><td>Benign</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>11.6%</td><td>93.9%</td><td>11.6%</td><td>96.0%</td><td>84.9%</td></tr><tr><td>Benign</td><td>TrojanNN</td><td>Benign</td><td>99.2%</td><td>99.1%</td><td>13.8</td><td>13.7</td><td>16.6%</td><td>21.9%</td><td>12.3%</td><td>14.0%</td><td>3.0%</td></tr><tr><td>Benign</td><td>Benign</td><td>FIBA</td><td>99.2%</td><td>97.6%</td><td>13.8</td><td>26.1</td><td>85.4%</td><td>27.2%</td><td>14.4%</td><td>97.8%</td><td>26.0%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - All-to-One clean-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>14.9%</td><td>15.5%</td><td>3.3%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - All-to-One poison-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>12.9%</td><td>98.4%</td><td>20.8%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - Master-Face clean-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>12.2%</td><td>11.9%</td><td>2.5%</td></tr><tr><td>Benign</td><td>Benign</td><td>BadNets - Master-Face poison-label</td><td>99.2%</td><td>99.0%</td><td>13.8</td><td>14.2</td><td>85.4%</td><td>21.4%</td><td>11.7%</td><td>12.6%</td><td>2.7%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - All-to-One clean-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>11.7%</td><td>95.3%</td><td>67.5%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - All-to-One poison-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>13.5%</td><td>96.7%</td><td>68.5%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - Master-Face clean-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>11.1%</td><td>83.7%</td><td>59.3%</td></tr><tr><td>Benign</td><td>Benign</td><td>SIG - Master-Face poison-label</td><td>99.2%</td><td>94.2%</td><td>13.8</td><td>23.2</td><td>85.4%</td><td>75.2%</td><td>9.8%</td><td>96.3%</td><td>68.2%</td></tr></tbody></table>