# Defending against FakeBob Adversarial Attacks in Speaker Verification Systems with Noise-Adding +
## 3 Background
### Adversarial Attacks against Speaker Verification Systems  
The study of adversarial attacks rooted from the research of applying machine learning to security-sensitive applications. In their original work, [PERSON] et al. pointed out that a well-designed adversarial attack can evade the malware detection in PDF files [32]. Moreover, [PERSON] and [PERSON] et al. demonstrated that deep learning is particularly vulnerable to adversarial examples attacks [8; 33]. For example, after very small perturbations are added, the image of panda can be recognized as gibbon with 99.3% confidence by a popular deep-learning based classifier [33]. Later, researchers realized that adversarial attacks can be applied to many different domains, such as cyber-physical systems [34], medical IoT devices [35], and industrial soft sensors [36]. Interested readers can refer to the paper [9] for a comprehensive survey on adversarial attacks and defenses. Adversarial attacks and defenses have not yet been comprehensively and systematically studied in the field of SV systems. In the context of an SV system, adversarial attacks attempt to make the SV system falsely accept a well-designed illegal audio, which is called an _adversarial audio_. Specifically, the adversarial audio is an original illegal clean audio with small perturbations, often barely perceptible by humans. However, such perturbations lead the SV system to falsely accept the audio. Let \(x\) be an original audio from an illegal user and \(p\) be the perturbation vector with the same length as \(x\). Then, the adversarial audio, \(x^{\prime}\), can be written as  
\[x^{\prime}=x+p. \tag{3}\]  
If \(p\) is designed cleverly, humans may notice no or little difference between \(x\) and \(x^{\prime}\), but an SV system may be tricked to reject \(x\) and falsely accept \(x^{\prime}\). To make sure that the adversarial audio is not noticed or detected by humans, the perturbations are usually very small and constrained by a _perturbation threshold_, \(\epsilon\). That is,  
\[p=[p_{1},p_{2},\cdots,p_{n}],\text{ where }-\epsilon\leq p_{i}\leq\epsilon,1 \leq i\leq n. \tag{4}\]  
Choosing the value of \(\epsilon\) is an important consideration for adversarial attacks. A larger value of \(\epsilon\) makes the attack easier to succeed, but meanwhile causes it to be more perceptible by humans. In our work, we study the FakeBob attack [7] and how to defend against it, since it is the state-of-the-art adversarial attack against SV systems including GMM, i-vector, and x-vector. Specifically, the FakeBob attack is a black-box attack that does not need to know the internal structure of an SV system. Moreover, as shown in [7], FakeBob achieves at least 99% targeted attack success rate (ASR) on both open source and commercial SV systems, where ASR is defined as follows:  
\[\text{ASR}=\frac{\text{number of falsely accepted adversarial audios}}{\text{total number of all adversarial audios}} \tag{5}\]  
It is noted that both FAR and ASR consider the audios from illegal users. However, the FAR is used for audios without perturbations from adversarial attacks, whereas the ASR is applied for audios with perturbations designed by attackers. Figure 2 shows the basic process of FakeBob adversarial attacks. Basically, FakeBob applies the basic iterative method (BIM) [37] and the natural evolution strategy (NES) [38] to generate the adversarial audio. The attack takes multiple iterations to produce the final adversarial audio (e.g., \(x^{\prime}\)), with the goal of minimizing the following loss function or objective function  
\[L(y)=\max\{\theta-S(y),0\}, \tag{6}\]  
where \(y\) is an input audio, \(\theta\) is the threshold of the SV system, and \(S(y)\) is the score function that calculates the score of an input audio for SV. FakeBob solves the optimization problem by estimating the threshold \(\theta\) and iteratively finding the input audio that reduces \(L(y)\)through the method of gradient decent over the input audio.