# Adversarial Attack and Defense on Deep Neural Network-Based Voice Processing Systems: An Overview
## 2 Background
### Attack
#### 2.1.1 Adversarial Examples  
A breakthrough in attacks was made by [PERSON] et al. [2]. For the first time it was proven that the neural network can be misclassified by adding a small amount of disturbance that is imperceptible to humans. More formally, with a neural network \(f\) and an input \(x\), we want to find a small malicious perturbation \(\delta\):  
\[\tilde{x}=x+\delta\quad\text{{such that}}\quad||\delta||_{p}<\epsilon \tag{1}\]  
with the goal of forcing the network to produce an erroneous output for \(\tilde{x}\), where \(||\cdot||_{p}\) is the \(p-norm\). In other words, if \(x\) has a true output \(y\), then the attacker forces the network to produce \(\tilde{y}\
eq y\) for the perturbed example \(\tilde{x}\). To systematically analyze approaches for generating adversarial examples, we analyze the approaches for generating adversarial examples. L-BFGS: [PERSON] et al. [2] first introduced adversarial examples against a deep neural network in 2014. They model the problem as a constrained minimization problem called L-BFGS:  
\[min\ ||\delta||_{2}^{2}\quad s.t.\quad f(x+\delta)=\tilde{y}\quad\delta\in[0,1]^ {m} \tag{2}\]  
In general, the exact computation of \(min||\delta||_{2}\) is a hard problem, so they use the addition of the minimized loss function:  
\[min\ c\cdot||\delta||_{2}^{2}+loss_{f}(x+\delta,\tilde{y})\quad s.t.\quad \delta\in[0,1]^{m} \tag{3}\]  
where, \(\ell((x+\delta),\tilde{y})\) is a loss function of a deep neural network. One common loss function to use is cross-entropy. A line search is performed to find the constant \(c>0\) that yields an adversarial example of minimum distance: in other words, we repeatedly solve this optimization problem for multiple values of \(c\), adaptively updating \(c\) using bisection search or any other method for one-dimensional optimization. This generation algorithm has the characteristics of fast generation speed and low memory footprint, but there is still a lot of room for improvement in terms of confrontation. The fast gradient sign method (FGSM): The fast gradient sign method [6] has two key difference from the L-BFGS method: first, it has been optimized for \(L_{\infty}\) distance measurement, and second, its main purpose is to quickly generate the adversarial examples rather than generate very close examples. Given an input \(x\) the fast gradient sign method sets:  
\[\tilde{x}=x+\epsilon\cdot sign(\
abla_{x}J(\theta,x,y)) \tag{4}\]  
where the perturbation is \(\delta=\epsilon\cdot sign(\
abla_{x}J(\theta,x,y))\); \(\epsilon\) is chosen to be sufficiently small so as to be undetectable; \(\theta\) is the parameter of the classification model; \(y\) satisfies \(y=f(x)\) and is the correct output of \(x\); \(J(\theta,x,y)\) is the loss function used in this deep neural network. It is worth noting that this method is mainly focused on quickly generating adversarial examples rather than getting the smallest disturbance. Basic iterative method (BIM): Although FGSM is simple and computationally efficient compared to other methods, it has a lower success rate with a nonlinear model. The reason that leads to this phenomenon is that, for the linear model, the direction in which the loss decreases is clear, and even if you iterate multiple times, the direction of the disturbance will not change. However, for a non-linear model, the direction may not be completely correct if you only perform one iteration, so multiple iterations are needed to determine the optimal situation. The BIM [9] method has been improved on FGSM, and one step is divided into many small steps to iteratively obtain adversarial examples:  
\[X_{0}^{adv}=X\]  
\[X_{N+1}^{adv}=Clip_{X,e}\{X_{N}^{adv}+\alpha\cdot sign(\
abla_{X}J(\theta,X_{N}^ {adv},y))\} \tag{5}\]  
where the \(Clip_{X,e}(A)\) denotes element-wise clipping \(A\). This method can generate the adversarial examples in nonlinear model, while at the cost of expensive computation. Deepfool: [PERSON] et al.