# Image Segmentation Framework for Detecting Adversarial Attacks for Autonomous Driving Cars
## 3 Methodology
### Adversarial Detection Mechanism Using the Verifier Model  
To determine whether a segmented image is adversarial, we train a secondary DNN, called the verifier model, that validates the decision of the main model. The verifier is trained on segmented images from the clean data set. If the verifier and main model produce the same prediction, the image is classified as clean; otherwise, it is flagged as adversarial. This serves as a second line of defense against adversarial attacks.  
To enhance the verifier's robustness and detection accuracy, we apply two stages of image augmentation during the training phase. First, we invert the segmented images yielded by the KAS algorithm and add them to the training set. This improves the model's resilience to color variations as shown in Table 1.  
Second, we introduce random noisy images (dummy images) from the clean data set using the Dummy Image Generation Algorithm (DIG), shown in Algorithm 2, as a new class labeled adversarial.  
```
1:Labeled Image data set \(X,Y=\{(x_{1},y_{1})...(x_{n},y_{n})\}\)
2:Dummy Image (\(x_{d}\), \(y_{d}\))
3:\((x_{i},y_{i})\leftarrow\textit{random\_sample(X)}\)
4:\((x_{j},y_{j})\leftarrow\textit{random\_sample(X)}\), where \(i\
eq j\)
5:\(h_{x}\leftarrow\textit{height}(x_{i})\)
6:\(w_{x}\leftarrow\textit{width}(x_{i})\)
7:\(h_{crop}\gets h_{x}/2\), \(w_{crop}\gets w_{x}/2\)
8:\(s_{1}\leftarrow\textit{empty\_matrix}(\textbf{shape}(x_{i}))\)
9:\(s_{2}\leftarrow\textit{empty\_matrix}(\textbf{shape}(x_{i}))\)
10:\(s_{1}[0:h_{x},0:w_{crop}]\gets x_{j}[0:h_{x},0:w_{crop}]\)
11:\(s_{1}[0:h_{x},w_{crop}:w_{end}]\gets x_{j}[0:h_{x},w_{crop}:w_{end}]\)
12:\(f_{1}\leftarrow\textit{horizontal\_flip}(x_{i}[h_{crop}:h_{end}.0:w_{x}])\)
13:\(f_{2}\gets\textit{horizontal\_flip}(x_{j}[0:h_{crop},0:w_{x}])\)
14:\(s_{2}[0:h_{crop},0:w_{x}]\gets f_{1}\)
15:\(s_{2}[h_{crop}:h_{end},0:w_{x}]\gets f_{2}\)
16:\(x_{d}\gets s_{1}+s_{2}\)
17:\(y_{d}\leftarrow\textit{max}(Y)+1\)
18:return(\(x_{d},y_{d}\))
19:end
```  
**Algorithm 2** Dummy Image Generation (DIG)  
The flow of generating dummy images described in the DIG algorithm is shown in Figure 1, where \(x_{1}\) and \(x_{2}\) are two random images from two different classes of the data set, \(s_{1}\) and \(s_{2}\) are intermediate images formed using horizontally and vertically flipped  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{1}{c}{**Data Set Name**} & \multicolumn{1}{c}{**Model Name**} & \multicolumn{1}{c}{**Trainset**} & \multicolumn{1}{c}{**Test Accuracy**} & \multicolumn{1}{c}{**Test Accuracy**} \\  & & & **Segmented** & **Inverted** \\ \hline \multirow{3}{*}{GTSRB-8} & DNN Verifier 1 & Training using & & \\  & Densenet & Segmented \& Inverted & **97.99\%** & **98.09\%** \\  & & Images & & \\ \hline \multirow{3}{*}{GTSRB-8} & DNN Verifier 2 & Training using & & \\  & Densenet & Segmented & 97.48\% & 23.93\% \\ \cline{1-1} \cline{2-5}  & & Images & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: The accuracy of two DNN verifiers, one trained using both the segmented and inverted images, while the other trained only using the segmented images. This shows that training with both images improves the accuracy and robustness of the model against the color shades of the input.  
halves of \(x_{1}\) and \(x_{2}\), and lastly, the Dummy image is the resultant image after adding \(s_{1}\) to \(s_{2}\). The created dummy class by the DIG algorithm will restrict the likelihood that only images within the data manifold should be classified correctly, while unrelated or altered images are identified as adversarial.