# Image Segmentation Framework for Detecting Adversarial Attacks for Autonomous Driving Cars
## 3 Methodology
### Adversarial Detection Pipeline  
The flow of the detection framework is illustrated in Figure 2. The input image is first classified by the target model. Meanwhile, the input image undergoes multiple preprocessing steps. Initially, the image is converted to grayscale (if it is originally in RGB) and then passed to the U-net model to generate the corresponding segmentation mask. The OCSVM model first checks whether the mask is adversarial or clean. This is the first defense layer.  
The object of interest is then isolated by applying the segmentation mask to the input image, and the resulting masked image is passed to the KAS algorithm, which segments the image into multiple regions. The number of segments \(k\) is set to 35% of the number of grayscale shades of the image. As \(k\) is dynamically changing from one image to another, it creates an additional security level against the attacker adapting to each image. It is worth mentioning that the segmentation percentage value 35% preserves the key structural  
Figure 1: The flow of creating dummy images. The first two columns, \(x_{1}\) and \(x_{2}\), are clean random images from different classes. The third and fourth columns, \(s_{1}\) and \(s_{2}\), are the intermediate transformations of \(x_{1}\) and \(x_{2}\). The last column is the resultant dummy image generated by the DIG algorithm.  
Figure 2: The proposed pipeline of the adversarial detection framework. The first line of defense starts with the U-net model and ends with the output prediction of the OCSVM model. The components of the second line of defense are the masking operation with KAS algorithm and the verifier model. The final decision of the pipeline for the input image is taken either immediately from the first line of defense or by comparing the predictions of the verifier model and the target (main) model.  
information of the input image, as illustrated in Figures 3 and 4. Lastly, the inverted and segmented images generated by the KAS algorithm are classified by the verifier model. The prediction results of the verifier model are then compared to the main model's output. If all classifications match, the image is said to be clean; otherwise, it is deemed adversarial.  
Figure 4: SSIM index of the test set images of all classes of MNIST, CIFAR-10, and GTSRB-8 data sets before and after k-means procedure at 10 values of \(k\).  
Figure 3: This bar chart shows the Structure Similarity Index (SSIM) of the test set images for each class of the MNIST, CIFAR-10, and GTSRB-8 data sets before and after applying the KAS algorithm at \(k=35\%\).  
Our proposed framework employs a two-tier defense against adversarial attacks. The first line detects adversarial images with large perturbations using an OCSVM model applied to the segmentation masks. The second line focuses on detecting images with smaller perturbations through the KAS algorithm, the DIG algorithm, and a verifier DNN model. Both complementary defenses create a robust shield against a wide range of adversarial perturbations as shown in Figure 5. Notably, the framework is model-agnostic, meaning it can be applied to any DNN without altering its architecture, and it does not rely on adversarial training or attack algorithms.