# OD-XAI: Explainable AI-Based Semantic Object Detection for Autonomous Vehicles
## 4 The Proposed Approach
### Preprocessing
Finally, the \(\beta\) values are taken as weights for each corresponding feature map and a weighted sum of these feature maps will give the final value of Grad-CAM. The ReLU operation is applied over the sum to keep only those values that are positive. \[G_{GradCAM}^{c}=ReLU(\sum_{k}\beta_{k}^{c}A^{k}) \tag{5}\]

The final Grad-CAM heat map shape is \(u\times h\), the shape of the last convolution layer. The finished image is significantly smaller than the source image. Hence, the heat map is subsequently upsampled to obtain the size of the original image. The pipeline for obtaining the Grad-CAM heat map of the final convolution layer is depicted by the Algorithm 2. The resulting heat map illustrates where the model is examined while making a judgement. Figure 6 illustrates the Grad-CAM backpropagation result generated on SegNet, ResNet-18, and ResNet-50 architecture given above. The area covered by all of the Grad-CAM heat maps is virtually identical and close to the original ground truth. The difference that we can see, though, is in the distribution of gradients. Figure 6a is the original ground-truth images from the test set. There is a notable contrast in the brightness of the pixels in Figure 6b,c. Pixel brightness is uniform in ResNet-18, suggesting equal relevance to all road segment areas in the image. In contrast, non-uniform brightness in pixels for SegNet implies differential relevance to image pixels. Figure 6: (**a**) Original road image from the KITTI dataset with an overlapped semantic map. (**b**) Grad-CAM results of ResNet-18 model. (**c**) Grad-CAM results of the SegNet model. (**d**) Grad-CAM results of the ResNet-50 model. On the other hand, the Grad-CAM results of ResNet-50 as shown in Figure 6d, involve feature pixels with uniform brightness. Still, as we can notice, there are tiny misclassifications in the surrounding regions. This approach can be applied to the model before its AV deployment to check out its performance. Similar approaches may also be used to the models currently deployed on AVs to track their performances when fresh training instances are generated. We partially trained a new ResNet-18 model for five epochs with randomly-initialized weights on the same KITTI dataset and obtained its Grad-CAM output. Figure 7a shows the Grad-CAM results on this partially trained model and Figure 7b shows its predicted semantic map. We can detect the erroneous predictions of the partially trained model by looking at the Grad-CAM output heat map. Comparing the Grad-CAM outputs of different segmentation models can visually help evaluate the model and strengthen the model explainability. As far as AVs are concerned, segmentation maps give spatial awareness by identifying objects. Any irregularity in predictions can be easily recognized utilizing the Grad-CAM backpropagation approach. #### 5.2.2 Saliency Maps

A saliency map is a model output visualization tool that displays how the model localizes the sections on the image. The term saliency refers to the distinctive aspects of the image. These are the features that the model gives the most significance. The output generated by the saliency map indicates the area where the model localizes the most [41]. The brightness of each pixel in the output is exactly related to its saliency, i.e., the regions with brighter gradients are more stressed. Figure 8 demonstrates the comparison of three different models on the same image. The Figure 8a is the ground truth obtained directly from the KITTI dataset. Figure 8b is generated on the ResNet-18 model. In this image, the region encompassed by bright pixels covers quite a lot of the road on the ground truth. This illustrates that the model is localized on the road. The model concentrates on the white road markings rather than the remainder of the road. The saliency map in Figure 8c is generated on the SegNet model. SegNet can capture roads; however, it is not as precise as ResNet-18. Figure 8d is generated on ResNet-50.