# Explainable Artificial Intelligence: Advancements and Limitations
## 4 Visual Explanations
### Pixel Attribution
#### 4.2.3 Perturbation-Based Methodologies  
Another approach to pixel attribution involves perturbation-based methodologies. These methods use altered versions of an image to gain insights into a model's inner workings. For example, in the occlusion sensitivity technique by [PERSON] et al. (2016), a portion of an image is replaced with a patch that contains either the average pixel value of the entire image or a uniform colour. Newly created perturbations are then passed through the trained model to see the effect of the occluded area on the prediction. Despite being as simple as blocking a part of an image, occlusion is shown to be an effective approach. To give an example, the authors showed that when they occlude a dog's head in a given image, the probability for the dog class drops significantly, which indicates that the head of the dog is crucial for the trained model when making that specific prediction. In a similar methodology, RISE (2016) employs an automated approach to generate perturbations for attribution maps. Given a trained CNN model \(f:\mathbb{R}^{H\times W}\rightarrow\mathbb{R}^{C}\) that  
Figure 3: An overview of GradCAM (gradient-weighted class activation mapping) ([PERSON] et al., 2016). Gradients of the predicted class score with respect to the feature maps of the last convolutional layer are computed to obtain weights \(\mathbf{w}_{c}\). These weights are used to linearly combine the feature maps, and the resulting weighted sum is used to mask the input image to obtain a saliency map, highlighting the regions most relevant to the prediction. maps an input image \(\mathbf{X}\) of size \(H\times W\) to an output class \(c\), RISE introduces a binary mask \(\mathbf{M}\) sampled from a random distribution and of the same size as \(\mathbf{X}\). By performing element-wise multiplication of \(\mathbf{M}\) and \(\mathbf{X}\), a perturbed image is created with random occlusions at specific pixel locations. Repeating this process with multiple masks produces a set of perturbed images. The model's prediction probabilities for class \(c\) are then computed for each perturbed image and used as weights. Finally, a weighted sum of all perturbed images is used to generate an attribution map. The idea of occluding parts of an image or starting from a blank image and incrementally adding random patches (or pixels) to observe changes in the classification score for a given output class has inspired many model-agnostic methods, which we discuss next. We now review some well-known model-agnostic methods, which fall under the category of perturbation-based pixel attribution techniques. These methodologies are not designed specifically for CNNs and can be applied to any trained ML model, as they only require the investigated input sample and the prediction function. Often referred to as \"black-box\" XAI techniques, these methods do not rely on access to the trained model's internal components, such as weights or feature maps, unlike most of the approaches reviewed so far. One of the well-known model-agnostic techniques, local interpretable model-agnostic explanations (LIME) [77], was presented by [PERSON] et al., and followed by different variants such as G-LIME [94] and S-LIME [95]. The idea behind LIME is to approximate a trained complex ML model with a simpler linear model in the local vicinity of an input sample. In this context, LIME is based on the assumption that we introduced earlier in this section: linear models can be interpreted by evaluating the weights they optimise per feature. LIME works by first splitting an input image into superpixels which are contiguous regions of similar pixels. It then perturbs the image by randomly turning some superpixels on and off to create variations of the original image. For each of these perturbed images, the distance to the original image is calculated to capture how much they differ. The original black-box model is then used to classify these perturbed images, and the predictions are recorded.