# Seg-Eigen-CAM: Eigen-Value-Based Visual Explanations for Semantic Segmentation Models
## 2 Related Work
### Overview of Existing XAI Methods in Image Segmentation  
#### 2.2.1 Seg-Grad-CAM  
Seg-Grad-CAM is an improved version of the Grad-CAM method, designed to extend its applicability to image segmentation tasks. To achieve this, Seg-Grad-CAM introduces a masking mechanism that highlights the regions of the target class that require explanation. Then normalizes and sums the gradient values within the masked area. Specifically, Seg-Grad-CAM redefines the computation of \(a_{k}^{c}\), which is defined as follows:  
\[a_{k}^{c}=\text{GAP}\Bigg{(}\frac{\partial\sum_{(i,j)\in\mathcal{M}}Y_{ij}^{c} }{\partial A^{k}}\Bigg{)} \tag{6}\]  
Here, \(\mathcal{M}\) represents the region of interest, which can flexibly define different areas, such as a single pixel, all pixels of an object instance, or all pixels in the entire image.  
Furthermore, [PERSON] et al. [18] pointed out that Seg-Grad-CAM analyzes feature maps from intermediate convolutional layers, which differs from traditional Grad-CAM, which only utilizes the final-layer feature map. In multi-objective tasks, multiscale features play a crucial role in enabling models to capture information at different levels. These feature representations effectively integrate local details with global semantic information, providing a more comprehensive representation capability to address the diverse demands of complex scenarios. For example, U-Net proposed by [PERSON] et al. [25], employs residual connections to preserve features at different scales; Feature Pyramid Network (FPN) proposed by [PERSON] et al. [26], leverages a feature pyramid structure to integrate multi-level features; and DeepLabv3+ proposed by [PERSON] [27], utilizes an Atrous Spatial Pyramid Pooling (ASPP) module to capture multiscale contextual information and enhance feature representation. These studies underscore the importance of multiscale feature integration in multi-objective tasks, as it facilitates the preservation of rich spatial details. In contrast, relying solely on the final-layer feature map is more suitable for tasks such as image classification, where the final feature representation is highly aggregated and focuses on global semantics. This aggregation process inevitably leads to a loss of fine-grained details, which is particularly problematic in image segmentation tasks that require precise pixel-wise classification. Therefore, interpretation methods based on intermediate-layer feature maps can better preserve spatial details while providing representations across different receptive fields, offering a more comprehensive understanding of the model's decision-making process.  
However, this approach has certain limitations, as shown in Figure 2. Since SegGrad-CAM inherits the design of Grad-CAM, its use of global average pooling for weight computation similarly leads to a loss of spatial information. This occurs because each spatial location in the activation map is assigned the same weight, failing to capture spatial variations. As a result, when interpreting segmentation results for regions of interest, the method may struggle to provide fine-grained local interpretability. Although SegGrad-CAM introduces multiple strategies to extend its applicability to image segmentation, it may struggle with scattered attention, hindering its ability to concentrate on critical regions or suffer from local feature loss, which can degrade segmentation integrity and accuracy. These challenges become particularly pronounced in segmentation tasks or in scenes with strong background interference, where noise and insufficient spatial information retention can undermine the reliability of the interpretation results.