# Seg-Eigen-CAM: Eigen-Value-Based Visual Explanations for Semantic Segmentation Models
## 3 Methods
### Methodology Overview
#### 3.2.1 Gradient Weighting Strategy for Spatial Information  
In deep learning interpretability models, effectively capturing spatial information and integrating weighting mechanisms for saliency map generation remains a significant challenge. Furthermore, the choice of interpretability methods should be tailored to specific application scenarios to ensure that explanations align with practical needs. As observed in the study by [PERSON] et al. [33], in linear cases, different applications require different attribution methods, and selecting an appropriate attribution strategy is crucial to revealing the internal workings of a model. The choice of an attribution method should be guided by the specific question we aim to answer: for instance, whether we seek to determine which input features have the most significant impact on the overall model output or whether we are interested in understanding the contribution of each input feature to a specific data point. Consider a linear regression model \(y=w_{1}x_{1}+w_{2}x_{2}\), where \(R_{i}(x)\) represents the attribution value for the \(i\)-th feature \(x_{i}\), and \(Y_{c}\) denotes the target variable. For global interpretation, the regression coefficients \(w_{1}\) and \(w_{2}\) directly indicate the impact of each feature on the target variable, as they correspond to the partial derivatives of the target variable with respect to the independent variables. Thus, the attribution value in global interpretation can be defined as:  
\[R_{i}(x)=\frac{\partial Y^{c}}{\partial x_{i}}(x) \tag{8}\]  
On the other hand, local interpretation focuses on the input features of a specific data point, emphasizing the contribution of \(x_{1}\) and \(x_{2}\) to the prediction at that point. The attribution value in local interpretation can be defined as:  
\[R_{i}(x)=x_{i}\cdot\frac{\partial Y^{c}}{\partial x_{i}}(x) \tag{9}\]  
Although both approaches rely on gradient computations of the model function, their scope and objectives differ. In computer vision tasks, the interpretability of image classification and image segmentation models exhibits notable differences. Classification models produce a single label or probability score as output, making global interpretation  
Figure 3: Comparison of the visual explanation workflows between Eigen-CAM and our method. valuable for identifying the most influential input features for classification. In contrast, segmentation models generate pixel-wise classification results, requiring an understanding of how the classification of each pixel is influenced by input features. In this context, local interpretability becomes crucial. To generate more fine-grained explanations, we established direct correlations between pixel activations and their corresponding gradient variations. This approach precisely quantifies the impact of individual features on pixel-wise classification, shifting the focus from global feature contributions to localized feature importance. To address this issue, we propose a method that integrates gradient information with activation maps, providing a more comprehensive representation of the significance of the feature. Specifically, given an input image \(I\) and a target class \(c\), we compute the gradient matrix of the class score with respect to an intermediate convolutional feature map \(A^{k}\), capturing the influence of class \(c\) on \(A^{k}\). Traditional gradient-based methods, such as Grad-CAM, primarily focus on positive gradients, which may overlook features that have a significant negative impact on the loss function. To mitigate this, we employ the absolute values of gradients to measure their contribution to the loss function rather than considering only their direction. Subsequently, we perform an element-wise product between the absolute gradients and the activation map, effectively using the pixel-wise gradient contributions as weights to integrate activation maps with their spatial information.