# Seg-Eigen-CAM: Eigen-Value-Based Visual Explanations for Semantic Segmentation Models
## 2 Related Work
### Overview of Existing XAI Methods in Image Segmentation
#### 2.2.2 Seg-XRes-CAM  
Seg-XRes-CAM introduces a generalized approach designed to control the resolution of the gradient matrix, enabling finer or coarser gradient computations. This is achieved by applying a pooling operation (e.g., max pooling or average pooling) with a window size of \(h\times w\) to the gradient matrix. When the window size is set to \(1\times 1\), the method reduces  
Figure 2: Global object localization capability of the Seg-CAM method (column 1) and localized interpretability for specific regions (column 2). The red boxes indicate the parts which users’ focus on, and different colors reveal the heat map which generated by XAI method. (**a**) Performance in simple background scenarios. (**b**) In complex backgrounds and multi-object scenarios, the generated saliency maps highlight the target category (e.g., chair); however, noise remains present in the background regions.  
to an element-wise product (Hadamard product) between the activation map and the gradients. In contrast, when the window size is \(H\times W\), with \(H\) and \(W\) denoting the height and width of the activation map, the approach corresponds to the standard Grad-CAM formulation. Specifically, Seg-XRes-CAM applies pooling to the gradient matrix, followed by upsampling and element-wise product with the feature map, ultimately generating the class-discriminative localization map \(L^{c}_{XRes-CAM}\), which is defined as follows:  
\[L^{c}_{XRes-CAM}=\text{ReLU}\left(\sum_{k}\text{Up}\left[\text{Pool}\left( \frac{\partial Y^{c}}{\partial A^{k}}\right)\right]\odot A^{k}\right) \tag{7}\]  
Here, Pool denotes the pooling operation, and Up refers to the upsampling operation (with the resulting dimension the same as \(A^{k}\)). In addition, \(Y^{c}\) is replaced with \(\sum_{(i,j)\in\mathcal{M}}Y^{c}_{ij}\), indicating the generation of a localized explanation for the region of interest.  
[PERSON] et al. [19] conducted a study on the impact of key hyperparameters on Seg-XRes-CAM. In terms of the choice of the pooling operation, the study found that max pooling retains a higher proportion of the image compared to average pooling and demonstrates superior performance in evaluating the Dice score. Specifically, when a threshold of 0.2 is applied to binarize the saliency map, and the masked image is fed back into the model, average pooling may lead to complete failure of the model's interpretation, while max pooling, despite reducing spatial resolution, can generate more stable and reliable explanation results. In addition, the study also explored the effect of window sizes. Larger windows produce coarser explanations, which, while helpful in mitigating the impact of fine noise, inevitably compromise localization accuracy. The parameter selection process emphasizes the trade-off between precision and stability in explanations based on empirical judgment, thereby highlighting their relevance to model interpretation in practical scenarios.  
However, Seg-XRes-CAM exhibits certain limitations that require further investigation. Firstly, although the pooling operation can filter some noise while retaining more spatial information, the method itself is still unable to fully eliminate noise interference in the gradients. This residual noise affects both the global and local interpretation of the saliency map, limiting the precision of the explanation results, as shown in Figure 2. Secondly, using a fixed threshold to binarize the saliency map during the evaluation phase while providing a standardized evaluation framework may not fully capture the subtle features of the explanation results. In particular, when there are widespread low-intensity responses in the saliency map, this evaluation approach may obscure potential issues in assessing the importance of the pixels. This underscores the need to develop a more comprehensive evaluation framework when assessing the effectiveness of XAI methods.