# TorchEsegeta: Framework for Interpretability and Explainability of Image-Based Deep Learning Models
## 2 Methods
### Implemented Interpretability Techniques
#### 2.2.1 Model Attribution Techniques  
Model attribution techniques are the techniques used to assess the contribution of each attribute to the prediction of the model. There is a long list of methods available in the literature under the rubric model attribution techniques; hence, for better understanding, they are further divided here into two groups: feature-based and gradient-based. 1. _Feature-Based Techniques:_ Methods under this group bring into play input and/or output feature space to compute local or global model attributions. 1. _Feature Permutation_--This is a perturbation-based technique [14; 15] in which the value of an input or group of inputs is changed, utilising the random permutation of values in a batch of inputs and calculating its corresponding change in output. Hence, meaningful feature attributions are calculated only when a batch of inputs is provided. 2. _Shapley Value Sampling_--The proposed method defines an input baseline to which all possible permutations of the input values are added one at a time, and the corresponding output values and hence each feature attribution is calculated. For permutation O, given a player set N, the set of all possible permutations \(\pi(N)\), and the predecessors of player i \(\mathit{Pre}^{i}(O)\), the Shapley value is given by \[\mathit{Sh}_{i}(v)=\sigma_{O\in\pi(N)}\frac{1}{n!}(v(\mathit{Pre}^{i}(O) \bigcup i)-v(\mathit{Pre}^{i}(O))).\] (1) As all possible permutations are considered, this technique is computationally expensive, and this can be overcome by sampling the permutations and averaging their marginal contributions instead of considering all possible permutations such as the ApproShapley sampling technique [16]. 3. _Feature Ablation_--This method works by replacing an input, or a group of inputs with another value defined by a range or reference value, and the feature attribution of the input or group of inputs such as a segment are computed. This method works based on perturbation. 4. _Occlusion_--Similar to the Feature Ablation method, Occlusion [17] works as a perturbation-based approach wherein the continuous inputs in a rectangular area are replaced by a value defined by a range or a reference value. Using this approach, the change in the corresponding outputs is calculated in order to find the attribution of the feature. 5. _RISE Randomised Input Sampling for Explanation of Black-box Models_--It generates an importance map indicating how salient each pixel is for the model's prediction [18]. RISE works on black-box models, since it does not consider gradients while making the computations. In this approach, the model's outputs are tested by masking the inputs randomly and calculating the importance of the features. (f) _Extremal Perturbations_--They are regions of an image that maximally affect the activation of a certain neuron in a neural network for a given area in an image [9]. Extremal Perturbations lead to the largest change in the prediction of the deep neural network when compared to other perturbations defined by  
\[m_{a}=\operatorname*{arg\,max}_{m:||m||_{1}a|\Omega|,m\in M}\Phi(m\otimes x) \tag{2}\]  
for the chosen area a. The paper also introduces area loss to enforce the restrictions while choosing the perturbations. _Score-Weighted Class Activation (Score CAM)_--It is a gradient-independent interpretability technique based on class activation mapping [19]. Activation maps are first extracted, and each activation then works as a mask on the original image, and its forward-passing score on the target class is obtained. Finally, the result can be generated by the linear combination of score-based weights and activation maps ([[https://github.com/haofanwang/Score-CAM](https://github.com/haofanwang/Score-CAM)]([https://github.com/haofanwang/Score-CAM](https://github.com/haofanwang/Score-CAM))).