## 5 Gradient Analysis
Merging both results, we obtain the two most important features the model considers for the prediction, positive and negative, as shown in Figure 7.  
More technically speaking, to obtain the heatmap \(L^{c}\), we have to study the relation between each output tensor \(A^{k}\) of the last convolutional layer and the output of a class \(y^{c}\), which is just the \(c\) component of vector \(y\). For each element of \(A^{k}\), we compute a measure of how that element contributes to \(y^{c}\). In particular, two options can be used: (i) relying on the gradients, i.e., for the \(i\)-th element of the tensor  
\[g^{k}_{c}(i)=\dfrac{\partial y^{c}}{\partial A^{k}_{i}}; \tag{1}\]  
(ii) or using guided gradients, i.e., for the \(i\)-th element of the tensor  
\[g^{k}_{c}(i)= \begin{cases}\dfrac{\partial y^{c}}{\partial A^{k}_{i}}&\text{ if }A^{k}_{i}>0\text{ and }\dfrac{\partial y^{c}}{\partial A^{k}_{i}}>0\\ 0&\text{otherwise.}\end{cases}  \tag{2}\]  
By using the first, we only consider the gradients in every part of \(A^{k}\), while in the second approach we only take into consideration elements whose value is positive, i.e., they would  
Figure 7: Relevance values given by GradCAM-1D when using both standard and counterfactual analysis. trigger the ReLU activation, and whose gradient is also positive, i.e., contributes towards class \(c\) in the classification decision. Using either of the previous gradients, we compute the average score of \(A^{k}\),  
\[a^{c}_{k}=\underset{i}{\text{mean}}\,g^{k}_{c}(i), \tag{3}\]  
which represents how much \(A^{k}\) contributes to class \(c\). Moreover, it is worth noting that this works for both 1D, 2D, or n-dimensional CNNs, since we compute the gradient at each element of \(A^{k}\) and they are averaged all together, ignoring their original position or the 2D/3D structure of \(A^{k}\). After obtaining the averaged gradient score, a partial linearization is made to only keep the parts that produce a positive response, that is,  
\[L^{c}=ReLU\left(\sum_{k}a^{c}_{k}A^{k}\right)=\max\left(0,\sum_{k}a^{c}_{k}A^{k }\right). \tag{4}\]  
Similarly, (3) can be modified to obtain counterfactual explanations, i.e., zones that discard a class, contributing negatively to \(y^{c}\). This is performed by computing the gradient of \(-y^{c}\). Similarly to (1),  
\[g^{k}_{c}(i)=\frac{\partial(-y^{c})}{\partial A^{k}_{i}}=-\frac{\partial y^{ c}}{\partial A^{k}_{i}}=-g^{k}_{c}(i), \tag{5}\]  
so, essentially, \(\bar{a}^{c}_{k}=-a^{c}_{k}\) when using gradients. However, for guided gradients, there is no relevant relation between these two quantities:  
\[g^{k}_{c}(i)= \begin{cases}-\frac{\partial y^{c}}{\partial A^{k}_{i}}&\text{ if }A^{k}_{i}>0\text{ and }\frac{\partial y^{c}}{\partial A^{k}_{i}}<0\\ 0&\text{ otherwise.}\end{cases}  \tag{6}\]  
In both cases, the weights of each \(A^{k}\) can be computed using the previous gradients  
\[\bar{a}^{c}_{k}=\underset{i}{\text{mean}}\,g^{k}_{c}(i), \tag{7}\]  
yielding the final expression for the counterfactual relevance map:  
\[\bar{L}^{c}=ReLU\left(\sum_{k}\bar{a}^{c}_{k}A^{k}\right). \tag{8}\]  
Together, \(L^{c}\) and \(L^{c}\) explain the decision of the model. On the one hand, \(L^{c}\) describes the regions that contribute to choose class \(c\). On the other hand, \(\bar{L}^{c}\) describes which regions hint not to choose class \(c\). Another consideration that must be made is that on multiclass applications like this one, this technique only presents data relevant to the predicted class. For example, if a packet has been identified as \"Chat\", the information that GradCAM will be able to provide will be related to whether that packet has been identified as \"Chat\" and not other classes.