## 5 Gradient Analysis  
As we have seen, proper separation in flows is needed in order to avoid overfitting or data leakage and reach a better overall performance. This conclusion was made with the network behavior analysis when exposed to data of the same flow. In order to prevent this type of problem in the future, we looked for a way to obtain an explanation of the behavior of the models. For this purpose, we applied GradCAM, known as gradient-weighted class activation mapping. This XAI algorithm was presented initially by [39] and used for image-based classification, captioning, and a visual question-answering model, such as ResNet [40]. However, this algorithm is applicable to a wider variety of model families, such as any convolutional neural network (CNN) model with fully connected layers, with structured output, or with multi-modal inputs. GradCAM is a generalization of CAM for CNN architectures [41]. The main concept behind GradCAM is deeper representation on a CNN capture higher-level visual construct; convolutional layers naturally retain spatial information, which is lost in fully connected layers. As such, the last convolutional layers should have a spatial representation of the features extracted from the input, so the algorithm uses the gradient information flowing into the last convolutional layer of the CNN to identify important regions considered by the model for the decision. We will now briefly explain the algorithm, visually shown in Figure 6.  
To obtain the class activation map or heatmap, we first need to find the CNN layer in our model that we want to extract the gradient from. In Figure 5, this is the convolutional layer when \(i=D\). As a result, we obtain (i) the inputs of the model \(X\); (ii) the output \(A=[A^{1},\dots,A^{k}]\) of the CNN layer, which consists of \(k\) tensors of the same size; and (iii) the output \(y\) of the softmax activations from the model. As an observation, GradCAM focuses on the feature extraction part of the model, not in the feature analysis part. In that sense, GradCAM is agnostic to the type of convolutional model as long as the input is fed into some type of CNN that retains spatial information, so that the extracted heatmap is spatially correlated to the input \(X\). \begin{table}
\begin{tabular}{c c} \hline \hline \multicolumn{2}{c}{**Common Parameters**} \\ \hline \multicolumn{2}{c}{2D model} \\ \hline Max-Pool Size & \(2\times 2\) \\ Number of Conv2D stages & 4 \\ Number of fully connected stages & 4 \\ Number of LSTM stages (when used) & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: _Cont._  
Figure 6: Histogram computation diagram in GradCAM. We fed the input \(X\), the bytes from the packet payload, into the model, while we retained information of the gradients of \(y\) with respect to \(A\) computed with backpropagation, i.e., \(\dfrac{\partial y}{\partial A}\). These gradients were used together with \(A\) to compute a heatmap for each class. Finally, the heatmaps were normalized and scaled up to the original size of the input \(X\). Usually, the last step is made when a representation with both the original image and the activation map is desired. However, for some cases where we want to extract the data as-is, we can skip the normalization step. In addition to the standard use of GradCAM, that is, to detect relevant areas of information for the model for a certain input, you can also obtain explanations that highlight support for regions that would make the model change the prediction. This is performed by using the gradient of the opposite of the score of each class with respect to \(A\). This is called counterfactual explanations, and further reinforces the insight the standard model might give. These counterfactual explanations are very useful in non-binary classification problems, as they show what features they are seeing to determine that a possible packet does not belong to a category.