# Integrating Explanations into CNNs by Adopting Spiking Attention Block for Skin Cancer Detection
## 3 Proposed Methodology
Additionally, \(\omega\) and \(\beta\) represent the corresponding weights and biases. For this work, the value of \(\omega\) is randomly initiated at the start, and \(\beta\) for all inputs is fixed at 1.05 after extensive experiments. For the network, \(\eta_{(t-1)}\) represents the preceding hidden state. The forget gate, represented as \(Gf_{t}\), determines which information should be disregarded. The input gate, after applying spatial attention, \(Gi_{t}\), is responsible for regulating the flow of information into the cell and includes an additional auxiliary input layer denoted as \(h_{t}\), which is affected by an additional activation peak \(\sigma_{2}\). A spiking convolution's output is defined by its cell state, \(h_{t}\), and output gate, \(Go_{t}\). The use of spiking convolution is made possible by improving the forget gate through the attention mechanism. The proposed method, known as the spiking convolution with the CSAM component, aims to dynamically incorporate previously extracted spatial features to refine the representation. When using spiking convolution, the forget gate is designed by merging the current pulse input with the hidden state of the previous red state, followed by segmentation and convolution. The proposed method derives the one-dimensional channel attention map and the two-dimensional space attention map from the CSAM component's forget gate. Because of this, the forget gate's channel properties and spatial aspects can be extracted. Additionally, spatial adjustments are applied to refine channel features based on their contextual relevance. The comprehensive process of attention can be succinctly summarized as  
\[Gf_{t}^{\prime}=\text{Mod}_{s}(\text{Gf}_{t})\odot\text{Gf}_{t} \tag{6}\]  
\[Gf_{t}^{\prime\prime}=\text{Mod}_{c}(\text{Gf}_{t}^{\prime})\odot\text{Gf}_{t}^ {\prime} \tag{7}\]  
The spatial attention module is referred to as \(Mod_{s}\), whereas the channel attention module is referred to as \(Mod_{c}\). Channel features and spatial features are extracted sequentially from the forget gate, resulting in the generation of \(Mod_{c}(\text{Gf}_{t})\) and \(Mod_{s}(\text{Gf}^{\prime}_{t})\). Gf\({}^{\prime}_{t}\) represents the output of the spiking CSAM. Figure 2 illustrates the computation method of each attention diagram. The CSAM channel attention component uses max pooling and average pooling to generate a one-dimensional channel attention map of the forget gate. The calculation for the channel attention is as follows:  
Figure 3:0  
Here, \(SConv^{(3\times 3)}\) denotes the spiking convolutional operation. The weights of the models in the average and max pooling feature maps are the same. After passing it through the convolution process, the spatial attention part of CSAM pools the results and does average and max pooling operations along the channel axis. The forget gate's two-dimensional spatial attention feature map is created by passing the result through the spike activation function. The calculation of spatial attention is as follows:  
Figure 3:1 Figure 3:2 (10) Figure 3:3  
Here, \(Conv^{(7\times 7)}\) denotes a convolution operation using a filter of size \(7\times 7\). As a result of these attention operations, the forget gate's spatial attention map and one-dimensional channel attention map are generated. These maps extract the spatial aspects of the forget gate, as well as its channel features. The gradient substitution approach employs the Heaviside step function in the forward-propagation process, represented as \(j=\theta(i)\). During the process of backpropagation, the substitution function utilized is \(\frac{dj}{dt}=\sigma(i)^{\prime}\). Here, \(\sigma(i)\) is a function that has a similar form to \(\theta(i)\) but is characterized by being smooth and continuous. The gradient of the Gaussian error function (gef) spike function is the main function used in backpropagation. The function that corresponds to this is  
Figure 3:4  
The parameter \(\tau\) determines the level of smoothness in the gradient throughout the process of backpropagation.