# CA-BSN: Mural Image Denoising Based on Cross-Attention Blind Spot Network
## 3 Methods
### Feature Extraction Network Construction
#### 3.1.2 Non-Local Attention Mechanism Introduction  
In order to enhance the model to extract and utilize the global information of the image and retain the texture information of the image, we introduce a non-local attention mechanism (NLA) to adjust the processing based on the dynamic weights of the input image feature. The structure of the non-local attention mechanism (NLA) is shown below in Figure 3.  
Figure 2: The structure of densely dilated residual block.  
Our non-local attention mechanism (NLA) is divided into two main steps. First, calculate the attention block; then, output the global feature. The specific steps are as follows.  
Step 1. Calculate the attention block.  
The attention block is a structure of interactive operations through Value (V), Key (K), Query (Q) matrices, as shown in Figure 4 below.  
In neural networks, deep convolution can use fewer parameters than normal convolution by performing a separate convolution operation for each channel. Therefore, after changing the number of feature channels in the \(1\times 1\) convolution, we use deep convolution instead of the previously used convolution for arithmetic processing.  
In attention block, first, for the input \(V\in R^{H\times W\times C}\), \(K\in R^{H\times W\times C}\) and \(Q\in R^{H\times W\times C}\) matrices, the feature are extracted using \(1\times 1\) convolution \(Conv_{V,K,Q}(.)\) and \(3\times 3\) deep convolution \(DConv_{V,K,Q}(.)\), respectively, and the \(V\), \(K\) and \(Q\) dimensions are changed to \(\{V^{\prime},K^{\prime},Q^{\prime}\}\in R^{(H\times W)\times C}\) using the Reshape operation \(R(.)\). The process is computed as shown in Equations (3)-(5):  
\[V^{\prime} =R(DConv_{V}(Conv_{V}(V))) \tag{3}\] \[K^{\prime} =R(DConv_{K}(Conv_{K}(K)))\] (4) \[Q^{\prime} =R(DConv_{Q}(Conv_{Q}(Q))) \tag{5}\]  
Then, the matrix \(\left(K^{\prime}\right)^{T}\) after transposition of \(K^{\prime}\) is multiplied with the matrix \(Q^{\prime}\), and the mapping operation is performed using the Softmax function to obtain the correlation  
Figure 4: The structure of attention block.  
Figure 3: The structure of non-local attention mechanism.  
weight \(A\in R^{C\times C}\) between the two input matrices. This step of the computation is shown in Equation (6):  
\[A=\text{Softmax}({(K^{\prime})}^{T}Q^{\prime}) \tag{6}\]  
Finally, the weight matrix is calculated using weight \(A\) and \(V^{\prime}\). The calculation process is shown in Equation (7):  
\[g_{A}(V,K,Q)=Conv(R(V^{\prime}A)) \tag{7}\]  
Step 2. Export global feature.  
The input feature is \(I_{nl}\in R^{H\times W\times C}\). Different components \(V_{nl}\), \(K_{nl}\) and \(Q_{nl}\) are input into the attention block to get the weight matrix to establish the correlation between the global features. We use the skip connection to ensure the learning performance of the network model to output global feature \(O_{nl}\in R^{H\times W\times C}\). The calculation process is shown in Equation (8):  
\[O_{nl}=g_{A}(V_{nl},K_{nl},Q_{nl})+I_{nl} \tag{8}\]  
By using non-local attention mechanism (NLA) based on the self-attention mechanism, the extraction of global feature information can be enhanced to make up for the shortcomings of densely dilated residual block (DDR) in global feature extraction and retain as much as possible the texture and edge information of the mural image that is easily lost in the process of local feature extraction.