# Image Deblurring Based on an Improved CNN-Transformer Combination Network
## 3 Proposed Approach
### Local Global Feature Combination Block
#### 3.1.1 Transformer Block  
The transformer consists of multi-head attention and feed-forward; in addition, it is well-known for establishing long-term dependence of an image. However, to some extent, the high computation and large memory usage restrict its wide application. In normal cases, the computation complexity is mainly attributable to the self-attention (SA) layer, which is the core of the multi-head attention part. To ameliorate this problem, we adopt multi-dconv head transposed attention (MDTA) motioned in [18], but with different numbers of attention heads. Compared with applying SA in the spatial dimension, MDTA employs SA between channels to generate attention feature maps and then encode the global features. Furthermore, we use depth-wise convolutions to obtain more contextual information ahead of obtaining attention maps, as depicted in Figure 2(a). Figure 2: The structure of the proposed Local Global Feature Combination Block. The MDTA first crosses a layer normalized and projects query (\(Q\)), key (\(K\)), and value (\(V\)) (\(Q,K,V\in\mathbb{R}^{C\times H\times W}\)) by using a 1 \(\times\) 1 convolutional layer. Subsequently, a 3 \(\times\) 3 depth-wise convolutional layer is used for obtaining spatial content at the channel-level. The input feature \(X\in R^{C\times H\times W}\) and \(\hat{X}\in\mathbb{R}^{C\times H\times W}\) are obtained from a layer-normalized operation. \[Q=H_{\text{donv}}^{3\times 3}(H_{\text{conv}}^{1\times 1}(\hat{X})), \tag{3}\]  
\[K=H_{\text{donv}}^{3\times 3}(H_{\text{conv}}^{1\times 1}(\hat{X})), \tag{4}\]  
\[V=H_{\text{donv}}^{3\times 3}(H_{\text{conv}}^{1\times 1}(\hat{X})), \tag{5}\]  
Next, we need to capture global attention by obtaining the correlation between \(Q\) and \(K\). The common method involves reshaping \(Q\) and \(K\) into \(\hat{Q}\in\mathbb{R}^{C\times HW}\hat{K}\in\mathbb{R}^{HW\times C}\). Applying a dot product interaction generates a transposed-attention map with a size of \(\mathbb{R}^{C\times C}\). Then, we can reshape V into \(\hat{V}\in\mathbb{R}^{C\times HW}\) and multiply the attention map with \(\hat{V}\) to obtain a feature map \(X_{weight}\in\mathbb{R}^{C\times HW}\). In the end, we reshaped \(X_{weight}\) into \(X_{weight}\in\mathbb{R}^{C\times H\times W}\) and adopted a 1 \(\times\) 1 convolutional layer. Before the softmax operation, we applied \(\sqrt{\alpha}\), which is a temperature parameter [18] used to control the magnitude of the dot product of \(\hat{Q}\) and \(\hat{K}\). The above procedure can be defined as:  
\[X_{weight}=Soft\text{max}(\hat{Q}\cdot\hat{K}/\sqrt{\alpha})\cdot\hat{V}, \tag{6}\]  
\[Y=H_{\text{conv}}^{1\times 1}(\text{Reshape}(X_{weight})), \tag{7}\]  
In the meantime, the depth-wise convolutional was also used in the gate-donv feed-forward network (GDFN). As shown in Figure 3b, after layer normalization, we adopted a depth-wise convolution to obtain more information from spatially nearby pixel positions. In addition, we also used the element-wise product of two parallel paths, one of which passes through a RELU activation function. Assuming that \(x\) is the input feature, the GDFN is:  
\[\hat{x}=H_{\text{donv}}^{3\times 3}(H_{\text{conv}}^{1\times 1}(x)), \tag{8}\]  
\[Y=H_{\text{conv}}^{1\times 1}(\hat{x}\cdot\textit{relu}(\hat{x})), \tag{9}\]  
(b) Multi-Scale Residual block  
As is well-known, both global features and local features are of great significance for image deblurring. In general, multi-scale receptive fields are beneficial for extracting finer features. Thus, inspired by the inception in [27; 28], we designed a multi-scale residual block to obtain multi-scale features, as shown in Figure 4.  
Different from the original inception structure, we replaced one 5 \(\times\) 5 convolutional layer with two 3 \(\times\) 3 convolutional layers and applied three 3 \(\times\) 3 convolutional layers  
Figure 4: The structure of Multi-Scale Residual Block. instead of one 7 \(\times\) 7 convolutional layer as well.