# Image Deblurring Based on an Improved CNN-Transformer Combination Network
## 3 Proposed Approach  
### Framework  
As shown in Figure 1, the model is a structure with three parts: the encoder path, the skip connection, and the decoder path. Like the MIMO-Unet+ [8], we also rescaled the original blur images to a different resolution, input them in different levels of encoder paths, and restored images at a different level of decoder path, respectively. In general, the encoder path is designed to extract local and global features. The decoder path is used for image reconstruction. Meanwhile, the skip connections are bridges to achieve feature integration between encoder and decoder paths. To better describe the model framework, we define \(\mathrm{I}_{1},\mathrm{I}_{2},\mathrm{I}_{3}\) as the input blurry images with three resolutions and \(\hat{\mathsf{S}}_{1},\hat{\mathsf{S}}_{2},\hat{\mathsf{S}}_{3}\) are the corresponding restoration images at each scale, respectively. 1. Encoder Path As mentioned above, our proposed model adopts a coarse-to-fine structure, which has multi-scale input and multi-scale output. This strategy has been employed by numer  
Figure 1: The whole structure of the proposed deblurring network. ous CNN-based deblurring models and has already demonstrated its effectiveness. Given a blurred image \(\text{I}_{1}\in\mathbb{R}^{H\times W\times 3}\), we can take rescale operations to obtain \(\text{I}_{2}\in\mathbb{R}^{\frac{H}{4}\times\frac{W}{2}\times 3}\) and \(\text{I}_{3}\in\mathbb{R}^{\frac{H}{4}\times\frac{W}{4}\times 3}\) successively. First, we input the \(\text{I}_{1}\) to the model and apply a \(3\times 3\) convolutional layer and four residual blocks as Encoder Block 1 (EB1) to extract features. After that, the extracted feature needs to pass two more complicated encoder blocks. Each encoder block includes a downsample layer, a shallow convolutional module (SCM), a channel attention fusion block (CAFB), and two local-global feature combination blocks (LGFCB). Next, we can start to introduce the complicated encoder block in detail. First, supposing \(\text{k}=2,3\), the previous encoder output \(\text{EB}^{\text{out}}_{\text{k}-1}\) passes a downsample operation which consists of a \(3\times 3\) kernel convolutional layer with a stride of two. Meanwhile, we can use \(\text{SCM}_{\text{k}-1}\) to extract shallow features of rescaled blur image \(\text{I}_{\text{k}}\). Then, to fuse the \(\text{SCM}^{\text{out}}_{\text{k}-1}\) and the downsampled \(\text{EB}^{\text{out}}_{\text{k}-1}\), we can design a channel attention fusion block (CAFB), which can selectively emphasize or suppress the feature from the previous level. After, different from MIMO-Unet+ [8], which employs eight residual blocks as the core part of each encoder level, we can specially design a local-global feature combination block (LGFCB) which can obtain local details and global features simultaneously. It is important to note that the SAM is a module used in MIMO-Unet+ [8]; the detailed structure of SAM is shown in Figure 1.  
(b) Skip Connection  
In general, skip connections exist in most conventional U-Net structures. However, they usually only transmit the current scale feature from an encoder to decoder paths. To achieve feature communication between different levels, our model adopts asymmetric feature fusion (AFF), as shown in Figure 1. Each AFF integrates all the encoder outputs (\(\text{EB}^{\text{out}}_{\text{k}},\text{k}=1,2,3\)) with convolutional layers and allows feature information to be transmitted from different scales. Then, the output of AFF is delivered to the corresponding decoder paths. The detailed process can be formulated as follows:  
\[\text{AFF}^{\text{out}}_{1}=\text{AFF}_{1}(\text{EB}^{\text{out}}_{1},(\text{ EB}^{\text{out}}_{2})^{\top},(\text{EB}^{\text{out}}_{3})^{\top}), \tag{1}\]  
\[\text{AFF}^{\text{out}}_{2}=\text{AFF}_{2}((\text{EB}^{\text{out}}_{1})^{\top },\text{EB}^{\text{out}}_{2},(\text{EB}^{\text{out}}_{3})^{\top}), \tag{2}\]  
The (\(\uparrow\)) represents upsample and the (\(\downarrow\)) is downsample operation.