Motion deblurring based on deep feature fusion attention and double-scale

目的 拍摄运动物体时，图像易出现运动模糊，这将影响计算机视觉任务的完成。为提升运动图像去模糊的质量，提出了基于深度特征融合注意力的双尺度去运动模糊网络。方法 首先，设计了双尺度网络，在网络结构上设计高低尺度通路，在低尺度上增加对模糊区域的注意力，在高尺度上提升网络的高频细节恢复能力，增强了模型去模糊效果。其次，设计了深度特征融合注意力模块，通过融合全尺度特征、构建通道注意力，将编码的全尺度特征与解码的同级特征进行拼接融合，进一步增强了网络的去模糊性能和细节恢复能力。最后，在双尺度的基础上，引入多尺度损失，使模型更加关注高频细节的恢复。结果 在3个数据集上，与12种去模糊方法进行了对比实验。在GoPro数据集上得到了最优结果，相比SRN （scale-recurrent network）方法，平均峰值信噪比提升了2.29 dB，能够恢复出更多的细节信息。在Kohler数据集上，得到了最高的峰值信噪比（29.91 dB）。在Lai数据集上，视觉上有最好的去模糊效果。结论 实验结果表明，本文方法可以有效去除运动模糊并恢复细节。;Objective When taking an image, the image tends to come out blurred due to interferences from shaky or out of focus lenses, dust, or atmospheric light. Image blurs can be categorized into atmospheric, defocus, and motion blurs. The most common type of blur is motion blur, which is caused by jittering objects during shooting and has a great impact on computer vision tasks, such as image classification, object detection, text recognition. Meanwhile, methods for motion deblurring can be grouped into non-blind and blind deblurring. On the one hand, non-blind deblurring has a known fuzzy kernel and can output a sharp image due to effect of deconvolution with fuzzy image directly after removing noise. This method is simple and effective, but its premise is that the fuzzy kernel is known. On the other hand, blind deblurring faces the serious problem where the fuzzy kernel is unknown. This method needs to calculate the value of the fuzzy kernel and then deblur using the same procedure as the non-blind deblurring method. The method is complicated, computationally expensive, and time consuming. However, the fuzzy kernel is unknown in real scenarios, and most cases of deblurring are actually blind deblurring. Therefore, this paper focuses on blind deblurring. Owing to its powerful feature extraction ability, a convolutional neural network can deblur an image after obtaining its fuzzy features from a dataset of sharp and blurred image pairs, hence presenting a breakthrough for the task of image deblurring. Method To address motion blur, a twoscale network based on deep feature fusion attention is proposed in this paper. First, a two-scale network is designed to extract different scales of spatial information. During the transformation from high to low scale, the state of blurred features in the motion blurred image changes from smooth to sharp. Therefore, the network pays further attention to those fuzzy areas in a low-scale image, hence allowing the network to obtain fuzzy features. This network not only improves the capability of recovering frequency details from the original scale image but also effectively uses the spatial information of the blurred image to enhance the deblurring effect of the model. Second, the deep feature fusion attention module is constructed. The main structure of the network is very similar to that of U-Net. After the encoding and decoding structure, the deep feature fusion attention module is constructed to obtain the best fusion feature information. The encoding and decoding structure can obtain multi-level information about blurred images. Such information is then fed into the module of fullscale features and squeeze-and-excite in the deep feature fusion attention module to produce full-scale features, which in turn are spliced and fused with the decoded features of the same level to further enhance the recovery performance of the network. Third, in order to make the network recover the high-frequency details effectively, the function of perception loss is replaced by the function of frequency loss. The loss function in this paper is composed of two parts, namely, content loss and frequency loss. The function of content loss uses the mean absolute error and combines multi-scale knowledge to calculate the absolute disparity of each pixel between the sharp and restored images and obtain the value of content loss. This procedure also marks the first step in calculating the 2D Fourier transform of the restored and sharp images and in measuring the frequency loss. After calculating the Fourier transform, the average absolute error between the restored and sharp images is determined. The multi-scale frequency loss is obtained by multiplying the multi-scale weight by the average absolute error. Our loss function can improve the sensitivity of the network to fuzzy features and enhance the recovery ability of the model in frequency detail. Result We compare the performance of our model with that of 12 other methods on 3 different datasets. We evaluate the performance of our model on the GoPro dataset by utilizing the peak signal-to-noise ratio(PSNR) and structural similarity(SSIM)between the restored and sharp images. Compared with the scale-recurrent network, the proposed method obtains 2. 29 dB higher PSNR, thus allowing this method to recover detailed information. We also compare the optimal results of each method. We test the generalization performance of the proposed method on the Kohler dataset, where this method achieves the highest PSNR of 29. 91 dB without retraining. Meanwhile, we compare the deblurring performance of these methods for real blurred images on the Lai dataset and determine the best results via subjective comparison. Conclusion To improve the quality of motion deblurring, a two-scale network based on deep feature fusion attention is proposed in this paper. This work offers three contributions, namely, a novel two-scale network, a deep feature fusion attention module, and multi-scale loss. Objective and subjective experimental results show that the proposed deblurring model can efficiently integrate image spatial information and feature semantic information, thus improving its deblurring performance, PSNR, and SSIM.