# Multi-Frame Blind Super-Resolution Based on Joint Motion Estimation and Blur Kernel Estimation
## 2 Related Work of MFSR  
Non-blind super-resolution Non-blind super-resolution methods include interpolation-based methods [4], reconstruction-based methods [5; 6; 7] and learning-based methods [8; 9; 10]. [PERSON] et al. [4] designed an extended dual weighting scheme. This method was based on Voronoi tessellation which relied on motion confidence weights and distance weights. [PERSON] et al. [6] developed an adaptive bilateral total variation (ABTV) regularization method and used the half-quadratic estimation to take error norm adaptively. [PERSON] et al. [7] proposed a method by combining Demons registration and regularized Bayesian reconstruction. [PERSON] et al. [8] reconstructed the HR image by learning texture details using local linear embedding. [PERSON] et al. [9] proposed a method based on sparse coding by optimizing a single objective function. [PERSON] et al. [10] proposed a method to obtain more high-frequency detail information. This method fused adjacent frames by combining the registration module based on Generative Adversarial Networks (GAN). Non-blind super-resolution methods are effective when the blur kernel is known, or the blur is not considered. When the blur kernel does not match, the quality of the super-resolved image is very poor. Figure 1 shows the influence of the blur kernel mismatch on the SR result, where \(\sigma_{blur}\) denotes the blur kernel width. If the blur kernel width is smaller than the real value, the SR result is over-smoothing, and the high frequency information is blurred. If the blur kernel width is larger than the real value, the SR result shows obvious ringing artifacts due to over-enhancing of the high-frequency edges. Figure 1: SR results using different blur kernels when the real blur kernel width is 1. (**a**) \(\sigma_{blur}=0.5\); (**b**) \(\sigma_{blur}=1\); (**c**) \(\sigma_{blur}=1.5\). Blind super-resolution Multi-frame blind super-resolution can fall into two categories according to whether reconstruction of the HR image and the deblurring are separated independently. [PERSON] et al. [11] proposed a method to first estimate the blur from some enhanced edges iteratively and then estimated the reconstructed HR image by the non-blind super-resolution method using the estimated blur kernel. [PERSON] and [PERSON] [12] proposed a blind super-resolution method, which was first to deblur each LR image by using the estimated blur kernel and then to apply MFSR to the deblurred images. [PERSON] et al. [13] proposed a method for MFSR with two steps. In the first stage, a non-linear filtering method was proposed to produce an unsampled but still blurry image using the inter-frame motion and exploiting spatiotemporal redundancy. In the second stage, a variational method for single image deblurring was proposed. [PERSON] and [PERSON] [14] first proposed an accurate blur kernel estimation-based matrix decomposition method and then presented a matrix-variable optimization method for blind super resolution. Such methods that reconstruct the HR image by using the first estimated blur kernel have a simple operation flow and the blur kernel is estimated only one. However, they make it easy to magnify the error caused by inaccurate estimation of the blur kernel, which leads to the unsatisfactory effect of the reconstructed HR image. Another kind of multi-frame blind super-resolution is to estimate the blur kernel and reconstruct the HR image simultaneously. [PERSON] et al. [15] proposed a method to perform image alignment, deblurring and MFSR by an assumption of projective motion path. [PERSON] et al. [16] proposed a Bayesian method for estimating the motion, blur kernel and noise simultaneously while reconstructing the original high-resolution frame. [PERSON] et al. [17] proposed an expectation maximization algorithm to guide motion blur estimation and high-resolution image reconstruction. [PERSON] et al. [18] proposed a novel method of joint image registration and Point Spread Function (PSF) estimation so as to produce the reconstructed HR image.