# Multi-Frame Blind Super-Resolution Based on Joint Motion Estimation and Blur Kernel Estimation
## 4 Proposed Multi-Frame Blind Super-Resolution Algorithm
### Image Reconstruction  
\(\mathbf{K}\) and \(\mathbf{F}\) are fixed in this image reconstruction step, the estimated HR image \(\mathbf{x}\) can be computed in the following way:  
\[\hat{\mathbf{x}}=\operatorname*{argmax}_{\mathbf{x}}p(\mathbf{y_{1}},\mathbf{y_{2}},\dots,\bm {y_{N}}|\mathbf{K},\mathbf{F},\mathbf{x})p(\mathbf{x}), \tag{3}\]  
We adopt the regularization technique to estimate the HR image \(\mathbf{x}\) which can be expounded as  
\[\hat{\mathbf{x}}=\operatorname*{argmin}_{\mathbf{x}}\{\sum_{n=1}^{N}\mathbf{W}\mathbf{\cdot}h (\mathbf{D}\mathbf{K}_{n}\mathbf{F}_{n}\mathbf{x}-\mathbf{y_{n}})+\lambda\ \gamma(\mathbf{x})\}, \tag{4}\]  
where \(\mathbf{W}\) is the confidence weights matrix, \(h(\mathbf{r})\) is the Huber loss function as the data fidelity term, \(\gamma(\mathbf{x})\) is the regularization term with respect to \(\mathbf{x}\), \(\lambda\) is the trade-off parameter between  
Figure 2: The observation model of MSFR.  
the fidelity term and the regularization term. We use cross validation [5] to determine the value of \(\lambda\).  
The larger the observation error, the smaller the corresponding confidence weight. This paper estimates the confidence weight matrix \(\mathbf{W}=\left(\mathbf{\beta_{1}},\ldots,\mathbf{\beta_{n}},\ldots,\mathbf{\beta_{N}}\right)^{T}\) by using the method proposed in our previous article [23]. \(\mathbf{\beta_{n}}\) represents the confidence weight of the \(nth\) LR image. \(\mathbf{\beta_{n}}\) can be expounded as  
\[\mathbf{\beta_{n}}=\begin{cases}\frac{\text{mean}(\mathbf{r_{n}})}{\mathbf{r_{n}}}&\quad \quad\quad\quad\quad\quad\quad\quad\text{if }|r_{n,i}|\leq c\sigma_{f}^{t}\\ \frac{\text{mean}(\mathbf{r_{n}})}{\mathbf{r_{n}}}\cdot\frac{c\sigma_{f}^{t}}{\left|r_ {n,i}\right|}&\quad\quad\text{otherwise},\end{cases} \tag{5}\]  
where \(\mathbf{r_{n}}=\mathbf{D}\mathbf{K_{n}}\mathbf{F_{n}}\mathbf{x}-\mathbf{y_{n}}\) represents the observation error of the \(nth\) LR image, \(\text{mean}(\mathbf{r_{n}})\) is the mean value of \(\mathbf{r_{n}}\). \(c\) represents a positive constant. We set \(c=2\). \(\sigma_{f}^{t}\) represents the scale parameter of iteration \(t\). For discriminating inliers and outliers adaptively, \(\sigma_{f}^{t}\) can be estimated by using the median absolute deviation (MAD) [24]. The equation can be expounded as  
\[\sigma_{f}^{t}=\sigma_{0}\cdot MAD\Big{(}r^{t-1}\Big{|}\beta^{t-1}\Big{)}, \tag{6}\]  
where \(\sigma_{0}=1.4826\).  
Huber loss function \(h(\mathbf{r})\) is used for the fidelity term. The Huber loss function \(h(\mathbf{r})\) can be defined as  
\[h(\mathbf{r})=\begin{cases}r^{2}&\quad\text{if }|r|\leq\delta\\ 2\delta|r|-\delta^{2}&\quad\text{otherwise},\end{cases} \tag{7}\]  
Bilateral Total Variation (BTV) [25] is used for the regularization term \(\gamma(\mathbf{x})\). The BTV with the image sparsity is used to handle outliers. The equation can be expounded as  
\[\gamma(\mathbf{x})=\sum\
olimits_{n=-P}^{P}\sum\
olimits_{m=-P}^{P}\alpha^{|m|+|n |}\parallel\mathbf{x}-S_{x}^{n}S_{y}^{m}\mathbf{x}\parallel_{1}, \tag{8}\]  
where \(\alpha\) represents the scaled weight parameter. \(\alpha\) can be computed by using the method in [19]. \(P\) is the size of the sliding window. \(S_{x}^{n}\) represents moving \(n\) pixels in the \(x\) direction. \(S_{y}^{m}\) represents moving \(m\) pixels in the \(y\) direction.  
Scaled Conjugate Gradient (SCG) [26] is employed for the purpose of solving the problem in the Equation (4).