# Image Restoration by Second-Order Total Generalized Variation and Wavelet Frame Regularization

Authors: Jianguang Zhu; Kai Li; Binbin Hao
Journal: Complexity
 (2019)

---

## Abstract

It has been proved that total generalized variation (TGV) can better preserve edges while suppressing staircase effect. In this paper, we propose an effective hybrid regularization model based on second-order TGV and wavelet frame. The proposed model inherits the advantages of TGV regularization and wavelet frame regularization, can eliminate staircase effect while protecting the sharp edge, and simultaneously has good capability of sparsely estimating the piecewise smooth functions. The alternative direction method of multiplier (ADMM) is employed to solve the new model. Numerical results show that our proposed model can preserve more details and get higher image visual quality than some current state-of-the-art methods.


---

## Body

## 1. Introduction

Image restoration refers to the problem of recovering image that satisfies people’s needs from an observed image that degraded by different blur and noise. The problem is mainly applied to remote sensing, medical image, video cameras, and other fields [1–5]. There are many factors that cause image degradation, such as atmospheric turbulence, camera shake, and relative motion between the camera and the object [6]. To obtain higher quality images, many methods have been proposed to solve the degenerate model, for instance, variational analysis [7]. In this paper, the degenerate model can be expressed as the following form:(1)g=Ku+nwhere g, u are degenerate and real images, respectively, K is a blur operator, we use the “fspecial” function to describe the blur operator “K” in this paper, and n is the additive white Gaussian noise with variance σ2 and zero-mean.Recoveringx from g is an inverse problem. In order to deal with this problem, scholars have done a lot of researches on this problem. One of the most effective ways to deal with this problem is adding some regularized terms to objective function. This leads to the following restoration model:(2)minu12Ku-g22+λΦuwhere ·2 denotes the Euclidean norm, Φ(u) is a regularized term which is used to regularize the solution, and λ is a positive regularization parameter which controls the two terms.The traditional regularization terms include the Tikhonov-like regularization [8] and the total variation (TV) regularization [9]. Tikhonov-like regularization: Φ(u)=Du22, where D is finite different operator. Due to the fact that Tikhonov-like regularization tends to make images overly smooth in the process of image processing, it fails to preserve sharp edges. On the contrary, TV regularization does better in protecting the sharp edge of image [10]. TV regularization: Φ(u)=∇u2 or Φ(u)=∇u1. In (2), Φ(u) is isotropic regularization term if the norm · is 2-norm, and Φ(u) is anisotropic regularization term if the norm · is 1-norm. TV regularization was first introduced by Rudin et al. in 1992 and widely used to solve problem (1). Although the TV regularization is better than the Tikhonov-like regularization in preserving sharp edges, it often produces staircase effects. The reason of causing the staircase effects is that TV regularization tends to transform the smooth regions of the solution into piecewise constant regions during solving the minimization problem [11, 12].The TV regularization effectively reduces the noise and preserves the sharp edges. But there still exist some undesired staircase effects in smooth regions of the restored images. To suppress the staircase effects, many improved models based on TV regularization term are proposed, such as high-order TV model [12–14], hybrid TV model [15–19], fractional order TV model [20–28], and total generalized variation model [29–34]. The TGV regularization is a useful tool to remove the staircase effect as well as preserve the sharp edge. In this paper, we focus on the total generalized variation regularization which can be seen as a generalization of total variation. The concept of TGV regularization was first proposed by Bredies et al. [29] as penalty function for image processing. It is worth noticing that TGV involves and balances higher-order derivatives of u. This results in the fact that the reconstruction by using TGV regularization can preserve edges while suppressing staircase effect. In order to solve the TGV model efficiently, many optimization algorithms have been proposed, such as Newton’s method, split Bregman method, alternating direction method of multipliers, and gradient descent method [35–43]. Experiments show that TGV has the superior performance to TV based regularization models in image reconstruction. In other words, TGV regularization model has some advantages in restraining the staircase effects caused in TV based regularization models.In recent years, sparse representation aroused people’s attention [44]. In most cases, images are usually sparse in some domain such as Fourier, cosine, wavelet, and wavelet frame [45, 46]. In fact, images can be sparsely approximated by proper wavelet frame [47–49]. So, the sparse representation based on the wavelet frame has become a hot topic in the process of image restoration. Correspondingly, the effective model is the l1-norm of the wavelet frame coefficients, because of its sparsity and convexity. Numerical experiments show that the models based on wavelet frame and variational methods can significantly improve the quality of images [50, 51]. However, the only fly in the ointment is that the Gibbs-like oscillations emerge frequently around the image discontinuities.Therefore, to better reconstruct the degraded image and simultaneously preserve image features, a new edgepreserving regularization scheme is reported in this work. Inspired by the above-mentioned advantages of wavelet frame based methods and TGV, and avoiding their main shortcomings, we concentrate on a novel hybrid regularizers model for image restoration by combining TGV and wavelet frame. Owing to the proposed model making good use of the advantages of wavelet frame and total generalized variation regularization, the new proposed model can not only protect the sharp edges of the images, but also make good use of the sparse prior information. In addition, the staircase effects are also effectively suppressed.Experimentally, we employ the ADMM technique to develop a restoration method to solve the proposed model. First, three auxiliary variables are introduced to transform the proposed model into a new constrained problem. Next, we utilize the variable splitting technique to transform the constrained problem into the unconstrained problem. Finally, the obtained unconstrained problem can be solved by the alternative direction method of multiplier. Numerical experiments show that the new proposed model is very effective compared to several state-of-the-art methods in eliminating staircase effects and recovering some details.The rest of the paper is organized as follows. In Section2, we briefly introduce the concept of TGV and wavelet frame. In Section 3, we present the proposed model and use the alternative direction method of multiplier (ADMM) to solve it. In Section 4, we show several numerical experiments to demonstrate the effectiveness of the proposed model. Finally, we summarize this article in Section 5.


## 2. Review of Total Generalized Variation and the Wavelet Frame

In this section, we will briefly introduce the concepts of the TGV and wavelet frame.


### 2.1. Total Generalized Variation

The concept of the TGV was first proposed by Bredies et al. [29], which is considered as a generalization of TV. Later, many scholars applied TGV to the field of image processing to suppress the staircase effects. The experimental results have certain advantages over the existing methods; see [30, 33]. The TGV model can be defined as (3)TGVαku=sup⁡∫Ωudivkqdx∣q∈CckΩ,symkRd,divlq∞≤αl,l=0,…,k-1,where symk(Rd)=ζ:Rd×Rd⋯×Rd︸ktimes→R∣ζisk-linearandsymmetric, it is the space of symmetric k-tensors on Rd; Ω∈R2 is image domain, Cck(Ω,symk(Rd)) is the space of compactly supported symmetric tensor field; ·∞ is the L∞ norm, and αl is fixed positive parameter; divk is the generalization of the divergence operator of k order to the tensor field. From the definition of TGVαk, we can see that TGV is a generalization of TV. When k=1, α0=1, TGVαk can be seen as the classical TV; for more details, see [29, 33, 34].In this paper, we focus on the second-order TGV, which can be written as(4)TGVα2u=sup⁡∫Ωudiv2qdx∣q∈Cc2Ω,Sd×d,divq∞≤αl,q∞≤α0,where Sd×d is the space of all symmetric matrices and Cc2(Ω,Sd×d) is the space of continuously differentiable symmetric matrix field with the compact support in Ω. For q∈Sd×d, the divergence of q is defined as (divq)i=∑j=1d∂qi,j/∂xj1≤i≤d; div2q=∑i,j=1d∂2qi,j/∂xi∂xj, and the infinity norm of q and divq is given by (5)q∞=supx∈Ω∑i,j=1dqi,jx21/2,divq∞=supx∈Ω∑i=1ddivqix21/2.In the following, we focus on the dimension d=2 and denote the spaces U=Cc2(Ω,R),V=Cc2(Ω,R2), and W=Cc2(Ω,S2×2). As noted in [29, 32], the discrete TGVα2(u) of u∈U can be rewritten as the following equivalent form:(6)TGVα2u=minp⁡α0εp1+α1∇u-p1,where ε(p)=1/2(∇p+∇pT) denotes symmetrized derivative, u∈Rmn represents the original m×n image, ∇=(∇x;∇y)∈R2mn×mn is the ordinary gradient operator, and ∇x,∇y∈Rmn×mn represent two first-order forward finite difference operators in directions x and y. Furthermore, we denote ∇i=(∇ix;∇iy)∈R2×mn, which is the ith row of ∇x and ∇y, and ∇iu∈R2 is the gradient of ui. Let p=(p1;p2)∈R2mn and pi=((p1)i;(p2)i)∈R2,i=1,2,…,mn. Then, we can discretize the symmetric operator ε as(7)εp=∇xp1∇yp1+∇xp22∇yp1+∇xp22∇yp2∈R2mn×2and the ith component of ε(p) is denoted as (8)εip=∇ixp1∇iyp1+∇ixp22∇iyp1+∇ixp22∇iyp2∈R2×2,i=1,2,…,mn.


### 2.2. The Wavelet Frame

In this subsection, we briefly introduce some notations of wavelet frames. In the discrete setting, letW stand for the fast tensor product framelet decomposition and WT be the fast reconstruction; then we obtain u=WTWu for any images x based on the unitary extension principle (UEP) [52] since WTW=I, where I is the identity matrix. The construction of framelets can be obtained according to the unitary extension principle (UEP) as well. In experiments, we will use the piecewise linear B-spline framelets constructed by [52] considering the balance of the time and quality. We denote an L-level framelet decomposition of u as (9)W=Wl,ju:0≤l≤L-1,j∈Jwhere I denotes the index set of all framelet bands and Wl,ju∈Rn is the wavelet frame coefficients of u in bands j at level l. We will also use α∈Rm to denote the frame coefficients; i.e., α=Wu, where (10)α=αl,j:0≤l≤L-1,j∈J,withαl,j=Wl,ju.For more details on discrete algorithms of framelet transforms, see [53, 54].To use the sparseness of the wavelet frame coefficients, there are several different wavelet frame based models proposed in the papers, i,e., the synthesis based approach [55, 56], the analysis based approach [57, 58], and the balanced approach [51, 53]. Although these approaches are different, they can guarantee a clear recovery from the unknown clean image as long as certain conditions are satisfied.


## 2.1. Total Generalized Variation

The concept of the TGV was first proposed by Bredies et al. [29], which is considered as a generalization of TV. Later, many scholars applied TGV to the field of image processing to suppress the staircase effects. The experimental results have certain advantages over the existing methods; see [30, 33]. The TGV model can be defined as (3)TGVαku=sup⁡∫Ωudivkqdx∣q∈CckΩ,symkRd,divlq∞≤αl,l=0,…,k-1,where symk(Rd)=ζ:Rd×Rd⋯×Rd︸ktimes→R∣ζisk-linearandsymmetric, it is the space of symmetric k-tensors on Rd; Ω∈R2 is image domain, Cck(Ω,symk(Rd)) is the space of compactly supported symmetric tensor field; ·∞ is the L∞ norm, and αl is fixed positive parameter; divk is the generalization of the divergence operator of k order to the tensor field. From the definition of TGVαk, we can see that TGV is a generalization of TV. When k=1, α0=1, TGVαk can be seen as the classical TV; for more details, see [29, 33, 34].In this paper, we focus on the second-order TGV, which can be written as(4)TGVα2u=sup⁡∫Ωudiv2qdx∣q∈Cc2Ω,Sd×d,divq∞≤αl,q∞≤α0,where Sd×d is the space of all symmetric matrices and Cc2(Ω,Sd×d) is the space of continuously differentiable symmetric matrix field with the compact support in Ω. For q∈Sd×d, the divergence of q is defined as (divq)i=∑j=1d∂qi,j/∂xj1≤i≤d; div2q=∑i,j=1d∂2qi,j/∂xi∂xj, and the infinity norm of q and divq is given by (5)q∞=supx∈Ω∑i,j=1dqi,jx21/2,divq∞=supx∈Ω∑i=1ddivqix21/2.In the following, we focus on the dimension d=2 and denote the spaces U=Cc2(Ω,R),V=Cc2(Ω,R2), and W=Cc2(Ω,S2×2). As noted in [29, 32], the discrete TGVα2(u) of u∈U can be rewritten as the following equivalent form:(6)TGVα2u=minp⁡α0εp1+α1∇u-p1,where ε(p)=1/2(∇p+∇pT) denotes symmetrized derivative, u∈Rmn represents the original m×n image, ∇=(∇x;∇y)∈R2mn×mn is the ordinary gradient operator, and ∇x,∇y∈Rmn×mn represent two first-order forward finite difference operators in directions x and y. Furthermore, we denote ∇i=(∇ix;∇iy)∈R2×mn, which is the ith row of ∇x and ∇y, and ∇iu∈R2 is the gradient of ui. Let p=(p1;p2)∈R2mn and pi=((p1)i;(p2)i)∈R2,i=1,2,…,mn. Then, we can discretize the symmetric operator ε as(7)εp=∇xp1∇yp1+∇xp22∇yp1+∇xp22∇yp2∈R2mn×2and the ith component of ε(p) is denoted as (8)εip=∇ixp1∇iyp1+∇ixp22∇iyp1+∇ixp22∇iyp2∈R2×2,i=1,2,…,mn.


## 2.2. The Wavelet Frame

In this subsection, we briefly introduce some notations of wavelet frames. In the discrete setting, letW stand for the fast tensor product framelet decomposition and WT be the fast reconstruction; then we obtain u=WTWu for any images x based on the unitary extension principle (UEP) [52] since WTW=I, where I is the identity matrix. The construction of framelets can be obtained according to the unitary extension principle (UEP) as well. In experiments, we will use the piecewise linear B-spline framelets constructed by [52] considering the balance of the time and quality. We denote an L-level framelet decomposition of u as (9)W=Wl,ju:0≤l≤L-1,j∈Jwhere I denotes the index set of all framelet bands and Wl,ju∈Rn is the wavelet frame coefficients of u in bands j at level l. We will also use α∈Rm to denote the frame coefficients; i.e., α=Wu, where (10)α=αl,j:0≤l≤L-1,j∈J,withαl,j=Wl,ju.For more details on discrete algorithms of framelet transforms, see [53, 54].To use the sparseness of the wavelet frame coefficients, there are several different wavelet frame based models proposed in the papers, i,e., the synthesis based approach [55, 56], the analysis based approach [57, 58], and the balanced approach [51, 53]. Although these approaches are different, they can guarantee a clear recovery from the unknown clean image as long as certain conditions are satisfied.


## 3. The Proposed Model and Algorithm

In this section, we will introduce a new hybrid regularization model based on TGV and wavelet frame. The new model can suppress the staircase effect while protecting the edge of the image. Alternating direction method of multipliers is used to solve the proposed minimization problem.


### 3.1. The New Model

Combining wavelet frame and TGV regularizations, we propose the following hybrid regularization model:(11)minx⁡12Ku-g22+TGVα2u+βWu1,where TGVα2(u) is the total generalized variation of u; its definition has been given in Section 2.1. Equation (11) can be rewritten as(12)minx⁡12Ku-g22+α0εp1+α1∇u-p1+α2Wu1,where α0,α1,α2 are positive regularization parameters that balance the four terms for minimization. W is the discrete wavelet frame transform; more details have been described in Section 2.2. Note that we name the proposed hybrid regularization model (12) as the TGVframe restoration model.


### 3.2. The Proposed Algorithm for the New Model

In this subsection, the alternating direction method of multipliers is employed to solve TGVframe restoration model (12). First, by introducing three auxiliary variables d=d1d3d3d2∈R2mn×2, v=(v1;v2)∈R2mn, z∈Rmn, we can transform model (12) into the following constrained problem: (13)minu,p,d,v,z12Ku-g22+α0d1+α1v1+α2z1s.t.d=εp,v=∇u-p,z=Wu.For the above constrained problem (13), using the classical quadratic penalty method, we can transform it into the following unconstrained problem:(14)minu,d,v,z,p⁡12Ku-g22+α0d1+α1v1+α2z1+β12d-εp22+β22v-∇u+p22+β32z-Wu22,where β1,β2,β3 are positive penalty parameters. For the above unconstrained optimization problem (14), its augmented Lagrange function is given by(15)LAu,d,z,v,p;ξ,η,λ=12Ku-g22+α0d1+α1v1+α2z1+β12d-εp22+β22v-∇u+p22+β32z-Wu22-ξ,d-εp-η,v-∇u+p-λ,z-Wu,where ξ=ξ1ξ3ξ3ξ2∈R2mn×2, η=(η1;η2)∈R2mn, λ∈Rmn are Lagrange multipliers, which control positive penalty parameters β1,β2,β3 going to infinity. According to the classical ADMM, we should solve the following iterative scheme: (16)uk+1=arg⁡minxLAu,dk,zk,vk,pk;ξk,ηk,λk,pk+1=arg⁡minpLAuk+1,dk,zk,vk,p;ξk,ηk,λk,dk+1=arg⁡minyLAuk+1,d,zk,vk,pk+1;ξk,ηk,λk,zk+1=arg⁡minzLAuk+1,dk+1,z,vk,pk+1;ξk,ηk,λk,vk+1=arg⁡minvLAuk+1,dk+1,zk+1,v,pk+1;ξk,ηk,λk,ξk+1=ξk-β1dk+1-εpk+1,ηk+1=ηk-β2vk+1-∇uk+1+pk+1,λk+1=λk-β3zk+1-Wuk+1.For theu-subproblem, it can be rewritten as(17)uk+1=arg⁡minu12Ku-g22+β22vk-∇u+pk-ηkβ222+β32zk-Wu-λkβ322Based on the optimality condition [59], the minimization problem (17) can be solved by the following linear equation:(18)KTK+β2∇T∇+β3Iuk+1=KTg+β2∇Tvk+pk-ηkβ3+β3WTzk-λkβ3,where ∇ refers to the first-order difference operator and ∇T is conjugate operator of ∇. Under the periodic boundary condition for u, ∇T∇=(∇x)T∇x+(∇y)T∇y and KTK are block circulant matrices [59]. So ∇T∇ and KTK can be diagonalized by the Fourier transform. The Fourier transform of u is defined as F(u). F-1(u) is the inverse Fourier transform of u. By using the Fourier transform, we can gain the solution of u as follows:(19)uk+1=F-1γ,where(20)γ=FKTg+β2∇Tvk+pk-ηk/β3+β3WTzk-λk/β3FKTK+β2∇T∇+β3I.Solving thep-subproblem is similar to u-subproblem, and the solution of p-subproblem can be given by(21)pk+1=arg⁡minpβ12yk-εp-ξkβ122+β22vk-∇uk+1+pk+1-ηkβ222.Because p=[p1;p2], we only need to get p1 and p2.Then, forp1k+1, we have(22)β1∇xT∇x+β12∇yT∇y+β2Ip1k+1=β1∇xTy1k-ξ1kβ1+∇yTy3k-ξ3kβ1-12∇xp2k+β2∇xuk+1+η1kβ2-v1k,for p2k+1,(23)β1∇yT∇y+β12∇xT∇x+β2Ip2k+1=β1∇yTy2k-ξ2kβ1+∇xTy3k-ξ3kβ1-12∇yp1k+β2∇yuk+1+η2kβ2-v2k.The solutions of problems (22) and (23) are similar to that of problem (18), so we can solve problems (22) and (23) with several FFTs and IFFTs [59] under the periodic boundary condition.For the minimization ofd in (16), the subproblem is given by(24)dk+1=arg⁡mindα0d1+β12d-εpk+1-ξkβ122.Problem (24) can be solved by the the following four-dimensional shrinkage operator:(25)dik+1=max⁡εipk+1+ξkiβ12-α0β1,0εipk+1+ξki/β1εipk+1+ξki/β12,i=1,2,…,mn.Forz-subproblem, it can be given by(26)zk+1=arg⁡minzα2z1+β32z-Wuk+1-λkβ322The solution of problem (26) is given by the one-dimensional shrinkage operation:(27)zik+1=max⁡Wuk+1i+λkiβ3-α2β3,0·signWuk+1i+λkiβ3,i=1,2,…,mn.Forv-subproblem, it can be written as(28)vk+1=arg⁡minvα1v1+β22v-∇uk+1+pk+1-ηkβ222.The solution of problem (28) can be given by the following two-dimensional shrinkage operation:(29)vik+1=max⁡∇iuk+1-pik+1+ηkiβ22-α1β2,0∇iuk+1-pik+1+ηki/β2∇iuk+1-pik+1+ηki/β22,i=1,2,…,mn.Finally, we update Lagrange multiplierξ,η,λ through the following formula, respectively, (30)ξk+1=ξk-β1dk+1-εpk+1,ηk+1=ηk-β2vk+1-∇uk+1+pk+1,λk+1=λk-β3zk+1-Wuk+1.In this paper, our algorithm is simply denoted as TGVframe. The TGVframe algorithm of the deblurring and denoising is summarized as shown in Algorithm1.Algorithm 1:ADMM for the TGVframe model.
1. Input:g, K, α0  α1>0, α2>0, β1>0, β2>0, β3>0. 2. Initialization:x0,p0,y0,z0,v0,ξ0,η0,λ0. 3.While “not converged”, Do 4. Computeuk+1 according to (19); 5. Computepk+1 according to (22) and (23); 6. Computedk+1 via (25); 7. Computezk+1 via (27); 8. Computevk+1 according to (29); 9. Updateξk+1,ηk+1,λk+1 from ξ k + 1 = ξ k - β 1 d k + 1 - ε p k + 1, η k + 1 = η k - β 2 v k + 1 - ∇ u k + 1 + p k + 1, λ k + 1 = λ k - β 3 z k + 1 - W u k + 1 . 10.End Do 11. Outputuk+1


## 3.1. The New Model

Combining wavelet frame and TGV regularizations, we propose the following hybrid regularization model:(11)minx⁡12Ku-g22+TGVα2u+βWu1,where TGVα2(u) is the total generalized variation of u; its definition has been given in Section 2.1. Equation (11) can be rewritten as(12)minx⁡12Ku-g22+α0εp1+α1∇u-p1+α2Wu1,where α0,α1,α2 are positive regularization parameters that balance the four terms for minimization. W is the discrete wavelet frame transform; more details have been described in Section 2.2. Note that we name the proposed hybrid regularization model (12) as the TGVframe restoration model.


## 3.2. The Proposed Algorithm for the New Model

In this subsection, the alternating direction method of multipliers is employed to solve TGVframe restoration model (12). First, by introducing three auxiliary variables d=d1d3d3d2∈R2mn×2, v=(v1;v2)∈R2mn, z∈Rmn, we can transform model (12) into the following constrained problem: (13)minu,p,d,v,z12Ku-g22+α0d1+α1v1+α2z1s.t.d=εp,v=∇u-p,z=Wu.For the above constrained problem (13), using the classical quadratic penalty method, we can transform it into the following unconstrained problem:(14)minu,d,v,z,p⁡12Ku-g22+α0d1+α1v1+α2z1+β12d-εp22+β22v-∇u+p22+β32z-Wu22,where β1,β2,β3 are positive penalty parameters. For the above unconstrained optimization problem (14), its augmented Lagrange function is given by(15)LAu,d,z,v,p;ξ,η,λ=12Ku-g22+α0d1+α1v1+α2z1+β12d-εp22+β22v-∇u+p22+β32z-Wu22-ξ,d-εp-η,v-∇u+p-λ,z-Wu,where ξ=ξ1ξ3ξ3ξ2∈R2mn×2, η=(η1;η2)∈R2mn, λ∈Rmn are Lagrange multipliers, which control positive penalty parameters β1,β2,β3 going to infinity. According to the classical ADMM, we should solve the following iterative scheme: (16)uk+1=arg⁡minxLAu,dk,zk,vk,pk;ξk,ηk,λk,pk+1=arg⁡minpLAuk+1,dk,zk,vk,p;ξk,ηk,λk,dk+1=arg⁡minyLAuk+1,d,zk,vk,pk+1;ξk,ηk,λk,zk+1=arg⁡minzLAuk+1,dk+1,z,vk,pk+1;ξk,ηk,λk,vk+1=arg⁡minvLAuk+1,dk+1,zk+1,v,pk+1;ξk,ηk,λk,ξk+1=ξk-β1dk+1-εpk+1,ηk+1=ηk-β2vk+1-∇uk+1+pk+1,λk+1=λk-β3zk+1-Wuk+1.For theu-subproblem, it can be rewritten as(17)uk+1=arg⁡minu12Ku-g22+β22vk-∇u+pk-ηkβ222+β32zk-Wu-λkβ322Based on the optimality condition [59], the minimization problem (17) can be solved by the following linear equation:(18)KTK+β2∇T∇+β3Iuk+1=KTg+β2∇Tvk+pk-ηkβ3+β3WTzk-λkβ3,where ∇ refers to the first-order difference operator and ∇T is conjugate operator of ∇. Under the periodic boundary condition for u, ∇T∇=(∇x)T∇x+(∇y)T∇y and KTK are block circulant matrices [59]. So ∇T∇ and KTK can be diagonalized by the Fourier transform. The Fourier transform of u is defined as F(u). F-1(u) is the inverse Fourier transform of u. By using the Fourier transform, we can gain the solution of u as follows:(19)uk+1=F-1γ,where(20)γ=FKTg+β2∇Tvk+pk-ηk/β3+β3WTzk-λk/β3FKTK+β2∇T∇+β3I.Solving thep-subproblem is similar to u-subproblem, and the solution of p-subproblem can be given by(21)pk+1=arg⁡minpβ12yk-εp-ξkβ122+β22vk-∇uk+1+pk+1-ηkβ222.Because p=[p1;p2], we only need to get p1 and p2.Then, forp1k+1, we have(22)β1∇xT∇x+β12∇yT∇y+β2Ip1k+1=β1∇xTy1k-ξ1kβ1+∇yTy3k-ξ3kβ1-12∇xp2k+β2∇xuk+1+η1kβ2-v1k,for p2k+1,(23)β1∇yT∇y+β12∇xT∇x+β2Ip2k+1=β1∇yTy2k-ξ2kβ1+∇xTy3k-ξ3kβ1-12∇yp1k+β2∇yuk+1+η2kβ2-v2k.The solutions of problems (22) and (23) are similar to that of problem (18), so we can solve problems (22) and (23) with several FFTs and IFFTs [59] under the periodic boundary condition.For the minimization ofd in (16), the subproblem is given by(24)dk+1=arg⁡mindα0d1+β12d-εpk+1-ξkβ122.Problem (24) can be solved by the the following four-dimensional shrinkage operator:(25)dik+1=max⁡εipk+1+ξkiβ12-α0β1,0εipk+1+ξki/β1εipk+1+ξki/β12,i=1,2,…,mn.Forz-subproblem, it can be given by(26)zk+1=arg⁡minzα2z1+β32z-Wuk+1-λkβ322The solution of problem (26) is given by the one-dimensional shrinkage operation:(27)zik+1=max⁡Wuk+1i+λkiβ3-α2β3,0·signWuk+1i+λkiβ3,i=1,2,…,mn.Forv-subproblem, it can be written as(28)vk+1=arg⁡minvα1v1+β22v-∇uk+1+pk+1-ηkβ222.The solution of problem (28) can be given by the following two-dimensional shrinkage operation:(29)vik+1=max⁡∇iuk+1-pik+1+ηkiβ22-α1β2,0∇iuk+1-pik+1+ηki/β2∇iuk+1-pik+1+ηki/β22,i=1,2,…,mn.Finally, we update Lagrange multiplierξ,η,λ through the following formula, respectively, (30)ξk+1=ξk-β1dk+1-εpk+1,ηk+1=ηk-β2vk+1-∇uk+1+pk+1,λk+1=λk-β3zk+1-Wuk+1.In this paper, our algorithm is simply denoted as TGVframe. The TGVframe algorithm of the deblurring and denoising is summarized as shown in Algorithm1.Algorithm 1:ADMM for the TGVframe model.
1. Input:g, K, α0  α1>0, α2>0, β1>0, β2>0, β3>0. 2. Initialization:x0,p0,y0,z0,v0,ξ0,η0,λ0. 3.While “not converged”, Do 4. Computeuk+1 according to (19); 5. Computepk+1 according to (22) and (23); 6. Computedk+1 via (25); 7. Computezk+1 via (27); 8. Computevk+1 according to (29); 9. Updateξk+1,ηk+1,λk+1 from ξ k + 1 = ξ k - β 1 d k + 1 - ε p k + 1, η k + 1 = η k - β 2 v k + 1 - ∇ u k + 1 + p k + 1, λ k + 1 = λ k - β 3 z k + 1 - W u k + 1 . 10.End Do 11. Outputuk+1


## 4. Numerical Experiments

In this section, in order to show the effectiveness of the proposed model in suppressing staircasing effect, protecting the sharp edges, and removing Gaussian noise in image restoration, we compare the proposed models with FTVd version 4.1 [10], TVframe [49], and TGV [29]. All the experiments were performed under Windows 10 and MATLAB 2012a running on a desktop with an core i5 Duo central processing unit at 2.50 GHz and 4 GB memory to ensure the fairness and effectiveness of the test.In the experiments, we select “Cameraman” (256×256), “Lena” (256×256), “House” (256×256), and “Barbara” (256×256) as the tested images; see Figure 1. For all experiments, the tested images are degraded by different blur kernel and additional Gaussian white noise with different standard deviation. We use signal-to-noise ratio (SNR) and structural similarity (SSIM) to measure the quality of the restored images. The larger the SNR is, the better the image quality is. The SNR and SSIM are defined as (31)SNR=10log10⁡f0-f¯22f0-f22,SSIM=2μf0μf+c12σf0f+c2μf02+μf2+c1σf02+σf2+c2.Figure 1
Test images.
(a)
Cameraman (b)
Lena (c)
House (d)
BarbaraHeref0, f are the ideal image and the restored image, respectively. f¯ is the mean intensity value of f0, and μf0 and μf are the mean value of the f0 and f, respectively. σf02 and σf2 are the variance of f0 and f, respectively, and σf0f is covariance of f0 and f. c1 and c2 are positive constants which can be seen as stabilizing constants for nearzero denominator values. The SSIM is an index used to measure the similarity between the restored image and the ideal image. For more details about SSIM, see [60].In order to have fair comparisons in all the experiments, we terminate all the algorithms by the following stopping condition:(32)uk+1-uk2uk2≤10-4.


### 4.1. Gaussian Blur Experiment

In this subsection, Gaussian blur experiment is considered. It is known that the size of blur kernel determines the quality of the recovered images to a certain extent. The larger the blur kernel is, the lower the recovered image quality is. In order to test the effectiveness of our method for different blur kernels, we do a test for the images with different size of blurring kernels. The values of SNR and SSIM by four different methods under different Gaussian blur kernels are shown in Table1. In this test, we select the well-known image “Cameraman” (256×256) which degraded by being blurred with a 9∗9 Gaussian kernel and contaminated by Gaussian noise with standard variance σ=0.1 as tested image displayed in Figure 2(b), and the recovered results are shown in Figure 2. In our method, we select parameters α0=5e-3, α1=3e-3, and α2=4.2e-3; μ = 300 and β = 10 in FTVd; α=5e-3, β=5e-5 in TVframe; and α0=1e-4, α1=2e-3 in TGV. Figure 3 shows the enlarged results of a part of Figure 2. From Figures 2 and 3 and Table 1, it is not difficult to see that our method still has good restoration when the size of blur kernel reached 11∗11, which shows that our method is robust to blur kernel. Meanwhile, our method can also better suppress the staircase effect. In order to show our proposed method is convergent, we also plot the changing curve of SNRs versus iteration numbers in Figure 4.Table 1
Restoration results of different methods under gaussian blur with noise levelσ=0.1.
FTVd TGV TVframe TGVframe Cameraman 7 ∗ 7 SNR 26.16 27.35 27.82 28.66 SSIM 0.6783 0.7005 0.7180 0.7591 9 ∗ 9 SNR 24.09 24.98 25.29 25.98 SSIM 0.6283 0.6421 0.6573 0.6807 11 ∗ 11 SNR 22.81 23.24 23.85 24.74 SSIM 0.5817 0.6066 0.6209 0.6794  Lena 7 ∗ 7 SNR 25.78 26.80 27.31 28.15 SSIM 0.8160 0.8300 0.8425 0.8667 9 ∗ 9 SNR 24.07 25.85 26.02 26.99 SSIM 0.7560 0.7819 0.7903 0.8262 11 ∗ 11 SNR 23.92 24.73 24.98 25.89 SSIM 0.7159 0.7430 0.7517 0.8009  House 7 ∗ 7 SNR 32.27 33.41 33.95 34.82 SSIM 0.6865 0.7206 0.7450 0.7765 9 ∗ 9 SNR 30.85 31.81 32.35 33.42 SSIM 0.6281 0.6425 0.6558 0.7174 11 ∗ 11 SNR 30.47 31.23 31.98 32.53 SSIM 0.5311 0.5751 0.5947 0.6578  Barbara 7 ∗ 7 SNR 23.21 24.32 25.16 26.03 SSIM 0.8068 0.8351 0.8538 0.8767 9 ∗ 9 SNR 22.01 23.08 23.73 24.53 SSIM 0.7690 0.7848 0.8003 0.8312 11 ∗ 11 SNR 21.33 22.41 23.13 23.62 SSIM 0.7250 0.7415 0.7762 0.8034Figure 2
Restoration results of the Cameraman image with Gaussian blur kernel. (a) Original image; (b) blurred and noisy image; (c) image deblurred by FTVd; (d) image deblurred by TGV; (e) image deblurred by TVframe; (f) image deblurred by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 3
The zoomed regions of the results of Figure2. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 4
The changes of SNRs versus iteration number for test images about Gaussian blur.
(a)
σ = 0.1, Gaussian(11, 11) (b)
σ = 0.02, Gaussian(9, 9)


### 4.2. Average Blur Experiment

In this experiment, we test the well-known image “house”(256×256) which has many texture details that help to show the effectiveness of our method. The “house” image is degraded by blurring kernel with 11∗11 average kernel and noise derivation σ=0.1; see Figure 5(b). The effect diagrams that are restored with FTVd, TVframe, TGV, and our method are shown in Figures 5(c)–5(f). In this test, the parameters α0=8e-4, α1=6e-4, and α2=8.2e-4 are used in TGVframe; in FTVd, μ=53 and λ=0.5; the parameters α=4e-6 and β=5e-6 are selected in TVframe; the parameters α0=6e-4, α1=1e-3 are used in TGV. The restoration results are shown in Figure 5. We have enlarged some details of the four restored images for a better visualization and the effects of the enlarged part are shown in Figure 6. As it is seen in the zoomed parts, our method outperforms the other three methods. The values of SNR and SSIM of the four images are listed in Table 2. From Figures 5 and 6 and Table 2, we can find that the restored images by FTVd, TGV, and TVframe have some piecewise constant regions, while our method does better and has better qualities with more details, textures, and less stair-casing effect.Table 2
Restoration results of different methods under average blur with noise levelσ=0.1.
FTVd TGV TVframe TGVframe Lena average(9) SNR 23.36 24.65 25.42 26.17 SSIM 0.7251 0.7538 0.7801 0.8038 average(11) SNR 22.37 23.62 24.26 25.59 SSIM 0.6824 0.7022 0.7609 0.7737  House average(9) SNR 30.40 31.78 32.36 33.23 SSIM 0.5743 0.6362 0.6440 0.6625 average(11) SNR 28.97 30.03 30.81 31.76 SSIM 0.4894 0.5273 0.5309 0.5828  Cameraman average(9) SNR 23.63 24.96 25.32 26.43 SSIM 0.6001 0.6423 0.6576 0.6956 average(11) SNR 22.55 23.48 24.25 24.59 SSIM 0.5510 0.5896 0.6114 0.6359  Barbara average(9) SNR 22.47 24.13 24.79 25.28 SSIM 0.7182 0.7801 0.8003 0.8331 average(11) SNR 21.27 23.00 23.87 24.53 SSIM 0.6966 0.7314 0.7523 0.7982Figure 5
Restoration results of the House image with average blur kernel. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 6
The zoomed regions of the results of Figure5. (a) Original image; (b) blurred and noisy image; (c) image deblurred by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)


### 4.3. Motion Blur Experiment

In this experiment, the image “Barbara” is degraded by blurring kernel with55∗135 motion kernel and noise derivation σ=0.02. In TGVframe, we select α0=8e-4, α1=2e-3, and α2=8.2e-3; the parameters μ = 400 and β = 15 are selected in FTVd; the parameters α = 2e-4 and β = 1e-4 are selected in TVframe; the parameters α0=2e-3, α1=2e-3 are used in TGV. The restoration results by FTVd, TVframe, TGV, and TGVframe are shown in Figures 7(c)–7(f). It is not difficult to find that the visual effect of the restored image by our method is quite competitive with those restored by the other three methods. Several zoomed regions of Barbara image are shown in Figure 8. We can easily find that our method performs better than the other methods in detail texture. The values of SNR and SSIM are listed in Table 3. The values of SNR and SSIM of the recovered image by our method are superior to those by the other methods: FTVd, TVframe, and TGV. To further illustrate the effectiveness of our method, we also verify the effect of our method for image restoration with different noise under the same blurring kernel. The experimental results are shown in Table 4. We can see that our method is also applicable to different noises.Table 3
Restoration results of different methods under motion blur with noise levelσ=0.02.
FTVd TGV TVframe TGVframe Cameraman motion(25, 35) SNR 29.12 31.21 31.96 32.94 SSIM 0.6096 0.6794 0.7253 0.7824  Barbara motion(55, 135) SNR 26.16 27.52 28.28 31.24 SSIM 0.8978 0.9198 0.9279 0.9630Table 4
Restoration results of different methods under motion blurring kernel with size 55∗135 with different noise levels.
σ = 0.001 σ = 0.02 σ = 0.1 SNR SSIM SNR SSIM SNR SSIM Cameraman FTVd 29.52 0.7412 28.29 0.7230 25.67 0.6727 TGV 31.61 0.7836 31.58 0.7851 27.56 0.7231 TVframe 33.88 0.8549 32.20 0.8274 28.15 0.7315 TGVframe 35.23 0.9053 33.57 0.8611 29.39 0.7642  Lena FTVd 31.20 0.8335 30.23 0.8075 25.11 0.5813 TGV 32.69 0.8578 32.59 0.8424 26.75 0.6248 TVframe 33.23 0.8959 33.61 0.8879 27.56 0.6612 TGVframe 35.22 0.9451 34.21 0.9195 28.97 0.7221  House FTVd 31.77 0.7487 28.87 0.6097 27.97 0.5813 TGV 34.47 0.8080 31.26 0.6946 29.83 0.6348 TVframe 35.58 0.8391 32.31 0.7321 31.30 0.6712 TGVframe 36.64 0.8748 33.44 0.8085 32.26 0.7221  Barbara FTVd 29.67 0.9023 26.16 0.8978 25.35 0.8031 TGV 31.23 0.9225 27.52 0.9198 26.47 0.8521 TVframe 32.82 0.9381 28.28 0.9279 27.19 0.9067 TGVframe 34.97 0.9700 31.24 0.9621 29.16 0.9312Figure 7
Restoration results of the Barbara image with motion blur kernel. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 8
The zoomed regions of the results of Figure7. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)


### 4.4. Out-of-Focus Blur Experiment

In this experiment, the image “Lena”(256×256) is degraded by out-of-focus blur kernel with radius 9. In this subsection, we use the MATLAB function “fspecial“ to characterize the out-of-focus blur. Besides, we add the Gaussian white noise with σ=0.1 to the image “Lena.” The parameters in the out-of-focus blur test are set as follows: we select α0=6e-4, α1=3e-5, α2=3e-4 in our method; μ = 300, β = 15 in FTVd; α=6e-6, β=5e-5 in TVframe; α0=1e-3, α1=1e-2 in TGV. In Table 5, we summarize the image Lena restoration results of our method, FTVd, TGV, and TVframe; and the recovered images are shown in Figures 9(c)–9(f). We can see that our method can get higher SNRs and SSIMs and higher image visual quality. In addition, the zoomed parts of “Lena” are listed in Figure 10. As it is seen in Figures 10(c)–10(f), our method can restore more details than the other methods and can overcome the staircase effects.Table 5
Restoration results of different methods under out-of-focus blur with noise levelσ=0.1.
FTVd TGV TVframe TGVframe Lena SNR 22.15 22.92 23.53 24.11 SSIM 0.6153 0.6415 0.6841 0.7225  Cameraman SNR 21.28 21.91 22.65 23.17 SSIM 0.4567 0.4611 0.4903 0.5120  House SNR 25.63 26.30 26.95 27.49 SSIM 0.4523 0.4633 0.4701 0.5001  Barbara SNR 21.01 21.53 22.19 22.78 SSIM 0.6582 0.6667 0.6846 0.7104Figure 9
Restoration results of the Lena image with out-of-focus blur kernel. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 10
The zoomed regions of the results of Figure9. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)


### 4.5. The Experiment Compared with BM3D

In this subsection, we do an experiment compared with BM3D [61] which is a better deblurring and denoising method. In this test, we select the image “Cameraman” (256×256) which degraded by 25∗25 pixel Gaussian blur with two different standard deviations 1.6,2.5 and contaminated by Gaussian noise with standard variance σ=0.1. The results are shown in Figure 11 and the enlarged parts are shown in Figure 12. It is easy to see that our method achieves a better recovery effect than BM3D.Figure 11
The results compared with BM3D. (a) Blurred and noisy image; (b) image recovered by BM3D; (c) image recovered by our method; (d) blurred and noisy image; (e) image recovered by BM3D; (f) image recovered by our method.
(a)
Degenerated image with standard deviation 1.6 (b)
SNR = 23.06, SSIM = 0.5937 (c)
SNR = 23.51, SSIM= 0.6152 (d)
Degenerated image with standard deviation 2.5 (e)
SNR = 22.23, SSIM = 0.5586 (f)
SNR = 22.85, SSIM = 0.5845Figure 12
The zoomed regions of Figure11.
(a) (b) (c) (d) (e) (f)


## 4.1. Gaussian Blur Experiment

In this subsection, Gaussian blur experiment is considered. It is known that the size of blur kernel determines the quality of the recovered images to a certain extent. The larger the blur kernel is, the lower the recovered image quality is. In order to test the effectiveness of our method for different blur kernels, we do a test for the images with different size of blurring kernels. The values of SNR and SSIM by four different methods under different Gaussian blur kernels are shown in Table1. In this test, we select the well-known image “Cameraman” (256×256) which degraded by being blurred with a 9∗9 Gaussian kernel and contaminated by Gaussian noise with standard variance σ=0.1 as tested image displayed in Figure 2(b), and the recovered results are shown in Figure 2. In our method, we select parameters α0=5e-3, α1=3e-3, and α2=4.2e-3; μ = 300 and β = 10 in FTVd; α=5e-3, β=5e-5 in TVframe; and α0=1e-4, α1=2e-3 in TGV. Figure 3 shows the enlarged results of a part of Figure 2. From Figures 2 and 3 and Table 1, it is not difficult to see that our method still has good restoration when the size of blur kernel reached 11∗11, which shows that our method is robust to blur kernel. Meanwhile, our method can also better suppress the staircase effect. In order to show our proposed method is convergent, we also plot the changing curve of SNRs versus iteration numbers in Figure 4.Table 1
Restoration results of different methods under gaussian blur with noise levelσ=0.1.
FTVd TGV TVframe TGVframe Cameraman 7 ∗ 7 SNR 26.16 27.35 27.82 28.66 SSIM 0.6783 0.7005 0.7180 0.7591 9 ∗ 9 SNR 24.09 24.98 25.29 25.98 SSIM 0.6283 0.6421 0.6573 0.6807 11 ∗ 11 SNR 22.81 23.24 23.85 24.74 SSIM 0.5817 0.6066 0.6209 0.6794  Lena 7 ∗ 7 SNR 25.78 26.80 27.31 28.15 SSIM 0.8160 0.8300 0.8425 0.8667 9 ∗ 9 SNR 24.07 25.85 26.02 26.99 SSIM 0.7560 0.7819 0.7903 0.8262 11 ∗ 11 SNR 23.92 24.73 24.98 25.89 SSIM 0.7159 0.7430 0.7517 0.8009  House 7 ∗ 7 SNR 32.27 33.41 33.95 34.82 SSIM 0.6865 0.7206 0.7450 0.7765 9 ∗ 9 SNR 30.85 31.81 32.35 33.42 SSIM 0.6281 0.6425 0.6558 0.7174 11 ∗ 11 SNR 30.47 31.23 31.98 32.53 SSIM 0.5311 0.5751 0.5947 0.6578  Barbara 7 ∗ 7 SNR 23.21 24.32 25.16 26.03 SSIM 0.8068 0.8351 0.8538 0.8767 9 ∗ 9 SNR 22.01 23.08 23.73 24.53 SSIM 0.7690 0.7848 0.8003 0.8312 11 ∗ 11 SNR 21.33 22.41 23.13 23.62 SSIM 0.7250 0.7415 0.7762 0.8034Figure 2
Restoration results of the Cameraman image with Gaussian blur kernel. (a) Original image; (b) blurred and noisy image; (c) image deblurred by FTVd; (d) image deblurred by TGV; (e) image deblurred by TVframe; (f) image deblurred by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 3
The zoomed regions of the results of Figure2. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 4
The changes of SNRs versus iteration number for test images about Gaussian blur.
(a)
σ = 0.1, Gaussian(11, 11) (b)
σ = 0.02, Gaussian(9, 9)


## 4.2. Average Blur Experiment

In this experiment, we test the well-known image “house”(256×256) which has many texture details that help to show the effectiveness of our method. The “house” image is degraded by blurring kernel with 11∗11 average kernel and noise derivation σ=0.1; see Figure 5(b). The effect diagrams that are restored with FTVd, TVframe, TGV, and our method are shown in Figures 5(c)–5(f). In this test, the parameters α0=8e-4, α1=6e-4, and α2=8.2e-4 are used in TGVframe; in FTVd, μ=53 and λ=0.5; the parameters α=4e-6 and β=5e-6 are selected in TVframe; the parameters α0=6e-4, α1=1e-3 are used in TGV. The restoration results are shown in Figure 5. We have enlarged some details of the four restored images for a better visualization and the effects of the enlarged part are shown in Figure 6. As it is seen in the zoomed parts, our method outperforms the other three methods. The values of SNR and SSIM of the four images are listed in Table 2. From Figures 5 and 6 and Table 2, we can find that the restored images by FTVd, TGV, and TVframe have some piecewise constant regions, while our method does better and has better qualities with more details, textures, and less stair-casing effect.Table 2
Restoration results of different methods under average blur with noise levelσ=0.1.
FTVd TGV TVframe TGVframe Lena average(9) SNR 23.36 24.65 25.42 26.17 SSIM 0.7251 0.7538 0.7801 0.8038 average(11) SNR 22.37 23.62 24.26 25.59 SSIM 0.6824 0.7022 0.7609 0.7737  House average(9) SNR 30.40 31.78 32.36 33.23 SSIM 0.5743 0.6362 0.6440 0.6625 average(11) SNR 28.97 30.03 30.81 31.76 SSIM 0.4894 0.5273 0.5309 0.5828  Cameraman average(9) SNR 23.63 24.96 25.32 26.43 SSIM 0.6001 0.6423 0.6576 0.6956 average(11) SNR 22.55 23.48 24.25 24.59 SSIM 0.5510 0.5896 0.6114 0.6359  Barbara average(9) SNR 22.47 24.13 24.79 25.28 SSIM 0.7182 0.7801 0.8003 0.8331 average(11) SNR 21.27 23.00 23.87 24.53 SSIM 0.6966 0.7314 0.7523 0.7982Figure 5
Restoration results of the House image with average blur kernel. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 6
The zoomed regions of the results of Figure5. (a) Original image; (b) blurred and noisy image; (c) image deblurred by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)


## 4.3. Motion Blur Experiment

In this experiment, the image “Barbara” is degraded by blurring kernel with55∗135 motion kernel and noise derivation σ=0.02. In TGVframe, we select α0=8e-4, α1=2e-3, and α2=8.2e-3; the parameters μ = 400 and β = 15 are selected in FTVd; the parameters α = 2e-4 and β = 1e-4 are selected in TVframe; the parameters α0=2e-3, α1=2e-3 are used in TGV. The restoration results by FTVd, TVframe, TGV, and TGVframe are shown in Figures 7(c)–7(f). It is not difficult to find that the visual effect of the restored image by our method is quite competitive with those restored by the other three methods. Several zoomed regions of Barbara image are shown in Figure 8. We can easily find that our method performs better than the other methods in detail texture. The values of SNR and SSIM are listed in Table 3. The values of SNR and SSIM of the recovered image by our method are superior to those by the other methods: FTVd, TVframe, and TGV. To further illustrate the effectiveness of our method, we also verify the effect of our method for image restoration with different noise under the same blurring kernel. The experimental results are shown in Table 4. We can see that our method is also applicable to different noises.Table 3
Restoration results of different methods under motion blur with noise levelσ=0.02.
FTVd TGV TVframe TGVframe Cameraman motion(25, 35) SNR 29.12 31.21 31.96 32.94 SSIM 0.6096 0.6794 0.7253 0.7824  Barbara motion(55, 135) SNR 26.16 27.52 28.28 31.24 SSIM 0.8978 0.9198 0.9279 0.9630Table 4
Restoration results of different methods under motion blurring kernel with size 55∗135 with different noise levels.
σ = 0.001 σ = 0.02 σ = 0.1 SNR SSIM SNR SSIM SNR SSIM Cameraman FTVd 29.52 0.7412 28.29 0.7230 25.67 0.6727 TGV 31.61 0.7836 31.58 0.7851 27.56 0.7231 TVframe 33.88 0.8549 32.20 0.8274 28.15 0.7315 TGVframe 35.23 0.9053 33.57 0.8611 29.39 0.7642  Lena FTVd 31.20 0.8335 30.23 0.8075 25.11 0.5813 TGV 32.69 0.8578 32.59 0.8424 26.75 0.6248 TVframe 33.23 0.8959 33.61 0.8879 27.56 0.6612 TGVframe 35.22 0.9451 34.21 0.9195 28.97 0.7221  House FTVd 31.77 0.7487 28.87 0.6097 27.97 0.5813 TGV 34.47 0.8080 31.26 0.6946 29.83 0.6348 TVframe 35.58 0.8391 32.31 0.7321 31.30 0.6712 TGVframe 36.64 0.8748 33.44 0.8085 32.26 0.7221  Barbara FTVd 29.67 0.9023 26.16 0.8978 25.35 0.8031 TGV 31.23 0.9225 27.52 0.9198 26.47 0.8521 TVframe 32.82 0.9381 28.28 0.9279 27.19 0.9067 TGVframe 34.97 0.9700 31.24 0.9621 29.16 0.9312Figure 7
Restoration results of the Barbara image with motion blur kernel. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 8
The zoomed regions of the results of Figure7. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)


## 4.4. Out-of-Focus Blur Experiment

In this experiment, the image “Lena”(256×256) is degraded by out-of-focus blur kernel with radius 9. In this subsection, we use the MATLAB function “fspecial“ to characterize the out-of-focus blur. Besides, we add the Gaussian white noise with σ=0.1 to the image “Lena.” The parameters in the out-of-focus blur test are set as follows: we select α0=6e-4, α1=3e-5, α2=3e-4 in our method; μ = 300, β = 15 in FTVd; α=6e-6, β=5e-5 in TVframe; α0=1e-3, α1=1e-2 in TGV. In Table 5, we summarize the image Lena restoration results of our method, FTVd, TGV, and TVframe; and the recovered images are shown in Figures 9(c)–9(f). We can see that our method can get higher SNRs and SSIMs and higher image visual quality. In addition, the zoomed parts of “Lena” are listed in Figure 10. As it is seen in Figures 10(c)–10(f), our method can restore more details than the other methods and can overcome the staircase effects.Table 5
Restoration results of different methods under out-of-focus blur with noise levelσ=0.1.
FTVd TGV TVframe TGVframe Lena SNR 22.15 22.92 23.53 24.11 SSIM 0.6153 0.6415 0.6841 0.7225  Cameraman SNR 21.28 21.91 22.65 23.17 SSIM 0.4567 0.4611 0.4903 0.5120  House SNR 25.63 26.30 26.95 27.49 SSIM 0.4523 0.4633 0.4701 0.5001  Barbara SNR 21.01 21.53 22.19 22.78 SSIM 0.6582 0.6667 0.6846 0.7104Figure 9
Restoration results of the Lena image with out-of-focus blur kernel. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)Figure 10
The zoomed regions of the results of Figure9. (a) Original image; (b) blurred and noisy image; (c) image recovered by FTVd; (d) image recovered by TGV; (e) image recovered by TVframe; (f) image recovered by TGVframe.
(a) (b) (c) (d) (e) (f)


## 4.5. The Experiment Compared with BM3D

In this subsection, we do an experiment compared with BM3D [61] which is a better deblurring and denoising method. In this test, we select the image “Cameraman” (256×256) which degraded by 25∗25 pixel Gaussian blur with two different standard deviations 1.6,2.5 and contaminated by Gaussian noise with standard variance σ=0.1. The results are shown in Figure 11 and the enlarged parts are shown in Figure 12. It is easy to see that our method achieves a better recovery effect than BM3D.Figure 11
The results compared with BM3D. (a) Blurred and noisy image; (b) image recovered by BM3D; (c) image recovered by our method; (d) blurred and noisy image; (e) image recovered by BM3D; (f) image recovered by our method.
(a)
Degenerated image with standard deviation 1.6 (b)
SNR = 23.06, SSIM = 0.5937 (c)
SNR = 23.51, SSIM= 0.6152 (d)
Degenerated image with standard deviation 2.5 (e)
SNR = 22.23, SSIM = 0.5586 (f)
SNR = 22.85, SSIM = 0.5845Figure 12
The zoomed regions of Figure11.
(a) (b) (c) (d) (e) (f)


## 5. Conclusion

In this paper, we propose an effective image restoration model based on total generalized variation and wavelet frame. In addition, we employ the alternating direction method of multipliers to solve it. Experimental results demonstrate that our method not only has better visual resolution, but also can suppress the staircase effect compared to some current state-of-the-art methods.



---

*Source: 3650128-2019-03-06.xml*