# Superresolution of Images Using Area Preserving Geometric Evolution Laws

Authors: Caroline Jäger; Simon Praetorius; Axel Voigt
Journal: ISRN Computer Graphics
 (2012)

---

## Abstract

PDE-based approaches are used to obtain superresolution of images. Using level set method for area constraint geometric evolution laws allows to remove pixelation in images without removing small-scale features. We compare mean curvature flow, surface diffusion, and Willmore flow for this purpose.


---

## Body

## 1. Introduction

The goal of superresolution is to construct out of a given low-resolution image an image with higher resolution. Superresolution is a common problem in image processing and numerous algorithms have been proposed [1]. Most of these algorithms are based on interpolation and lead to an improvement of the image but also to imperfect reconstructions with false high-frequency components. For interpolation methods based on PDEs, which are mostly adapted from inpainting problems, we refer to [2–6]. These approaches try to reconstruct the geometric properties of the images by evolving their level curves. In order to maintain fidelity to the original low-resolution image, different approaches are used. A standard level set method to modify pixel intensity which is based on evolution of mean curvature flow ∂tϕ=H∥∇ϕ∥, where ϕ is the intensity and H the mean curvature, as used for example, in [7, 8] for edge-preserving smoothing, noise removal, and other image enhancement, smoothes away feature objects, as all level curves are shortening and thus will not lead to appropriate superresolution images. In [6], this is circumvented by imposing constraints that preserve accuracy to the original image using unchanging anchor pixels defining explicit topology constraints. In [4], the problem is attacked from a different point of view, by constraining the mean curvature flow problem in order to preserve the area enclosed by each level curve. We will follow this approach and extend it to other geometric evolution problems of higher order. Higher-order PDEs have been applied to various problems in image processing, for example, [9–11]. They are, however, not yet been used for superresolution. Only in [12] a higher-order model based on a Cahn-Hilliard equation is applied to superresolution of binary images, especially text.The paper is organized as follows: in Section2, we describe a level set formulation for area conserving mean curvature flow, surface diffusion, and Willmore flow. In Section 3, we discuss an appropriate discretization which conserves all level curves simultaneously. In Section 4, the algorithms are applied to test problems. Thereby we start from a given 256×256 image, lower the resolution, and use bicubic interpolation to create an image that is again 256×256. Our three area preserving evolution laws are applied to this image for a short time and the results are compared with the original image. We also apply the algorithms to text images which are only known at a low resolution. In Section 5, we draw conclusions.


## 2. Level Set Method

For a given initial curveℳ0, a geometric evolution law defines a family of curves ℳ(t), t≥0 with ℳ(0)=ℳ0. If ℳ(t) is described implicitly as a specific level curve of a function ϕ(t), we can evolve ℳ(t) by evolving ϕ(t) by solving ∂tϕ+v∥∇ϕ∥=0, with v a given normal velocity. If v is determined through a geometric evolution law, we can implicitly evolve the curve ℳ(t) by solving this level set equation [13]. To simultaneously evolve all level curves ℳc of a level set function ϕ, we can define a global energy
(1)E[ϕ]=∫Re[Mc]dc=∫Ω‖∇ϕ‖fdx,
using the coarea formula. Thereby, we consider
(2)e[Mc]=∫Mcfds
with f a given energy density on the curve ℳc and an appropriate gradient flow with respect to a specific metric g(ℳc)(3)v=-gradg(Mc)e[Mc].
Following [14], we interpret ϕ as an element of the manifold ℒ of level set ensembles and identify a tangent vector s=∂tϕ on ℒ with a motion velocity v of the corresponding level curve ℳc via the classical level set equation s+v∥∇ϕ∥=0. This allows to define the corresponding metric on ℒ which reads for the L2 metric(4)gϕL(s1,s2)=∫R∫Mcv1⋅v2dsdc=∫Ωs1‖∇ϕ‖s2‖∇ϕ‖‖∇ϕ‖dx=∫Ωs1s2‖∇ϕ‖-1dx
and for the H-1 metric(5)gϕH-1(s1,s2)=∫R∫Mcv1⋅v2dsdc=∫Ω(ΔM)-1(s1‖∇ϕ‖)s2‖∇ϕ‖‖∇ϕ‖dx=∫Ωs1s2‖∇ϕ‖-1dx.
For both cases, we are now able to rewrite the simultaneous gradient flow of all level curves in terms of the level set functionϕ(6)∂tϕ=-gradgϕE[ϕ],
which is equivalent to(7)gϕ(∂tϕ,η)=-(E′[ϕ],η)
for all functions η∈C0∞(Ω). From (7) the weak form can be obtained for various geometric evolution laws in level set form by choosing appropriate energy density f, see Table 1 (e.g., [14, 15]).Table 1
Construction of geometric evolution laws in level set form, withh the mean curvature.
Mean curvature flowf=1gϕ = gϕL2Surface diffusionf=1gϕ = gϕH-1Willmore flowf=h2gϕ = gϕL2To define area preserving versions of these geometric evolutions laws, we introduce a Lagrange multiplierλc and define the Lagrangien
(8)l[Mc,λc]=e[Mc]-λc(area(t)-area(0))
with
(9)area(t)=∫Ωc(t)dx
the area of the interior domain Ωc inside of ℳc. We thus obtain
(10)v=-gradg(Mc)l[Mc,λc]=-gradg(Mc)e[Mc]+λc
with λc uniquely determined by the constraint ∫ℳc∂txds=0:
(11)λc=∫Mcgradg(Mc)e[Mc]ds∫Mcds.
The corresponding evolution laws read as follows:(i)
area preserving mean curvature flow(12)v=h+λc,λc=∫Mchds∫Mcds,(ii)
surface diffusion (which is already area preserving)(13)v=ΔMh,
with Δℳ the surface Laplacian;(iii)
area preserving Willmore flow(14)v=ΔMh+h(‖S‖22-12h2)+λc,λc=-∫McΔMh+h(‖S‖22-(1/2)h2)ds∫Mcds,
thereby S denotes the shape operator on ℳc and ∥·∥2 the Frobenius norm.To construct a level set formulation to simultaneously evolve all level curvesℳc under the constraint, that their enclosed area is preserved, we follow the same lines as for the unconstraint problem and define
(15)L[ϕ,λ]=∫Rl[Mc,λc]dc
to obtain
(16)∂tϕ=-gradgϕL[ϕ,λ]
or equivalently(17)gϕ(∂tϕ,η)=-(L′[ϕ,λ],η)
for all functions η∈C0∞(Ω) and L′=Lϕ. For area preserving mean curvature flow (f=1 and gϕ=gϕL2), we thus obtain(18)∫Ω∂tϕ‖∇ϕ‖ηdx=∫Ωhη-ληdx,∫Ωhηdx=-∫Ω∇ϕ‖∇ϕ‖⋅∇ηdx,λ(x)=∫Ωδ(ϕ(x′)-ϕ(x))h(x′)‖∇ϕ(x′)‖dx′∫Ωδ(ϕ(x′)-ϕ(x))‖∇ϕ(x′)‖dx′.
The same formulation can be obtained by rewriting the formulation in [4] in weak form. For surface diffusion (f=1 and gϕ=gϕH-1), we obtain(19)∫Ω∂tϕηdx=∫ΩP∇h⋅∇η‖∇ϕ‖dx,∫Ωhηdx=-∫Ω∇ϕ‖∇ϕ‖⋅∇ηdx,
with P=I-∇ϕ/∥∇ϕ∥⊗∇ϕ/∥∇ϕ∥ the projection onto the tangent space 𝒯xℳc defined for every x on ℳc. For area preserving Willmore flow (f=h2 and gϕ=gϕL2), we obtain(20)∫Ω∂tϕ‖∇ϕ‖ηdx=∫Ωμη-ληdx,∫Ωμηdx=-∫Ωω22‖∇ϕ‖3∇ϕ⋅∇η-P∇ω‖∇ϕ‖⋅∇ηdx,∫Ωω‖∇ϕ‖ηdx=∫Ω∇ϕ‖∇ϕ‖⋅∇ηdx,λ(x)=∫Ωδ(ϕ(x′)-ϕ(x))μ(x′)‖∇ϕ(x′)‖dx′∫Ωδ(ϕ(x′)-ϕ(x))‖∇ϕ(x′)‖dx′,
with ω=-∥∇ϕ∥h a weighted curvature.


## 3. Semi-Implicit Finite Element Discretization

We discretize the constraint evolution problems first in space using piecewise linear finite elements and then in time based on a semi-implicit backward Euler scheme. We thereby follow the discretizations of the unconstrained problems [11, 14] and treat the Lagrange multiplier explicit by evaluating
(21)λc=∫Ωδ(ϕ(x′)-c)α(x′)‖∇ϕ(x′)‖dx′∫Ωδ(ϕ(x′)-c)‖∇ϕ(x′)‖dx′
(with α=h,μ) for different discrete values of c and then interpolating to obtain λ, see [4]. The algorithms are implemented in the adaptive finite element toolbox AMDiS [16].We first demonstrate the smoothing and area preserving properties of the algorithms. We, therefore, evolve a perturbed circle. Thereby the radius ofℳ0 is initially defined as r=r0(1+epscos(freqθ)) with angle θ∈[0,2π). Figure 1 shows the initial curve and the evolution of its averaged curvature over time towards the constant value h=1/r0. We here only use the constraint for ℳ0 and let all other level lines evolve accordingly, thus only λ0 is computed and λ=λ0. For all cases, the area remains constant; however, the smoothing differes from which a different behaviour if applied to images is expected.Figure 1
Area preserving evolution laws for perturbed circle withr0=0.5, eps=0.075, and freq=16.0. Different time steps are used for the evolution laws.


## 4. Numerical Results

We start with a simple application, namely, the superresolution of text images. Therefor, we use the logo of our university and apply area preserving Willmore flow, mean curvature flow, and surface diffusion to smooth the jagged parts of the text given at a low resolution. Figure2 shows the results and the improvement if compared with bicubic interpolation. The differences obtained with the three evolution laws are minor.Superresolution of text (a) bicubic interpolation, (b) Willmore flow, (c) mean curvature flow, (d) surfaces diffusion, (e) zoom from left to right: bicubic interpolation, Willmore flowm mean curvature flow, and surface diffusion. The evolution results are shown after 10 time steps. The number of time steps used results from our individual taste for the best result.
(a)(b)(c)(d)(e)The described algorithms can be extended to color images by applying them separately to the individual color planes. As already mentioned in [6], the possible separation of such an approach, due to the independent evolution, is not of significant for the obtained results.We consider two images, the “Lena’’ image and the “Monarch’’ image. For both images, originally given with a resolution256×256, the resolution is reduced to 75×75 and bicubic interpolations is used to produce again a 256×256 image. We apply our three level set algorithms to the resulting images. Figure 3 shows results on the “Lena’’ image.“Lena’’ image from left to right: (a) bicubic interpolation, (b) Willmore flow, (c) mean curvature flow, (d) surfaces diffusion. The evolution results are shown after 20 time steps in the case of Willmore flow and surface diffusion and 8 time steps for mean curvature flow. The number of time steps used results from our individual taste for the best result.
(a)(b)(c)(d)Applying the same conserved evolution laws to the “Monarch’’ image results in Figure4.“Monarch’’ image from left to right: (a) bicubic interpolation, (b) Willmore flow, (c) mean curvature flow, (d) surfaces diffusion. The evolution results are shown after 16 time steps in the case of Willmore flow and surface diffusion and 6 time steps for mean curvature flow. The number of time steps used results from our individual taste for the best result.
(a)(b)(c)(d)In both cases, all area conserved evolution laws result in smoother contours, which is directly visible at the shoulder and the hat in the “Lena’’ image and along the white stripes in the “Monarch’’ image. The jagged parts remaining after bicubic interpolations are removed. Without the area constraint, the evolution would oversmooth the images and significant detail would be lost. The applied constraints help to maintain these details. The difference in the three evolution laws is most significant for mean curvature flow, which reduces the sharpness of the edges. The results for Willmore flow and surface diffusion are comparable.


## 5. Conclusion

We consider PDE bases approaches to reconstruct geometric properties of images by evolving their level curves. Applying these level set reconstructions to obtain superresolution images had already been shown to lead to improved results, as jagged parts of the image can be smoothed. However, unconstrained evolutions will lead to oversmoothing. The goal thus is to constrain the evolution laws such that visually significant properties remain. A simple attempt in this direction, which does not require image-specific treatments is to constrain the area of each level curve. We have shown that such constrained geometric evolution laws can improve results of other methods for image magnification. The resulting PDEs turn to become nonlocal which requires special treatment. For all cases of area conserved mean curvature flow, surface diffusion and area conserved Willmore flow we use a semi-implicit discretization in which the Lagrange multiplier is treated explicitly which drastically reduced the complexity of the discretization scheme of the higher-order PDEs. A comparison of the three evolution laws shows an improvement if fourth-order PDEs are used. Small-scale details seem to better remain than for the second-order PDE. The difference between area conserved Willmore flow and surface diffusion is only marginally in the considered examples. As surface diffusion is already area conserving and thus computationally less expensive, it should be the method of choice.



---

*Source: 846980-2011-12-15.xml*