# Compressed Sensing, Pseudodictionary-Based, Superresolution Reconstruction

Authors: Chun-mei Li; Ka-zhong Deng; Jiu-yun Sun; Hui Wang
Journal: Journal of Sensors
 (2016)

---

## Abstract

The spatial resolution of digital images is the critical factor that affects photogrammetry precision. Single-frame, superresolution, image reconstruction is a typical underdetermined, inverse problem. To solve this type of problem, a compressive, sensing, pseudodictionary-based, superresolution reconstruction method is proposed in this study. The proposed method achieves pseudodictionary learning with an available low-resolution image and uses theK-SVD algorithm, which is based on the sparse characteristics of the digital image. Then, the sparse representation coefficient of the low-resolution image is obtained by solving the norm of l
0 minimization problem, and the sparse coefficient and high-resolution pseudodictionary are used to reconstruct image tiles with high resolution. Finally, single-frame-image superresolution reconstruction is achieved. The proposed method is applied to photogrammetric images, and the experimental results indicate that the proposed method effectively increase image resolution, increase image information content, and achieve superresolution reconstruction. The reconstructed results are better than those obtained from traditional interpolation methods in aspect of visual effects and quantitative indicators.


---

## Body

## 1. Introduction

High-quality images are required for high-precision measurements, and their spatial resolution is the key indicator in the assessment of digital image quality; an image with a higher spatial resolution contains more information content. The level of image detail has a decisive role in absolute photogrammetry precision and the range of applications for which a given image can be used. However, limited by flight condition and cost as well as optical diffraction, certain difficulties exist in directly acquiring high-resolution images. To satisfy the demand of image resolution using single-frame-photogrammetry of local geospatial data, superresolution image reconstruction was investigated in this study.There are two types of single-frame-image, superresolution reconstruction methods: interpolation and learning. Traditional interpolation methods achieve high resolutions using an interpolation kernel function and include methods such as the nearest neighbor, bilinear interpolation, bicubic interpolation, and spline function methods. Scholars later proposed edge-oriented interpolation methods. For example, Jensen and Anastassiou [1] improved visual effects by template-fitting after image edge detection; Carey et al. [2] proposed a wavelet-based interpolation method; and Chang et al. [3] used a wavelet transform to describe edge points, infer extreme point coefficients at finer scales, and reconstruct high-resolution images. Li and Orchard [4] also proposed an edge-guide interpolation algorithm based on the least squares method, where the edge feature of an interpolated image was maintained based on the edge-guide feature of the adaptive covariance. Zhou et al. [5] proposed a multisurface fitting-based interpolation algorithm that could increase the edge clarity of an interpolated image; that algorithm had good robustness against noise. The interpolation method is simple and applicable to many situations; however, it lacks prior information, can only enhance image visual effects, and typically has difficulty in recovering high-frequency information that is lost in low-resolution images, producing blurred reconstructed images.Learning-based superresolution reconstruction is achieved by establishing corresponding relationships between high- and low-resolution images using prior information. The example-based superresolution method proposed by Freeman et al. [6] obtained prior information between high- and low-resolution images via Markov random field learning; this type of method achieved excellent reconstruction results in human-facial [7] and word-processing applications. The sparse-representation-based SR algorithm proposed by Yang et al. [8] used the sparse characteristics of natural images and linear programing to solve for the sparse representation of the low-resolution images and then combined the obtained representation coefficient with a high-resolution dictionary to create high-resolution image tiles. Dong et al. [9] proposed an image interpolation algorithm based on a nonlocal autoregression model under the framework of sparse representation. This method used similar image blocks that were common in natural images to construct a nonlocal autoregression model; it then used self-similarity in the image structure as additional information to reconstruct high-resolution images.The premise of the learning method is to obtain prior knowledge of the high-resolution images. Studies typically start with existing images, produce corresponding low-resolution images via blurring and downsampling, and then use a reconstruction algorithm to approximate the original images. The reconstructed images typically do not exhibit substantial improvements in image quality or resolution. Although the amount of information in the interpolated, superresolution, image reconstruction did not increase, the resolution can be increased markedly. Due to the limitations of traditional reconstruction methods, this study uses the local, geometrically consistent, spatial relationship between the interpolated and original images; organically combines the interpolation and learning methods; and proposes a compressed sensing, pseudodictionary-based, superresolution reconstruction method.


## 2. Method and Model

The proposed method treats an existing image as a low-resolution image, uses bicubic interpolation images as guides, and creates a joint dictionary for image training based on the sparse characteristics of digital images by requiring that the high- and low-resolution images have the same sparse representation in the corresponding high- and low-resolution dictionary. Then, the related rule between the high- and low-resolution images is applied to the bicubic interpolation image, and the optimization problem is solved using compressed sensing theory. The proposed method can effectively retrieve more detailed high-frequency information while producing a superresolution image.


### 2.1. Training Sample Extraction and Pseudodictionary Learning

The goal of the superresolution reconstruction of a single-frame image is to restore a given low-resolution image to a high-resolution image of the same scenery. To resolve this ill-posed problem, the requirement of a consistent observation model for high- and low-resolution images must be satisfied [10]. Based on the digital imaging mechanism, the low-resolution image can be obtained by sampling the high-resolution image, and the observation model can be expressed as follows:(1)
y
l
i
=
B
s
y
h
i
+
v
=
L
y
h
i
+
v
,where B and s describe the blurring and downsampling operations on the image, respectively; v is the Gaussian noise; y
h
i and y
l
i are the high- and low-resolution images, respectively; and the approximation image y
^
h
i of the original high-resolution image can be reconstructed by processing y
l
i. To avoid complexity induced by the differences in the high- and low-resolutions of the images, the low-resolution image is upsampled to the same dimension as the high-resolution image via interpolation, thus obtaining high- and low-resolution image sample sets y
d
l
i and y
h
i, respectively.The preprocessing of the samples is first performed after the construction of the training samples. Because the images’ structural characteristics are primarily represented in the high-frequency range, the image interpolation valuee
h
i
=
y
h
i
-
y
d
l
i is used as the high-frequency information of the high-resolution image. The preprocessing of the low-resolution image is performed by a convolution integral of degree K on the geometric structure components; the characteristic vector components are combined to form the characteristic vector of the low-resolution image. Finally, the characteristic vectors of the high- and low-resolution images at the same position are formed into a training sample data pair P
=
p
l
k
,
p
h
k.The compressed sensing, pseudodictionary-based, superresolution reconstruction problem requires the simultaneous use of an over-defined dictionary pairA
h and A
l of the high- and low-resolution image tiles. Because the pair belongs to a heterogeneous data type, the following equation must be solved for the training data to have the same sparse representation in both dictionaries [11]:(2)
m
i
n
A
l
,
A
h
,
S
∑
i
y
l
-
A
l
s
i
2
2
+
λ
0
y
h
-
A
h
s
i
2
2
+
λ
s
i
0
,where A
h and A
l are the high- and low-resolution image tile dictionaries, respectively; s
i is the sparse coefficient that simultaneously satisfies y
h in dictionary A
h and y
l in dictionary A
l; and λ
0 and λ are the regularization parameters of the second and third terms, respectively. The joint dictionary receives the correlation between the high- and low-resolution training samples in full consideration, and learning is required to use prior knowledge of the high-resolution image to effectively guide the superresolution reconstruction of other low-resolution images. This study uses the K-SVD method [12] to solve an over-defined dictionary.


### 2.2. Mathematical Model of Compressed Sensing

The other key element in superresolution reconstruction is to reconstruct the low-resolution dictionary’s sparse representation with respect to the low-resolution image tiles. Compressed sensing is a new theory in the field of signal processing.N dimensional signal X
∈
R
n can be expressed by the superposition of a set of orthonormal basis as follows [13, 14]:(3)
X
=
Ψ
S
.If S has only k nonzero elements or k elements that are far greater than other points, then the signal X is called k sparse signal, where k
≤
N, and Ψ is the sparse domain of X. The sampling process of compressed sensing is achieved using a sensing matrix Φ that is not correlated to the basis [13]:(4)
Y
=
Φ
X
=
Φ
Ψ
S
=
A
S
,where Y
=
(
Y
1
,
Y
2
,
…
,
Y
M
)
T is an M-dimensional sample matrix, M
≪
N, and A
=
[
A
1
,
A
2
,
…
,
A
M
] is the sensing matrix, which is also called the sparse dictionary.Signal reconstruction is the core of compressed sensing. For ak sparse signal X, the sparse coefficient S can be reversely derived with a high probability from the M-dimensional observation value Y and the k-sparsity constraint condition when the sensing matrix satisfies the isometric condition. The original signal X can then be restored with a high probability based on the reversibility of the orthogonal transformation. Thus the signal reconstruction problem can be considered as a search for the sparsest solution to the underdetermined equation set Y
=
A
S.


### 2.3. Flow of the Proposed Superresolution Reconstruction Algorithm

This study uses the bicubic interpolation image of an existing image as the high-resolution imageZ
h. Then, Z
h is processed by the same blurring and downsampling as in the training stage, to obtain the corresponding low-resolution Z
l. Based on the detailed characteristics of the bicubic interpolation that remain unchanged in spatial location, the pseudodictionary obtained in the training stage is directly applied to the higher level bicubic interpolation image. The superresolution reconstruction problem can be described as the problem of solving for the optimum approximation z
^
c of the superresolution image z
c with the given Z
l. After completing the high- and low-resolution pseudodictionaries, the superresolution reconstruction process can be summarized as follows:(1) Use the interpolation operator to perform a bicubic interpolation ofZ
l, producing Z
d
l, which has the same resolution as Z
c.(2) Use the same method as that used in image training to decompose the image into tiles, producingP
=
p
z
l
k
,
p
z
h
k.(3) Use the image tile sparsity as prior information. Because the natural information is compressible, the superresolution construction of the compressed sensing can be solved usingl
0 norm optimization. The mathematical model of this process is shown as follows:(5)
m
i
n
q
0
s.t.
p
z
l
k
=
A
l
q
k
.However, this is NP-Hard problem, and the l
0 norm and l
1 norm have equivalency under certain conditions [15, 16]. Thus, the above equation can be converted into the convex optimization problem under a minimum l
1 norm:(6)
m
i
n
q
1
s.t.
p
z
l
k
=
A
l
q
k
.This study estimates the sparse representation coefficientq
k of the low-resolution characteristic block p
z
l
k with respect of the pseudodictionary A
l via the optimal orthogonal matching pursuit (OMP) algorithm for the l
0 norm.(4) Based on the same sparse representation in the high- and low-resolution images in their respective sparse pseudodictionaries, their superresolution image tiles are reconstructed byp
^
z
h
k
=
A
h
q
k, and the superresolution image z
^
c
=
p
^
z
h
k
k
=
1
K is then created.


### 2.4. Quality Evaluation Method of Reconstruction Results

This study evaluates the superresolution reconstruction results from both subjective and objective perspectives. Image information entropy (InEn) and average contrast (AC) are used as the objective quality evaluation index (NR-IQA) [17] for the superresolution reconstruction results of a no-reference image:(7)
InEn
=
-
∑
i
=
0
255
p
i
l
o
g
2
⁡
p
i
,
AC
⁡
=
1
2
A
C
x
2
+
A
C
y
2
,where p
(
i
) represents the distribution probability of the ith gray/color scale; A
C
x and A
C
y are the image average contrasts in the x- and y-directions, respectively; and A
C
x can be calculated by the following equation:(8)
A
C
x
=
1
M
-
1
N
-
1
∑
y
=
0
N
-
2
∑
x
=
0
M
-
2
Gray
x
,
y
-
Gray
x
+
1
,
y
,where Gray
(
x
,
y
) is the gray-scale of pixel (x
,
y) and M and N are the number of pixels in the x- and y-directions, respectively. The calculation method for A
C
y is the same as that for A
C
x.Whenp
(
i
)
=
0, let l
o
g
2
p
(
i
)
=
0, where InEn
∈
[
0,8
]. The information entropy and the average contrast of the color image can be obtained from the three-channel information entropy normalization. The information entropy is a measure of the information richness contained in the image; a higher information entropy indicates richer information contained in the image, and a higher average contrast indicates a clearer image.


## 2.1. Training Sample Extraction and Pseudodictionary Learning

The goal of the superresolution reconstruction of a single-frame image is to restore a given low-resolution image to a high-resolution image of the same scenery. To resolve this ill-posed problem, the requirement of a consistent observation model for high- and low-resolution images must be satisfied [10]. Based on the digital imaging mechanism, the low-resolution image can be obtained by sampling the high-resolution image, and the observation model can be expressed as follows:(1)
y
l
i
=
B
s
y
h
i
+
v
=
L
y
h
i
+
v
,where B and s describe the blurring and downsampling operations on the image, respectively; v is the Gaussian noise; y
h
i and y
l
i are the high- and low-resolution images, respectively; and the approximation image y
^
h
i of the original high-resolution image can be reconstructed by processing y
l
i. To avoid complexity induced by the differences in the high- and low-resolutions of the images, the low-resolution image is upsampled to the same dimension as the high-resolution image via interpolation, thus obtaining high- and low-resolution image sample sets y
d
l
i and y
h
i, respectively.The preprocessing of the samples is first performed after the construction of the training samples. Because the images’ structural characteristics are primarily represented in the high-frequency range, the image interpolation valuee
h
i
=
y
h
i
-
y
d
l
i is used as the high-frequency information of the high-resolution image. The preprocessing of the low-resolution image is performed by a convolution integral of degree K on the geometric structure components; the characteristic vector components are combined to form the characteristic vector of the low-resolution image. Finally, the characteristic vectors of the high- and low-resolution images at the same position are formed into a training sample data pair P
=
p
l
k
,
p
h
k.The compressed sensing, pseudodictionary-based, superresolution reconstruction problem requires the simultaneous use of an over-defined dictionary pairA
h and A
l of the high- and low-resolution image tiles. Because the pair belongs to a heterogeneous data type, the following equation must be solved for the training data to have the same sparse representation in both dictionaries [11]:(2)
m
i
n
A
l
,
A
h
,
S
∑
i
y
l
-
A
l
s
i
2
2
+
λ
0
y
h
-
A
h
s
i
2
2
+
λ
s
i
0
,where A
h and A
l are the high- and low-resolution image tile dictionaries, respectively; s
i is the sparse coefficient that simultaneously satisfies y
h in dictionary A
h and y
l in dictionary A
l; and λ
0 and λ are the regularization parameters of the second and third terms, respectively. The joint dictionary receives the correlation between the high- and low-resolution training samples in full consideration, and learning is required to use prior knowledge of the high-resolution image to effectively guide the superresolution reconstruction of other low-resolution images. This study uses the K-SVD method [12] to solve an over-defined dictionary.


## 2.2. Mathematical Model of Compressed Sensing

The other key element in superresolution reconstruction is to reconstruct the low-resolution dictionary’s sparse representation with respect to the low-resolution image tiles. Compressed sensing is a new theory in the field of signal processing.N dimensional signal X
∈
R
n can be expressed by the superposition of a set of orthonormal basis as follows [13, 14]:(3)
X
=
Ψ
S
.If S has only k nonzero elements or k elements that are far greater than other points, then the signal X is called k sparse signal, where k
≤
N, and Ψ is the sparse domain of X. The sampling process of compressed sensing is achieved using a sensing matrix Φ that is not correlated to the basis [13]:(4)
Y
=
Φ
X
=
Φ
Ψ
S
=
A
S
,where Y
=
(
Y
1
,
Y
2
,
…
,
Y
M
)
T is an M-dimensional sample matrix, M
≪
N, and A
=
[
A
1
,
A
2
,
…
,
A
M
] is the sensing matrix, which is also called the sparse dictionary.Signal reconstruction is the core of compressed sensing. For ak sparse signal X, the sparse coefficient S can be reversely derived with a high probability from the M-dimensional observation value Y and the k-sparsity constraint condition when the sensing matrix satisfies the isometric condition. The original signal X can then be restored with a high probability based on the reversibility of the orthogonal transformation. Thus the signal reconstruction problem can be considered as a search for the sparsest solution to the underdetermined equation set Y
=
A
S.


## 2.3. Flow of the Proposed Superresolution Reconstruction Algorithm

This study uses the bicubic interpolation image of an existing image as the high-resolution imageZ
h. Then, Z
h is processed by the same blurring and downsampling as in the training stage, to obtain the corresponding low-resolution Z
l. Based on the detailed characteristics of the bicubic interpolation that remain unchanged in spatial location, the pseudodictionary obtained in the training stage is directly applied to the higher level bicubic interpolation image. The superresolution reconstruction problem can be described as the problem of solving for the optimum approximation z
^
c of the superresolution image z
c with the given Z
l. After completing the high- and low-resolution pseudodictionaries, the superresolution reconstruction process can be summarized as follows:(1) Use the interpolation operator to perform a bicubic interpolation ofZ
l, producing Z
d
l, which has the same resolution as Z
c.(2) Use the same method as that used in image training to decompose the image into tiles, producingP
=
p
z
l
k
,
p
z
h
k.(3) Use the image tile sparsity as prior information. Because the natural information is compressible, the superresolution construction of the compressed sensing can be solved usingl
0 norm optimization. The mathematical model of this process is shown as follows:(5)
m
i
n
q
0
s.t.
p
z
l
k
=
A
l
q
k
.However, this is NP-Hard problem, and the l
0 norm and l
1 norm have equivalency under certain conditions [15, 16]. Thus, the above equation can be converted into the convex optimization problem under a minimum l
1 norm:(6)
m
i
n
q
1
s.t.
p
z
l
k
=
A
l
q
k
.This study estimates the sparse representation coefficientq
k of the low-resolution characteristic block p
z
l
k with respect of the pseudodictionary A
l via the optimal orthogonal matching pursuit (OMP) algorithm for the l
0 norm.(4) Based on the same sparse representation in the high- and low-resolution images in their respective sparse pseudodictionaries, their superresolution image tiles are reconstructed byp
^
z
h
k
=
A
h
q
k, and the superresolution image z
^
c
=
p
^
z
h
k
k
=
1
K is then created.


## 2.4. Quality Evaluation Method of Reconstruction Results

This study evaluates the superresolution reconstruction results from both subjective and objective perspectives. Image information entropy (InEn) and average contrast (AC) are used as the objective quality evaluation index (NR-IQA) [17] for the superresolution reconstruction results of a no-reference image:(7)
InEn
=
-
∑
i
=
0
255
p
i
l
o
g
2
⁡
p
i
,
AC
⁡
=
1
2
A
C
x
2
+
A
C
y
2
,where p
(
i
) represents the distribution probability of the ith gray/color scale; A
C
x and A
C
y are the image average contrasts in the x- and y-directions, respectively; and A
C
x can be calculated by the following equation:(8)
A
C
x
=
1
M
-
1
N
-
1
∑
y
=
0
N
-
2
∑
x
=
0
M
-
2
Gray
x
,
y
-
Gray
x
+
1
,
y
,where Gray
(
x
,
y
) is the gray-scale of pixel (x
,
y) and M and N are the number of pixels in the x- and y-directions, respectively. The calculation method for A
C
y is the same as that for A
C
x.Whenp
(
i
)
=
0, let l
o
g
2
p
(
i
)
=
0, where InEn
∈
[
0,8
]. The information entropy and the average contrast of the color image can be obtained from the three-channel information entropy normalization. The information entropy is a measure of the information richness contained in the image; a higher information entropy indicates richer information contained in the image, and a higher average contrast indicates a clearer image.


## 3. Experiment and Analysis

### 3.1. Data Source

The foremost task in achieving superresolution reconstruction is to construct a high-resolution training sample. In this study, 96 randomly selected natural images are used as training samples. To verify the effectiveness of the superresolution reconstruction model and algorithm and analyze the results of the superresolution reconstruction of photogrammetric images using the proposed method, the 1 : 5000 photogrammetric image of Xuzhou city (resolution = 5 cm), Hammer Aerial Photographic data (resolution = 375 cm), and a close-up image (resolution = 6 mm) collected by a Canon EOS 5D nonmetric digital camera are used in the superresolution reconstruction experiment in this study. The results are compared to those produced by the bicubic interpolation method. Because all experimental data are colored, the conversion from RGB to YUV space is performed first in a MATLAB environment; the superresolution reconstruction ofY component uses proposed method, while the other two channels use bicubic interpolation method and finally reconvert the colored image to RGB space. To increase the speed of the experiment, a test image with a size of 512 × 512 pixels is used for this superresolution experiment.


### 3.2. Parameter Setting

The training image is used as a high-resolution image, and the corresponding low-resolution image is obtained by downsampling, where the downsampling factorS is 3, and a one-dimension filter of [1, 3.4, 3, 1]/12 is used to blur the training image in the horizontal and vertical directions, respectively. The image is then magnified using the bicubic interpolation method, and the training sample set is composed of 50,000 image tile pairs that were extracted during the experiment. The extraction of low-resolution image characteristics is achieved using first- and second-order gradient operators:(9)
f
h
1
=
-
1,0
,
1
,
f
v
1
=
-
1,0
,
1
T
,
f
h
2
=
1,0
,
-
2,0
,
1
,
f
v
2
=
1,0
,
-
2,0
,
1
T
.The size of a low-resolution image tile is 3 × 3 pixels, and the overlap is 1 pixel. The number of iterations of the pseudodictionary-pair learning process using theK-SVD algorithm is 30, and the dictionary size is 1024. In the testing stage, the bicubic interpolation method is used to magnify the image three times, creating the high-resolution test image Z
h. Then, the proposed method is used to achieve a reconstruction that has a superresolution that is three times larger. The iteration termination value of the OMP reconstruction algorithm is ε = 0.05.


### 3.3. Results and Analysis

Experiment 1 (impact of ground-feature type on reconstruction accuracy).
To analyze the impact of different ground-feature types on the image reconstruction results, the Xuzhou 1 : 5000 aerial photogrammetric image is used as source data for capturing images of four typical ground features (a building, vegetation, bare soil, and a body of water) used as test data in this study. The reconstruction results are shown in Figure1. For comparison, the original image is simply magnified using the nearest neighbor interpolation method.
To evaluate the quality of a color image, the human objective effect is strong. As shown in the resulting images, the image produced by the traditional bicubic interpolation method is blurred, and the reconstructed image lacks clear details at edges and exhibits an irregular structural region with serious losses of texture. The reconstructed image produced by the compressed sensing method is clearer; the locally magnified image that is 10 times larger shows more detailed textures and an improved edge effect.
The quantitative index values of the reconstructed images produced by the bicubic interpolation method and the proposed method are shown in Table1. Compared to the bicubic interpolation method, the compressed sensing method exhibits certain improvements in both the information entropy and average contrast. The information entropy of the four typical features (i.e., building, vegetation, bare soil, and water body) increases by 0.0284, 0.0342, 0.0425, and 0.0505, respectively; the image contrast also increases by 0.5014, 1.2047, 0.8966, and 0.4926, respectively. The information entropy of the image reconstructed by the proposed method increases by 0.0347, and the average image contrast increases by 0.7128.
To describe the degree of enhancement of various ground-feature image reconstruction indices, the increased values are drawn into related curves based on ground-feature types, producing Figure2.
With regard to image information entropy, the entropy gradually increases from building to vegetation, bare soil, and water body. The increase in entropy is most marked for the water body. In terms of image contrast, the increase in vegetation is the most prevalent, while that of the water body is least prevalent.Table 1
Quantitative index values of the reconstruction result.
Ground-feature type algorithm
Index
Bicubic interpolation method
Proposed method
Δ
Building
InEn
7.7930
7.8214
0.0284
AC
2.3480
2.8494
0.5014
Vegetation
InEn
7.1726
7.2068
0.0342
AC
4.3220
5.5267
1.2047
Bare soil
InEn
6.7129
6.7554
0.0425
AC
3.1622
4.0588
0.8966
Water
InEn
6.9424
6.9929
0.0505
AC
1.7332
2.2258
0.4926Figure 1
Superresolution reconstruction results of various ground-feature types in a photogrammetric image.
(a)
Original LR image
(b)
Bicubic interpolation method
(c)
Proposed methodFigure 2
Relation curves based on the impact of various ground-feature types on reconstruction results.Experiment 2 (impact of image resolution on reconstruction accuracy).
Image spatial resolution is an important index when evaluating image quality; it directly reflects the level of detail of the information of the imaged object. A higher spatial resolution indicates that the image has more detailed information, including edges. The purpose of superresolution reconstruction is to increase the spatial resolution of an image and restore more image information. To examine the impact of the original image resolution on the superresolution reconstruction results, the reconstruction of a 375 mm resolution Hammer photogrammetric image, a 50 mm resolution photogrammetric image of Xuzhou city, and a 6 mm resolution close-up image captured by a nonmetric digital camera are compared; the reconstruction results are shown in Figure3.
Considering the visual effect, the images reconstructed by the proposed method show more detailed high-frequency information, and their geometric textures are more prevalent. The corresponding quantitative indices are listed in Table2. To mitigate the influence of different ground-feature types on the reconstruction results, the averages of building, vegetation, bare soil, and water body in the reconstructed image of Xuzhou city were determined. As shown in Table 2, for images with resolutions of 375, 50, and 6 mm, the information entropy of the reconstructed images produced by the proposed method increased by 0.0417, 0.0389, and 0.0232, respectively, compared to those produced by the bicubic interpolation method; additionally, the image contrast increased by 1.131, 0.7739, and 0.5659, respectively.
To describe the impact of image resolution on the reconstruction accuracy in more detail, the reconstruction results in Table2 are used to create quantitative indices of improvement curves with regard to image resolution, as shown in Figure 4. Because the original image’s resolution continues to decrease, the improvement in the accuracy of the reconstructed image shows a continuously increasing trend.Table 2
Comparison of quantitative indices of reconstruction results.
Image resolution algorithm
Index
Bicubic interpolation method
Proposed method
Δ
6 mm
InEn
6.2986
6.3218
0.0232
AC
2.2326
2.7985
0.5659
50 mm
InEn
7.4953
7.5247
0.0294
AC
2.4916
3.2048
0.7132
375 mm
InEn
6.9261
6.9678
0.0417
AC
2.9165
4.0475
1.131Figure 3
Superresolution reconstruction results of photogrammetric images of different resolutions.Figure 4
Effect of image resolution on the quantitative indices of reconstructed images.


## 3.1. Data Source

The foremost task in achieving superresolution reconstruction is to construct a high-resolution training sample. In this study, 96 randomly selected natural images are used as training samples. To verify the effectiveness of the superresolution reconstruction model and algorithm and analyze the results of the superresolution reconstruction of photogrammetric images using the proposed method, the 1 : 5000 photogrammetric image of Xuzhou city (resolution = 5 cm), Hammer Aerial Photographic data (resolution = 375 cm), and a close-up image (resolution = 6 mm) collected by a Canon EOS 5D nonmetric digital camera are used in the superresolution reconstruction experiment in this study. The results are compared to those produced by the bicubic interpolation method. Because all experimental data are colored, the conversion from RGB to YUV space is performed first in a MATLAB environment; the superresolution reconstruction ofY component uses proposed method, while the other two channels use bicubic interpolation method and finally reconvert the colored image to RGB space. To increase the speed of the experiment, a test image with a size of 512 × 512 pixels is used for this superresolution experiment.


## 3.2. Parameter Setting

The training image is used as a high-resolution image, and the corresponding low-resolution image is obtained by downsampling, where the downsampling factorS is 3, and a one-dimension filter of [1, 3.4, 3, 1]/12 is used to blur the training image in the horizontal and vertical directions, respectively. The image is then magnified using the bicubic interpolation method, and the training sample set is composed of 50,000 image tile pairs that were extracted during the experiment. The extraction of low-resolution image characteristics is achieved using first- and second-order gradient operators:(9)
f
h
1
=
-
1,0
,
1
,
f
v
1
=
-
1,0
,
1
T
,
f
h
2
=
1,0
,
-
2,0
,
1
,
f
v
2
=
1,0
,
-
2,0
,
1
T
.The size of a low-resolution image tile is 3 × 3 pixels, and the overlap is 1 pixel. The number of iterations of the pseudodictionary-pair learning process using theK-SVD algorithm is 30, and the dictionary size is 1024. In the testing stage, the bicubic interpolation method is used to magnify the image three times, creating the high-resolution test image Z
h. Then, the proposed method is used to achieve a reconstruction that has a superresolution that is three times larger. The iteration termination value of the OMP reconstruction algorithm is ε = 0.05.


## 3.3. Results and Analysis

Experiment 1 (impact of ground-feature type on reconstruction accuracy).
To analyze the impact of different ground-feature types on the image reconstruction results, the Xuzhou 1 : 5000 aerial photogrammetric image is used as source data for capturing images of four typical ground features (a building, vegetation, bare soil, and a body of water) used as test data in this study. The reconstruction results are shown in Figure1. For comparison, the original image is simply magnified using the nearest neighbor interpolation method.
To evaluate the quality of a color image, the human objective effect is strong. As shown in the resulting images, the image produced by the traditional bicubic interpolation method is blurred, and the reconstructed image lacks clear details at edges and exhibits an irregular structural region with serious losses of texture. The reconstructed image produced by the compressed sensing method is clearer; the locally magnified image that is 10 times larger shows more detailed textures and an improved edge effect.
The quantitative index values of the reconstructed images produced by the bicubic interpolation method and the proposed method are shown in Table1. Compared to the bicubic interpolation method, the compressed sensing method exhibits certain improvements in both the information entropy and average contrast. The information entropy of the four typical features (i.e., building, vegetation, bare soil, and water body) increases by 0.0284, 0.0342, 0.0425, and 0.0505, respectively; the image contrast also increases by 0.5014, 1.2047, 0.8966, and 0.4926, respectively. The information entropy of the image reconstructed by the proposed method increases by 0.0347, and the average image contrast increases by 0.7128.
To describe the degree of enhancement of various ground-feature image reconstruction indices, the increased values are drawn into related curves based on ground-feature types, producing Figure2.
With regard to image information entropy, the entropy gradually increases from building to vegetation, bare soil, and water body. The increase in entropy is most marked for the water body. In terms of image contrast, the increase in vegetation is the most prevalent, while that of the water body is least prevalent.Table 1
Quantitative index values of the reconstruction result.
Ground-feature type algorithm
Index
Bicubic interpolation method
Proposed method
Δ
Building
InEn
7.7930
7.8214
0.0284
AC
2.3480
2.8494
0.5014
Vegetation
InEn
7.1726
7.2068
0.0342
AC
4.3220
5.5267
1.2047
Bare soil
InEn
6.7129
6.7554
0.0425
AC
3.1622
4.0588
0.8966
Water
InEn
6.9424
6.9929
0.0505
AC
1.7332
2.2258
0.4926Figure 1
Superresolution reconstruction results of various ground-feature types in a photogrammetric image.
(a)
Original LR image
(b)
Bicubic interpolation method
(c)
Proposed methodFigure 2
Relation curves based on the impact of various ground-feature types on reconstruction results.Experiment 2 (impact of image resolution on reconstruction accuracy).
Image spatial resolution is an important index when evaluating image quality; it directly reflects the level of detail of the information of the imaged object. A higher spatial resolution indicates that the image has more detailed information, including edges. The purpose of superresolution reconstruction is to increase the spatial resolution of an image and restore more image information. To examine the impact of the original image resolution on the superresolution reconstruction results, the reconstruction of a 375 mm resolution Hammer photogrammetric image, a 50 mm resolution photogrammetric image of Xuzhou city, and a 6 mm resolution close-up image captured by a nonmetric digital camera are compared; the reconstruction results are shown in Figure3.
Considering the visual effect, the images reconstructed by the proposed method show more detailed high-frequency information, and their geometric textures are more prevalent. The corresponding quantitative indices are listed in Table2. To mitigate the influence of different ground-feature types on the reconstruction results, the averages of building, vegetation, bare soil, and water body in the reconstructed image of Xuzhou city were determined. As shown in Table 2, for images with resolutions of 375, 50, and 6 mm, the information entropy of the reconstructed images produced by the proposed method increased by 0.0417, 0.0389, and 0.0232, respectively, compared to those produced by the bicubic interpolation method; additionally, the image contrast increased by 1.131, 0.7739, and 0.5659, respectively.
To describe the impact of image resolution on the reconstruction accuracy in more detail, the reconstruction results in Table2 are used to create quantitative indices of improvement curves with regard to image resolution, as shown in Figure 4. Because the original image’s resolution continues to decrease, the improvement in the accuracy of the reconstructed image shows a continuously increasing trend.Table 2
Comparison of quantitative indices of reconstruction results.
Image resolution algorithm
Index
Bicubic interpolation method
Proposed method
Δ
6 mm
InEn
6.2986
6.3218
0.0232
AC
2.2326
2.7985
0.5659
50 mm
InEn
7.4953
7.5247
0.0294
AC
2.4916
3.2048
0.7132
375 mm
InEn
6.9261
6.9678
0.0417
AC
2.9165
4.0475
1.131Figure 3
Superresolution reconstruction results of photogrammetric images of different resolutions.Figure 4
Effect of image resolution on the quantitative indices of reconstructed images.


## 4. Discussion

Currently, many studies of the superresolution reconstruction of single-frame images using the sparse representation and dictionary-learning methods have been published in the literature [8, 18, 19]. These studies generally use methods such as the adaptive sparse domain and adaptive regularization parameters [20] to improve reconstruction accuracy using the improved dictionary-learning method [21, 22], and they typically produce good reconstruction results. However, these studies used existing images as high-resolution images, and the superresolution reconstruction process is merely a signal restoration of the downsampled images; the image resolution did not receive essential improvement. For this reason, this study used compressed sensing as a theoretical construct along with the similarity of the high- and low-resolution image characteristics to train high- and low-resolution image dictionaries using the K-SVD method. Using a bicubic interpolation image as prior information, the proposed method performed a superresolution reconstruction of the existing images, which resulted in an image with a resolution triple that of the original image. The experimental results of this study indicated that the image obtained by the proposed reconstruction method increased both in visual effect and quantitative index compared to the image reconstructed using the bicubic interpolation method.The superresolution reconstruction of single-frame images has been successfully applied in the reconstruction of remote-sensing images [23], medical imaging, “Resource Satellite III” images [24], Moon Rover images [25], and SAR images [26]. High-resolution imagery is necessary for high-precision photogrammetry and can increase the absolute measurement precision of photogrammetry. Thus, this study of the superresolution reconstruction of a single-frame photogrammetric image is important. This study introduces the superresolution reconstruction concept into the photogrammetry field, producing superresolution reconstructions of single-frame photogrammetric image experiments from the MATLAB platform and superresolution images that exhibit improved visual effect and accuracy compared to those produced by the bicubic interpolation method.The current studies in the field all concentrate on improving the sparse domain and dictionary-learning methods to improve algorithm accuracy. However, the impact of different ground-feature types and the image resolution of the photogrammetric images on the reconstruction accuracy is marked; this study conducted detailed studies of these phenomena. The texture structure and gray-scale variation of different types of ground features are reflected differently on photogrammetric images, which inevitably induce differences in reconstruction accuracy. Thus, to achieve superresolution reconstruction, this study used a 5 cm resolution image of Xuzhou city as an experimental object to create a superresolution reconstruction of four typical ground features in the photogrammetric image. This study then conducted an accuracy comparison with the traditional bicubic interpolation method based on the quantitative index analysis. The experimental results of this study indicate that the building image has the most improved reconstruction result due to the clarity and richness of the image. Additionally, the four resulting images all exhibit certain degrees of improvement, and the improvement in the image information entropy exhibits an increasing trend from building, vegetation, and bare soil to water body. In general, lower original image entropy produces more marked improvements. Conversely, the increase in image contrast is marked because vegetation exhibits the richest color information and the most frequent variation in gray-scale. The texture of the water-body image is the most monotone, and thus, its increase in contrast is the lowest. Additionally, the increment in image contrast of different ground features maintains a consistent relationship with the self-contrast of the images of these features.Image spatial resolution is an important index for judging the amount of detailed information in an image. The direct purpose of superresolution reconstruction is to increase image resolution; however, the reconstruction process is built on the basis of existing images, and thus, the resolution of the existing image will inevitably influence the reconstruction accuracy. Thus, this study performed a superresolution reconstruction experiment on three images of different resolutions. The results of this study indicate that as the original image resolution decreases, the reconstructed images exhibit increases in the quantitative indices. Thus, the proposed superresolution reconstruction method is particularly suitable for low-resolution images.


## 5. Conclusions

Blurry images produced during image reconstruction are common when using the traditional interpolation method. Additionally, the learning method requires prior information of the resulting high-resolution image to perform superresolution reconstruction; reconstructed images with superresolutions are thus not produced. This study uses compressed sensing as a theoretical framework; organically combines the interpolation method with the learning method, guided by bicubic interpolation images; and produces a superresolution reconstruction of an existing image at three times the scale. This study uses the classical OMP reconstruction algorithm to solve the sparse optimization problem and effectively increases image resolution. Compared to traditional bicubic interpolation method, the reconstruction based on the proposed method increases the image information content and clarity. The proposed method provides an effective way to improve the application range and accuracy of aviation and aerospace photogrammetric images. In the future, the sparse optimization method will be improved further to produce even better reconstruction results.



---

*Source: 1250538-2016-08-03.xml*