# Self-Similarity Superresolution for Resource-Constrained Image Sensor Node in Wireless Sensor Networks

Authors: Yuehai Wang; Weidong Wang; Shiying Cao; Shiju Li; Li Xie; Baocang Ding
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

Wireless sensor networks, in combination with image sensors, open up a grand sensing application field. It is a challenging problem to recover a high resolution(
HR
) image from its low resolution (
LR
) counterpart, especially for low-cost resource-constrained image sensors with limited resolution. Sparse representation-based techniques have been developed recently and increasingly to solve this ill-posed inverse problem. Most of these solutions are based on an external dictionary learned from huge image gallery, consequently needing tremendous iteration and long time to match. In this paper, we explore the self-similarity inside the image itself, and propose a new combined self-similarity superresolution (
SR
) solution, with low computation cost and high recover performance. In the self-similarity image super resolution model (
SSIR
), a small size sparse dictionary is learned from the image itself by the methods such as KSVD. The most similar patch is searched and specially combined during the sparse regulation iteration. Detailed information, such as edge sharpness, is preserved more faithfully and clearly. Experiment results confirm the effectiveness and efficiency of this double self-learning method in the image super resolution.


---

## Body

## 1. Introduction

Wireless sensor networks, in combination with image sensors, open up a grand sensing application field. Visual information provided by image sensor is the most intuitive information perceived by human, especially for recognition, monitoring, and surveillance. Low-cost and resource-constrained image sensors with limited resolution are mainly employed [1–3]. Recovery from low resolution to high resolution is the pressing need for image sensor node. Image super resolution (
SR
) receives more and more interests recently, which has lots of applications in image sensor, digital cameras, mobile phone, image enhancement, high definition TV [4–6], and so forth. It aims to reconstruct a high-resolution (
HR
) image from the low-resolution (
LR
) one based on reasonable assumptions or prior knowledge. From the view of the target HR image, the LR image can be generated after downsampling and some blurring operator. Hence, the SR work has always been formulated as an inverse problem:
(1)
Y
=
Φ
H
X
+
N
,
where X is the HR image to be recovered, Y is the known LR image, Φ is the downsampling operator, H is the blurring operator that minimizes the high frequency aliasing effect, and N is the noise. Traditionally, the downsampling operator Φ and blurring operator H are conducted at the same time. Hence, we can use the following formulation (2) instead of (1):
(2)
Y
=
ℋ
X
+
N
,
where ℋ
=
Φ
H is the generalized blurring and downsampling operator. However, the detailed information, especially the high frequency part, is lost after these two operations. Hence, image super resolution has become a highly under-determined reconstruction problem.The classicalSR solutions are interpolation-based methods, including bilinear, bicubic, spline interpolation and some other improved versions [7, 8]. These methods tend to generate overly smooth HR images with ringing and jaggy effects. Their visual clarity is very limited. Edge preserving and directional interpolators have been proposed to improve the reconstruction image’s visual clarity [9–11]. However, the blurring and noises are still obstacles to overcome.Sparse representation-basedSR methods are becoming more popular recently since the issue of sparse representation is consistent with (2). Sparse representation provides a different perspective in solving the underdetermined problems [12–15]. This powerful and promising tool has proven to be effective for a wide range of problems, such as sub-Nyquist sensing of signals and coding, image denoising, and deblurring [16–23]. Several sparse representation based SR algorithms have been proposed with superior results reported [12, 22, 24, 25]. Most of them need training dictionaries based on a large scale external image gallery, which have limited matching degree to the target image and time consuming. Another issue is that the external dictionary depends on the blurring modal with less generality. Self-learning SR algorithms, lately emerged, show that the internal statistics in the image itself often have stronger prediction power than the external statistics and can give more powerful image-specific priors [26, 27].In this paper, we explore the self-similarity inside the image and propose a new combined self-similarity super resolution(
SR
) solution, which successfully restores the missing detailed image information. In this self-similarity image super resolution model (
SSIR
), the patches from the LR image are downsampled firstly to form smaller LR patches (
SLR
). Small-sized sparse dictionary is learned from the image itself by methods such as KSVD. Then, a most similar patch for the unrecovered LR patch is searched and combined, during the sparse iteration, to preserve the faithful detailed information. Experiment results confirm the effectiveness and efficiency of the double self-similarity learning method in the image super resolution.The rest of this paper is organized as follows. Section2 describes our approach of SSIR framework with self-learning dictionary. In Section 3, experiments are taken to compare the proposed method with other ones. The conclusions are finally given in Section 4.


## 2. The Proposed Self-Similarity-Based Image Super Resolution Approach

### 2.1. Sparse Representation of Image Super Resolution

For sparse representation-basedSR methods, high resolution image X can be represented by sparse coefficients α under dictionary D as follows:
(3)
X
=
D
α
.Hence, theHR image recovering procedure can be seen as the minimization of the l
1-norm problem:
(4)
α
^
=
argmin
∥
α
∥
1
s
.
t
.
Y
=
ℋ
X
=
ℋ
D
α
,
where Y is the LR image and ℋ is the generalized blurring and downsampling degradation matrix. The quality of recover HR image is always determined by the details, such as edges and contrast. However, such details are lost when the HR image is downsampled. Hence, small patch based recovery is more popular than the whole image based ones to prevent large scale details losing. We follow the patch based learning strategy in our approach. For N
×
N sized LR image, the atoms in D are learned by patches sized by n
×
n, where n can be 8, 10, and so forth. Then the sparse representation (4) can be rewritten as
(5)
α
^
k
=
argmin
∥
α
k
∥
1
s
.
t
.
y
k
=
ℋ
x
k
=
ℋ
D
α
k
,
where y is the LR patch with size of (
1
/
scale
)
×
n
×
n, x
k is the HR patch, α
k is the coefficient of the patch, and ℋ and D are corresponding patch with the size of n
×
n. The image reconstruction scheme based on self-learning dictionary can be presented more intuitively by
(6)
α
^
=
argmin
∥
y
-
ℋ
D
α
∥
F
2
+
λ
∥
α
∥
1
.


### 2.2. Internal Dictionary Learning

Most of the sparse representationSR methods are based on dictionary learning from the external image library [12, 22, 25]. The number of the atoms in dictionary D should be huge enough to confirm the sparsity of α and avoid image hallucination and blurring [16]. Normally, the dimension of external dictionary should be above thousand and the recovery time is huge. For various natural images, especially the high-gradient ones, high recover performance could not be easily and fast reached if the dictionary is learned from the outside image gallery. External dictionary approaches are not suitable for the resource-constrained image sensor node. A different idea is that we should make full use of the information inside of the image itself as shown in [26, 27]. The feature of the same structure textures or patterns can be more easily found within the image. For the destination image, the dictionary does not need to be tremendous to mate different kinds of natural images. Inspired by [26, 27], the dictionary D is learned firstly from LR image in our approach to classify the local structures.The internal training patches are extracted fromLR image and then used to generate an overcomplete dictionary D
∈
R
n
×
n
×
K which contains K atoms. It is assumed that a training patch y
TS can be represented as y
TS
=
D
β, which satisfies ∥
y
TS
-
D
β
∥
<
ε. Hence, the training dictionary is the solution of
(7)
[
D
,
β
]
=
argmin
∥
y
TS
-
D
β
∥
s
.
t
.
∥
β
∥
0
<
K
.Iterative optimization is used to solve this dictionary training problem. The iteration consists of two basic steps:(
1
) sparse coding: fix the dictionary D and search for the sparse representation of β and (
2
) dictionary update: update the dictionary atoms {
d
j
}
j
=
1
K and their corresponding coefficients β one by one. Inspired by [28, 29], we use orthogonal matching pursuit (
OMP
) algorithm in the sparse coding step and K-singular value decomposition (
K
-
SVD
) based iterative optimization in dictionary update step, respectively. These two steps run iteratively until the maximum iteration or the convergence is reached.Typically, the self-leaning dictionary sizeK is set below 256 in our approach, and we get similar recovery performance with the external dictionary. Detailed comparison is illustrated in Section 3.


### 2.3. Self-Similarity Regulation Scheme

Local image structures inLR image can be classified by the patch dictionary learned from itself. However, detailed information, such as sharp edges and corners, could not be clustered perfectly by limit atoms and may be lost for some extent after downsampled from the HR patch. The following Table 1 demonstrates a real HR patch in Lena, its corresponding LR patch, and reconstruction patch by self-learning KSVD dictionary with 256 atoms. From Table 1, the rich variation between the HR pixels is omitted in LR patch and smoothed in the reconstruction patch. The reason of smooth effect under KSVD dictionary is mainly that the dictionary atoms are trained not only for the special patch, but also for all the patches in the image.Table 1
HR patch, corresponding LR patch, and reconstruction patch under KSVD dictionary.
HR patch
LR patch
Reconstruction patch underKSVD dictionary
100
99
96
91
90
88
85
83
99
95
89
84
100
99
96
94
91
88
84
84
98
100
98
93
90
87
84
83
98
93
88
85
101
99
96
92
90
86
83
84
99
99
96
92
89
86
85
84
99
94
90
84
100
98
95
92
88
85
84
85
100
96
93
91
89
87
85
85
99
94
87
85
99
97
94
92
90
87
84
85
99
97
95
95
91
88
85
83
100
97
94
93
92
88
85
85
101
99
95
91
91
89
86
84
101
99
96
93
91
88
85
84
101
99
95
91
89
88
86
85
100
99
96
92
89
87
85
84
98
97
96
93
88
85
84
84
99
98
96
92
88
86
86
85Hence, accurate reconstruction for each patch is tough even under the sparse self-learning dictionary. More prior information should be incorporated into the recover procedure to improve theHR image quality. Several additional parameters have been studied such as frequency, histogram, low-pass, nonlocal means constraints [22, 25]. Unlike these statistic constraints, we consider true information inside of the image as the regulation index.As aforementioned, distinct edges and corners become blur after downsample operation. The information loss phenomenon appears when theHR image is downsampled to LR image. Similar information loss phenomenon also appears when the LR image is down-sampled to an even lower resolution image. The lost information during the latter procedure can be recovered from the image before down-sample. It provides a learning way to recover more realistic HR patches. A new self-similarity regulation scheme is proposed based on finding image patch similar to the destination HR patch. The new sparse regulation scheme can be formulated as
(8)
α
^
=
argmin
∥
y
-
ℋ
D
α
∥
F
2
+
λ
∥
α
∥
1
+
γ
similar
∥
J
(
α
)
∥
1
,
where γ
similar is the regulation threshold and J
(
α
) is the similarity prior. We divide the whole sparse regulation into two steps: self-similarity regulation and sparse dictionary regulation. The self-similarity regulation step can be seen as an internal regulation step to compensate the sharpness of the edges. The sparse dictionary regulation step provides the basic framework to enlarge the LR image.The detailed self-similarity regulation step is described in Figure1. Firstly, the input unrecovered LR patch, named as y
LR, is upscaled by bicubic operator. Then, a similar HR patch of the same up-scaled size, named as S
HR, is searched around the LR patch y
LR inside of LR image Y. If a similar HR patch S
HR is found, we can get its corresponding down-sampled LR patch S
LR. The true HR patch x
HR is approximated by the similar HR patch S
HR. This recovered HR patch x
~
HR coming from real pixels can be closer to the ground truth x
HR than that recovered by statistic constraints studied previously. During approximation, the similar down-sampled LR patch S
LR is firstly subtracted from the unrecovered LR patch y
LR. Then, the above difference is estimated from the residual R
S
LR by the self-learning sparse dictionary, which is named as R
S
HR. At last, the recovered HR patch x
~
HR is computed by adding the similar patch S
HR and the difference estimation R
S
HR. The well-known sparse regulation methods, like IRLS and OMP, can be used in the recovery procedure [27–30].Figure 1
Self-similarity regulation step.The above self-similarity regulation step can be represented as(9)
R
S
HR
l
=
D
H
*
IRLS
(
y
LR
-
ℋ
S
HR
l
,
D
L
)
,
x
~
HR
l
+
1
/
2
=
S
HR
l
+
R
S
HR
l
,
where l is the current iteration index, S
HR
l is the most similar patch found in lth iteration, R
S
HR
l is the recovered difference between S
HR
l and x
~
HR
,
x
~
HR
l
+
1
/
2 represents updated x
~
HR, and D
L and D
H are dictionary trained for low-resolution patch and high-resolution patch, respectively.We introduced sum square error(
SSE
) as the self-similarity prior J
(
α
) and use it to decide which patch is the most matching one. The definition of the SSE is given by
(10)
J
(
α
)
=
SSE
=
∑
l
∑
k
(
B
l
k
-
B
^
l
k
)
2
,
where B
l
k is the pixels taken from y
LR neighbor patch in the searching zone and B
^
l
k is the pixels taken from the bicubic up-scaled patch y
HR. Both have the same size as the output HR patch x
~
HR. The patches we searched for come from the LR image, so the fidelity can be guaranteed.Sparse thresholdγ
similar is used to decide whether a patch is similar to destination HR patch. γ
similar is adaptive to y
LR, instead of being a fixed value. The adaptive threshold γ
similar is defined as α
Var
⁡
+
β, where Var
⁡is variance of the processing patch y
LR and α
,
β are associated parameters. If the minimum J
(
α
) within the searching zone is smaller than γ
similar, its corresponding patch is named as the most similar patch S
HR.The sparse dictionary regulation step is then performed under self-learned dictionary, which can be represented by(11)
x
~
HR
l
+
1
=
x
~
HR
l
+
1
/
2
+
λ
D
H
*
IRLS
(
y
LR
-
ℋ
x
~
HR
l
+
1
/
2
,
D
L
)
.The above two regulation steps are performed until the maximum iteration times or the convergence is reached.The procedure of self-similarity regulation scheme is described in detail by Algorithm1.Algorithm 1:Self-similarity regulation scheme.
Input: LR imageY, LR image patches’ size m and HR image patches’ size n, the degradation matrix ℋ.
Output: HR imageX
Step  1.  Extract patches y
LR
∈
R
m from LR image Y, follow the raster-scan order, and start from the upper-left corner
(some pixel overlap in each direction is allowed).
Step  2.  Recover HR image patches x
HR iteratively by Steps  2.1 and  2.2, until the maximum iteration times
or convergence is reached.
Step  2.1 Self-similarity regulation step:
Step  2.1.1.  Use bicubic method to up scale the unrecovered LR patch y
LR to the same size n as HR patch, defined as y
HR.
Step  2.1.2.  Searching for a similar n sized patch in y
LR’s neighbor:
Step  2.1.2.1.  Compute each searching patch’s SSE as the self-similarity prior J
(
α
),
J
(
α
)
=
SSE
=
∑
l
∑
k
(
B
l
k
-
B
^
l
k
)
2
Step  2.1.2.2.  Find the least SSE patch, and compare its SSE with the adaptive threshold
α
Var
⁡
+
β. If SSE
<
α
Var
⁡
+
β, define this least SSE patch as the similar patch S
HR.
Step  2.1.3.  Use degradation matrix ℋ to down sample similar patch S
HR, define as S
LR.
Step  2.1.4.  Subtract S
LR from LR patch y
LR, and get the residual R
S
LR
=
y
LR
-
S
LR.
Step  2.1.5.  Recover the residual R
S
LR to R
S
HR using IRLS algorithm according (9).
Step  2.1.6.  Add the R
S
HR to S
HR, according to (10).
Step  2.2  Sparse dictionary regulation step: update x
~
HR
l
+
1 according to (11).
Step  3.  Ensemble all x
~
HR to recover HR image X (if there is pixel overlap, the weighted average method is needed).


### 2.4. Overall Diagram of Self-Similarity Based Image Super Resolution Approach

After all the analyses above, the overall diagram of self-similarity based image super resolution approach is shown in Figure2. Firstly, the input LR image Y, regarded as a down-sampled version from corresponding HR image X, is segmented into patches y
LR. Then the sparse representation dictionaries D
L and D
H are trained by these internal patches. Next, the self-similarity regulation scheme is applied to find a matching patch S
HR. Afterwards, HR patch x
^
HR is recovered by sparse regulation based on the self-learning dictionary. At last, we ensemble all these recovered HR patches x
^
HR to get a high-quality HR image X.Figure 2
The overall diagram of the self-similarity-based image super resolution.


## 2.1. Sparse Representation of Image Super Resolution

For sparse representation-basedSR methods, high resolution image X can be represented by sparse coefficients α under dictionary D as follows:
(3)
X
=
D
α
.Hence, theHR image recovering procedure can be seen as the minimization of the l
1-norm problem:
(4)
α
^
=
argmin
∥
α
∥
1
s
.
t
.
Y
=
ℋ
X
=
ℋ
D
α
,
where Y is the LR image and ℋ is the generalized blurring and downsampling degradation matrix. The quality of recover HR image is always determined by the details, such as edges and contrast. However, such details are lost when the HR image is downsampled. Hence, small patch based recovery is more popular than the whole image based ones to prevent large scale details losing. We follow the patch based learning strategy in our approach. For N
×
N sized LR image, the atoms in D are learned by patches sized by n
×
n, where n can be 8, 10, and so forth. Then the sparse representation (4) can be rewritten as
(5)
α
^
k
=
argmin
∥
α
k
∥
1
s
.
t
.
y
k
=
ℋ
x
k
=
ℋ
D
α
k
,
where y is the LR patch with size of (
1
/
scale
)
×
n
×
n, x
k is the HR patch, α
k is the coefficient of the patch, and ℋ and D are corresponding patch with the size of n
×
n. The image reconstruction scheme based on self-learning dictionary can be presented more intuitively by
(6)
α
^
=
argmin
∥
y
-
ℋ
D
α
∥
F
2
+
λ
∥
α
∥
1
.


## 2.2. Internal Dictionary Learning

Most of the sparse representationSR methods are based on dictionary learning from the external image library [12, 22, 25]. The number of the atoms in dictionary D should be huge enough to confirm the sparsity of α and avoid image hallucination and blurring [16]. Normally, the dimension of external dictionary should be above thousand and the recovery time is huge. For various natural images, especially the high-gradient ones, high recover performance could not be easily and fast reached if the dictionary is learned from the outside image gallery. External dictionary approaches are not suitable for the resource-constrained image sensor node. A different idea is that we should make full use of the information inside of the image itself as shown in [26, 27]. The feature of the same structure textures or patterns can be more easily found within the image. For the destination image, the dictionary does not need to be tremendous to mate different kinds of natural images. Inspired by [26, 27], the dictionary D is learned firstly from LR image in our approach to classify the local structures.The internal training patches are extracted fromLR image and then used to generate an overcomplete dictionary D
∈
R
n
×
n
×
K which contains K atoms. It is assumed that a training patch y
TS can be represented as y
TS
=
D
β, which satisfies ∥
y
TS
-
D
β
∥
<
ε. Hence, the training dictionary is the solution of
(7)
[
D
,
β
]
=
argmin
∥
y
TS
-
D
β
∥
s
.
t
.
∥
β
∥
0
<
K
.Iterative optimization is used to solve this dictionary training problem. The iteration consists of two basic steps:(
1
) sparse coding: fix the dictionary D and search for the sparse representation of β and (
2
) dictionary update: update the dictionary atoms {
d
j
}
j
=
1
K and their corresponding coefficients β one by one. Inspired by [28, 29], we use orthogonal matching pursuit (
OMP
) algorithm in the sparse coding step and K-singular value decomposition (
K
-
SVD
) based iterative optimization in dictionary update step, respectively. These two steps run iteratively until the maximum iteration or the convergence is reached.Typically, the self-leaning dictionary sizeK is set below 256 in our approach, and we get similar recovery performance with the external dictionary. Detailed comparison is illustrated in Section 3.


## 2.3. Self-Similarity Regulation Scheme

Local image structures inLR image can be classified by the patch dictionary learned from itself. However, detailed information, such as sharp edges and corners, could not be clustered perfectly by limit atoms and may be lost for some extent after downsampled from the HR patch. The following Table 1 demonstrates a real HR patch in Lena, its corresponding LR patch, and reconstruction patch by self-learning KSVD dictionary with 256 atoms. From Table 1, the rich variation between the HR pixels is omitted in LR patch and smoothed in the reconstruction patch. The reason of smooth effect under KSVD dictionary is mainly that the dictionary atoms are trained not only for the special patch, but also for all the patches in the image.Table 1
HR patch, corresponding LR patch, and reconstruction patch under KSVD dictionary.
HR patch
LR patch
Reconstruction patch underKSVD dictionary
100
99
96
91
90
88
85
83
99
95
89
84
100
99
96
94
91
88
84
84
98
100
98
93
90
87
84
83
98
93
88
85
101
99
96
92
90
86
83
84
99
99
96
92
89
86
85
84
99
94
90
84
100
98
95
92
88
85
84
85
100
96
93
91
89
87
85
85
99
94
87
85
99
97
94
92
90
87
84
85
99
97
95
95
91
88
85
83
100
97
94
93
92
88
85
85
101
99
95
91
91
89
86
84
101
99
96
93
91
88
85
84
101
99
95
91
89
88
86
85
100
99
96
92
89
87
85
84
98
97
96
93
88
85
84
84
99
98
96
92
88
86
86
85Hence, accurate reconstruction for each patch is tough even under the sparse self-learning dictionary. More prior information should be incorporated into the recover procedure to improve theHR image quality. Several additional parameters have been studied such as frequency, histogram, low-pass, nonlocal means constraints [22, 25]. Unlike these statistic constraints, we consider true information inside of the image as the regulation index.As aforementioned, distinct edges and corners become blur after downsample operation. The information loss phenomenon appears when theHR image is downsampled to LR image. Similar information loss phenomenon also appears when the LR image is down-sampled to an even lower resolution image. The lost information during the latter procedure can be recovered from the image before down-sample. It provides a learning way to recover more realistic HR patches. A new self-similarity regulation scheme is proposed based on finding image patch similar to the destination HR patch. The new sparse regulation scheme can be formulated as
(8)
α
^
=
argmin
∥
y
-
ℋ
D
α
∥
F
2
+
λ
∥
α
∥
1
+
γ
similar
∥
J
(
α
)
∥
1
,
where γ
similar is the regulation threshold and J
(
α
) is the similarity prior. We divide the whole sparse regulation into two steps: self-similarity regulation and sparse dictionary regulation. The self-similarity regulation step can be seen as an internal regulation step to compensate the sharpness of the edges. The sparse dictionary regulation step provides the basic framework to enlarge the LR image.The detailed self-similarity regulation step is described in Figure1. Firstly, the input unrecovered LR patch, named as y
LR, is upscaled by bicubic operator. Then, a similar HR patch of the same up-scaled size, named as S
HR, is searched around the LR patch y
LR inside of LR image Y. If a similar HR patch S
HR is found, we can get its corresponding down-sampled LR patch S
LR. The true HR patch x
HR is approximated by the similar HR patch S
HR. This recovered HR patch x
~
HR coming from real pixels can be closer to the ground truth x
HR than that recovered by statistic constraints studied previously. During approximation, the similar down-sampled LR patch S
LR is firstly subtracted from the unrecovered LR patch y
LR. Then, the above difference is estimated from the residual R
S
LR by the self-learning sparse dictionary, which is named as R
S
HR. At last, the recovered HR patch x
~
HR is computed by adding the similar patch S
HR and the difference estimation R
S
HR. The well-known sparse regulation methods, like IRLS and OMP, can be used in the recovery procedure [27–30].Figure 1
Self-similarity regulation step.The above self-similarity regulation step can be represented as(9)
R
S
HR
l
=
D
H
*
IRLS
(
y
LR
-
ℋ
S
HR
l
,
D
L
)
,
x
~
HR
l
+
1
/
2
=
S
HR
l
+
R
S
HR
l
,
where l is the current iteration index, S
HR
l is the most similar patch found in lth iteration, R
S
HR
l is the recovered difference between S
HR
l and x
~
HR
,
x
~
HR
l
+
1
/
2 represents updated x
~
HR, and D
L and D
H are dictionary trained for low-resolution patch and high-resolution patch, respectively.We introduced sum square error(
SSE
) as the self-similarity prior J
(
α
) and use it to decide which patch is the most matching one. The definition of the SSE is given by
(10)
J
(
α
)
=
SSE
=
∑
l
∑
k
(
B
l
k
-
B
^
l
k
)
2
,
where B
l
k is the pixels taken from y
LR neighbor patch in the searching zone and B
^
l
k is the pixels taken from the bicubic up-scaled patch y
HR. Both have the same size as the output HR patch x
~
HR. The patches we searched for come from the LR image, so the fidelity can be guaranteed.Sparse thresholdγ
similar is used to decide whether a patch is similar to destination HR patch. γ
similar is adaptive to y
LR, instead of being a fixed value. The adaptive threshold γ
similar is defined as α
Var
⁡
+
β, where Var
⁡is variance of the processing patch y
LR and α
,
β are associated parameters. If the minimum J
(
α
) within the searching zone is smaller than γ
similar, its corresponding patch is named as the most similar patch S
HR.The sparse dictionary regulation step is then performed under self-learned dictionary, which can be represented by(11)
x
~
HR
l
+
1
=
x
~
HR
l
+
1
/
2
+
λ
D
H
*
IRLS
(
y
LR
-
ℋ
x
~
HR
l
+
1
/
2
,
D
L
)
.The above two regulation steps are performed until the maximum iteration times or the convergence is reached.The procedure of self-similarity regulation scheme is described in detail by Algorithm1.Algorithm 1:Self-similarity regulation scheme.
Input: LR imageY, LR image patches’ size m and HR image patches’ size n, the degradation matrix ℋ.
Output: HR imageX
Step  1.  Extract patches y
LR
∈
R
m from LR image Y, follow the raster-scan order, and start from the upper-left corner
(some pixel overlap in each direction is allowed).
Step  2.  Recover HR image patches x
HR iteratively by Steps  2.1 and  2.2, until the maximum iteration times
or convergence is reached.
Step  2.1 Self-similarity regulation step:
Step  2.1.1.  Use bicubic method to up scale the unrecovered LR patch y
LR to the same size n as HR patch, defined as y
HR.
Step  2.1.2.  Searching for a similar n sized patch in y
LR’s neighbor:
Step  2.1.2.1.  Compute each searching patch’s SSE as the self-similarity prior J
(
α
),
J
(
α
)
=
SSE
=
∑
l
∑
k
(
B
l
k
-
B
^
l
k
)
2
Step  2.1.2.2.  Find the least SSE patch, and compare its SSE with the adaptive threshold
α
Var
⁡
+
β. If SSE
<
α
Var
⁡
+
β, define this least SSE patch as the similar patch S
HR.
Step  2.1.3.  Use degradation matrix ℋ to down sample similar patch S
HR, define as S
LR.
Step  2.1.4.  Subtract S
LR from LR patch y
LR, and get the residual R
S
LR
=
y
LR
-
S
LR.
Step  2.1.5.  Recover the residual R
S
LR to R
S
HR using IRLS algorithm according (9).
Step  2.1.6.  Add the R
S
HR to S
HR, according to (10).
Step  2.2  Sparse dictionary regulation step: update x
~
HR
l
+
1 according to (11).
Step  3.  Ensemble all x
~
HR to recover HR image X (if there is pixel overlap, the weighted average method is needed).


## 2.4. Overall Diagram of Self-Similarity Based Image Super Resolution Approach

After all the analyses above, the overall diagram of self-similarity based image super resolution approach is shown in Figure2. Firstly, the input LR image Y, regarded as a down-sampled version from corresponding HR image X, is segmented into patches y
LR. Then the sparse representation dictionaries D
L and D
H are trained by these internal patches. Next, the self-similarity regulation scheme is applied to find a matching patch S
HR. Afterwards, HR patch x
^
HR is recovered by sparse regulation based on the self-learning dictionary. At last, we ensemble all these recovered HR patches x
^
HR to get a high-quality HR image X.Figure 2
The overall diagram of the self-similarity-based image super resolution.


## 3. Experimental Results

### 3.1. Experimental Background

In this section, several experimental results for the proposed method are given. All the simulations are conducted in MATLAB 7.5 onPC with Intel Core2/1.6 GHz/1 GB. The test LR images include several typical 256
×
256 natural images. We aim to recover their 512
×
512
HR images. The input LR images with different degradation matrix ℋ (direct downsampling degradation matrix ℋ
d and blur down-sampling degradation matrix ℋ
b) are tested. Every experiment is evaluated from the luminance peak signal-to-noise ratio (
Y
_
PSNR
) and SSIM and is compared with the state of the art methods such as Yang et al.’s [12, 24], Dong et al.’s  [25]. We thank the above authors to provide their program codes.


### 3.2. Experiments on Different Downsampled Image

In this test, our method is tested on several512
×
512 common experimental natural images such as Lena, Plane, and Pepper. The input 256
×
256  LR image is down-sampled from the original 512
×
512
HR image. We use both direct downsampling degradation matrix ℋ
d and blur downsampling degradation matrix ℋ
b to test the algorithm’s adaptability. At first, a sparse dictionary is trained by the 8
×
8  patches taken from input LR image. The dictionary has 128 atoms. Hence, the dictionary is a 64
×
128  matrix. Then, the 8
×
8  HR image patches are recovered by 4
×
4  LR image patches under our self-similarity based SR approach. We set 3 pixels overlap in LR patches by default. The neighbor searching zone is set to 10
×
10.Figure3 shows the experiment on the image Lena under different downsampling matrix. Figure 3(a) plots the original Lena image. Figures 3(b)–3(d) plot the HR Lena images recovered from ℋ
b down-sampled LR image, respectively, by Bicubic, Yang et al.’s [24], and our proposed methods. Recovered image by Dong et al.’s [25] NCSR method is also illustrated in Figure 3(e), which uses the elaborate Gaussian low-pass filter. Figures 3(e)–3(g) show the recovered HR Lena images from ℋ
d down-sampled LR image, respectively, by Bicubic, Yang et al.’s [24] method, and the proposed method. Dong et al.’s [25] NCSR method cannot get acceptable performance without Gaussian low-pass filter, which is not illustrated in Figure 3. These experimental results show that our method has better performance than the state of the art methods [12, 24, 25] in both cases. The Bicubic method could not recover the high frequency details in both cases. Although Yang et al.’s method [24] can recover the blur downsampled-LR image very well but produce too much artifact and fake high frequency details in the direct downsampling case.Experiment results on image Lena, from left to right: (a) original512
×
512  HR image, (b) ℋ
b sampled Bicubic (
PSNR
=
3
4.87
dB
), and (c) ℋ
b sampled Yang et al.’s [24] (
PSNR
=
35.7
5
dB
), (d) ℋ
b sampled Proposed method (
PSNR
=
35.8
0
dB
), (e) Dong et al.’s [25] NCSR  (
PSNR
=
35.45
dB
), (f) ℋ
d sampled Bicubic (
PSNR
=
30
.
96
dB
), (g) ℋ
d sampled Yang et al.’s [24] (
PSNR
=
29.09
dB
), and (h) ℋ
d sampled proposed method (
PSNR
=
33.82
dB
).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)Experiment result on image Pepper is shown in Figure4. Pepper has lots of edge, which is a preferable image to test the recover effect about edge. Similar result is derived. The edge recovered by Yang et al.’s method [24] is not clear when LR image is down-sampled by ℋ
d. This failure may be caused by the inconsistency between Yang et al.’s [24] pair of HR and LR dictionaries. In comparison, our proposed method can preserve the edge’s sharpness well. Besides the edge’s sharpness, recovered information by self-learning is more faithful to the true HR details.Experiment results on image Pepper, from left to right: (a) original512
×
512
HR image, (b) ℋ
b sampled Bicubic (
PSNR
=
33.08
dB
), (c) ℋ
b sampled Yang et al.’s [24] (
PSNR
=
33.84
dB
), (d) ℋ
b sampled proposed method (
PSNR
=
35.68
dB
), (e) Dong et al.’s [25] NCSR
(
PSNR
=
34
.
92
dB
)  (f), ℋ
d sampled bicubic (
PSNR
=
29.78
dB
), (g) ℋ
d sampled Yang et al.’s [24] (
PSNR
=
28.3943
dB
), and (h) ℋ
d sampled proposed method (
PSNR
=
33.25
dB
).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)More bench-mark comparisons are illustrated in Table2. Our proposed method shows high recovery performance under both kinds of downsampling degradation matrix. The comparison shows that self-similarity is a powerful image-specific prior for sparse representation SR method.Table 2
Comparison results of differentSR methods.
Image
Downsampling matrix
Measures
Methods
Bicubic
Yang et al. [24]
Dong et al. [25]
Proposed
Lena
Blur
PSNR (dB)
34.8671
35.7477
35.46
35.8
SSIM
0.8538
0.8586
0.9067
0.9237
Direct
PSNR (dB)
30.964
29.0875
33.8206
SSIM
0.7873
0.7322
0.8098
Plane
Blur
PSNR (dB)
32.6525
33.5788
34.05
34.9828
SSIM
0.9341
0.9369
0.9487
0.9629
Direct
PSNR (dB)
28.8573
27.5349
31.5616
SSIM
0.8985
0.8525
0.9141
Pepper
Blur
PSNR (dB)
33.0847
33.8412
34.92
35.6808
SSIM
0.8382
0.8474
0.9028
0.9173
Direct
PSNR (dB)
29.7824
28.3943
33.2525
SSIM
0.7712
0.7229
0.7754
Sailboat
Blur
PSNR (dB)
31.065
31.6411
32.05
32.1125
SSIM
0.8143
0.8294
0.8826
0.8979
Direct
PSNR (dB)
27.7443
26.0675
30.0478
SSIM
0.7537
0.6893
0.756
Baboon
Blur
PSNR (dB)
25.0694
25.0966
25.13
25.4071
SSIM
0.7068
0.7388
0.7375
0.7479
Direct
PSNR (dB)
22.661
20.5052
23.6535
SSIM
0.611
0.5595
0.6217Images produced by industrial environment sensors are tested too, as shown below in Figures5 and 6. Recovered high resolution images in Figure 6 show the effectiveness of our approach.Figure 5
Low resolution test images from industrial environment sensors.Figure 6
Recovered high resolution test images from industrial environment sensorsscale
factor
=
2.Furthermore, we do experiments on Forman video sequence to test the stability of our algorithm. All the frames are processed as an image. Figure7 shows the PSNR comparison between the proposed method and Bicubic method. The proposed approach stably outperforms the Bicubic method. From about the 210
th frame, recovery performance decays rapidly, since the followed frames are full of wild high frequency details.Figure 7
Self-similarity basedSR performance on Foreman video sequence.


### 3.3. Influence of Different Parameters

To further observe different parameter’s impact, several comparison experiments are conducted.


#### 3.3.1. Influence of Dictionary Size

Another advantage of the proposed approach is that the sparse dictionary only needs a small amount of atoms. 128 atoms are enough to get a favorable result for the proposed method. Meanwhile, Yang et al.’s method [12, 24] needs to train external dictionaries at least 512 atoms. In [22], Yang et al. propose a CS-based SR method, which also needs to train a dictionary with 500 atoms by external database. Comparison experiments are conducted on gray 512
×
512 natural images, including Lena, Pepper, and Boat. Table 3 shows the recovery PSNR of three sparse based SR methods with different dictionary sizes. The proposed method can recover favorable HR images by the smallest dictionary. Test results show that the proposed self-similarity learning method is more suitable for resource-constrained image sensor node.Table 3
Recovery PSNR of three sparse basedSR methods with different dictionary sizes.
Image
Bicubic (dB)
Yang et al. [24] (dB)1024 atoms
Yang et al. [22] (dB)500 atoms
Proposed (dB)128 atoms
Lena
29.8545
33.4116
33.4302
33.6360
Pepper
28.8546
31.4435
31.5071
31.9216
Boat
26.7158
30.4966
30.5984
30.8878For external dictionary basedSR method, the recovery performance gets better as the dictionary size is growing larger. Figure 8 shows another comparison on Lena between Yang et al.’s method [24] and the proposed method. Yang et al.’s method [24] is conducted by a series of dictionary sizes of 256, 512, 1024, and 2048. The proposed method is conducted by different dictionary sizes of 64, 128, 256, and 512. We use the increment PSNR to Bicubic method as the comparison index. As PSNR growth curve shown in Figure 8, we can see that the recovery performance of Yang et al.’s method [24] relies much more on the dictionary size. Its dictionary size should be three times larger than the dictionary size in the proposed method. By contrast, our approach gives a stable performance on different dictionary sizes.Figure 8
Recovery performance comparison over different dictionary sizes.


#### 3.3.2. Influence of Self-Similarity Searching Zone

Self-similarity is introduced as the sparse regulation prior in our approach. The above tests show its effectiveness and stability in preserving the detailed information such as edge sharpness. The size of self-similarity searching zone is tested here, using test image Tank from8
×
8 to 14
×
14 neighborhood. Results are illustrated in Table 4 and Figure 9. The similar patches found are shown in Figure 10. The experiment tells us that more edge patches can be found, and the recovery performance gets better, when the searching zone size increases.Table 4
Recovery effects of different size searching zones on image Tank.
Searching zone
8
×
8
10
×
10
12
×
12
14
×
14
PSNR (dB)
34.4982
34.6764
34.8211
34.8985
SSIM
0.9057
0.9085
0.9106
0.9114Recovery effects of different sized searching zones on image Tank, from left to right: (a)8
×
8, (b) 10
×
10, (c) 12
×
12, and (d) 14
×
14.
(a)
(b)
(c)
(d)Similar patches found by different sized searching zones, from left to right: (a)8
×
8, (b) 10
×
10, (c) 12
×
12, and (d) 14
×
14.
(a)
(b)
(c)
(d)


### 3.4. Limitation and Further Research Direction

Although we have shown the outstanding performance of the proposed self-similarity super resolution approach, there are still some limitation that should be considered. The proposed method assumes that the blur matrix is known as mostSR methods. Further research should consider how to estimate the optimal blur kernel under the blind circumstance. Another point is that the SSE self-similarity prior used in the proposed algorithm is quite simple. We will use more delicate prior such as Parzen window estimation [31], BM3D [32], and so forth, to get a better match with the destination HR patch.


## 3.1. Experimental Background

In this section, several experimental results for the proposed method are given. All the simulations are conducted in MATLAB 7.5 onPC with Intel Core2/1.6 GHz/1 GB. The test LR images include several typical 256
×
256 natural images. We aim to recover their 512
×
512
HR images. The input LR images with different degradation matrix ℋ (direct downsampling degradation matrix ℋ
d and blur down-sampling degradation matrix ℋ
b) are tested. Every experiment is evaluated from the luminance peak signal-to-noise ratio (
Y
_
PSNR
) and SSIM and is compared with the state of the art methods such as Yang et al.’s [12, 24], Dong et al.’s  [25]. We thank the above authors to provide their program codes.


## 3.2. Experiments on Different Downsampled Image

In this test, our method is tested on several512
×
512 common experimental natural images such as Lena, Plane, and Pepper. The input 256
×
256  LR image is down-sampled from the original 512
×
512
HR image. We use both direct downsampling degradation matrix ℋ
d and blur downsampling degradation matrix ℋ
b to test the algorithm’s adaptability. At first, a sparse dictionary is trained by the 8
×
8  patches taken from input LR image. The dictionary has 128 atoms. Hence, the dictionary is a 64
×
128  matrix. Then, the 8
×
8  HR image patches are recovered by 4
×
4  LR image patches under our self-similarity based SR approach. We set 3 pixels overlap in LR patches by default. The neighbor searching zone is set to 10
×
10.Figure3 shows the experiment on the image Lena under different downsampling matrix. Figure 3(a) plots the original Lena image. Figures 3(b)–3(d) plot the HR Lena images recovered from ℋ
b down-sampled LR image, respectively, by Bicubic, Yang et al.’s [24], and our proposed methods. Recovered image by Dong et al.’s [25] NCSR method is also illustrated in Figure 3(e), which uses the elaborate Gaussian low-pass filter. Figures 3(e)–3(g) show the recovered HR Lena images from ℋ
d down-sampled LR image, respectively, by Bicubic, Yang et al.’s [24] method, and the proposed method. Dong et al.’s [25] NCSR method cannot get acceptable performance without Gaussian low-pass filter, which is not illustrated in Figure 3. These experimental results show that our method has better performance than the state of the art methods [12, 24, 25] in both cases. The Bicubic method could not recover the high frequency details in both cases. Although Yang et al.’s method [24] can recover the blur downsampled-LR image very well but produce too much artifact and fake high frequency details in the direct downsampling case.Experiment results on image Lena, from left to right: (a) original512
×
512  HR image, (b) ℋ
b sampled Bicubic (
PSNR
=
3
4.87
dB
), and (c) ℋ
b sampled Yang et al.’s [24] (
PSNR
=
35.7
5
dB
), (d) ℋ
b sampled Proposed method (
PSNR
=
35.8
0
dB
), (e) Dong et al.’s [25] NCSR  (
PSNR
=
35.45
dB
), (f) ℋ
d sampled Bicubic (
PSNR
=
30
.
96
dB
), (g) ℋ
d sampled Yang et al.’s [24] (
PSNR
=
29.09
dB
), and (h) ℋ
d sampled proposed method (
PSNR
=
33.82
dB
).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)Experiment result on image Pepper is shown in Figure4. Pepper has lots of edge, which is a preferable image to test the recover effect about edge. Similar result is derived. The edge recovered by Yang et al.’s method [24] is not clear when LR image is down-sampled by ℋ
d. This failure may be caused by the inconsistency between Yang et al.’s [24] pair of HR and LR dictionaries. In comparison, our proposed method can preserve the edge’s sharpness well. Besides the edge’s sharpness, recovered information by self-learning is more faithful to the true HR details.Experiment results on image Pepper, from left to right: (a) original512
×
512
HR image, (b) ℋ
b sampled Bicubic (
PSNR
=
33.08
dB
), (c) ℋ
b sampled Yang et al.’s [24] (
PSNR
=
33.84
dB
), (d) ℋ
b sampled proposed method (
PSNR
=
35.68
dB
), (e) Dong et al.’s [25] NCSR
(
PSNR
=
34
.
92
dB
)  (f), ℋ
d sampled bicubic (
PSNR
=
29.78
dB
), (g) ℋ
d sampled Yang et al.’s [24] (
PSNR
=
28.3943
dB
), and (h) ℋ
d sampled proposed method (
PSNR
=
33.25
dB
).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)More bench-mark comparisons are illustrated in Table2. Our proposed method shows high recovery performance under both kinds of downsampling degradation matrix. The comparison shows that self-similarity is a powerful image-specific prior for sparse representation SR method.Table 2
Comparison results of differentSR methods.
Image
Downsampling matrix
Measures
Methods
Bicubic
Yang et al. [24]
Dong et al. [25]
Proposed
Lena
Blur
PSNR (dB)
34.8671
35.7477
35.46
35.8
SSIM
0.8538
0.8586
0.9067
0.9237
Direct
PSNR (dB)
30.964
29.0875
33.8206
SSIM
0.7873
0.7322
0.8098
Plane
Blur
PSNR (dB)
32.6525
33.5788
34.05
34.9828
SSIM
0.9341
0.9369
0.9487
0.9629
Direct
PSNR (dB)
28.8573
27.5349
31.5616
SSIM
0.8985
0.8525
0.9141
Pepper
Blur
PSNR (dB)
33.0847
33.8412
34.92
35.6808
SSIM
0.8382
0.8474
0.9028
0.9173
Direct
PSNR (dB)
29.7824
28.3943
33.2525
SSIM
0.7712
0.7229
0.7754
Sailboat
Blur
PSNR (dB)
31.065
31.6411
32.05
32.1125
SSIM
0.8143
0.8294
0.8826
0.8979
Direct
PSNR (dB)
27.7443
26.0675
30.0478
SSIM
0.7537
0.6893
0.756
Baboon
Blur
PSNR (dB)
25.0694
25.0966
25.13
25.4071
SSIM
0.7068
0.7388
0.7375
0.7479
Direct
PSNR (dB)
22.661
20.5052
23.6535
SSIM
0.611
0.5595
0.6217Images produced by industrial environment sensors are tested too, as shown below in Figures5 and 6. Recovered high resolution images in Figure 6 show the effectiveness of our approach.Figure 5
Low resolution test images from industrial environment sensors.Figure 6
Recovered high resolution test images from industrial environment sensorsscale
factor
=
2.Furthermore, we do experiments on Forman video sequence to test the stability of our algorithm. All the frames are processed as an image. Figure7 shows the PSNR comparison between the proposed method and Bicubic method. The proposed approach stably outperforms the Bicubic method. From about the 210
th frame, recovery performance decays rapidly, since the followed frames are full of wild high frequency details.Figure 7
Self-similarity basedSR performance on Foreman video sequence.


## 3.3. Influence of Different Parameters

To further observe different parameter’s impact, several comparison experiments are conducted.


### 3.3.1. Influence of Dictionary Size

Another advantage of the proposed approach is that the sparse dictionary only needs a small amount of atoms. 128 atoms are enough to get a favorable result for the proposed method. Meanwhile, Yang et al.’s method [12, 24] needs to train external dictionaries at least 512 atoms. In [22], Yang et al. propose a CS-based SR method, which also needs to train a dictionary with 500 atoms by external database. Comparison experiments are conducted on gray 512
×
512 natural images, including Lena, Pepper, and Boat. Table 3 shows the recovery PSNR of three sparse based SR methods with different dictionary sizes. The proposed method can recover favorable HR images by the smallest dictionary. Test results show that the proposed self-similarity learning method is more suitable for resource-constrained image sensor node.Table 3
Recovery PSNR of three sparse basedSR methods with different dictionary sizes.
Image
Bicubic (dB)
Yang et al. [24] (dB)1024 atoms
Yang et al. [22] (dB)500 atoms
Proposed (dB)128 atoms
Lena
29.8545
33.4116
33.4302
33.6360
Pepper
28.8546
31.4435
31.5071
31.9216
Boat
26.7158
30.4966
30.5984
30.8878For external dictionary basedSR method, the recovery performance gets better as the dictionary size is growing larger. Figure 8 shows another comparison on Lena between Yang et al.’s method [24] and the proposed method. Yang et al.’s method [24] is conducted by a series of dictionary sizes of 256, 512, 1024, and 2048. The proposed method is conducted by different dictionary sizes of 64, 128, 256, and 512. We use the increment PSNR to Bicubic method as the comparison index. As PSNR growth curve shown in Figure 8, we can see that the recovery performance of Yang et al.’s method [24] relies much more on the dictionary size. Its dictionary size should be three times larger than the dictionary size in the proposed method. By contrast, our approach gives a stable performance on different dictionary sizes.Figure 8
Recovery performance comparison over different dictionary sizes.


### 3.3.2. Influence of Self-Similarity Searching Zone

Self-similarity is introduced as the sparse regulation prior in our approach. The above tests show its effectiveness and stability in preserving the detailed information such as edge sharpness. The size of self-similarity searching zone is tested here, using test image Tank from8
×
8 to 14
×
14 neighborhood. Results are illustrated in Table 4 and Figure 9. The similar patches found are shown in Figure 10. The experiment tells us that more edge patches can be found, and the recovery performance gets better, when the searching zone size increases.Table 4
Recovery effects of different size searching zones on image Tank.
Searching zone
8
×
8
10
×
10
12
×
12
14
×
14
PSNR (dB)
34.4982
34.6764
34.8211
34.8985
SSIM
0.9057
0.9085
0.9106
0.9114Recovery effects of different sized searching zones on image Tank, from left to right: (a)8
×
8, (b) 10
×
10, (c) 12
×
12, and (d) 14
×
14.
(a)
(b)
(c)
(d)Similar patches found by different sized searching zones, from left to right: (a)8
×
8, (b) 10
×
10, (c) 12
×
12, and (d) 14
×
14.
(a)
(b)
(c)
(d)


## 3.3.1. Influence of Dictionary Size

Another advantage of the proposed approach is that the sparse dictionary only needs a small amount of atoms. 128 atoms are enough to get a favorable result for the proposed method. Meanwhile, Yang et al.’s method [12, 24] needs to train external dictionaries at least 512 atoms. In [22], Yang et al. propose a CS-based SR method, which also needs to train a dictionary with 500 atoms by external database. Comparison experiments are conducted on gray 512
×
512 natural images, including Lena, Pepper, and Boat. Table 3 shows the recovery PSNR of three sparse based SR methods with different dictionary sizes. The proposed method can recover favorable HR images by the smallest dictionary. Test results show that the proposed self-similarity learning method is more suitable for resource-constrained image sensor node.Table 3
Recovery PSNR of three sparse basedSR methods with different dictionary sizes.
Image
Bicubic (dB)
Yang et al. [24] (dB)1024 atoms
Yang et al. [22] (dB)500 atoms
Proposed (dB)128 atoms
Lena
29.8545
33.4116
33.4302
33.6360
Pepper
28.8546
31.4435
31.5071
31.9216
Boat
26.7158
30.4966
30.5984
30.8878For external dictionary basedSR method, the recovery performance gets better as the dictionary size is growing larger. Figure 8 shows another comparison on Lena between Yang et al.’s method [24] and the proposed method. Yang et al.’s method [24] is conducted by a series of dictionary sizes of 256, 512, 1024, and 2048. The proposed method is conducted by different dictionary sizes of 64, 128, 256, and 512. We use the increment PSNR to Bicubic method as the comparison index. As PSNR growth curve shown in Figure 8, we can see that the recovery performance of Yang et al.’s method [24] relies much more on the dictionary size. Its dictionary size should be three times larger than the dictionary size in the proposed method. By contrast, our approach gives a stable performance on different dictionary sizes.Figure 8
Recovery performance comparison over different dictionary sizes.


## 3.3.2. Influence of Self-Similarity Searching Zone

Self-similarity is introduced as the sparse regulation prior in our approach. The above tests show its effectiveness and stability in preserving the detailed information such as edge sharpness. The size of self-similarity searching zone is tested here, using test image Tank from8
×
8 to 14
×
14 neighborhood. Results are illustrated in Table 4 and Figure 9. The similar patches found are shown in Figure 10. The experiment tells us that more edge patches can be found, and the recovery performance gets better, when the searching zone size increases.Table 4
Recovery effects of different size searching zones on image Tank.
Searching zone
8
×
8
10
×
10
12
×
12
14
×
14
PSNR (dB)
34.4982
34.6764
34.8211
34.8985
SSIM
0.9057
0.9085
0.9106
0.9114Recovery effects of different sized searching zones on image Tank, from left to right: (a)8
×
8, (b) 10
×
10, (c) 12
×
12, and (d) 14
×
14.
(a)
(b)
(c)
(d)Similar patches found by different sized searching zones, from left to right: (a)8
×
8, (b) 10
×
10, (c) 12
×
12, and (d) 14
×
14.
(a)
(b)
(c)
(d)


## 3.4. Limitation and Further Research Direction

Although we have shown the outstanding performance of the proposed self-similarity super resolution approach, there are still some limitation that should be considered. The proposed method assumes that the blur matrix is known as mostSR methods. Further research should consider how to estimate the optimal blur kernel under the blind circumstance. Another point is that the SSE self-similarity prior used in the proposed algorithm is quite simple. We will use more delicate prior such as Parzen window estimation [31], BM3D [32], and so forth, to get a better match with the destination HR patch.


## 4. Conclusion

This paper has presented a novel double self-similarity super resolution approach for the resource-constrained image sensor node in the wireless sensor networks. The proposed method does not need external database and only uses theLR image itself as the training sample for sparse representation dictionary with a small number of atoms. Self-similarity sparse prior is combined in the regulation iteration to preserve the detailed information. Experiments are conducted on bench-mark test images. The effects of different parameters have been surveyed. Comparative tests show the effectiveness and stability of the proposed method over the state of the art sparse based SR methods.



---

*Source: 719408-2014-01-21.xml*