# Neural Network for Sparse Reconstruction

Authors: Qingfa Li; Yaqiu Liu; Liangkuan Zhu
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

We construct a neural network based on smoothing approximation techniques and projected gradient method to solve a kind of sparse reconstruction problems. Neural network can be implemented by circuits and can be seen as an important method for solving optimization problems, especially large scale problems. Smoothing approximation is an efficient technique for solving nonsmooth optimization problems. We combine these two techniques to overcome the difficulties of the choices of the step size in discrete algorithms and the item in the set-valued map of differential inclusion. In theory, the proposed network can converge to the optimal solution set of the given problem. Furthermore, some numerical experiments show the effectiveness of the proposed network in this paper.


---

## Body

## 1. Introduction

Sparse reconstruction is the term used to describe the process of extracting some underlying original source signals from a number of observed mixture signals, where the mixing model is either unknown or the knowledge about the mixing process is limited. The problem of recovering a sparse signal from noisy linear observation arises in many real world sensing applications [1–5]. Mathematically, a signal recovery problem can be formulated as estimating the original signal based on noisy linear observations, which can be expressed as
(1)
b
=
A
x
+
η
,
where A
∈
ℝ
m
×
n is the mixing matrix, x
∈
ℝ
n is the original signal, b
∈
ℝ
m is the observed signal, and η
∈
ℝ
m is the noise. In many cases, A is a matrix of block Toeplitz with Toeplitz blocks (BTTB) when zero boundary conditions are applied and block Toeplitz-plus-Hankel with Toeplitz-plus-Hankel blocks (BTHTHB) when Neumann boundary conditions are used [6]. Then, this problem can be viewed as a linear inverse problem. A standard approach to solve linear inverse problems is to define a suitable objective function and to minimize it. It is often divided into two steps to solve this problem, which are estimation of mixture matrix A and recovery of original signal x. In this paper, we focus on the study of the second step, where we assume that we have known mixture matrix A.Generally, finding a solution with few nonzero entries for an underdetermined linear system with noise is often modeled as the regularization problem:(2)
min
⁡
∥
A
x
-
b
∥
2
+
λ
∥
x
∥
0
,
where λ
>
0 and ∥
x
∥
0 is defined by the number of nonzero entries in x. However, the l
2
-
l
0 regularized problem (2) is difficult to deal with because of the discrete structure of the l
0 norm, which derives researchers to pay attention to the continuous l
2
-
l
1 minimization problem:
(3)
min
⁡
∥
A
x
-
b
∥
2
+
λ
∥
x
∥
1
.
The first term in (3) is often called as date fitting term, which forces the solutions of (3) closeness to the data, and the second term in it is often called as regularization or potential term, which is to push the solutions to exhibit some prior expected features. Under certain conditions, l
2
-
l
1 problem and l
2
-
l
0 problem have the same solution sets [7]. The l
2
-
l
1 problem is a continuous convex optimization problem and can be efficiently solved, which is known as Lasso [8].A class of signal recovery problems can be formulated as(4)
min
⁡
∥
A
x
-
b
∥
2
+
λ
∥
D
x
∥
1
s
.
t
.
x
∈
Ω
,
where D is a linear operator, λ is the regularization parameter that controls the trade-off between the regularization term and the data-fitting term, and constraint set Ω is a closed convex subset of ℝ
n.Optimization problems arise in a variety of scientific and engineering applications and they really need real time solutions. Since the computing time greatly depends on the dimension and the structure of the optimization problems, numerical algorithms are usually less effective in large scale or real time optimization problems. In many applications, real time optimal solutions are usually imperative, such as on-board signal processing and robot motion planning and control. One promising approach to handle these problems is to employ artificial neural network. During recent decades, neural dynamical method for solving optimization problems has been a major area in neural network research based on circuit implementation [9–12]. First, the structure of a neural network can be implemented physically by designated hardware such as application-specific integrated circuits where the computational procedure is distributed and parallel. This lets the neural network approach solve optimization problems in running time at the order of magnitude much faster than conventional optimization algorithms executed on general-purpose digital computers. Second, neural networks can solve many optimization problems with time-varying parameters. Third, the dynamical and ODE techniques can be applied to the continuous-time neural networks. And recent reports have shown that the global convergence can be obtained by the neural network approach under some weaker conditions.Since the neural network was first proposed for solving linear [13, 14] and nonlinear [15] programming problems, many researchers were inspired to develop neural networks for optimization. Many types of neural networks have been proposed to solve various optimization problems, for example, the recurrent neural network, the Lagrangian network, the deterministic annealing network, the projection-type neural network, a generalized neural network, and so forth. In [16], Chong et al. proposed a neural network for linear programming problem with finite time convergence. In [17], a generalized neural network was presented for solving a class of nonsmooth convex optimization problems. In [18], a neural network was defined by using the penalty function method and differential inclusion for solving a class of nonsmooth convex optimization problems. In fact, in many important applications, neural network built by a differential inclusion is an important method to solve a class of nonsmooth optimization problems. One has to mention that the optimization problems are not differentiable in many important applications. Moreover, the neural networks for smooth optimization problems required the gradients of the objective and constrained functions in such neural networks. So these networks cannot solve nonsmooth optimization problems. Using smoothing techniques in neural network is an effective method for solving nonsmooth optimization problems [19, 20]. The main feature of smoothing method is to approximate the nonsmooth functions by parameterized smooth functions [21, 22]. By smoothing approximations, we can give a class of smooth functions, which converge to the original nonsmooth function and whose gradients converge to the subgradient of nonsmooth function. For solving many constrained optimization problems, projection is a simple and effective method for solving the constraints. In [23, 24], projection had been used in neural networks for solving some kind of constrained optimization problems.Basing on the advantages of the neural networks, in this paper, we will propose a neural network and use some mathematical techniques to solve optimization problem (4). The problem (4) is nonsmooth. Many neural networks are modeled by differential inclusions, which have the difficulty in the choice of the right set-valued map. In this paper, we will introduce a smoothing function to overcome this problem. Using smoothing techniques into neural network is an interesting and promise method for solving (4).Notation. Throughout this paper, ∥
·
∥ denotes the l
2 norm and ∥
·
∥
1 denotes the l
1 norm.


## 2. Preliminary Results

In this section, we will introduce several basic definitions and lemmas, which are used in the development.Definition 1.
Suppose thatf is Lipschitz near x; the generalized directional derivative of f at x in the direction v
∈
ℝ
n is given by
(5)
f
0
(
x
;
v
)
=
limsup
⁡
y
→
x
;
r
→
0
+
f
(
y
+
r
v
)
-
f
(
y
)
r
.
Furthermore, the Clarke generalized gradient of f at x is defined as
(6)
∂
f
(
x
)
=
{
ξ
∈
ℝ
n
:
f
0
(
x
;
v
)
≥
〈
v
,
ξ
〉
,
∀
v
∈
ℝ
n
}
.Moreover, iff
:
ℝ
n
→
ℝ is a convex function, then it has the following properties as well.Proposition 2.
Iff
:
ℝ
n
→
ℝ is a convex function, the following property holds:
(7)
f
(
x
)
-
f
(
y
)
≤
〈
p
,
x
-
y
〉
,
∀
x
,
y
∈
ℝ
n
,
∀
p
∈
∂
f
(
x
)
.Since the constraint set of (4) is a closed convex subset of ℝ
n, then we use the projection operator to handle the constraint. The projection operator of x to the closed convex subset Ω is defined by
(8)
P
Ω
(
x
)
=
arg
min
⁡
u
∈
Ω
∥
u
-
x
∥
.The projection operator has the following properties.Proposition 3.
Consider the following:(9)
〈
v
-
P
Ω
(
v
)
,
P
Ω
(
v
)
-
u
〉
≥
0
,
∀
v
∈
ℝ
n
,
u
∈
Ω
,
∥
P
Ω
(
u
)
-
P
Ω
(
v
)
∥
≤
∥
u
-
v
∥
,
∀
u
,
v
∈
ℝ
n
.Definition 4.
Leth
:
ℝ
n
→
ℝ be a locally Lipschitz function. We call h
~
:
ℝ
n
×
[
0
,
+
∞
)
→
ℝ a smoothing function of h, if h
~ satisfies the following conditions. (i)
For any fixedμ
>
0, h
~
(
·
,
μ
) is continuously differentiable in ℝ
n, and for any fixed x
∈
ℝ
n, h
~
(
x
,
·
) is differentiable in [
0
,
+
∞
).
(ii)
For any fixedx
∈
ℝ
n, lim
⁡
μ
↓
0
h
~
(
x
,
μ
)
=
h
(
x
).
(iii)
There is a positive constantκ
h
~
>
0 such that
(10)
|
∇
μ
h
~
(
x
,
μ
)
|
≤
κ
h
~
,
∀
μ
∈
[
0
,
+
∞
)
,
x
∈
ℝ
n
.
(iv)
{
lim
⁡
z
→
x
,
μ
↓
0
∇
z
h
~
(
z
,
μ
)
}
⊆
∂
h
(
x
).From the above definition, we can get that for any fixedx
∈
ℝ
n,
(11)
lim
⁡
z
→
x
,
μ
↓
0
h
~
(
z
,
μ
)
=
h
(
x
)
,
|
h
~
(
x
,
μ
)
-
h
(
x
)
|
≤
κ
h
~
μ
,
∀
μ
∈
[
0
,
+
∞
)
,
x
∈
ℝ
n
.Next, we present a smoothing function of the absolute value function, which is defined by(12)
φ
(
y
,
μ
)
=
{
|
y
|
if
|
y
|
≥
μ
,
y
2
2
μ
+
μ
2
if
|
y
|
<
μ
.Proposition 5 (see [21]).
Consider the following:(i)
φ
(
y
,
μ
)
is continuously differentiable about
y
in
ℝ
for any fixed
μ
>
0
and differentiable about
μ
for any fixed
y
∈
ℝ
;
(ii)
0
≤
∇
μ
φ
(
y
,
μ
)
≤
1
, for all
y
∈
ℝ
, for all
μ
∈
(
0,1
]
;
(iii)
0
≤
φ
(
y
,
μ
)
-
|
y
|
≤
μ
/
2
, for all
y
∈
ℝ
, for all
μ
∈
(
0,1
]
;
(iv)
φ
(
y
,
μ
)
is convex about
y
for any fixed
μ
and
{
lim
⁡
μ
↓
0
∇
y
φ
(
y
,
μ
)
}
⊆
∂
|
y
|.


## 3. Theoretical Results

In (4), D
∈
ℝ
m
×
n can be rewritten as D
=
(
d
1
,
d
2
,
…
,
d
m
)
T, where d
i
(
i
=
1,2
,
…
,
m
) is an n dimensional vector. Then (4) can be rewritten as
(13)
min
⁡
∥
A
x
-
b
∥
2
+
λ
∑
i
=
1
m
‍
|
d
i
T
x
|
s
.
t
.
x
∈
Ω
.In the following, we use∑
i
=
1
m
‍
φ
(
d
i
T
x
,
μ
) to approximate ∑
i
=
1
m
‍
|
d
i
T
x
|. From the idea of the projected gradient method, we construct our neural network as follows:
(14)
x
˙
(
t
)
=
-
x
(
t
)
+
P
Ω
[
∑
i
=
1
m
‍
x
(
t
)
-
2
A
T
(
A
x
(
t
)
-
b
)
-
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
]
,
where x
(
0
)
=
x
0, μ
(
t
)
=
e
-
t, and P
Ω is the projection operator on Ω.Next, we will give some analysis on the proposed neural network (14).Theorem 6.
For any initial pointx
0
∈
Ω, there is a global and uniformly bounded solution of (14).Proof.
The right hand of (14) is continuous about x and t, then there is a local solution of (14) with x
0
∈
Ω. And we assume that [
0
,
T
) is the maximal existence interval of t. First, we prove that x
(
t
)
∈
Ω for all t
∈
[
0
,
T
). Obviously, (14) can be rewritten as
(15)
x
˙
(
t
)
+
x
(
t
)
=
P
Ω
[
∑
i
=
1
m
‍
x
(
t
)
-
2
A
T
(
A
x
(
t
)
-
b
)
-
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
]
.
From the integration about the above differential equation, we have(16)
x
(
t
)
=
e
-
t
x
0
+
(
1
-
e
t
)
∫
0
t
‍
k
(
s
)
e
s
e
t
-
1
d
s
,
where k
(
t
)
=
P
Ω
[
x
(
t
)
-
2
A
T
(
A
x
(
t
)
-
b
)
-
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
].
Since∫
0
t
‍
(
e
s
/
(
e
t
-
1
)
)
d
s
=
1, x
0
∈
Ω, and Ω is a closed convex subset, we confirm that
(17)
x
(
t
)
∈
Ω
,
∀
t
∈
[
0
,
T
)
.
Differentiating∥
A
x
(
t
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
) along this solution of (14), we obtain
(18)
d
d
t
[
∥
A
x
(
t
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
]
=
〈
2
A
T
(
A
x
(
t
)
-
b
)
,
x
˙
(
t
)
〉
+
λ
∑
i
=
1
m
‍
[
〈
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
,
x
˙
(
t
)
〉
+
∇
μ
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
μ
˙
(
t
)
]
≤
〈
2
A
T
(
A
x
(
t
)
-
b
)
+
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
,
x
˙
(
t
)
〉
.
Using the inequality of project operator to (14), we obtain
(19)
〈
2
A
T
(
A
x
(
t
)
-
b
)
+
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
,
x
˙
(
t
)
〉
≤
-
∥
x
˙
(
t
)
∥
2
.
Thus,(20)
d
d
t
[
∥
A
x
(
t
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
]
≤
-
∥
x
˙
(
t
)
∥
2
,
which follows that ∥
A
x
(
t
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
) is nonincreasing along the solution of (14). On the other hand, by Proposition 5, we know that
(21)
∥
A
x
(
0
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
0
)
,
μ
(
0
)
)
≥
∥
A
x
(
t
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
≥
∥
A
x
(
t
)
-
b
∥
2
+
λ
∥
D
x
(
t
)
∥
1
.
Thus,x
(
t
) is bounded on [
0
,
T
). Using the extension theorem, the solution of (14) is globally existent and uniformly bounded.Theorem 7.
For any initial pointx
0
∈
Ω, the solution of (14) is unique and satisfies the following:(i)
∥
x
˙
(
t
)
∥ is nonincreasing on [
0
,
+
∞
) and lim
⁡
t
→
+
∞
∥
x
˙
(
t
)
∥
=
0;
(ii)
the solution of (14) is convergent to the optimal solution set of (4).Proof.
Suppose that there exist two solutionsx
:
[
0
,
∞
)
→
ℝ
n and y
:
[
0
,
∞
)
→
ℝ
n of (14) with initial point x
0
=
y
0, which means that
(22)
x
˙
(
t
)
=
-
x
(
t
)
+
P
Ω
[
∑
i
=
1
m
‍
x
(
t
)
-
2
A
T
(
A
x
(
t
)
-
b
)
-
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
]
,
y
˙
(
t
)
=
-
y
(
t
)
+
P
Ω
[
∑
i
=
1
m
‍
y
(
t
)
-
2
A
T
(
A
x
(
t
)
-
b
)
-
λ
∑
i
=
1
m
‍
∇
y
φ
(
d
i
T
y
(
t
)
,
μ
(
t
)
)
]
.
Thus,
(23)
d
d
t
1
2
∥
x
(
t
)
-
y
(
t
)
∥
2
=
〈
x
(
t
)
-
y
(
t
)
,
x
˙
(
t
)
-
y
˙
(
t
)
〉
=
-
∥
x
(
t
)
-
y
(
t
)
∥
2
+
〈
x
(
t
)
-
y
(
t
)
,
P
Ω
[
ξ
1
(
t
)
]
-
P
Ω
[
ξ
2
(
t
)
]
〉
,
where ξ
1
(
t
)
=
x
(
t
)
-
2
A
T
(
A
x
(
t
)
-
b
)
-
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
) and ξ
2
(
t
)
=
y
(
t
)
-
2
A
T
(
A
x
(
t
)
-
b
)
-
λ
∑
i
=
1
m
‍
∇
y
φ
(
d
i
T
y
(
t
)
,
μ
(
t
)
).
From the expression ofP
Ω and x
(
t
)
,
y
(
t
)
∈
Ω, for all t
≥
0, we get
(24)
〈
ξ
1
(
t
)
-
P
Ω
[
ξ
1
(
t
)
]
,
x
(
t
)
-
y
(
t
)
〉
=
0
,
∀
t
≥
0
,
〈
ξ
2
(
t
)
-
P
Ω
[
ξ
2
(
t
)
]
,
x
(
t
)
-
y
(
t
)
〉
=
0
,
∀
t
≥
0
.
Thus, we have
(25)
〈
x
(
t
)
-
y
(
t
)
,
P
Ω
[
ξ
1
(
t
)
]
-
P
Ω
[
ξ
2
(
t
)
]
〉
=
〈
x
(
t
)
-
y
(
t
)
,
ξ
1
(
t
)
-
ξ
2
(
t
)
〉
=
∥
x
(
t
)
-
y
(
t
)
∥
2
-
2
∥
A
(
x
(
t
)
-
y
(
t
)
)
∥
2
-
λ
∑
i
=
1
m
‍
〈
x
(
t
)
-
y
(
t
)
,
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
-
∇
y
φ
(
d
i
T
y
(
t
)
,
μ
(
t
)
)
〉
.
Sinceφ
(
d
i
T
y
,
μ
), (
i
=
1,2
,
…
,
m
) is convex about y for any fixed μ, we have
(26)
〈
x
-
y
,
∇
x
φ
(
d
i
T
x
,
μ
)
-
∇
y
φ
(
d
i
T
y
,
μ
)
〉
≥
0
,
∀
x
,
y
∈
ℝ
n
.
Thus, for all t
≥
0,
(27)
∑
i
=
1
m
‍
〈
x
(
t
)
-
y
(
t
)
,
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
-
∇
y
φ
(
d
i
T
y
(
t
)
,
μ
(
t
)
)
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
〉
≥
0
.
Using (25) and (27) into (23), we have
(28)
d
d
t
1
2
∥
x
(
t
)
-
y
(
y
)
∥
2
iii
=
-
2
∥
A
(
x
(
t
)
-
y
(
t
)
)
∥
2
iiiiiii
-
〈
x
(
t
)
-
y
(
t
)
,
λ
∑
i
=
1
m
‍
[
∇
x
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
-
∇
y
φ
(
d
i
T
y
(
t
)
,
μ
(
t
)
)
]
∑
i
=
1
m
‍
〉
≤
0
,
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
∀
t
∈
[
0
,
+
∞
)
.
By integrating (28) from 0 to t, we derive that
(29)
sup
⁡
t
≥
0
∥
x
(
t
)
-
y
(
t
)
∥
≤
∥
x
0
-
y
0
∥
.
Therefore, x
(
t
)
=
y
(
t
), for all t
≥
0, when x
0
=
y
0, which derives the uniqueness of the solution of (14).
Lety
(
t
)
=
x
(
t
+
h
), where h
>
0; (29) implies that
(30)
∥
x
(
t
+
h
)
-
x
(
t
)
∥
≤
∥
x
(
h
)
-
x
(
0
)
∥
,
∀
t
≥
0
.
Therefore t
→
∥
x
˙
(
t
)
∥ is nonincreasing.
From (20), we obtain that ∥
A
x
(
t
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
) is nonincreasing and bounded form below on [
0
,
+
∞
); therefore we have that
(31)
lim
⁡
t
→
+
∞
[
∥
A
x
(
t
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
]
exists
.
Using Proposition 5 to the above result, we obtain that
(32)
lim
⁡
t
→
+
∞
[
∥
A
x
(
t
)
-
b
∥
2
+
λ
∥
D
x
(
t
)
∥
1
]
exists
.
Moreover, we have that(33)
lim
⁡
t
→
+
∞
d
d
t
[
∥
A
x
(
t
)
-
b
∥
2
+
λ
∑
i
=
1
m
‍
φ
(
d
i
T
x
(
t
)
,
μ
(
t
)
)
]
=
0
.
Combining (20) and (33), we confirm that
(34)
lim
⁡
t
→
+
∞
∥
x
˙
(
t
)
∥
=
0
.
Sincex
(
t
) is uniformly bounded on the global interval, there is a cluster point of it, denoted as x
*, which follows that there exists an increasing sequence t
n such that
(35)
lim
⁡
n
→
+
∞
t
n
=
+
∞
,
lim
⁡
n
→
+
∞
x
(
t
n
)
=
x
*
.
Using the expression of (14) and lim
⁡
t
→
+
∞
x
˙
(
t
)
=
0, we have
(36)
x
*
=
P
Ω
[
x
*
-
lim
⁡
n
→
+
∞
(
∑
i
=
1
m
‍
2
A
T
(
A
x
(
t
n
)
-
b
)
+
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
n
)
,
μ
(
t
n
)
)
)
]
.
Using Proposition3 in the above equation, we have
(37)
〈
lim
⁡
n
→
+
∞
(
2
A
T
(
A
x
(
t
n
)
-
b
)
+
λ
∑
i
=
1
m
‍
∇
x
φ
(
d
i
T
x
(
t
n
)
,
μ
(
t
n
)
)
)
,
v
-
x
*
∑
i
=
1
m
‍
〉
≥
0
,
∀
v
∈
Ω
.
From Proposition5, there exists ξ
∈
∂
[
∥
A
x
(
t
)
-
b
∥
2
+
λ
∥
D
x
(
t
)
∥
1
] such that
(38)
〈
ξ
,
v
-
x
*
〉
≥
0
,
∀
v
∈
Ω
.
Therefore,x
* is a Clarke stationary point of (4). Since (4) is a convex programming, x
* is an optimal solution of (4). Owning to the random cluster point of x
(
t
), we know that any cluster point of x
(
t
) is an optimal solution of (4), which means that the solution of (14) converges to the optimal solution set of (4).


## 4. Numerical Experiments

In this section, we will give two numerical experiments to validate the theoretical results obtained in this paper and the good performance of the proposed neural network in solving the sparse reconstruction problems.Example 1.
In this experiment, we will test an experiment for the signal recovered with noise. Every original signal with sparsity 1 means that there is only one sound at time point. We use the following MATLAB codes to generate a 100 length original signalx
∈
ℝ
5
×
100, mixing matrix A
∈
ℝ
5
×
4, observed signal b
∈
ℝ
4
×
100, and noise n
∈
ℝ
4
×
100:s=zeros(5,100);
for l=1:100;
q=randperm(5);
s(q(1:2),l) = (2*randn(2,1));
end
A=randn(3,5);
n = 0.05 * randn(3,100);
b=A*s-n.
We denotes
* as the recovered signal using our method. Figures 1(a)-2(a) show the original, observed, and recovered signals using (14). Figure 2(b) presents the convergence of signal-to-noise ratio (SNR) along the solution of the proposed neural network. From this figure, we see that our method recovers this random original effectively. And we should state that the SNR of the recovered signal is 22.15 dB, where
(39)
SNR
=
∑
i
=
1
L
‍
-
1
L
20
lg
(
∥
s
*
(
l
)
-
s
(
l
)
∥
2
∥
s
(
l
)
∥
2
)
.(a) Original signals; (b) observed signals.
(a)
(b)(a) Recovered signals; (b) the convergence ofSNR
(
x
(
t
)
).
(a)
(b)Example 2.
In this experiment, we perform the proposed network (14) on the restoration of 20
×
20 circle image. The observed image is distorted from the unknown true image mainly by two factors: the blurring and the random noise. The blurring is a 2
D Gaussian function:
(40)
h
(
i
,
j
)
=
e
-
2
(
i
/
3
)
2
-
2
(
j
/
3
)
2
,
which is truncated such that the function has a support of 7
×
7. A Gaussian noise with zero mean and standard derivation of 0.05
dB is added to the blurred image. Figures 3(a) and 3(b) present the original and the observed images, respectively. The peak signal-to-noise ratio (PSNR) of the observed image is 16.87
dB. Denote x
o and x
b as the original and corresponding observed images and use the PSNR to evaluate the quality of the restored image; that is,
(41)
PSNR
(
x
)
=
-
10
log
⁡
10
∥
x
-
x
o
∥
20
×
20
.(a) Original image; (b) observed image; (c) recovered image.
(a)
(b)
(c)We use problem (13) to recover this image, where we let λ
=
0.017, Ω
=
{
x
:
0
≤
x
≤
e
}, and
(42)
D
=
(
L
1
⊗
I
I
⊗
L
1
)
with
L
1
=
(
1
-
1
1
-
1
⋱
⋱
1
-
1
)
.Choosex
0
=
P
Ω
(
b
). The recovered image by (14) with x
0 is figured in Figure 3(c) with PSNR
=
19.65 dB. The convergence of the objective value and PSNR(x
(
t
)) along the solution of (14) with initial point x
0 is presented in Figures 4(a) and 4(b). From Figures 4(a) and 4(b), we find that the objective value is monotonely decreasing and the PSNR is monotonely increasing along the solution of (14).(a) Convergence of the objective value; (b) convergence ofPSNR
(
x
(
t
)
).
(a)
(b)


## 5. Conclusion

Basing on the smoothing approximation technique and projected gradient method, we construct a neural network modeled by a differential equation to solve a class of constrained nonsmooth convex optimization problems, which have wide applications in sparse reconstruction. The proposed network has a unique and bounded solution with any initial point in the feasible region. Moreover, the solution of proposed network converges to the solution set of the optimization problem. Simulation results on numerical examples are elaborated upon to substantiate the effectiveness and performance of the neural network.



---

*Source: 107620-2014-03-31.xml*