# Minimax Robust Optimal Estimation Fusion for Distributed Multisensor Systems with a Relative Entropy Uncertainty

Authors: Hua Li; Jie Zhou
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

This paper considers the robust estimation fusion problem for distributed multisensor systems with uncertain correlations of local estimation errors. For an uncertain class characterized by the Kullback-Leibler (KL) divergence from the actual model to nominal model of local estimation error covariance, the robust estimation fusion problem is formulated to find a linear minimum variance unbiased estimator for the least favorable model. It is proved that the optimal fuser under nominal correlation model is robust while the estimation error has a relative entropy uncertainty.


---

## Body

## 1. Introduction

During the past decades, multisensor systems have received much attention in many applications, such as signal processing, communication, target tracking, and remote sensing [1–7]. In distributed estimation fusion frame, each sensor observes some local information of the physical environment, estimates the state or parameter of system in terms of some optimization criterion, and then transmits it to fusion center. The fusion center fuses the data from local sensors and obtains a better estimate. Generally, distributed fusion may have a less computational burden, lower communication rates, and higher survivability and be more flexible and reliable than centralized fusion which processes directly observation data from local sensors.There is much work on developing distributed estimation fusion algorithms. In [8, 9], a distributed Kalman filtering fusion formula was presented which has the same estimation performance as centralized Kalman filtering fusion. For the general systems with known auto- and cross-correlations of estimation errors from local sensors, in [6, 10–12], the optimal linear estimation fusion formulas were proposed in the sense of linear minimum variance (LMV). In practice, the cross-correlations of estimation errors among the sensors may be completely or partially unknown. Some fusion methods have been developed for the systems with various uncertain correlations. The simple convex combination approach ignores the cross-correlations. The covariance intersection (CI) method linearly combines the local estimates and considers a conservative estimate of the estimation error covariance matrix (see [13–16]). An information-theoretic justification for CI was presented in [16] to minimize the Chernoff information. A robust estimation fuser was developed in [17] by finding the Chebyshev center, that is, center of the minimum radius ball enclosing the intersection of ellipsoid sets. For a class of estimation error covariance matrices with norm-bounded uncertainties, a robust fusion model to minimize the worst-case mean square error (MSE) was proposed in [18]. In [19, 20], a generalized convex combination method was discussed. Recently, by employing random matrix to describe the uncertainty of cross-correlation, a chance-constrained programming approach was presented in [21]. In [22], a robust estimation problem was addressed for a linear model in which the unknown parameter vector is norm-bounded and noise covariance matrix is uncertain with a special structure. It was to get a linear estimator which minimizes the worst-case MSE over all vectors and noise covariance matrices in a specified uncertain region.This paper considers the robust estimation fusion approach for distributed multisensor systems with uncertain correlations among local estimation errors. Assuming the true joint probability density of local estimators belonging to a neighborhood of the nominal model, the robust estimation fusion can be modeled as a minimax problem that minimizes the variance for the least favorable model in an uncertain class. In this paper, the neighborhood is specified by the Kullback-Leibler (KL) divergence. It is worth noting that the use of KL divergence is rather natural as a metric for model mismatch in the statistics and information societies. From an information geometric viewpoint, it was argued in [23] that the KL divergence is a natural geometric “distance” between systems.The remainder of the paper is organized as follows. In Section2, we formulate the robust distributed estimation fusion as a min-max optimization problem. Then, we convert it equivalently into a min-min optimization problem by Lagrangian duality method in Section 3. Moreover, in Section 4, we give the analytical solution of original min-max problem and conclude that the linear minimum variance unbiased estimation (LMVUE) fusion is robust for the least favorable statistical model. Section 5 gives a conclusion.Notationsℝ
n and ℝ
m
×
n stand for the set of all n-dimensional real vectors and the set of all m-by-n real matrices, respectively. For a matrix A, A
T, A
†, and ∥
A
∥ denote the transpose, Moore-Penrose generalized inverse, and spectral norm of A, respectively. A
≻
0 means A is symmetric and positive definite. The symbol I represents the identity matrix with appropriate dimension. The expectation of a random variable (or random vector) is denoted by 𝔼
[
·
].


## 2. Problem Formulation

Consider anl-sensor distributed system in which x
∈
ℝ
n is the state or parameter to be estimated, and x
^
(
1
)
,
…
,
x
^
(
l
) are local unbiased estimates of xavailable at fusion center. The relationship between the local unbiased estimates and the state or parameter can be formulated as follows:
(1)
y
=
A
x
+
ε
,
where
(2)
y
=
(
x
^
(
1
)
⋮
x
^
(
l
)
)
∈
ℝ
n
l
is augmented local unbiased estimates and can be viewed as the observation at fusion center, A
=
[
I
,
…
,
I
]
T
∈
ℝ
n
l
×
n, and ε
∈
ℝ
n
l is the augmented error vectors of all local estimates. It is clear that ε has mean zero and covariance matrix
(3)
V
=
(
V
(
11
)
⋯
V
(
1
l
)
⋮
⋮
V
(
l
1
)
⋯
V
(
l
l
)
)
,
where V
(
i
j
)
=
𝔼
[
(
x
^
(
i
)
-
x
)
(
x
^
(
j
)
-
x
)
T
] for i
,
j
=
1
,
…
,
l.In order to find an unbiased linear estimation fusionx
^
=
B
+
W
T
y, where B and W are compatible vector and matrix, respectively, a necessary and sufficient condition is B
=
0, and A
T
W
=
I. Therefore, the fused estimate becomes x
^
=
W
T
y.When the error covariance matrixV is accurately given, a general linear fusion result is presented in [6, 11], which is optimal in the sense of LMV; that is, it has the LMV among all linear unbiased estimation fusion rules. In this case, the LMVUE fusion is uniquely given with probability one as follows:
(4)
x
^
=
1
l
A
T
(
I
-
V
(
P
V
P
)
†
)
y
,
where P
=
I
-
A
A
† is an orthogonal projector (i.e., symmetric and idempotent matrix). Moreover, if V is nonsingular, then the optimal estimation fusion becomes
(5)
x
^
=
(
A
T
V
-
1
A
)
-
1
A
T
V
-
1
y
.However, the estimation error covariance matrix cannot be accurately obtained in practice. Denote the actual and nominal error covariance matrices byV and V
^, respectively. For measurement model (1), we assume that the noise ε has the n
l-dimensional Gaussian distribution with zero mean. It is well known that the Gaussianity assumption is reasonable in many practical applications. The actual and nominal probability density functions of observation y are N
(
A
x
,
V
) and N
(
A
x
,
V
^
), respectively, where the matrices V and V
^ are assumed to be invertible. Therefore, the probability density functions of the actual and nominal models are, respectively, given by
(6)
p
(
y
)
=
1
(
2
π
)
n
l
/
2
(
det
⁡
V
)
1
/
2
·
exp
⁡
(
-
(
y
-
A
x
)
T
V
-
1
(
y
-
A
x
)
/
2
)
,
p
^
(
y
)
=
1
(
2
π
)
n
l
/
2
(
det
⁡
V
^
)
1
/
2
·
exp
⁡
(
-
(
y
-
A
x
)
T
V
^
-
1
(
y
-
A
x
)
/
2
)
.
In order to measure the “distance” between the two models, we adopt the KL divergence from the actual model to nominal model:
(7)
D
(
V
,
V
^
)
≜
2
KL
(
p
(
y
)
,
p
^
(
y
)
)
=
2
∫
‍
p
(
y
)
ln
⁡
⁡
(
p
(
y
)
p
^
(
y
)
)
d
y
=
tr
⁡
(
V
^
-
1
V
-
I
)
-
ln
⁡
⁡
det
⁡
(
V
^
-
1
V
)
.
Although it is not a conventional distance, since it is not symmetric and does not obey the triangle inequality, the KL divergence satisfies D
(
V
,
V
^
)
≥
0 with equality if and only if V
^
=
V.Let(8)
𝒞
=
{
V
≻
0
:
D
(
V
,
V
^
)
≤
c
}
denote the set of all possible estimation error covariance matrices such that the KL divergence from corresponding model to actual model is less than or equal to c
≥
0. Note that 𝒞 is a convex set since D
(
V
,
V
^
) is a convex function of V. The minimax robust estimation fusion problem is to seek for an optimal weighting matrix W
* to minimize the MSE in the worst case. It can be formulated as the following minimax problem:
(9)
W
*
=
arg
min
⁡
W
∈
𝒲
⁡
max
⁡
V
∈
𝒞
⁡
𝔼
∥
x
^
-
x
∥
2
,
where 𝒲
=
{
W
∈
ℝ
n
l
×
n
:
A
T
W
=
I
}. From
(10)
𝔼
∥
x
^
-
x
∥
2
=
𝔼
[
(
W
T
y
-
x
)
T
(
W
T
y
-
x
)
]
=
𝔼
[
(
W
T
y
-
W
T
A
x
)
T
(
W
T
y
-
W
T
A
x
)
]
=
𝔼
[
(
y
-
A
x
)
T
W
W
T
(
y
-
A
x
)
]
=
𝔼
[
tr
⁡
(
W
T
(
y
-
A
x
)
(
y
-
A
x
)
T
W
)
]
=
tr
⁡
(
W
T
V
W
)
,
the optimization problem (9) is equivalent to the following optimization problem:
(11)
W
*
=
arg
min
⁡
W
∈
𝒲
⁡
max
⁡
V
∈
𝒞
⁡
tr
⁡
(
W
T
V
W
)
.


## 3. Problem Conversion

Consider the inner maximization problem in (11); that is,
(12)
max
⁡
V
≻
0
⁡
tr
⁡
(
W
T
V
W
)
s
.
t
.
h
D
(
V
,
V
^
)
≤
c
.
To find the optimal solution, we construct the Lagrangian function
(13)
L
(
W
,
V
,
λ
)
=
tr
⁡
(
W
T
V
W
)
+
λ
(
c
-
tr
⁡
(
V
^
-
1
V
-
I
)
+
ln
⁡
⁡
det
⁡
⁡
(
V
^
-
1
V
)
)
,
where λ
≥
0 is the Lagrange multiplier associated with the inequality constraint. Let G
(
W
,
λ
) be the Lagrange dual function sup
⁡
V
≻
0
L
(
W
,
V
,
λ
) while the latter is finite.Lemma 1.
For problem (12), the following strong Lagrangian duality identity holds:
(14)
max
⁡
V
∈
𝒞
⁡
tr
⁡
(
W
T
V
W
)
=
min
⁡
λ
>
∥
W
T
V
^
W
∥
G
(
W
,
λ
)
.Proof.
The functionL
(
W
,
V
,
λ
) is the sum of a linear function and a logarithm of determinant which is concave, so L
(
W
,
V
,
λ
) is a concave function of V.
If0
≤
λ
≤
∥
W
T
V
^
W
∥, then, noticing the invertibility of W
T
V
^
W because W is full column rank from A
T
W
=
I, we have the following eigenvalue decomposition:
(15)
V
^
1
/
2
W
W
T
V
^
1
/
2
=
Q
T
Λ
Q
,
where Q is an orthogonal matrix and
(16)
Λ
=
diag
⁡
(
η
1
,
…
,
η
n
,
0
,
…
,
0
)
∈
ℝ
n
l
×
n
l
with η
1
≥
⋯
≥
η
n
>
0 and η
1
≥
λ. Let
(17)
Σ
=
diag
⁡
(
μ
,
1
,
…
,
1
)
∈
ℝ
n
l
×
n
l
,
μ
>
0
,
V
=
V
^
1
/
2
Q
T
Σ
Q
V
^
1
/
2
≻
0
.
Then,
(18)
L
(
W
,
V
,
λ
)
=
tr
⁡
(
(
V
^
1
/
2
W
W
T
V
^
1
/
2
-
λ
I
)
V
^
-
1
/
2
V
V
^
-
1
/
2
)
+
λ
(
c
+
tr
⁡
I
+
ln
⁡
⁡
det
⁡
(
V
^
-
1
V
)
)
=
tr
⁡
(
(
Λ
-
λ
I
)
Σ
)
+
λ
(
c
+
n
l
+
ln
⁡
⁡
det
⁡
(
Σ
)
)
=
(
η
1
-
λ
)
μ
+
∑
i
=
2
n
‍
(
η
i
-
λ
)
+
λ
(
c
+
n
l
+
ln
⁡
⁡
μ
)
can approach +
∞ while μ
→
+
∞.
Ifλ
>
∥
W
T
V
^
W
∥, then all eigenvalues of matrix W
T
V
^
W or V
^
1
/
2
W
W
T
V
^
1
/
2 are smaller than λ. So, λ
I
-
V
^
1
/
2
W
W
T
V
^
1
/
2
≻
0; that is, V
^
-
1
-
(
1
/
λ
)
W
W
T
≻
0. Using the formulas for derivatives of traces and determinants of matrices (see, e.g., [24, pp. 178, 182]), we have
(19)
∂
tr
⁡
(
W
T
V
W
)
∂
V
=
W
W
T
,
∂
tr
⁡
(
V
^
-
1
V
)
∂
V
=
V
^
-
1
,
∂
ln
⁡
⁡
det
⁡
(
V
^
-
1
V
)
∂
V
=
V
^
-
1
(
(
V
^
-
1
V
)
T
)
)
-
1
=
V
-
1
.
Then,
(20)
∂
L
(
W
,
V
,
λ
)
∂
V
=
∂
tr
⁡
(
W
T
V
W
)
∂
V
-
λ
(
∂
tr
⁡
(
V
^
-
1
V
)
∂
V
-
∂
ln
⁡
⁡
det
⁡
(
V
^
-
1
V
)
∂
V
)
=
W
W
T
-
λ
(
V
^
-
1
-
V
-
1
)
.
Taking ∂
L
(
W
,
V
,
λ
)
/
∂
V as zero, we can obtain the positive definite matrix maximizing L
(
W
,
V
,
λ
) as follows:
(21)
V
*
(
λ
)
=
(
V
^
-
1
-
1
λ
W
W
T
)
-
1
,
and G
(
W
,
λ
)
=
L
(
W
,
V
*
(
λ
)
,
λ
)
<
+
∞.
In summary, we obtain the Lagrange dual function(22)
sup
⁡
V
≻
0
L
(
W
,
V
,
λ
)
=
{
G
(
W
,
λ
)
,
λ
>
∥
W
T
V
^
W
∥
,
+
∞
,
otherwise
.
Therefore, for the dual problem of problem (12), it is sufficient and necessary to consider only the case of λ
>
∥
W
T
V
^
W
∥.
Since the primal problem (12) involves the maximization of a linear function tr
⁡
(
W
T
V
W
) over a bounded convex set 𝒞, the optimal objective value is finite. In addition, there exists V
=
V
^
≻
0 with D
(
V
,
V
^
)
=
0
<
c. Therefore, the Slater conditions are satisfied. From Proposition  6.4
.
3 in [25], strong Lagrangian duality holds.From Lemma1, the original min-max problem (9) or (11) is converted equivalently into the following min-min problem:
(23)
min
⁡
W
∈
𝒲
min
⁡
λ
>
∥
W
T
V
^
W
∥
G
(
W
,
λ
)
.
Furthermore, from (21), we have
(24)
W
W
T
=
λ
(
V
^
-
1
-
(
V
*
(
λ
)
)
-
1
)
,
V
^
-
1
V
*
(
λ
)
=
(
I
-
1
λ
W
W
T
V
^
)
-
1
.
Then,
(25)
G
(
W
,
λ
)
=
tr
⁡
(
W
T
V
*
(
λ
)
W
)
+
λ
(
c
-
tr
⁡
(
V
^
-
1
V
*
(
λ
)
-
I
)
)
+
λ
ln
⁡
⁡
det
⁡
(
V
^
-
1
V
*
(
λ
)
)
=
λ
c
+
λ
ln
⁡
⁡
det
⁡
⁡
(
V
^
-
1
V
*
(
λ
)
)
=
λ
c
-
λ
ln
⁡
⁡
det
⁡
⁡
(
I
-
1
λ
W
T
V
^
W
)
.


## 4. Solution of the Robust Optimization Problem

The objective functionG
(
W
,
λ
) of problem (23) is convex in (
W
,
λ
) since L
(
W
,
V
,
λ
) is convex in (
W
,
λ
) and G
(
W
,
λ
) is obtained by maximizing L
(
W
,
V
,
λ
) over V (see [25, Proposition 1.2
.
4
c]). In addition, the set
(26)
𝒟
=
{
(
W
,
λ
)
:
A
T
W
=
I
,
∥
W
T
V
^
W
∥
<
λ
}
is convex because it is the intersection of a linear constraint and a semidefinite programming (SDP) constraint. Consequently, the problem (23) is a convex optimization problem. Next, we pursue the analytical solution of problem (23).Lemma 2.
IfM
≻
0 and Q is an orthogonal projector with compatible dimensions, then
(27)
(
Q
M
Q
)
†
(
Q
M
Q
)
=
Q
,
(28)
(
I
-
(
Q
M
Q
)
†
M
)
(
I
-
Q
)
=
I
-
(
Q
M
Q
)
†
M
.Proof.
Using the properties of Moore-Penrose generalized inverse (see, e.g., [26]), we have
(29)
Q
(
Q
M
Q
)
†
=
(
Q
M
Q
)
†
Q
=
(
Q
M
Q
)
†
.
Therefore, (
Q
M
Q
)
†
(
Q
M
Q
) and Q
-
(
Q
M
Q
)
†
(
Q
M
Q
) are orthogonal projectors. From the properties of orthogonal projector, we have
(30)
rank
⁡
(
Q
-
(
Q
M
Q
)
†
(
Q
M
Q
)
)
=
tr
⁡
(
Q
-
(
Q
M
Q
)
†
(
Q
M
Q
)
)
=
tr
⁡
(
Q
)
-
tr
⁡
(
(
Q
M
Q
)
†
(
Q
M
Q
)
)
=
rank
⁡
(
Q
)
-
rank
⁡
(
(
Q
M
Q
)
†
(
Q
M
Q
)
)
=
rank
⁡
(
Q
)
-
rank
⁡
(
Q
M
Q
)
=
0
.
Equation (27) thus holds, and then (28) is direct.Theorem 3.
The optimal weighting matrix of the robust fusion problem (9) or (11) is given by
(31)
W
*
=
V
^
-
1
A
(
A
T
V
^
-
1
A
)
-
1
.Proof.
The derivative ofG
(
W
,
λ
) given by (25) with respect to λ is
(32)
∂
G
(
W
,
λ
)
∂
λ
=
c
-
ln
⁡
⁡
det
⁡
(
I
-
1
λ
W
T
V
^
W
)
-
λ
·
∂
∂
λ
ln
⁡
⁡
det
⁡
(
I
-
1
λ
W
T
V
^
W
)
=
c
-
ln
⁡
⁡
det
⁡
(
I
-
1
λ
W
T
V
^
W
)
-
tr
⁡
(
(
I
-
1
λ
W
T
V
^
W
)
-
1
1
λ
W
T
V
^
W
)
.
Let η
1
≥
⋯
≥
η
n be the eigenvalues of W
T
V
^
W. It is easy to show that
(33)
∂
G
(
W
,
λ
)
∂
λ
=
0
is equivalent to
(34)
c
-
∑
i
=
1
n
‍
(
η
i
λ
-
η
i
+
ln
⁡
⁡
(
1
-
η
i
λ
)
)
=
0
.
The above equation has only one solution λ
*
>
η
1
=
∥
W
T
V
^
W
∥ for any nonzero matrix W since
(35)
h
(
x
)
=
∑
i
=
1
n
‍
(
η
i
x
-
η
i
+
ln
⁡
⁡
(
1
-
η
i
x
)
)
is a monotonically decreasing continuous function in (
η
1
,
+
∞
) and
(36)
lim
⁡
x
→
η
1
+
h
(
x
)
=
+
∞
,
lim
⁡
x
→
+
∞
h
(
x
)
=
0
.
FromA
T
W
=
I and the result on derivatives of determinants of matrices (see, e.g., [24, p. 182]), the derivative of function G
(
W
,
λ
) with respect to W is
(37)
∂
G
(
W
,
λ
)
∂
W
=
-
λ
∂
∂
W
ln
⁡
⁡
det
⁡
(
W
T
(
A
A
T
-
1
λ
V
^
)
W
)
=
-
2
λ
(
A
A
T
-
1
λ
V
^
)
W
(
W
T
(
A
A
T
-
1
λ
V
^
)
W
)
-
1
=
-
2
λ
(
A
-
1
λ
V
^
W
)
(
I
-
1
λ
W
T
V
^
W
)
-
1
.
The solution of the matrix equation A
T
W
=
I given by equality constraint in the optimization problem (23) can be expressed as
(38)
W
=
1
l
A
+
P
Z
,
where P
=
I
-
A
A
† and Z
∈
ℝ
n
l
×
n is any matrix. Next, we seek the matrix Z to optimize the weighting matrix W. Redenote the objective function G
(
W
,
λ
) given by (25) as G
(
Z
,
λ
). From P
A
=
0, the derivative of G
(
Z
,
λ
) with respect to Z is
(39)
∂
G
(
Z
,
λ
)
∂
Z
=
P
∂
G
(
W
,
λ
)
∂
W
=
2
P
V
^
W
(
I
-
1
λ
W
T
V
^
W
)
-
1
.
Thus, from the invertibility of I
-
1
λ
W
T
V
^
W,
(40)
∂
G
(
Z
,
λ
)
∂
Z
=
0
if and only if
(41)
P
V
^
(
1
l
A
+
P
Z
)
=
0
.
That is,
(42)
P
V
^
P
Z
=
-
1
l
P
V
^
A
.
Because V
^ is invertible, from (27), the solution of matrix equation (42) is
(43)
Z
*
=
-
1
l
(
P
V
^
P
)
†
P
V
^
A
+
(
I
-
P
)
ξ
,
where ξ
∈
ℝ
n
l
×
n is any matrix. Substituting Z
* into (38) and using P
(
I
-
P
)
=
0, we have
(44)
W
*
=
1
l
(
I
-
(
P
V
^
P
)
†
V
^
)
A
.
From the convexity of problem (23) and Proposition  4.7
.
1 in [25], the pair (
W
*
,
λ
*
)
∈
𝒟 is the global minimum of (23). Moreover, from the definition of P and (28), we can obtain
(45)
1
l
(
I
-
(
P
V
^
P
)
†
V
^
)
A
A
T
V
^
-
1
A
=
(
I
-
(
P
V
^
P
)
†
V
^
)
(
I
-
P
)
V
^
-
1
A
=
(
I
-
P
-
(
P
V
^
P
)
†
V
^
+
(
P
V
^
P
)
†
V
^
P
)
V
^
-
1
A
=
(
I
-
(
P
V
^
P
)
†
V
^
)
V
^
-
1
A
=
V
^
-
1
A
-
(
P
V
^
P
)
†
P
A
=
V
^
-
1
A
.As a result, by the invertibility of A
T
V
^
-
1
A, we have
(46)
1
l
(
I
-
(
P
V
^
P
)
†
V
^
)
A
=
V
^
-
1
A
(
A
T
V
^
-
1
A
)
-
1
.
The proof of this theorem is completed.Remark 4.
The optimal weighting matrixW
* given in Theorem 3 does not depend on the parameter c. However, it is worth noting that the optimum value of the objective function is increasing as c increases. In fact, the optimal Lagrange multiplier λ
* depends on c.
From Theorem3, although the least favorable distribution given by covariance V
* is not the nominal distribution, the robust estimation fusion formula (
W
*
)
T
yis the same as the fusion formulas given by (4) and (5) which are derived from the nominal distribution. It concludes that the usual LMVUE fusion for a Gaussian nominal model is robust.


## 5. Conclusion

A minimax optimization problem is formulated to find the robust estimation fusion for multisensor systems with uncertain correlations of local estimation errors characterized by the relative entropy. By rigorous mathematical deduction, it has been proved that the usual LMVUE fusion for a Gaussian nominal model is robust no matter how far the KL divergence is from the actual model to nominal model. It is an interesting and significant discovery.



---

*Source: 910971-2014-05-12.xml*