# Memory Distributed LMS for Wireless Sensor Networks

Authors: Fangmin Xu; Chenyang Zheng; Haiyan Cao
Journal: Mathematical Problems in Engineering
 (2018)

---

## Abstract

Due to the limited communication resource and power, it is usually infeasible for sensor networks to gather data to a central processing node. Distributed algorithms are an efficient way to resolve this problem. In the algorithms, each sensor node deals with its own input data and transmits the local results to its neighbors. Each node fuses the information from neighbors and its own to get the final results. Different from the existing work, in this paper, we present an approach for distributed parameter estimation in wireless sensor networks based on the use of memory. The proposed approach consists of modifying the cost function by adding extra statistical information. A distributed least-mean squares (d-LMS) algorithm, called memory d-LMS, is then derived based on the cost function and analyzed. The theoretical performances of mean and mean square are analyzed. Moreover, simulation results show that the proposed algorithm outperforms the traditional d-LMS algorithm in terms of convergence rate and mean square error (MSE) performance.


---

## Body

## 1. Introduction

Wireless Sensor Networks (WSNs) consist of a large number of sensor nodes that can be deployed for monitoring unattainable areas, such as deep oceans, forest fires, and air pollution [1–4]. To estimate some parameter of interest from data which is collected at nodes distributed over a geographic area, it is becoming more and more important to design and analyze estimation algorithms for the networks.Generally, there are two kinds of estimation algorithms: centralized estimation [5–8] and distributed estimation [9–12]. In the former one, all nodes transmit their measurements to a central fusion center for processing, and the final estimate is sent back to the nodes. This method can obtain the global solution, but it requires a large amount of energy and communication. What is more, it will be nonrobust if the fusion center is out of work. In the latter one, each node only communicates with its immediate neighbor and the signal is processed locally. Compared to the centralized estimation, the distributed estimation can achieve good estimation performance whilst reducing complexity.Due to the scalability, robustness, and low cost, distributed estimation algorithms have been receiving more and more attention. The paper [10] proposes a diffusion LMS to obtain the distributed estimation of the networks. The paper [11] proposes a distributed incremental RLS solution. The papers [12, 13] propose LMS algorithms for censor data. To the authors’ knowledge, there is no existing work dealing with the past information of the nodes for distributed estimation. Motivated by this fact, in this paper, we state a new cost function where the past information is considered in the cost function. By optimizing the proposed cost function, a diffusion memory-LMS is proposed in this paper.This paper is organized as follows. Section2 describes the mathematical problem of the network and memory d-LMS. In Section 3, mean convergence is analyzed. Simulation results are provided in Section 4. Finally, we conclude the paper in Section 5.Notation.We use boldface uppercase letters to denote matrices and boldface lowercase letters to denote vectors. We use E(·) to denote the expectation operator, D(·) for variance, ·T for complex conjugate-transposition, Tr(·) for the trace of a matrix, and ⊗ for Kronecker product. The notation col{·} stands for a vector obtained by stacking the specified vectors. vec(·) stands for a linear transformation which converts a matrix into a column vector. Similarly, we use diag{·} to denote the diagonal matrix consisting of the specified vectors or matrices. Other notations will be introduced as necessary.


## 2. The Mathematical Problem of the Network and Memoryd-LMS

We consider a sensor network which is composed ofK sensor nodes distributed over some geographic regions and used to monitor a physical phenomenon, shown in Figure 1. At every time i, each node k can get a scalar measurement yk,i. Here yk,i is a 1 × 1 observations vector, and it can be expressed in terms of the following linear combinations:(1)yk,i=xk,iTω0+ek,i,where xk,i is an M×1 known regression vector which is corresponding to a realization of a stochastic process xk,i. ω0 is an M×1 nonrandom unknown parameter to be estimated. ek,i is a random error in observing yk,i and is assumed to be spatially and temporally independent. E(ek,i)=0 and D(ek,i)=σk,i2. To get a global solution of ω0, it requires access to the information {yk,i,xk,i} across all nodes in the network. Therefore, we explore a distributed algorithm in this section.Figure 1
Wireless sensor networks with each node taking a measurement at timei and storing the measurements from time i-i0+1 to time i.


### 2.1. Traditional Cost Function andd-LMS

Generally, thed-LMS algorithms [10–12] are derived from the following problem function:(2)Jglobalω≜∑k=1K∑l∈Nkcl,kEyl,i-xl,iTω2=∑k=1KJklocalω,where ≜ is used to denote Jklocal(ω) as ∑l∈Nkcl,kEyl,i-xl,iTω2, Jklocal(ω)≜∑l∈Nkcl,kEyl,i-xl,iTω2 is the local cost function of node k. cl,k is a set of nonnegative weight coefficients between node l and node k and satisfies(3)∑l=1Kcl,k=∑k=1Kcl,k=1.Here, C=cl,kK∗K, and cl,k=0 when node l and node k are not connected.By minimizing (2), one can obtain the Adapt-then-Combine (ATC) diffusion LMS as follows [10]:(4)φk,i=ωk,i-1+μk∑l∈Nkcl,kxl,iyl,i-xl,iTωk,i-1ωk,i=∑l∈Nkal,kφl,i,where μk is the step size and al,k is a set of nonnegative weight coefficients between node l and node k and satisfies(5)al,k=0,ifl∉Nk,1TA=1T.Here,A is an N×N matrix with individual entries al,k.The Adapt-then-Combine (ATC)d-LMS can be described as follows. Firstly, at time i, every node k obtains its local estimation φk,i by exchanging the measurements {yl,i,xl,i} across all its neighbor nodes. Then, all nodes combine the estimation from the first step with their neighbors. The traditional d-LMS algorithms obtain their estimation at time i by considering the measurements across its neighbor nodes only at time i. Rather than relying solely on current data and on the data shared with the neighbors at a particular time instant [9–12], we propose a distributed least-mean squares (d-LMS) algorithm called memory d-LMS in the following subsection.


### 2.2. Cost Function and Memoryd-LMS with the Past Measurement Information

To use the past measurements more sufficiently, we first propose cost functions with the past information. At timei, the global cost function with the past information is given by(6)fglobalω≜∑k=1K∑j=i-i0+1iEyl,j-xl,jTω2=∑k=1K∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTω2=∑k=1Kfklocalω,where(7)fklocalω=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTω2is the local cost function with the past information. cl,k is a set of nonnegative weight coefficients between node l and node k and satisfies(8)∑l=1Kcl,k=∑k=1Kcl,k=1.Here,C=cl,kK×K, cl,k=0 when node l and node k are not connected. i0 (1≤i0≤i) is a constant to be designed by requirements of the network. (6) is simplified into (2) if i0=1.By directly minimizingfglobal(ω) in (6), we obtain aglobald-LMS recursion(9)ωi=ωi-1+μ∑k=1K∑j=i-i0+1iRyx,k,j-Rxx,k,jωi-1≈ωi-1+μ∑k=1K∑j=i-i0+1iyk,jxk,j-xk,jxk,jTωi-1,where Ryx,l,j=Exl,jyl,j, Rxx,l,j=Exl,jxl,jT, and μ is the step size. The moments Ryx,l,j and Rxx,l,j are usually unavailable in practice. Thus, we replace them by their instantaneous approximations yk,jxk,j and xk,jxk,jT, respectively [13].Equation (9) is not distributed because it requires access to measurements across the whole network.Now, we focus on a distributed solution ofω0 by minimizing a local cost function with the past information. Notice that (6) can be rewritten as(10)fglobalω=fklocalω+∑l=1,l≠kKfllocalω,where fklocal(ω) can be rewritten as(11)fklocalω=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTω2=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTωklocal+xl,jTωklocal-xl,jTω2=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTωklocal2+ω-ωklocalT∑l∈Nkcl,k∑j=i-i0+1iExl,jxl,jTω-ωklocal=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTωklocal2+ω-ωklocalΛk2,where ω-ωklocalΛk2=ω-ωklocalTΛkω-ωklocal is a weighted vector norm and Λk=∑l∈Nkcl,k∑j=i-i0+1iExl,jxl,jT, ωklocal is the local estimate of node k.Substituting (11) into (10) leads to(12)fglobalω=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTω2+∑l≠kK∑l′∈Nlcl′,l∑j=i-i0+1iEyl′,j-xl′,jTωllocal2+∑l≠kKω-ωllocalΛl2.Minimizing (12) with respect to ω, we have(13)dfglobalωdω=2∑l∈Nkcl,k∑j=i-i0+1iExl,jyl,j-xl,jxl,jTω+2∑l≠kKΛlωllocal-ω.Notice that estimation ofω0 based on (13) still requires access to information across all nodes in the network. In response to this, we replace Λl by another kind of weighting coefficient γl,k, where(14)γl,k=0,ifl∉Nk.Then, we have(15)dfglobalωdω≈2∑l∈Nkcl,k∑j=i-i0+1iExl,jyl,j-xl,jxl,jTω+2∑l≠kKγl,kωllocal-ω=2∑l∈Nkcl,k∑j=i-i0+1iRyx,l,j-Rxx,l,jω+2∑l∈Nk/kγl,kωllocal-ω.Hence, we get an iterative steepest-descent estimation forω0: that is,(16)ωk,i=ωk,i-1+μk∑l∈Nkcl,k∑j=i-i0+1iRyx,l,j-Rxx,l,jωk,i-1+ηk∑l∈Nk/kγl,kωllocal-ωk,i-1,where μk and ηk are step size parameters.We replaceωllocal by ωl,i-1 which is available at node l in time i-1. Then, (15) is rewritten as(17)ωk,i=ωk,i-1+μk∑l∈Nkcl,k∑j=i-i0+1iRyx,l,j-Rxx,l,jωk,i-1+ηk∑l∈Nk/kγl,kωk,i-1-ωl,i-1.Now, we use a two-step estimation forω0(18)φk,i=ωk,i-1+μk∑l∈Nkcl,k∑j=i-i0+1iRyx,l,j-Rxx,l,jωk,i-1,(19)ωk,i=φk,i+ηk∑l≠k,l∈Nkγl,kωl,i-1-ωk,i-1=φk,i+ηk∑l∈Nk/kγl,kφl,i-φk,i=φk,i-ηk∑l∈Nkγl,kφk,i+ηkγk,kφk,i+ηk∑l∈Nk/kγl,kφl,i=1-ηk∑l∈Nkγl,k+ηkγk,kφk,i+ηk∑l∈Nk/kγl,kφl,i≜ak,kφk,i+∑l∈Nk/kal,kφl,i,where the last step of (19) follows from a change of variables(20)ak,k=1-ηk∑l∈Nkγl,k+ηkγk,k,al,k=ηkγl,k,l≠k.Form (13) and (20), we have al,k=ηkγl,k=0 if l∉Nk.Furthermore, from (20), we have ∑l=1Nal,k=1 directly.To obtain an adaptive implementation, we replace the second-order moments by instantaneous values in (18): that is,(21)Ryx,l,j≈yl,jxl,j(22)Rxx,l,j≈xl,jxl,jT.Now, we obtain a novel distributed algorithm calledmemoryd-LMS:(23)φk,i=ωk,i-1+μk∑l∈Nkcl,kαl,i-βl,iωk,i-1(24)ωk,i=∑l∈Nkal,kφl,i,where al,k=cl,k=0, if l∉Nk.(25)∑l=1Kcl,k=∑k=1Kcl,k=1,∑l=1Kal,k=1,αl,i=∑j=i-i0+1iyl,jxl,j;βl,i=∑j=i-i0+1ixl,jxl,jT.


### 2.3. Implementation of the Proposed Memoryd-LMS Algorithm

According to the abovementioned memoryd-LMS, we implement the algorithm with three steps: time diffusion, adaptation, and node diffusion. In detail, it will be described as follows.For every nodek, start with ωk,0=0, select an appropriate value for the step size μk, and set the nonnegative coefficients al,k, cl,k satisfying (24), for each time i, and repeat.Step 1.
Time diffusion step with respect to past measurements:(26)αl,i=∑j=i-i0+1iyl,jxl,j;βl,i=∑j=i-i0+1ixl,jxl,jT.
In this step, at theith iteration, every node l summarizes cross-correlations yl,jxl,j from time i-i0+1 to time i and autocorrelations xl,jxl,jT from time i-i0+1 to time i, respectively.Step 2.
Adaptation step is(27)φk,i=ωk,i-1+μk∑l∈Nkcl,kαl,i-βl,iωk,i-1.Step 3.
Node diffusion step with respect to preestimations of all nodes(28)ωk,i=∑l∈Nkal,kφl,i.
In this step, every nodek combines its preestimation at Step 2 with preestimations shared by the neighbor nodes to obtain estimation ωk,i.


## 2.1. Traditional Cost Function andd-LMS

Generally, thed-LMS algorithms [10–12] are derived from the following problem function:(2)Jglobalω≜∑k=1K∑l∈Nkcl,kEyl,i-xl,iTω2=∑k=1KJklocalω,where ≜ is used to denote Jklocal(ω) as ∑l∈Nkcl,kEyl,i-xl,iTω2, Jklocal(ω)≜∑l∈Nkcl,kEyl,i-xl,iTω2 is the local cost function of node k. cl,k is a set of nonnegative weight coefficients between node l and node k and satisfies(3)∑l=1Kcl,k=∑k=1Kcl,k=1.Here, C=cl,kK∗K, and cl,k=0 when node l and node k are not connected.By minimizing (2), one can obtain the Adapt-then-Combine (ATC) diffusion LMS as follows [10]:(4)φk,i=ωk,i-1+μk∑l∈Nkcl,kxl,iyl,i-xl,iTωk,i-1ωk,i=∑l∈Nkal,kφl,i,where μk is the step size and al,k is a set of nonnegative weight coefficients between node l and node k and satisfies(5)al,k=0,ifl∉Nk,1TA=1T.Here,A is an N×N matrix with individual entries al,k.The Adapt-then-Combine (ATC)d-LMS can be described as follows. Firstly, at time i, every node k obtains its local estimation φk,i by exchanging the measurements {yl,i,xl,i} across all its neighbor nodes. Then, all nodes combine the estimation from the first step with their neighbors. The traditional d-LMS algorithms obtain their estimation at time i by considering the measurements across its neighbor nodes only at time i. Rather than relying solely on current data and on the data shared with the neighbors at a particular time instant [9–12], we propose a distributed least-mean squares (d-LMS) algorithm called memory d-LMS in the following subsection.


## 2.2. Cost Function and Memoryd-LMS with the Past Measurement Information

To use the past measurements more sufficiently, we first propose cost functions with the past information. At timei, the global cost function with the past information is given by(6)fglobalω≜∑k=1K∑j=i-i0+1iEyl,j-xl,jTω2=∑k=1K∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTω2=∑k=1Kfklocalω,where(7)fklocalω=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTω2is the local cost function with the past information. cl,k is a set of nonnegative weight coefficients between node l and node k and satisfies(8)∑l=1Kcl,k=∑k=1Kcl,k=1.Here,C=cl,kK×K, cl,k=0 when node l and node k are not connected. i0 (1≤i0≤i) is a constant to be designed by requirements of the network. (6) is simplified into (2) if i0=1.By directly minimizingfglobal(ω) in (6), we obtain aglobald-LMS recursion(9)ωi=ωi-1+μ∑k=1K∑j=i-i0+1iRyx,k,j-Rxx,k,jωi-1≈ωi-1+μ∑k=1K∑j=i-i0+1iyk,jxk,j-xk,jxk,jTωi-1,where Ryx,l,j=Exl,jyl,j, Rxx,l,j=Exl,jxl,jT, and μ is the step size. The moments Ryx,l,j and Rxx,l,j are usually unavailable in practice. Thus, we replace them by their instantaneous approximations yk,jxk,j and xk,jxk,jT, respectively [13].Equation (9) is not distributed because it requires access to measurements across the whole network.Now, we focus on a distributed solution ofω0 by minimizing a local cost function with the past information. Notice that (6) can be rewritten as(10)fglobalω=fklocalω+∑l=1,l≠kKfllocalω,where fklocal(ω) can be rewritten as(11)fklocalω=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTω2=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTωklocal+xl,jTωklocal-xl,jTω2=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTωklocal2+ω-ωklocalT∑l∈Nkcl,k∑j=i-i0+1iExl,jxl,jTω-ωklocal=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTωklocal2+ω-ωklocalΛk2,where ω-ωklocalΛk2=ω-ωklocalTΛkω-ωklocal is a weighted vector norm and Λk=∑l∈Nkcl,k∑j=i-i0+1iExl,jxl,jT, ωklocal is the local estimate of node k.Substituting (11) into (10) leads to(12)fglobalω=∑l∈Nkcl,k∑j=i-i0+1iEyl,j-xl,jTω2+∑l≠kK∑l′∈Nlcl′,l∑j=i-i0+1iEyl′,j-xl′,jTωllocal2+∑l≠kKω-ωllocalΛl2.Minimizing (12) with respect to ω, we have(13)dfglobalωdω=2∑l∈Nkcl,k∑j=i-i0+1iExl,jyl,j-xl,jxl,jTω+2∑l≠kKΛlωllocal-ω.Notice that estimation ofω0 based on (13) still requires access to information across all nodes in the network. In response to this, we replace Λl by another kind of weighting coefficient γl,k, where(14)γl,k=0,ifl∉Nk.Then, we have(15)dfglobalωdω≈2∑l∈Nkcl,k∑j=i-i0+1iExl,jyl,j-xl,jxl,jTω+2∑l≠kKγl,kωllocal-ω=2∑l∈Nkcl,k∑j=i-i0+1iRyx,l,j-Rxx,l,jω+2∑l∈Nk/kγl,kωllocal-ω.Hence, we get an iterative steepest-descent estimation forω0: that is,(16)ωk,i=ωk,i-1+μk∑l∈Nkcl,k∑j=i-i0+1iRyx,l,j-Rxx,l,jωk,i-1+ηk∑l∈Nk/kγl,kωllocal-ωk,i-1,where μk and ηk are step size parameters.We replaceωllocal by ωl,i-1 which is available at node l in time i-1. Then, (15) is rewritten as(17)ωk,i=ωk,i-1+μk∑l∈Nkcl,k∑j=i-i0+1iRyx,l,j-Rxx,l,jωk,i-1+ηk∑l∈Nk/kγl,kωk,i-1-ωl,i-1.Now, we use a two-step estimation forω0(18)φk,i=ωk,i-1+μk∑l∈Nkcl,k∑j=i-i0+1iRyx,l,j-Rxx,l,jωk,i-1,(19)ωk,i=φk,i+ηk∑l≠k,l∈Nkγl,kωl,i-1-ωk,i-1=φk,i+ηk∑l∈Nk/kγl,kφl,i-φk,i=φk,i-ηk∑l∈Nkγl,kφk,i+ηkγk,kφk,i+ηk∑l∈Nk/kγl,kφl,i=1-ηk∑l∈Nkγl,k+ηkγk,kφk,i+ηk∑l∈Nk/kγl,kφl,i≜ak,kφk,i+∑l∈Nk/kal,kφl,i,where the last step of (19) follows from a change of variables(20)ak,k=1-ηk∑l∈Nkγl,k+ηkγk,k,al,k=ηkγl,k,l≠k.Form (13) and (20), we have al,k=ηkγl,k=0 if l∉Nk.Furthermore, from (20), we have ∑l=1Nal,k=1 directly.To obtain an adaptive implementation, we replace the second-order moments by instantaneous values in (18): that is,(21)Ryx,l,j≈yl,jxl,j(22)Rxx,l,j≈xl,jxl,jT.Now, we obtain a novel distributed algorithm calledmemoryd-LMS:(23)φk,i=ωk,i-1+μk∑l∈Nkcl,kαl,i-βl,iωk,i-1(24)ωk,i=∑l∈Nkal,kφl,i,where al,k=cl,k=0, if l∉Nk.(25)∑l=1Kcl,k=∑k=1Kcl,k=1,∑l=1Kal,k=1,αl,i=∑j=i-i0+1iyl,jxl,j;βl,i=∑j=i-i0+1ixl,jxl,jT.


## 2.3. Implementation of the Proposed Memoryd-LMS Algorithm

According to the abovementioned memoryd-LMS, we implement the algorithm with three steps: time diffusion, adaptation, and node diffusion. In detail, it will be described as follows.For every nodek, start with ωk,0=0, select an appropriate value for the step size μk, and set the nonnegative coefficients al,k, cl,k satisfying (24), for each time i, and repeat.Step 1.
Time diffusion step with respect to past measurements:(26)αl,i=∑j=i-i0+1iyl,jxl,j;βl,i=∑j=i-i0+1ixl,jxl,jT.
In this step, at theith iteration, every node l summarizes cross-correlations yl,jxl,j from time i-i0+1 to time i and autocorrelations xl,jxl,jT from time i-i0+1 to time i, respectively.Step 2.
Adaptation step is(27)φk,i=ωk,i-1+μk∑l∈Nkcl,kαl,i-βl,iωk,i-1.Step 3.
Node diffusion step with respect to preestimations of all nodes(28)ωk,i=∑l∈Nkal,kφl,i.
In this step, every nodek combines its preestimation at Step 2 with preestimations shared by the neighbor nodes to obtain estimation ωk,i.


## 3. Performance Analysis

In this section, we analyze the asymptotic performance of the proposed algorithm.Denote that(29)φ~k,i=ω0-φk,i,∀k,iω~k,i=ω0-ωk,iω~i=col⁡ω~1,i,ω~2,i,…,ω~K,iω~i-1=col⁡ω~1,i-1,ω~2,i-1,…,ω~K,i-1φ~i=col⁡φ~1,i,φ~2,i,…,φ~K,iVi=col⁡∑j=i-i0+1ix1,je1,j,…,∑j=i-i0+1ixK,jeK,jM=diag⁡μ1IM,μ2IM,…,μKIMΩi=diag⁡∑l∈N1cl,1∑j=i-i0+1ixl,jxl,jT,…,∑l∈NKcl,K∑j=i-i0+1ixl,jxl,jT,Α=A⊗IM=a11IM⋯a1KIM⋮⋱⋮aK1IM⋯aKKIM,whereA=al,kK∗KC=C⊗IM=c11IM⋯c1KIM⋮⋱⋮cK1IM⋯cKKIM,whereC=cl,kK∗K.Then, from (22) and (23), we have(30)φ~k,i=ω~k,i-1-μk∑l∈Nkcl,k∑j=i-i0+1ixl,jxl,jTω~l,i-1-μk∑l∈Nkcl,k∑j=i-i0+1ixl,jel,j,ω~k,i=∑l∈Nkal,kφ~l,i.Furthermore, the above expressions can be rewritten as(31)φ~i=ω~i-1-MiΩiω~i-1-CTMiVi,(32)ω~i=ΑTφ~i.Inserting (31) into (32), we have(33)ω~i=ΑTI-MΩiω~i-1-ΑTCTMVi.In the following of this section, we will give the asymptotic performance ofω~i and prove it under some assumptions.


### 3.1. Convergence Analysis ofEωk,i

Assumption 1.
All regressorsxk,i are independent of ek,i for all node k and time i.Lemma 2 (see [10]).
LetP and Xdenote arbitrary K×K matrices, where 1TP=1T. Then the matrix PTX is stable for any choice of P if and only if X is stable.Theorem 3.
Under Assumption1, mean of the memory d-LMS estimation based on the past information converges to the true value ω0, if(34)0<i0μk<2λmaxΣk,k=1,…,K,where Σk=1/i0∑j=i-i0+1ixk,jxk,jT.Proof.
Taking the expectation of both sides of (34) yields(35)Eω~i=ΑTI-MEΩiEω~i-1=ΑTI-M·i0·diag⁡Σ1,…,ΣKEω~i-1=ΑTI-M·i0·diag⁡Σ1,…,ΣKiω0
The right side of (35) converges to 0, if ΑTI-M·i0·diag⁡Σ1,…,ΣK is stable. This requires I-M·i0·diag⁡Σ1,…,ΣK to be stable or equivalently(36)0<i0·μk<2λmaxΣk,k=1,…,K.
Theorem3 shows that the mean of the estimation convergences to the true value when proper μk is chosen.


### 3.2. The Mean Squared Errors Performance

Now, we study the mean squared errors of the memoryd-LMS.(37)Eω~iΓ2=Eω~i-1I-MΩiΑΓΑTI-MΩi2+EViTMCΑΓΑTCTMVi=aEω~i-1I-MΩiΑΓΑTI-MΩi2+TrΓΑTCTEMViViTMCΑ=bEω~i-1I-MΩiΑΓΑTI-MΩi2+vec⁡ΑTCTEMViViTMCΑTvec⁡Γ.Let τ=vec⁡(Γ) and Δ=vec⁡ΑTCTE(MViViTM)CΑT in (37), by using the property(38)vec⁡UΓV=VT⊗Uvec⁡Γ.We rewrite (37) as(39)Eω~iτ2=Eω~0Fiτ2+Δ∑j=0i-1Fjτ=Eω~0Fiτ2+ΔI-FiI-F-1τ,where(40)F=EI-MΩiΑ⊗I-MΩiΑT=EI-MΩi⊗I-MΩiΑT⊗Α.For sufficiently large i and stable matrix F, (39) can be rewritten as(41)Eω~iτ2=ΔI-F-1τ.Substituting τ=vec⁡(diag⁡(ek)⊗IM) into (41), we can obtain the mean square deviation (MSD) as follows:(42)Eω~k,∞2=ΔI-F-1vec⁡diag⁡ek⊗IM,where ek is a K×1 vector with a single entry 1 at position k and zeros elsewhere.


## 3.1. Convergence Analysis ofEωk,i

Assumption 1.
All regressorsxk,i are independent of ek,i for all node k and time i.Lemma 2 (see [10]).
LetP and Xdenote arbitrary K×K matrices, where 1TP=1T. Then the matrix PTX is stable for any choice of P if and only if X is stable.Theorem 3.
Under Assumption1, mean of the memory d-LMS estimation based on the past information converges to the true value ω0, if(34)0<i0μk<2λmaxΣk,k=1,…,K,where Σk=1/i0∑j=i-i0+1ixk,jxk,jT.Proof.
Taking the expectation of both sides of (34) yields(35)Eω~i=ΑTI-MEΩiEω~i-1=ΑTI-M·i0·diag⁡Σ1,…,ΣKEω~i-1=ΑTI-M·i0·diag⁡Σ1,…,ΣKiω0
The right side of (35) converges to 0, if ΑTI-M·i0·diag⁡Σ1,…,ΣK is stable. This requires I-M·i0·diag⁡Σ1,…,ΣK to be stable or equivalently(36)0<i0·μk<2λmaxΣk,k=1,…,K.
Theorem3 shows that the mean of the estimation convergences to the true value when proper μk is chosen.


## 3.2. The Mean Squared Errors Performance

Now, we study the mean squared errors of the memoryd-LMS.(37)Eω~iΓ2=Eω~i-1I-MΩiΑΓΑTI-MΩi2+EViTMCΑΓΑTCTMVi=aEω~i-1I-MΩiΑΓΑTI-MΩi2+TrΓΑTCTEMViViTMCΑ=bEω~i-1I-MΩiΑΓΑTI-MΩi2+vec⁡ΑTCTEMViViTMCΑTvec⁡Γ.Let τ=vec⁡(Γ) and Δ=vec⁡ΑTCTE(MViViTM)CΑT in (37), by using the property(38)vec⁡UΓV=VT⊗Uvec⁡Γ.We rewrite (37) as(39)Eω~iτ2=Eω~0Fiτ2+Δ∑j=0i-1Fjτ=Eω~0Fiτ2+ΔI-FiI-F-1τ,where(40)F=EI-MΩiΑ⊗I-MΩiΑT=EI-MΩi⊗I-MΩiΑT⊗Α.For sufficiently large i and stable matrix F, (39) can be rewritten as(41)Eω~iτ2=ΔI-F-1τ.Substituting τ=vec⁡(diag⁡(ek)⊗IM) into (41), we can obtain the mean square deviation (MSD) as follows:(42)Eω~k,∞2=ΔI-F-1vec⁡diag⁡ek⊗IM,where ek is a K×1 vector with a single entry 1 at position k and zeros elsewhere.


## 4. Simulations

In what follows, we present computer simulations to evaluate the performance of the proposed memoryd-LMS algorithm and to verify the theoretical results introduced in Section 3.Figure2 gives the network topology with K=20 nodes considered in this section. Besides, noise power levels, that is, σk,i2, vary across the nodes within [10-10,1]. Correspondingly, SNRs vary within [0,10] dB which is shown in Figure 3, where SNR=-log10⁡(σk,i2). The network data are generated based on model (1), and the unknown vector ω0=[-1,3]T. The weights of al,k and cl,k are defined according to the relative degree and Metropolis rule [10, 13]: that is,(43)al,k=degl∑n∈Nkdegn,ifl∈Nk0,otherwise,cl,k=1max⁡degl,degkifl∈Nk,l≠k1-∑l≠kcl,kifl=k0,otherwise,where degl denotes the number of neighbors of node l or the degree of node l.Figure 2
Network topology withK=20 nodes.Figure 3
SNR levels varying across the nodes within[0,10] dB.We show the transient network mean square deviation (MSD):∑k=1KE(βk-βk,i)2/K with different step-sizes μ and i0=20 in Figure 4. Figure 4 illustrates that the estimate is convergent. As observed, the smaller μ is, the more accurate the estimate is (the slower the convergence rate).Figure 4
Transient network MSDs of the memoryd-LMS for estimating the parameter ω0 with different step sizes; i0=20.Figure5 demonstrates the network transient behavior in terms of mean square deviation (MSD): ∑k=1KE(βk-βk,i)2/K for the proposed memory d-LMS algorithm, d-LMS algorithm [9], and memory global LMS (3). The values are obtained by averaging over 200 independent experiments. As the results indicate, the performance of the proposed memory d-LMS is closed to that of the global solution. We also observe that the performance of the proposed algorithm exceeds that of d-LMS [9] by more than 4 dB.Figure 5
Transient network MSDs for different schemes withi0=i.Figure6 shows tracking results of the changes in the parameter for the memory d-LMS with different i0 and traditional d-LMS. Additionally, step size of memory d-LMS is 0.08/i0. Step size of d-LMS is 0.08. In the beginning, ω0=[-1,3]T, and after 200 iterations, it is set to [1,5]T. It can be observed that the proposed tracing performance of the memory d-LMS outperforms that of traditional d-LMS both with i0=20 and i0=50. It also shows that the larger i0 is, the more accurate the estimate is. This is because that more statistical estimation information is considered in the estimation algorithm.Figure 6
Transient network MSDs of the different algorithms for estimating the parameter. An abrupt change occurs at the 200th iteration.As observed, the smallerμ is, the more accurate the estimate is and the slower the convergence rate is. Hence, it is necessary to choose the small μ and the large i0 according to (34) for systems that have strict request on estimate accuracy but are insensitive to convergence rate. Otherwise, choose the large μ and the small i0 according to (34).


## 5. Conclusion

In this paper, a distributed estimation algorithm called memoryd-LMS is derived and analyzed. The proposed algorithm consists of three steps. Firstly, at time i, for each node, a set of neighbor nodes collect and process their local information from time i-i0+1 to time i, to obtain their preestimates. Secondly, all nodes transmit the preestimates to their neighbors. Finally, each node combines the collected information together with its local preestimate to generate improved estimates. Simulation results show that the proposed algorithm improves the convergence rate and reduces the MSE effectively.



---

*Source: 9831378-2018-03-04.xml*