convergence of consensus models with stochastic disturbances we consider consensus algorithms in their most general setting and provide conditions under which such algorithms are guaranteed to converge almost surely to a consensus let a t b t rn n be possibly stochastic nonstationary matrices and x t m t 6 rn 1 be state and perturbation vectors respectively for any consensus algorithm of the form x t 1 a t x t b t m t we provide conditions under which consensus is achieved almost surely i e pr limt x t c1 1 for some c r moreover we show that this general result subsumes recently reported results for specific consensus algorithms classes including sum preserving nonsum preserving quantized and noisy gossip algorithms also provided are the e converging time for any such converging iterative algorithm i e the earliest time at which the vector x t is e close to consensus and sufficient conditions for convergence in expectation to the average of the initial node measurements finally mean square error bounds of any consensus algorithm of the form discussed above are presented

fast distributed average consensus algorithms based on advection diffusion processes distributed consensus algorithms have recently gained large interest in sensor networks as a way to achieve globally optimal decisions in a totally decentralized way that is without the need of sending all the data collected by the sensors to a fusion center however distributed algorithms are typically iterative and they suffer from convergence time and energy consumption in this paper we show that introducing appropriate asymmetric interaction mechanisms with time varying weights on each edge it is possible to provide a substantial increase of convergence rate with respect to the symmetric time invariant case the basic idea underlying our approach comes from modeling the average consensus algorithm as an advection diffusion process governing the homogenization of fluid mixtures exploiting such a conceptual link we show how introducing interaction mechanisms among nearby nodes mimicking suitable advection processes yields a substantial increase of convergence rate moreover we show that the homogenization enhancement induced by the advection term produces a qualitatively different scaling law of the convergence rate versus the network size with respect to the symmetric case

shape from texture estimation isotropy and moments a theory is proposed for the interpretation of 3d textures with oriented elements it builds on two previous theories a statistical one due to witkin and kanatani s buffon transform we show that the statistical estimation of orientation of a textured plane is one of a general family of optimal backprojection problems which also includes maximal compactness general uniqueness and convergence results are obtained for an efficient algorithm employing second moment feedback this framework greatly enhances the usefulness of surface orientation estimation for texture in two ways first an error distribution is derived for the estimator and this is crucial for integration of shape information second hypotheses about intrinsic texture statistics can be verified by a 2 test which if failed warns that the orientation estimator is not to be believed finally it is argued that these results suggest a general moment tensor approach to analysis of 3d texture

optimization and analysis of distributed averaging with short node memory distributed averaging describes a class of network algorithms for the decentralized computation of aggregate statistics initially each node has a scalar data value and the goal is to compute the average of these values at every node the so called average consensus problem nodes iteratively exchange information with their neighbors and perform local updates until the value at every node converges to the initial network average much previous work has focused on algorithms where each node maintains and updates a single value every time an update is performed the previous value is forgotten convergence to the average consensus is achieved asymptotically the convergence rate is fundamentally limited by network connectivity and it can be prohibitively slow on topologies such as grids and random geometric graphs even if the update rules are optimized in this paper we provide the first theoretical demonstration that adding a local prediction component to the update rule can significantly improve the convergence rate of distributed averaging algorithms we focus on the case where the local predictor is a linear combination of the node s current and previous values i e two memory taps and our update rule computes a combination of the predictor and the usual weighted linear combination of values received from neighboring nodes we derive the optimal mixing parameter for combining the predictor with the neighbors values and conduct a theoretical analysis of the improvement in convergence rate that can be achieved using this acceleration methodology for a chain topology on n nodes this leads to a factor of n improvement over standard consensus and for a two dimensional grid our approach achieves a factor of n improvement

regularity of quasi minimizers for non uniformly elliptic integrals abstract in this paper we consider a class of non uniformly elliptic integral functionals and we prove the local boundedness of the quasi minimizers our approach is based on a suitable adaptation of the celebrated de giorgi proof and it relies on an appropriate caccioppoli type inequality

dqm decentralized quadratically approximated alternating direction method of multipliers this paper considers decentralized consensus optimization problems where nodes of a network have access to different summands of a global objective function nodes cooperate to minimize the global objective by exchanging information with neighbors only a decentralized version of the alternating directions method of multipliers dadmm is a common method for solving this category of problems dadmm exhibits linear convergence rate to the optimal objective for strongly convex functions but its implementation requires solving a convex optimization problem at each iteration this can be computationally costly and may result in large overall convergence times the decentralized quadratically approximated admm algorithm dqm which minimizes a quadratic approximation of the objective function that dadmm minimizes at each iteration is proposed here the consequent reduction in computational time is shown to have minimal effect on convergence properties convergence still proceeds at a linear rate with a guaranteed factor that is asymptotically equivalent to the dadmm linear convergence rate factor numerical results demonstrate advantages of dqm relative to dadmm and other alternatives in a logistic regression problem

dsa decentralized double stochastic averaging gradient algorithm this paper considers optimization problems where nodes of a network have access to summands of a global objective each of these local objectives is further assumed to be an average of a finite set of functions the motivation for this setup is to solve large scale machine learning problems where elements of the training set are distributed to multiple computational elements the decentralized double stochastic averaging gradient dsa algorithm is proposed as a solution alternative that relies on i the use of local stochastic averaging gradients ii determination of descent steps as differences of consecutive stochastic averaging gradients strong convexity of local functions and lipschitz continuity of local gradients is shown to guarantee linear convergence of the sequence generated by dsa in expectation local iterates are further shown to approach the optimal argument for almost all realizations the expected linear convergence of dsa is in contrast to the sublinear rate characteristic of existing methods for decentralized stochastic optimization numerical experiments on a logistic regression problem illustrate reductions in convergence time and number of feature vectors processed until convergence relative to these other alternatives

consensus in ad hoc wsns with noisy links part ii distributed estimation and smoothing of random signals distributed algorithms are developed for optimal estimation of stationary random signals and smoothing of even nonstationary dynamical processes based on generally correlated observations collected by ad hoc wireless sensor networks wsns maximum a posteriori map and linear minimum mean square error lmmse schemes well appreciated for centralized estimation are shown possible to reformulate for distributed operation through the iterative alternating direction method of multipliers sensors communicate with single hop neighbors their individual estimates as well as multipliers measuring how far local estimates are from consensus when iterations reach consensus the resultant distributed d map and lmmse estimators converge to their centralized counterparts when inter sensor communication links are ideal the d map estimators do not require the desired estimator to be expressible in closed form the d lmmse ones are provably robust to communication or quantization noise and both are particularly simple to implement when the data model is linear gaussian for decentralized tracking applications distributed kalman filtering and smoothing algorithms are derived for any time mmse optimal consensus based state estimation using wsns analysis and corroborating numerical examples demonstrate the merits of the novel distributed estimators

sensor networks with random links topology design for distributed consensus in a sensor network in practice the communication among sensors is subject to 1 errors that can cause failures of links among sensors at random times 2 costs and 3 constraints such as power data rate or communication since sensors and networks operate under scarce resources the paper studies the problem of designing the topology i e assigning the probabilities of reliable communication among sensors or of link failures to maximize the rate of convergence of average consensus when the link communication costs are taken into account and there is an overall communication budget constraint we model the network as a bernoulli random topology and establish necessary and sufficient conditions for mean square sense mss and almost sure a s convergence of average consensus when network links fail in particular a necessary and sufficient condition is for the algebraic connectivity of the mean graph topology to be strictly positive with these results we show that the topology design with random link failures link communication costs and a communication cost constraint is a constrained convex optimization problem that can be efficiently solved for large networks by semidefinite programming techniques simulations demonstrate that the optimal design improves significantly the convergence speed of the consensus algorithm and can achieve the performance of a non random network at a fraction of the communication cost

quantized consensus by the admm probabilistic versus deterministic quantizers this paper develops efficient algorithms for distributed average consensus with quantized communication using the alternating direction method of multipliers admm we first study the effects of probabilistic and deterministic quantizations on a distributed admm algorithm with probabilistic quantization this algorithm yields linear convergence to the desired average in the mean sense with a bounded variance when deterministic quantization is employed the distributed admm converges to a consensus within 3 lceil log 1 delta omega rceil iterations where delta 0 depends on the network topology and omega is a polynomial fraction depending on the quantization resolution the agents data and the network topology a tight upper bound on the consensus error is also obtained which depends only on the quantization resolution and the average degree of the graph this bound is much preferred in large scale networks over existing algorithms whose consensus errors are increasing in the range of agents data the quantization resolution and the number of agents we finally propose our algorithm which combines both probabilistic and deterministic quantizations simulations show that the consensus error of our algorithm is typically less than one quantization resolution for all connected networks where agents data can be of arbitrary magnitudes

decentralized maximum likelihood estimation for sensor networks composed of nonlinearly coupled dynamical systems in this paper we propose a decentralized sensor network scheme capable to reach a globally optimum maximum likelihood ml estimate through self synchronization of nonlinearly coupled dynamical systems each node of the network is composed of a sensor and a first order dynamical system initialized with the local measurements nearby nodes interact with each other exchanging their state value and the final estimate is associated to the state derivative of each dynamical system we derive the conditions on the coupling mechanism guaranteeing that if the network observes one common phenomenon each node converges to the globally optimal ml estimate we prove that the synchronized state is globally asymptotically stable if the coupling strength exceeds a given threshold acting on a single parameter the coupling strength we show how in the case of nonlinear coupling the network behavior can switch from a global consensus system to a spatial clustering system finally we show the effect of the network topology on the scalability properties of the network and we validate our theoretical findings with simulation results

a fast proximal gradient algorithm for decentralized composite optimization over directed networks abstract this paper proposes a fast decentralized algorithm for solving a consensus optimization problem defined in a directed networked multi agent system where the local objective functions have the smooth nonsmooth composite form and are possibly nonconvex examples of such problems include decentralized compressed sensing and constrained quadratic programming problems as well as many decentralized regularization problems we extend the existing algorithms pg extra and extrapush to a new algorithm pg extrapush for composite consensus optimization over a directed network this algorithm takes advantage of the proximity operator like in pg extra to deal with the nonsmooth term and employs the push sum protocol like in extrapush to tackle the bias introduced by the directed network with a proper step size we show that pg extrapush converges to an optimal solution at a linear rate under some regular assumptions we conduct a series of numerical experiments to show the effectiveness of the proposed algorithm specifically with a proper step size pg extrapush performs linear rates in most of cases even in some nonconvex cases and is significantly faster than subgradient push even if the latter uses a hand optimized step size the established theoretical results are also verified by the numerical results

meaningful matches in stereovision this paper introduces a statistical method to decide whether two blocks in a pair of images match reliably the method ensures that the selected block matches are unlikely to have occurred just by chance the new approach is based on the definition of a simple but faithful statistical background model for image blocks learned from the image itself a theorem guarantees that under this model not more than a fixed number of wrong matches occurs on average for the whole image this fixed number the number of false alarms is the only method parameter furthermore the number of false alarms associated with each match measures its reliability this a contrario block matching method however cannot rule out false matches due to the presence of periodic objects in the images but it is successfully complemented by a parameterless self similarity threshold experimental evidence shows that the proposed method also detects occlusions and incoherent motions due to vehicles and pedestrians in nonsimultaneous stereo

on the problem of reconstructing an unknown topology via locality properties of the wiener filter determining interrelatedness structure of various entities from multiple time series data is of significant interest to many areas knowledge of such a structure can aid in identifying cause and effect relationships clustering of similar entities identification of representative elements and model reduction the majority of existing results are based on correlation based indices which effectively assume a static relationship between the time series data and are not suitable for detecting interrelatedness when the time series are dynamically related or when the time series involve loops in this paper a methodology for identifying the interrelatedness structure of dynamically related time series data is presented that also allows for the presence of loops in the connectivity structure a linear dynamic graph model is presented where it is assumed that each time series data is the sum of an independent stochastic noise source and a dynamically weighted sum of other time series data a link is assumed to be present between two time series if the weight of a time series which is a linear time invariant filter is nonzero in the formation of the other reconstruction of the link connectivity structure under various scenarios is considered it is shown that when the linear dynamic graph is allowed to admit non causal weights then the links structure can be recovered with the possibility of identifying spurious connections however it is shown that the spurious links remain local where a spurious link is restricted to be within one hop of a true link furthermore strategies for exact reconstruction of the link structure when the weights are restricted to be causal are developed the main tools for determining the network topology are based on variations of wiener filtering a significant insight provided by the article is that in the class of network models considered in the paper the wiener filter estimating a stochastic process based on other processes remains local in the sense that the wiener filter utilizes only measurements local to the node being estimated

stochastic link activation for distributed filtering under sensor power constraint we consider the problem of link activation for distributed estimation with power constraint to satisfy the requirement of power consumption we propose a stochastic link activation scheme where each sensor equipped with a distributed estimator sends data to its neighboring sensors according to different probabilities first we design the optimal estimator gain of each sensor to minimize the state estimation error covariance then we find an upper bound of the expected state estimation error covariance and provide a sufficient condition to guarantee the stability of the proposed estimator finally we formulate the link activation problem as an optimization problem and convert it to a convex optimization

matching two perspective views a computational approach to image matching is described it uses multiple attributes associated with each image point to yield a generally overdetermined system of constraints taking into account possible structural discontinuities and occlusions in the algorithm implemented intensity edgeness and cornerness attributes are used in conjunction with the constraints arising from intraregional smoothness field continuity and discontinuity and occlusions to compute dense displacement fields and occlusion maps along the pixel grids the intensity edgeness and cornerness are invariant under rigid motion in the image plane in order to cope with large disparities a multiresolution multigrid structure is employed coarser level edgeness and cornerness measures are obtained by blurring the finer level measures the algorithm has been tested on real world scenes with depth discontinuities and occlusions a special case of two view matching is stereo matching where the motion between two images is known the algorithm can be easily specialized to perform stereo matching using the epipolar constraint

distributed parametric and nonparametric regression with on line performance bounds computation in this paper we focus on collaborative multi agent systems where agents are distributed over a region of interest and collaborate to achieve a common estimation goal in particular we introduce two consensus based distributed linear estimators the first one is designed for a bayesian scenario where an unknown common finite dimensional parameter vector has to be reconstructed while the second one regards the nonparametric reconstruction of an unknown function sampled at different locations by the sensors both of the algorithms are characterized in terms of the trade off between estimation performance communication computation and memory complexity in the finite dimensional setting we derive mild sufficient conditions which ensure that a distributed estimator performs better than the local optimal ones in terms of estimation error variance in the nonparametric setting we introduce an on line algorithm that allows the agents to simultaneously compute the function estimate with small computational communication and data storage efforts as well as to quantify its distance from the centralized estimate given by a regularization network one of the most powerful regularized kernel methods these results are obtained by deriving bounds on the estimation error that provide insights on how the uncertainty inherent in a sensor network such as imperfect knowledge on the number of agents and the measurement models used by the sensors can degrade the performance of the estimation process numerical experiments are included to support the theoretical findings

stochastic consensus over noisy networks with markovian and arbitrary switches this paper considers stochastic consensus problems over lossy wireless networks we first propose a measurement model with a random link gain additive noise and markovian lossy signal reception which captures uncertain operational conditions of practical networks for consensus seeking we apply stochastic approximation and derive a markovian mode dependent recursive algorithm mean square and almost sure i e probability one convergence analysis is developed via a state space decomposition approach when the coefficient matrix in the algorithm satisfies a zero row and column sum condition subsequently we consider a model with arbitrary random switching and a common stochastic lyapunov function technique is used to prove convergence finally our method is applied to models with heterogeneous quantizers and packet losses and convergence results are proved

adaptive filter algorithms for accelerated discrete time consensus in many distributed systems the objective is to reach agreement on values acquired by the nodes in a network a common approach to solve such problems is the iterative weighted linear combination of those values to which each node has access methods to compute appropriate weights have been extensively studied but the resulting iterative algorithms still require many iterations to provide a fairly good estimate of the consensus value in this study we show that a good estimate of the consensus value can be obtained with few iterations of conventional consensus algorithms by filtering the output of each node with set theoretic adaptive filters we use the adaptive projected subgradient method to derive a set theoretic filter requiring only local information available to each node and being robust to topology changes and erroneous information about the network numerical simulations show the good performance of the proposed method

a distributed framework for dimensionality reduction and denoising with the recent upsurge in the amount of data transmission and use in networks a lot of emphasis is placed on designing efficient methods to extract useful information by reducing the size of large data sets a major approach in data dimensionality reduction relies on the estimation of the principal data covariance eigenspace in this paper a distributed algorithmic framework is put forth for finding the principal eigenspace of spatially scattered sensor data toward this end the standard principal component analysis framework is reformulated as a separable constrained minimization problem which is solved by utilizing coordinate descent techniques combined with the alternating direction method of multipliers computationally simple local updating recursions are obtained that involve only single hop intersensor communications the proposed distributed algorithm is shown to be robust even in the presence of inter sensor communication noise and converge to the principal eigenspace when inter sensor links are ideal the proposed distributed framework is used in a data denoising setting while extensive numerical tests using both synthetic and real data demonstrate a fast convergence rate better steady state performance of the novel algorithm over existing alternatives and communication noise resilience

perturbed proximal primal dual algorithm for nonconvex nonsmooth optimization in this paper we propose a perturbed proximal primal dual algorithm pprox pda for an important class of linearly constrained optimization problems whose objective is the sum of smooth possibly nonconvex and convex possibly nonsmooth functions this family of problems can be used to model many statistical and engineering applications such as high dimensional subspace estimation and the distributed machine learning the proposed method is of the uzawa type in which a primal gradient descent step is performed followed by an approximate dual gradient ascent step one distinctive feature of the proposed algorithm is that the primal and dual steps are both perturbed appropriately using past iterates so that a number of asymptotic convergence and rate of convergence results to first order stationary solutions can be obtained finally we conduct extensive numerical experiments to validate the effectiveness of the proposed algorithm

stochastic proximal gradient consensus over random networks we consider solving a convex optimization problem with possibly stochastic gradient and over a randomly time varying multiagent network each agent has access to some local objective function and it only has unbiased estimates of the gradients of the smooth component we develop a dynamic stochastic proximal gradient consensus algorithm with the following key features 1 it works for both the static and certain randomly time varying networks 2 it allows the agents to utilize either the exact or stochastic gradient information 3 it is convergent with provable rate in particular the proposed algorithm converges to a global optimal solution with a rate of mathcal o 1 r resp mathcal o 1 sqrt r when the exact resp stochastic gradient is available where r is the iteration counter interestingly the developed algorithm establishes a close connection among a number of seemingly unrelated distributed algorithms such as the extra the pg extra the ic idc admm the dlm and the classical distributed subgradient method

distributed estimation and detection for sensor networks using hidden markov random field models we develop a hidden markov random field hmrf framework for distributed signal processing in sensor network environments under this framework spatially distributed observations collected at the sensors form a noisy realization of an underlying random field that has a simple structure with markovian dependence we derive iterated conditional modes icm algorithms for distributed estimation of the hidden random field from the noisy measurements we consider both parametric and nonparametric measurement error models the proposed distributed estimators are computationally simple applicable to a wide range of sensing environments and localized implying that the nodes communicate only with their neighbors to obtain the desired results we also develop a calibration method for estimating markov random field model parameters from training data and discuss initialization of the icm algorithms the hmrf framework and icm algorithms are applied to event region detection numerical simulations demonstrate the performance of the proposed approach

adaptive robust distributed learning in diffusion sensor networks in this paper the problem of adaptive distributed learning in diffusion networks is considered the algorithms are developed within the convex set theoretic framework more specifically they are based on computationally simple geometric projections onto closed convex sets the paper suggests a novel combine project adapt protocol for cooperation among the nodes of the network such a protocol fits naturally with the philosophy that underlies the projection based rationale moreover the possibility that some of the nodes may fail is also considered and it is addressed by employing robust statistics loss functions such loss functions can easily be accommodated in the adopted algorithmic framework all that is required from a loss function is convexity under some mild assumptions the proposed algorithms enjoy monotonicity asymptotic optimality asymptotic consensus strong convergence and linear complexity with respect to the number of unknown parameters finally experiments in the context of the system identification task verify the validity of the proposed algorithmic schemes which are compared to other recent algorithms that have been developed for adaptive distributed learning

weight optimization for consensus algorithms with correlated switching topology we design the weights in consensus algorithms for spatially correlated random topologies these arise with 1 networks with spatially correlated random link failures and 2 networks with randomized averaging protocols we show that the weight optimization problem is convex for both symmetric and asymmetric random graphs with symmetric random networks we choose the consensus mean square error mse convergence rate as the optimization criterion and explicitly express this rate as a function of the link formation probabilities the link formation spatial correlations and the consensus weights we prove that the mse convergence rate is a convex nonsmooth function of the weights enabling global optimization of the weights for arbitrary link formation probabilities and link correlation structures we extend our results to the case of asymmetric random links we adopt as optimization criterion the mean square deviation msdev of the nodes states from the current average state we prove that msdev is a convex function of the weights simulations show that significant performance gain is achieved with our weight design method when compared with other methods available in the literature

decentralized quasi newton methods we introduce the decentralized broyden fletcher goldfarb shanno d bfgs method as a variation of the bfgs quasi newton method for solving decentralized optimization problems decentralized quasi newton methods are of interest in problems that are not well conditioned making first order decentralized methods ineffective and in which second order information is not readily available making second order decentralized methods impossible d bfgs is a fully distributed algorithm in which nodes approximate curvature information of themselves and their neighbors through the satisfaction of a secant condition we additionally provide a formulation of the algorithm in asynchronous settings convergence of d bfgs is established formally in both the synchronous and asynchronous settings and strong performance advantages relative to existing methods are shown numerically

chebyshev polynomials in distributed consensus applications in this paper we analyze the use of chebyshev polynomials in distributed consensus applications it is well known that the use of polynomials speeds up the convergence to the consensus in a significant way however existing solutions only work for low degree polynomials and require the topology of the network to be fixed and known we propose a distributed algorithm based on the second order difference equation that describes the chebyshev polynomials of first kind the contributions of our algorithm are three i since the evaluation of chebyshev polynomials is stable there is no limitation in the degree of the polynomial ii instead of the knowledge of the whole network topology it only requires a partial knowledge or an approximation to it iii it can be applied to time varying topologies in the paper we characterize the main properties of the algorithm for both fixed and time varying communication topologies theoretical results as well as experiments with synthetic data show the benefits of using our algorithm compared to existing methods

distributed parameter estimation with quantized communication via running average in this paper we consider the problem of parameter estimation over sensor networks in the presence of quantized data and directed communication links we propose a two stage distributed algorithm aiming at achieving the centralized sample mean estimate in a distributed manner different from the existing algorithms a running average technique is utilized in the proposed algorithm to smear out the randomness caused by the probabilistic quantization scheme with the running average technique it is shown that the centralized sample mean estimate can be achieved both in the mean square and almost sure senses which is not observed in the standard consensus algorithms in addition the rates of convergence are given to quantify the mean square and almost sure performances finally simulation results are presented to illustrate the effectiveness of the proposed algorithm and highlight the improvements by using running average technique

coordination and consensus of networked agents with noisy measurements stochastic algorithms and asymptotic behavior this paper considers the coordination and consensus of networked agents where each agent has noisy measurements of its neighbors states for consensus seeking we propose stochastic approximation type algorithms with a decreasing step size and introduce the notions of mean square and strong consensus although the decreasing step size reduces the detrimental effect of the noise it also reduces the ability of the algorithm to drive the individual states towards each other the key technique is to ensure a trade off for the decreasing rate of the step size by following this strategy we first develop a stochastic double array analysis in a two agent model which leads to both mean square and strong consensus and extend the analysis to a class of well studied symmetric models subsequently we consider a general network topology and introduce stochastic lyapunov functions together with the so called direction of invariance to establish mean square consensus finally we apply the stochastic lyapunov analysis to a leader following scenario

a necessary and sufficient condition for consensus over random networks we consider the consensus problem for stochastic discrete time linear dynamical systems the underlying graph of such systems at a given time instance is derived from a random graph process independent of other time instances for such a framework we present a necessary and sufficient condition for almost sure asymptotic consensus using simple ergodicity and probabilistic arguments this easily verifiable condition uses the spectrum of the average weight matrix finally we investigate a special case for which the linear dynamical system converges to a fixed vector with probability 1

distributed linear parameter estimation asymptotically efficient adaptive strategies this paper considers the problem of distributed adaptive linear parameter estimation in multiagent inference networks local sensing model information is only partially available at the agents and interagent communication is assumed to be unpredictable the paper develops a generic mixed time scale stochastic procedure consisting of simultaneous distributed learning and estimation in which the agents adaptively assess their relative observation quality over time and fuse the innovations accordingly under rather weak assumptions on the statistical model and the interagent communication it is shown that by properly tuning the consensus potential with respect to the innovation potential the asymptotic information rate loss incurred in the learning process may be made negligible as such it is shown that the agent estimates are asymptotically efficient in that their asymptotic covariance coincides with that of a centralized estimator the inverse of the centralized fisher information rate for gaussian

convergence rates of distributed nesterov like gradient methods on random networks we consider distributed optimization in random networks where n nodes cooperatively minimize the sum i 1n fi x of their individual convex costs existing literature proposes distributed gradient like methods that are computationally cheap and resilient to link failures but have slow convergence rates in this paper we propose accelerated distributed gradient methods that 1 are resilient to link failures 2 computationally cheap and 3 improve convergence rates over other gradient methods we model the network by a sequence of independent identically distributed random matrices w k drawn from the set of symmetric stochastic matrices with positive diagonals the network is connected on average and the cost functions are convex differentiable with lipschitz continuous and bounded gradients we design two distributed nesterov like gradient methods that modify the d ng and d nc methods that we proposed for static networks we prove their convergence rates in terms of the expected optimality gap at the cost function let k and k be the number of per node gradient evaluations and per node communications respectively then the modified d ng achieves rates o logk k and o logk k and the modified d nc rates o 1 k2 and o 1 k2 where 0 is arbitrarily small for comparison the standard distributed gradient method cannot do better than 1 k2 3 and 1 k2 3 on the same class of cost functions even for static networks simulation examples illustrate our analytical findings

consensus seeking in multi agent systems with multiplicative measurement noises abstract in this paper the consensus problems of the continuous time integrator systems under noisy measurements are considered the measurement noises which appear when agents measure their neighbors states are modeled to be multiplicative by multiplication of the noises here the noise intensities are proportional to the absolute value of the relative states of an agent and its neighbor by using known distributed protocols for integrator agent systems the closed loop system is described in the vector form by a singular stochastic differential equation for the fixed and switching network topology cases constant consensus gains are properly selected such that mean square consensus and strong consensus can be achieved especially exponential mean square convergence of agents states to the common value is derived for the fixed topology case in addition asymptotic unbiased mean square average consensus and asymptotic unbiased strong average consensus are also studied simulations shed light on the effectiveness of the proposed theoretical results

distributed recursive least squares for consensus based in network adaptive estimation recursive least squares rls schemes are of paramount importance for reducing complexity and memory requirements in estimating stationary signals as well as for tracking nonstationary processes especially when the state and or data model are not available and fast convergence rates are at a premium to this end a fully distributed d rls algorithm is developed for use by wireless sensor networks wsns whereby sensors exchange messages with one hop neighbors to consent on the network wide estimates adaptively the wsns considered here do not necessarily possess a hamiltonian cycle while the inter sensor links are challenged by communication noise the novel algorithm is obtained after judiciously reformulating the exponentially weighted least squares cost into a separable form which is then optimized via the alternating direction method of multipliers if powerful error control codes are utilized and communication noise is not an issue d rls is modified to reduce communication overhead when compared to existing noise unaware alternatives numerical simulations demonstrate that d rls can outperform existing approaches in terms of estimation performance and noise resilience while it has the potential of performing efficient tracking

polynomial filtering for fast convergence in distributed consensus in the past few years the problem of distributed consensus has received a lot of attention particularly in the framework of ad hoc sensor networks most methods proposed in the literature address the consensus averaging problem by distributed linear iterative algorithms with asymptotic convergence of the consensus solution the convergence rate of such distributed algorithms typically depends on the network topology and the weights given to the edges between neighboring sensors as described by the network matrix in this paper we propose to accelerate the convergence rate for given network matrices by the use of polynomial filtering algorithms the main idea of the proposed methodology is to apply a polynomial filter on the network matrix that will shape its spectrum in order to increase the convergence rate such an algorithm is equivalent to periodic updates in each of the sensors by aggregating a few of its previous estimates we formulate the computation of the coefficients of the optimal polynomial as a semidefinite program that can be efficiently and globally solved for both static and dynamic network topologies we finally provide simulation results that demonstrate the effectiveness of the proposed solutions in accelerating the convergence of distributed consensus averaging problems

consensus based distributed multiple model ukf for jump markov nonlinear systems this note studies the problem of distributed estimation for jump markov nonlinear systems jmnlss in a not fully connected sensor network based on the consensus theory a distributed unscented kalman filter ukf is first derived for nonlinear systems without jumping parameters and then it is extended to develop a distributed multiple model ukf for jmnlss the proposed filtering algorithm is illustrated via a simulation example involving tracking a maneuvering target

linear convergence rate of a class of distributed augmented lagrangian algorithms we study distributed optimization where nodes cooperatively minimize the sum of their individual locally known convex costs f i x s x in bbr d is global distributed augmented lagrangian al methods have good empirical performance on several signal processing and learning applications but there is limited understanding of their convergence rates and how it depends on the underlying network this paper establishes globally linear geometric convergence rates of a class of deterministic and randomized distributed al methods when the f i s are twice continuously differentiable and have a bounded hessian we give explicit dependence of the convergence rates on the underlying network parameters simulations illustrate our analytical findings

stochastic approximation for consensus a new approach via ergodic backward products this paper considers both synchronous and asynchronous consensus algorithms with noisy measurements for stochastic approximation based consensus algorithms the existing convergence analysis with dynamic topologies heavily relies on quadratic lyapunov functions whose existence however may be difficult to ensure for switching directed graphs our main contribution is to introduce a new analytical approach we first show a fundamental role of ergodic backward products for mean square consensus in algorithms with additive noise subsequently we develop ergodicity results for backward products of degenerating stochastic matrices converging to a 0 1 matrix via a discrete time dynamical system approach these results complement the existing ergodic theory of stochastic matrices and provide an effective tool for analyzing consensus problems under a joint connectivity assumption our approach may deal with switching topologies delayed measurements and correlated noises and it does not require the double stochasticity condition typically assumed for the existence of quadratic lyapunov functions

kernel based structural equation models for topology identification of directed networks structural equation models sems have been widely adopted for inference of causal interactions in complex networks recent examples include unveiling topologies of hidden causal networks over which processes such as spreading diseases or rumors propagate the appeal of sems in these settings stems from their simplicity and tractability since they typically assume linear dependencies among observable variables acknowledging the limitations inherent to adopting linear models the present paper put forth nonlinear sems which account for possible nonlinear dependencies among network nodes the advocated approach leverages kernels as a powerful encompassing framework for nonlinear modeling and an efficient estimator with affordable tradeoffs is put forth interestingly pursuit of the novel kernel based approach yields a convex regularized estimator that promotes edge sparsity a property exhibited by most real world networks and the resulting optimization problem is amenable to proximal splitting optimization methods to this end solvers with complementary merits are developed by leveraging the alternating direction method of multipliers and proximal gradient iterations experiments conducted on simulated data demonstrate that the novel approach outperforms linear sems with respect to edge detection errors furthermore tests on a real gene expression dataset unveil interesting new edges that were not revealed by linear sems which could shed more light on regulatory behavior of human genes

fast consensus by the alternating direction multipliers method the alternating direction multipliers method admm has been recently proposed as a practical and efficient algorithm for distributed computing we discuss its applicability to the average consensus problem in this paper by carefully relaxing admm augmentation coefficients we are able to analytically investigate its properties and to propose simple and strict analytical bounds these provide a clear indication on how to choose system parameters for optimized performance we prove both analytically and via simulations that the proposed approach exhibits convergence speed between the best in the literature classical and optimized solutions while providing the most powerful resilience to noise

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters

bandwidth constrained distributed estimation for wireless sensor networks part ii unknown probability density function wireless sensor networks wsns deployed to perform surveillance and monitoring tasks have to operate under stringent energy and bandwidth limitations these motivate well distributed estimation scenarios where sensors quantize and transmit only one or a few bits per observation for use in forming parameter estimators of interest in a companion paper we developed algorithms and studied interesting tradeoffs that emerge even in the simplest distributed setup of estimating a scalar location parameter in the presence of zero mean additive white gaussian noise of known variance herein we derive distributed estimators based on binary observations along with their fundamental error variance limits for more pragmatic signal models i known univariate but generally non gaussian noise probability density functions pdfs ii known noise pdfs with a finite number of unknown parameters iii completely unknown noise pdfs and iv practical generalizations to multivariate and possibly correlated pdfs estimators utilizing either independent or colored binary observations are developed and analyzed corroborating simulations present comparisons with the clairvoyant sample mean estimator based on unquantized sensor observations and include a motivating application entailing distributed parameter estimation where a wsn is used for habitat monitoring

fast and versatile algorithm for nearest neighbor search based on a lower bound tree in this paper we present a fast and versatile algorithm which can rapidly perform a variety of nearest neighbor searches efficiency improvement is achieved by utilizing the distance lower bound to avoid the calculation of the distance itself if the lower bound is already larger than the global minimum distance at the preprocessing stage the proposed algorithm constructs a lower bound tree lb tree by agglomeratively clustering all the sample points to be searched given a query point the lower bound of its distance to each sample point can be calculated by using the internal node of the lb tree to reduce the amount of lower bounds actually calculated the winner update search strategy is used for traversing the tree for further efficiency improvement data transformation can be applied to the sample and the query points in addition to finding the nearest neighbor the proposed algorithm can also i provide the k nearest neighbors progressively ii find the nearest neighbors within a specified distance threshold and iii identify neighbors whose distances to the query are sufficiently close to the minimum distance of the nearest neighbor our experiments have shown that the proposed algorithm can save substantial computation particularly when the distance of the query point to its nearest neighbor is relatively small compared with its distance to most other samples which is the case for many object recognition problems

distributed estimation of gaussian correlations we study a distributed estimation problem in which two remotely located parties alice and bob observe an unlimited number of i i d samples corresponding to two different parts of a random vector alice can send k bits on average to bob who in turn wants to estimate the cross correlation matrix between the two parts of the vector in the case where the parties observe jointly gaussian scalar random variables with an unknown correlation rho we obtain two constructive and simple unbiased estimators attaining a variance of 1 rho 2 2k ln 2 which coincides with a known but non constructive random coding result of zhang and berger we extend our approach to the vector gaussian case which has not been treated before and construct an estimator that is uniformly better than the scalar estimator applied separately to each of the correlations we then show that the gaussian performance can essentially be attained even when the distribution is completely unknown this in particular implies that in the general problem of distributed correlation estimation the variance can decay at least as o 1 k with the number of transmitted bits this behavior however is not tight we give an example of a rich family of distributions for which local samples reveal essentially nothing about the correlations and where a slightly modified estimator attains a variance of 2 omega k

stereo matching as a nearest neighbor problem we propose a representation of images called intrinsic curves that transforms stereo matching from a search problem into a nearest neighbor problem intrinsic curves are the paths that a set of local image descriptors trace as an image scanline is traversed from left to right intrinsic curves are ideally invariant with respect to disparity stereo correspondence then becomes a trivial lookup problem in the ideal case we also show how to use intrinsic curves to match real images in the presence of noise brightness bias contrast fluctuations moderate geometric distortion image ambiguity and occlusions in this case matching becomes a nearest neighbor problem even for very large disparity values

extensions of fast lipschitz optimization the need of fast distributed solvers for optimization problems in networked systems has motivated the recent development of the fast lipschitz optimization framework in such an optimization problems satisfying certain qualifying conditions such as monotonicity of the objective function and contractivity of the constraints have a unique optimal solution obtained via fast distributed algorithms that compute the fixed point of the constraints this paper extends the set of problems for which the fast lipschitz framework applies existing assumptions on the problem form are relaxed and new and generalized qualifying conditions are established by novel results based on lagrangian duality it is shown for which cases of more constraints than decision variables and less constraints than decision variables fast lipschitz optimization applies new results are obtained by imposing non strict monotonicity of the objective functions the extended fast lipschitz framework is illustrated by a number of examples including network optimization and optimal control problems

decentralized dynamic optimization through the alternating direction method of multipliers this paper develops the application of the alternating direction method of multipliers admm to optimize a dynamic objective function in a decentralized multi agent system at each time slot agents in the network observe local functions and cooperate to track the optimal time varying argument of the sum objective this cooperation is based on maintaining local primal variables that estimate the value of the optimal argument and auxiliary dual variables that encourage proximity with neighboring estimates primal and dual variables are updated by an admm iteration that can be implemented in a distributed manner whereby local updates require access to local variables and the most recent primal variables from adjacent agents for objective functions that are strongly convex and have lipschitz continuous gradients the distances between the primal and dual iterates to their corresponding time varying optimal values are shown to converge to a steady state gap this gap is explicitly characterized in terms of the condition number of the objective function the condition number of the network that is defined as the ratio between the largest and smallest nonzero laplacian eigenvalues and a bound on the drifts of the optimal primal variables and the optimal gradients numerical experiments corroborate theoretical findings and show that the results also hold for non differentiable and non strongly convex primal objectives

convergence rate of distributed admm over networks we propose a new distributed algorithm based on alternating direction method of multipliers admm to minimize sum of locally known convex functions using communication over a network this optimization problem emerges in many applications in distributed machine learning and statistical estimation our algorithm allows for a general choice of the communication weight matrix which is used to combine the iterates at different nodes we show that when functions are convex both the objective function values and the feasibility violation converge with rate o 1 t where t is the number of iterations we then show that when functions are strongly convex and have lipschitz continuous gradients the sequence generated by our algorithm converges linearly to the optimal solution in particular an epsilon optimal solution can be computed with o left sqrt kappa f log 1 epsilon right iterations where kappa f is the condition number of the problem our analysis highlights the effect of network and communication weights on the convergence rate through degrees of the nodes the smallest nonzero eigenvalue and operator norm of the communication matrix

on the convergence of decentralized gradient descent consider the consensus problem of minimizing f x sum i 1 n f i x where x in mathbb r p and each f i is only known to the individual agent i in a connected network of n agents to solve this problem and obtain the solution all the agents collaborate with their neighbors through information exchange this type of decentralized computation does not need a fusion center offers better network load balance and improves data privacy this paper studies the decentralized gradient descent method a nedic and a ozdaglar ieee trans automat control 54 2009 pp 48 61 in which each agent i updates its local variable x i in mathbb r n by combining the average of its neighbors with a local negative gradient step alpha nabla f i x i the method is described by the iteration x i k 1 gets sum j 1 n w ij x j k alpha nabla f i x i k text for each agent i where w ij is nonzero only if i and j are neighbors or i j and the matrix

distributed average consensus with quantization refinement we consider the problem of distributed average consensus in a sensor network where sensors exchange quantized information with their neighbors we propose a novel quantization scheme that exploits the increasing correlation between the values exchanged by the sensors throughout the iterations of the consensus algorithm a low complexity uniform quantizer is implemented in each sensor and refined quantization is achieved by progressively reducing the quantization intervals during the convergence of the consensus algorithm we propose a recurrence relation for computing the quantization parameters that depend on the network topology and the communication rate we further show that the recurrence relation can lead to a simple exponential model for the quantization step size over the iterations whose parameters can be computed a priori finally simulation results demonstrate the effectiveness of the progressive quantization scheme that leads to the consensus solution even at low communication rate

a saddle point algorithm for networked online convex optimization an algorithm to learn optimal actions in convex distributed online problems is developed learning is online because cost functions are revealed sequentially and distributed because they are revealed to agents of a network that can exchange information with neighboring nodes only learning is measured in terms of the global network regret which is defined here as the accumulated loss of causal prediction with respect to a centralized clairvoyant agent to which the information of all times and agents is revealed at the initial time a variant of the arrow hurwicz saddle point algorithm is proposed to control the growth of global network regret this algorithm uses lagrange multipliers to penalize the discrepancies between agents and leads to an implementation that relies on local operations and exchange of variables between neighbors we show that decisions made with this saddle point algorithm lead to regret whose order is not larger than o sqrt t where t is the total operating time numerical behavior is illustrated for the particular case of distributed recursive least squares an application to computer network security in which service providers cooperate to detect the signature of malicious users is developed to illustrate the practical value of the proposed algorithm

adding a single state memory optimally accelerates symmetric linear maps previous papers have proposed to add memory registers to the dynamics of discrete time linear systems in order to accelerate their convergence in particular it has been proved that adding one memory slot per agent allows faster convergence towards average consensus we here prove that this situation cannot be improved by adding more memory slots when the knowledge about the self adjoint linear map to be accelerated reduces to bounds on its extreme eigenvalues

success and failure of adaptation diffusion algorithms with decaying step size in multiagent networks this paper investigates the problem of distributed stochastic approximation in multiagent systems the algorithm under study consists of two steps a local stochastic approximation step and a diffusion step which drives the network to a consensus the diffusion step uses row stochastic matrices to weight the network exchanges as opposed to previous works exchange matrices are not supposed to be doubly stochastic and may also depend on the past estimate we prove that nondoubly stochastic matrices generally influence the limit points of the algorithm nevertheless the limit points are not affected by the choice of the matrices provided that the latter are doubly stochastic in expectation this conclusion legitimates the use of broadcast like diffusion protocols which are easier to implement next by means of a central limit theorem we prove that doubly stochastic protocols perform asymptotically as well as centralized algorithms and we quantify the degradation caused by the use of nondoubly stochastic matrices throughout this paper a special emphasis is put on the special case of distributed nonconvex optimization as an illustration of our results

detection of surface orientation and motion from texture by a stereoogical technique abstract a new approach is given to detect the surface orientation and motion from the texture on the surface by making use of a mathematical principle called stereology information about the surface orientation is contained in features computed by scanning the image by parallel lines and counting the number of intersections with the curves of the texture a synthetic example is given to illustrate the technique this scheme can also detect surface motions relative to the viewer by computing features of its texture at one time and a short time later the motion is specified by explicit formulae of the computed features

proximity without consensus in online multiagent optimization we consider stochastic optimization problems in multiagent settings where a network of agents aims to learn parameters that are optimal in terms of a global convex objective while giving preference to locally observed streaming information to do so we depart from the canonical decentralized optimization framework where agreement constraints are enforced and instead formulate a problem where each agent minimizes a global convex objective while enforcing network proximity constraints this formulation includes online consensus optimization as a special case but allows for the more general hypothesis that there is data heterogeneity across the network to solve this problem we propose using a stochastic saddle point algorithm inspired by arrow and hurwicz this method yields a decentralized algorithm for processing observations sequentially received at each node of the network using lagrange multipliers to penalize the discrepancy between them only neighboring nodes exchange model information we establish that under a constant step size regime the time average suboptimality and constraint violation are contained in a neighborhood whose radius vanishes with increasing number of iterations as a consequence we prove that the time average primal vectors converge to the optimal objective while satisfying the network proximity constraints we apply this method to the problem of sequentially estimating a correlated random field in a sensor network as well as an online source localization problem both of which demonstrate the empirical validity of the aforementioned convergence results

consensus algorithms with state dependent weights we provide an analysis of a distributed consensus type algorithm with weights dependent only on the received data our approach does not require any global knowledge of the network which distinguishes our approach from most other contributions in general the weights may be further modified by any function this in turn may model various network behaviors e g link failures and time varying networks we provide general convergence conditions prove several important properties derive steady states for such locally derived weights as well as analyze the convergence rate of the algorithm applications of such weight model include distributed algorithms which do not require exact average and have no global information about the network topology

distributed maximum likelihood sensor network localization we propose a class of convex relaxations to solve the sensor network localization problem based on a maximum likelihood ml formulation this class as well as the tightness of the relaxations depends on the noise probability density function pdf of the collected measurements we derive a computational efficient edge based version of this ml convex relaxation class and we design a distributed algorithm that enables the sensor nodes to solve these edge based convex programs locally by communicating only with their close neighbors this algorithm relies on the alternating direction method of multipliers admm it converges to the centralized solution it can run asynchronously and it is computation error resilient finally we compare our proposed distributed scheme with other available methods both analytically and numerically and we argue the added value of admm especially for large scale networks

consensus in ad hoc wsns with noisy links part i distributed estimation of deterministic signals we deal with distributed estimation of deterministic vector parameters using ad hoc wireless sensor networks wsns we cast the decentralized estimation problem as the solution of multiple constrained convex optimization subproblems using the method of multipliers in conjunction with a block coordinate descent approach we demonstrate how the resultant algorithm can be decomposed into a set of simpler tasks suitable for distributed implementation different from existing alternatives our approach does not require the centralized estimator to be expressible in a separable closed form in terms of averages thus allowing for decentralized computation even of nonlinear estimators including maximum likelihood estimators mle in nonlinear and non gaussian data models we prove that these algorithms have guaranteed convergence to the desired estimator when the sensor links are assumed ideal furthermore our decentralized algorithms exhibit resilience in the presence of receiver and or quantization noise in particular we introduce a decentralized scheme for least squares and best linear unbiased estimation blue and establish its convergence in the presence of communication noise our algorithms also exhibit potential for higher convergence rate with respect to existing schemes corroborating simulations demonstrate the merits of the novel distributed estimation algorithms

a theory for multiresolution signal decomposition the wavelet representation multiresolution representations are effective for analyzing the information content of images the properties of the operator which approximates a signal at a given resolution were studied it is shown that the difference of information between the approximation of a signal at the resolutions 2 sup j 1 and 2 sup j where j is an integer can be extracted by decomposing this signal on a wavelet orthonormal basis of l sup 2 r sup n the vector space of measurable square integrable n dimensional functions in l sup 2 r a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi x this decomposition defines an orthogonal multiresolution representation called a wavelet representation it is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters wavelet representation lies between the spatial and fourier domains for images the wavelet representation differentiates several spatial orientations the application of this representation to data compression in image coding texture discrimination and fractal analysis is discussed