a discrete nodal domain theorem for trees let g be a connected graph with n vertices and let x x1 xn be a real vector a positive negative sign graph of the vector x is a maximal connected subgraph of g on vertices xi 0 xi 0 for an eigenvalue of a generalized laplacian of a tree we characterize the maximal number of sign graphs of an eigenvector we give an o n2 time algorithm to find an eigenvector with maximum number of sign graphs and we show that finding an eigenvector with minimum number of sign graphs is an np complete problem author s abstract

asynchronous distributed algorithms for solving linear algebraic equations two asynchronous distributed algorithms are presented for solving a linear equation of the form ax b with at least one solution the equation is simultaneously and asynchronously solved by m agents assuming that each agent knows only a subset of the rows of the partitioned matrix a b the estimates of the equation s solution generated by its neighbors and nothing more neighbor relationships are characterized by a time dependent directed graph whose vertices correspond to agents and whose arcs depict neighbor relationships each agent recursively updates its estimate of a solution at its own event times by utilizing estimates generated by its neighbors which are transmitted with delays the event time sequences of different agents are not assumed to be synchronized it is shown that for any matrix vector pair a b for which the equation has a solution and any repeatedly jointly strongly connected sequence of neighbor graphs defined on the merged sequence of all agents event times the algorithms cause all agents estimates to converge exponentially fast to the same solution to ax b the first algorithm requires a specific initialization step at each agent and the second algorithm works for arbitrary initializations explicit expressions for convergence rates are provided and a relation between local initializations and limiting consensus solutions is established which is used to solve the least 2 norm solution

distributed subgradient methods for convex optimization over random networks we consider the problem of cooperatively minimizing the sum of convex functions where the functions represent local objective functions of the agents we assume that each agent has information about his local function and communicate with the other agents over a time varying network topology for this problem we propose a distributed subgradient method that uses averaging algorithms for locally sharing information among the agents in contrast to previous works on multi agent optimization that make worst case assumptions about the connectivity of the agents such as bounded communication intervals between nodes we assume that links fail according to a given stochastic process under the assumption that the link failures are independent and identically distributed over time possibly correlated across links we provide almost sure convergence results for our subgradient algorithm

on the rectangularity of nonlinear block codes we give simple sufficient conditions for a code to be rectangular and show that large families of well known nonlinear codes are rectangular these include hadamard 1893 levenshtein 1964 delsarte goethals 1975 kerdock 1972 and nordstrom robinson 1967 codes being rectangular each of these codes has a unique minimal trellis that can be used for soft decision maximum likelihood decoding

pure adaptive search in global optimization pure adaptive seach iteratively constructs a sequence of interior points uniformly distributed within the corresponding sequence of nested improving regions of the feasible space that is at any iteration the next point in the sequence is uniformly distributed over the region of feasible space containing all points that are strictly superior in value to the previous points in the sequence the complexity of this algorithm is measured by the expected number of iterations required to achieve a given accuracy of solution we show that for global mathematical programs satisfying the lipschitz condition its complexity increases at mostlinearly in the dimension of the problem

exponential convergence of a distributed algorithm for solving linear algebraic equations abstract in a recent paper a distributed algorithm was proposed for solving linear algebraic equations of the form a x b assuming that the equation has at least one solution the equation is presumed to be solved by m agents assuming that each agent knows a subset of the rows of the matrix a b the current estimates of the equation s solution generated by each of its neighbors and nothing more neighbor relationships are represented by a time dependent directed graph n t whose vertices correspond to agents and whose arcs characterize neighbor relationships sufficient conditions on n t were derived under which the algorithm can cause all agents estimates to converge exponentially fast to the same solution to a x b these conditions were also shown to be necessary for exponential convergence provided the data about a b available to the agents is non redundant the aim of this paper is to relax this non redundant assumption this is accomplished by establishing exponential convergence under conditions which are the weakest possible for the problem at hand the conditions are based on a new notion of graph connectivity an improved bound on the convergence rate is also derived

distributed nonconvex constrained optimization over time varying digraphs this paper considers nonconvex distributed constrained optimization over networks modeled as directed possibly time varying graphs we introduce the first algorithmic framework for the minimization of the sum of a smooth nonconvex nonseparable function the agent s sum utility plus a difference of convex function with nonsmooth convex part this general formulation arises in many applications from statistical machine learning to engineering the proposed distributed method combines successive convex approximation techniques with a judiciously designed perturbed push sum consensus mechanism that aims to track locally the gradient of the smooth part of the sum utility sublinear convergence rate is proved when a fixed step size possibly different among the agents is employed whereas asymptotic convergence to stationary solutions is proved using a diminishing step size numerical results show that our algorithms compare favorably with current schemes on both convex and nonconvex problems

minimum vertex covers and the spectrum of the normalized laplacian on trees abstract we show that in the graph spectrum of the normalized graph laplacian on trees the eigenvalue 1 and eigenvalues near 1 are strongly related to minimum vertex covers in particular for the eigenvalue 1 its multiplicity is related to the size of a minimum vertex cover and zero entries of its eigenvectors correspond to vertices in minimum vertex covers while for eigenvalues near 1 their distance to 1 can be estimated from minimum vertex covers and for the largest eigenvalue smaller than 1 the sign graphs of its eigenvectors take vertices in a minimum vertex cover as representatives

distributed non convex first order optimization and information processing lower complexity bounds and rate optimal algorithms we consider a class of popular distributed non convex optimization problems in which agents connected by a network mathcal g collectively optimize a sum of smooth possibly non convex local objective functions we address the following question if the agents can only access the gradients of local functions what are the fastest rates that any distributed algorithms can achieve and how to achieve those rates first we show that there exist difficult problem instances such that it takes a class of distributed first order methods at least mathcal o 1 sqrt xi mathcal g times bar l epsilon communication rounds to achieve certain epsilon solution where xi mathcal g denotes the spectral gap of the graph laplacian matrix and bar l is some lipschitz constant second we propose near optimal methods whose rates match the developed lower rate bound up to a ploylog factor the key in the algorithm design is to properly embed the classical polynomial filtering techniques into modern first order algorithms to the best of our knowledge this is the first time that lower rate bounds and optimal methods have been developed for distributed non convex optimization problems

communication efficient algorithms for decentralized and stochastic optimization we present a new class of decentralized first order methods for nonsmooth and stochastic optimization problems defined over multiagent networks considering that communication is a major bottleneck in decentralized optimization our main goal in this paper is to develop algorithmic frameworks which can significantly reduce the number of inter node communications our major contribution is to present a new class of decentralized primal dual type algorithms namely the decentralized communication sliding dcs methods which can skip the inter node communications while agents solve the primal subproblems iteratively through linearizations of their local objective functions by employing dcs agents can find an epsilon solution both in terms of functional optimality gap and feasibility residual in mathcal o 1 epsilon resp mathcal o 1 sqrt epsilon communication rounds for general convex functions resp strongly convex functions while maintaining the mathcal o 1 epsilon 2 resp mathcal o 1 epsilon bound on the total number of intra node subgradient evaluations we also present a stochastic counterpart for these algorithms denoted by sdcs for solving stochastic optimization problems whose objective function cannot be evaluated exactly in comparison with existing results for decentralized nonsmooth and stochastic optimization we can reduce the total number of inter node communication rounds by orders of magnitude while still maintaining the optimal complexity bounds on intra node stochastic subgradient evaluations the bounds on the stochastic subgradient evaluations are actually comparable to those required for centralized nonsmooth and stochastic optimization under certain conditions on the target accuracy

dsa decentralized double stochastic averaging gradient algorithm this paper considers optimization problems where nodes of a network have access to summands of a global objective each of these local objectives is further assumed to be an average of a finite set of functions the motivation for this setup is to solve large scale machine learning problems where elements of the training set are distributed to multiple computational elements the decentralized double stochastic averaging gradient dsa algorithm is proposed as a solution alternative that relies on i the use of local stochastic averaging gradients ii determination of descent steps as differences of consecutive stochastic averaging gradients strong convexity of local functions and lipschitz continuity of local gradients is shown to guarantee linear convergence of the sequence generated by dsa in expectation local iterates are further shown to approach the optimal argument for almost all realizations the expected linear convergence of dsa is in contrast to the sublinear rate characteristic of existing methods for decentralized stochastic optimization numerical experiments on a logistic regression problem illustrate reductions in convergence time and number of feature vectors processed until convergence relative to these other alternatives

perron frobenius type results and discrete versions of nodal domain theorems abstract we prove discrete versions of nodal domain theorems in particular an eigenvector corresponding to the s th smallest eigenvalue of a graph laplacian has at most s nodal domains we compare our results to those of courant and pleijel on nodal domains of continuous laplacians and to those of fiedler on nonnegative regions of graph laplacians

decentralized optimization over time varying directed graphs with row and column stochastic matrices in this article we provide a distributed optimization algorithm termed as tv mathcal ab that minimizes a sum of convex functions over time varying random directed graphs contrary to the existing work the algorithm we propose does not require eigenvector estimation to estimate the non mathbf 1 perron eigenvector of a stochastic matrix instead the proposed approach relies on a novel information mixing approach that exploits both row and column stochastic weights to achieve agreement toward the optimal solution when the underlying graph is directed we show that tv mathcal ab converges linearly to the optimal solution when the global objective is smooth and strongly convex and the underlying time varying graphs exhibit bounded connectivity i e a union of every c consecutive graphs is strongly connected we derive the convergence results based on the stability analysis of a linear system of inequalities along with a matrix perturbation argument simulations confirm the findings in this article

constrained consensus algorithms with fixed step size for distributed convex optimization over multiagent networks in this technical note we are concerned with constrained consensus algorithms for distributed convex optimization with a sum of convex objective functions subject to local bound and equality constraints in multiagent networks each agent has its own data on objective function and constraints all the agents cooperatively find the minimizer while each agent can only communicate with its neighbors the consensus of multiagent networks with time invariant and undirected graphs is proven by the lyapunov method compared with existing consensus algorithms for distributed optimization with diminishing step sizes the proposed algorithms with fixed step size have better convergence rate simulation results on a numerical example are presented to substantiate the performance and characteristics of the proposed algorithms

a proximal gradient algorithm for decentralized composite optimization this paper proposes a decentralized algorithm for solving a consensus optimization problem defined in a static networked multi agent system where the local objective functions have the smooth nonsmooth composite form examples of such problems include decentralized constrained quadratic programming and compressed sensing problems as well as many regularization problems arising in inverse problems signal processing and machine learning which have decentralized applications this paper addresses the need for efficient decentralized algorithms that take advantages of proximal operations for the nonsmooth terms we propose a proximal gradient exact first order algorithm pg extra that utilizes the composite structure and has the best known convergence rate it is a nontrivial extension to the recent algorithm extra at each iteration each agent locally computes a gradient of the smooth part of its objective and a proximal map of the nonsmooth part as well as exchanges information with its neighbors the algorithm is exact in the sense that an exact consensus minimizer can be obtained with a fixed step size whereas most previous methods must use diminishing step sizes when the smooth part has lipschitz gradients pg extra has an ergodic convergence rate of o left 1 over k right in terms of the first order optimality residual when the smooth part vanishes pg extra reduces to p extra an algorithm without the gradients so no g in the name which has a slightly improved convergence rate at o left 1 over k right in a standard non ergodic sense numerical experiments demonstrate effectiveness of pg extra and validate our convergence results

a fast proximal gradient algorithm for decentralized composite optimization over directed networks abstract this paper proposes a fast decentralized algorithm for solving a consensus optimization problem defined in a directed networked multi agent system where the local objective functions have the smooth nonsmooth composite form and are possibly nonconvex examples of such problems include decentralized compressed sensing and constrained quadratic programming problems as well as many decentralized regularization problems we extend the existing algorithms pg extra and extrapush to a new algorithm pg extrapush for composite consensus optimization over a directed network this algorithm takes advantage of the proximity operator like in pg extra to deal with the nonsmooth term and employs the push sum protocol like in extrapush to tackle the bias introduced by the directed network with a proper step size we show that pg extrapush converges to an optimal solution at a linear rate under some regular assumptions we conduct a series of numerical experiments to show the effectiveness of the proposed algorithm specifically with a proper step size pg extrapush performs linear rates in most of cases even in some nonconvex cases and is significantly faster than subgradient push even if the latter uses a hand optimized step size the established theoretical results are also verified by the numerical results

distributed linearized alternating direction method of multipliers for composite convex consensus optimization given an undirected graph mathcal g mathcal n mathcal e of agents mathcal n lbrace 1 ldots n rbrace connected with edges in mathcal e we study how to compute an optimal decision on which there is consensus among agents and that minimizes the sum of agent specific private convex composite functions lbrace phi i rbrace i in mathcal n where phi i triangleq xi i f i belongs to agent i assuming only agents connected by an edge can communicate we propose a distributed proximal gradient algorithm dpga for consensus optimization over both unweighted and weighted static undirected communication networks in one iteration each agent i computes the prox map of xi i and gradient of f i and this is followed by local communication with neighboring agents we also study its stochastic gradient variant sdpga which can only access to noisy estimates of nabla f i at each agent i this computational model abstracts a number of applications in distributed sensing machine learning and statistical inference we show ergodic convergence in both suboptimality error and consensus violation for the dpga and sdpga with rates mathcal o 1 t and mathcal o 1 sqrt t respectively

the preparata and goethals codes trellis complexity and twisted squaring constructions the trellis complexity of the preparata and goethals codes is examined it is shown that at least for a given set of permutations these codes are rectangular upper bounds on the state complexity profiles of the preparata and goethals codes are given the upper bounds on the state complexity of the preparata and goethals codes are determined by the dimension length profiles dlp of the extended primitive double and triple error correcting bch codes respectively a twisted squaring construction for the preparata and goethals codes is given based on the double and triple error correcting extended primitive bch codes respectively

ordered binary decision diagrams and minimal trellises ordered binary decision diagrams obdds are graph based data structures for representing boolean functions they have found widespread use in computer aided design and in formal verification of digital circuits minimal trellises are graphical representations of error correcting codes that play a prominent role in coding theory this paper establishes a close connection between these two graphical models as follows let spl cscr be a binary code of length n and let f sub c x sub 1 x sub n be the boolean function that takes the value 0 at x sub 1 x sub n if and only if x sub 1 x sub n spl isin spl cscr given this natural one to one correspondence between boolean functions and binary codes we prove that the minimal proper trellis for a code spl cscr with minimum distance d 1 is isomorphic to the single terminal obdd for its boolean indicator function f sub c x sub 1 x sub n prior to this result the extensive research during the past decade on binary decision diagrams in computer engineering and on minimal trellises in coding theory has been carried out independently as outlined in this work the realization that binary decision diagrams and minimal trellises are essentially the same data structure opens up a range of promising possibilities for transfer of ideas between these disciplines

minimal trellises for block codes basic concepts in the study of trellises of block codes are defined it is shown that minimal proper trellises exists for all block codes bounds on the size of such trellises are established these bounds are shown to be exact for maximum distance separable codes and nearly so for perfect codes

algebraic connectivity and degree sequences of trees we investigate the structure of trees that have minimal algebraic connectivity among all trees with a given degree sequence we show that such trees are caterpillars and that the vertex degrees are non decreasing on every path on non pendant vertices starting at the characteristic set of the fiedler vector author s abstract

graphs of given order and size and minimum algebraic connectivity the structure of connected graphs of given size and order that have minimal algebraic connectivity is investigated it is shown that they must consist of a chain of cliques moreover an upper bound for the number of maximal cliques of size 2 or larger is derived

data dependent convergence for consensus stochastic optimization we study a distributed consensus based stochastic gradient descent sgd algorithm and show that the rate of convergence involves the spectral properties of two matrices the standard spectral gap of a weight matrix from the network topology and a new term depending on the spectral norm of the sample covariance matrix of the data this data dependent convergence rate shows that distributed sgd algorithms perform better on datasets with small spectral norm our analysis method also allows us to find data dependent convergence rates as we limit the amount of communication spreading a fixed amount of data across more nodes slows convergence for asymptotically growing datasets we show that adding more machines can help when minimizing twice differentiable losses

perturbed proximal primal dual algorithm for nonconvex nonsmooth optimization in this paper we propose a perturbed proximal primal dual algorithm pprox pda for an important class of linearly constrained optimization problems whose objective is the sum of smooth possibly nonconvex and convex possibly nonsmooth functions this family of problems can be used to model many statistical and engineering applications such as high dimensional subspace estimation and the distributed machine learning the proposed method is of the uzawa type in which a primal gradient descent step is performed followed by an approximate dual gradient ascent step one distinctive feature of the proposed algorithm is that the primal and dual steps are both perturbed appropriately using past iterates so that a number of asymptotic convergence and rate of convergence results to first order stationary solutions can be obtained finally we conduct extensive numerical experiments to validate the effectiveness of the proposed algorithm

stochastic proximal gradient consensus over random networks we consider solving a convex optimization problem with possibly stochastic gradient and over a randomly time varying multiagent network each agent has access to some local objective function and it only has unbiased estimates of the gradients of the smooth component we develop a dynamic stochastic proximal gradient consensus algorithm with the following key features 1 it works for both the static and certain randomly time varying networks 2 it allows the agents to utilize either the exact or stochastic gradient information 3 it is convergent with provable rate in particular the proposed algorithm converges to a global optimal solution with a rate of mathcal o 1 r resp mathcal o 1 sqrt r when the exact resp stochastic gradient is available where r is the iteration counter interestingly the developed algorithm establishes a close connection among a number of seemingly unrelated distributed algorithms such as the extra the pg extra the ic idc admm the dlm and the classical distributed subgradient method

the z sub 4 linearity of kerdock preparata goethals and related codes certain notorious nonlinear binary codes contain more codewords than any known linear code these include the codes constructed by nordstrom robinson 1967 kerdock 1972 preparata 1968 goethals 1974 and delsarte goethals 1975 it is shown here that all these codes can be very simply constructed as binary images under the gray map of linear codes over z sub 4 the integers mod 4 although this requires a slight modification of the preparata and goethals codes the construction implies that all these binary codes are distance invariant duality in the z sub 4 domain implies that the binary images have dual weight distributions the kerdock and preparata codes are duals over z sub 4 and the nordstrom robinson code is self dual which explains why their weight distributions are dual to each other the kerdock and preparata codes are z sub 4 analogues of first order reed muller and extended hamming codes respectively all these codes are extended cyclic codes over z sub 4 which greatly simplifies encoding and decoding an algebraic hard decision decoding algorithm is given for the preparata code and a hadamard transform soft decision decoding algorithm for the i kerdock code binary first and second order reed muller codes are also linear over z sub 4 but extended hamming codes of length n spl ges 32 and the golay code are not using z sub 4 linearity a new family of distance regular graphs are constructed on the cosets of the preparata code

distributed stochastic gradient tracking methods in this paper we study the problem of distributed multi agent optimization over a network where each agent possesses a local cost function that is smooth and strongly convex the global objective is to find a common solution that minimizes the average of all cost functions assuming agents only have access to unbiased estimates of the gradients of their local cost functions we consider a distributed stochastic gradient tracking method dsgt and a gossip like stochastic gradient tracking method gsgt we show that in expectation the iterates generated by each agent are attracted to a neighborhood of the optimal solution where they accumulate exponentially fast under a constant stepsize choice under dsgt the limiting expected error bounds on the distance of the iterates from the optimal solution decrease with the network size n which is a comparable performance to a centralized stochastic gradient algorithm moreover we show that when the network is well connected gsgt incurs lower communication cost than dsgt while maintaining a similar computational cost numerical example further demonstrates the effectiveness of the proposed methods

coset codes ii binary lattices and related codes for pt i see ibid vol 34 no 5 p 1123 51 1988 the family of barnes wall lattices including d sub 4 and e sub 8 of lengths n 2 sup n and their principal sublattices which are useful in constructing coset codes are generated by iteration of a simple construction called the squaring construction the closely related reed muller codes are generated by the same construction the principal properties of these codes and lattices are consequences of the general properties of iterated squaring constructions which also exhibit the interrelationships between codes and lattices of different lengths an extension called the cubing construction generates good codes and lattices of lengths n 3 2 sup n including the golay code and leech lattice with the use of special bases for 8 space another related construction generates the nordstrom robinson code and an analogous 16 dimensional nonlattice packing these constructions are represented by trellis diagrams that display their structure and interrelationships and that lead to efficient maximum likelihood decoding algorithms

on the complexity of steepest descent newton s and regularized newton s methods for nonconvex unconstrained optimization problems it is shown that the steepest descent and newton s methods for unconstrained nonconvex optimization under standard assumptions may both require a number of iterations and function evaluations arbitrarily close to o epsilon 2 to drive the norm of the gradient below epsilon this shows that the upper bound of o epsilon 2 evaluations known for the steepest descent is tight and that newton s method may be as slow as the steepest descent method in the worst case the improved evaluation complexity bound of o epsilon 3 2 evaluations known for cubically regularized newton s methods is also shown to be tight

pure adaptive search in monte carlo optimization pure adaptive search constructs a sequence of points uniformly distributed within a corresponding sequence of nested regions of the feasible space at any stage the next point in the sequence is chosen uniformly distributed over the region of feasible space containing all points that are equal or superior in value to the previous points in the sequence we show that for convex programs the number of iterations required to achieve a given accuracy of solution increases at most linearly in the dimension of the problem this compares to exponential growth in iterations required for pure random search

on the bcjr trellis for linear block codes in this semi tutorial paper we will investigate the computational complexity of an abstract version of the viterbi algorithm on a trellis and show that if the trellis has e edges the complexity of the viterbi algorithm is spl theta e this result suggests that the best trellis representation for a given linear block code is the one with the fewest edges we will then show that among all trellises that represent a given code the original trellis introduced by bahl cocke jelinek and raviv in 1974 and later rediscovered by wolf 1978 massey 1978 and forney 1988 uniquely minimizes the edge count as well as several other figures of merit following forney and kschischang and sorokine 1995 we will also discuss trellis oriented or minimal span generator matrices which facilitate the calculation of the size of the bcjr trellis as well as the actual construction of it

dextra a fast algorithm for optimization over directed graphs this paper develops a fast distributed algorithm termed dextra to solve the optimization problem when n agents reach agreement and collaboratively minimize the sum of their local objective functions over the network where the communication between the agents is described by a directed graph existing algorithms solve the problem restricted to directed graphs with convergence rates of o ln k sqrt k for general convex objective functions and o ln k k when the objective functions are strongly convex where k is the number of iterations we show that with the appropriate step size dextra converges at a linear rate o tau k for 0 given that the objective functions are restricted strongly convex the implementation of dextra requires each agent to know its local out degree simulation examples further illustrate our findings

graph homomorphisms and nodal domains in this paper we derive some necessary spectral conditions for the existence of graph homomorphisms in which we also consider some parameters related to the corresponding eigenspaces such as nodal domains in this approach we consider the combinatorial laplacian and co laplacian as well as the adjacency matrix also we present some applications in graph decompositions where we prove a general version of fisher s inequality for g designs

convergence analysis of alternating direction method of multipliers for a family of nonconvex problems the alternating direction method of multipliers admm is widely used to solve large scale linearly constrained optimization problems convex or nonconvex in many engineering fields however there is a general lack of theoretical understanding of the algorithm when the objective function is nonconvex in this paper we analyze the convergence of the admm for solving certain nonconvex consensus and sharing problems we show that the classical admm converges to the set of stationary solutions provided that the penalty parameter in the augmented lagrangian is chosen to be sufficiently large for the sharing problems we show that the admm is convergent regardless of the number of variable blocks our analysis does not impose any assumptions on the iterates generated by the algorithm and is broadly applicable to many admm variants involving proximal update rules and various flexible block selection rules

efficient monte carlo procedures for generating points uniformly distributed over bounded regions we consider the monte carlo problem of generating points uniformly distributed within an arbitrary bounded measurable region the class of markovian methods considered generate points asymptotically uniformly distributed within the region computational experience suggests the methods are potentially superior to conventional rejection techniques for large dimensional regions

linear convergence rate of a class of distributed augmented lagrangian algorithms we study distributed optimization where nodes cooperatively minimize the sum of their individual locally known convex costs f i x s x in bbr d is global distributed augmented lagrangian al methods have good empirical performance on several signal processing and learning applications but there is limited understanding of their convergence rates and how it depends on the underlying network this paper establishes globally linear geometric convergence rates of a class of deterministic and randomized distributed al methods when the f i s are twice continuously differentiable and have a bounded hessian we give explicit dependence of the convergence rates on the underlying network parameters simulations illustrate our analytical findings

the dynamics of group codes state spaces trellis diagrams and canonical encoders a group code c over a group g is a set of sequences of group elements that itself forms a group under a component wise group operation a group code has a well defined state space sigma sub k at each time k each code sequence passes through a well defined state sequence the set of all state sequences is also a group code the state code of c the state code defines an essentially unique minimal realization of c the trellis diagram of c is defined by the state code of c and by labels associated with each state transition the set of all label sequences forms a group code the label code of c which is isomorphic to the state code of c if c is complete and strongly controllable then a minimal encoder in controller canonical feedbackfree form may be constructed from certain sets of shortest possible code sequences called granules the size of the state space sigma sub k is equal to the size of the state space of this canonical encoder which is given by a decomposition of the input groups of c at each time k if c is time invariant and nu controllable then mod sigma sub k mod pi sub 1

interlacing property of zeros of eigenvectors of schrodinger operators on trees we prove an analogue for trees of sturm s oscillation theorem on the interlacing property of zeros of eigenfunctions of a schrodinger operator let be a finite tree and a combinatorial schrodinger operator on if the eigenvectors of are ordered according to increasing eigenvalues and the vertices corresponding to zero coordinates are of degree at most two then the zeros of the linear extensions of eigenvectors have the interlacing property

convergence rate of distributed admm over networks we propose a new distributed algorithm based on alternating direction method of multipliers admm to minimize sum of locally known convex functions using communication over a network this optimization problem emerges in many applications in distributed machine learning and statistical estimation our algorithm allows for a general choice of the communication weight matrix which is used to combine the iterates at different nodes we show that when functions are convex both the objective function values and the feasibility violation converge with rate o 1 t where t is the number of iterations we then show that when functions are strongly convex and have lipschitz continuous gradients the sequence generated by our algorithm converges linearly to the optimal solution in particular an epsilon optimal solution can be computed with o left sqrt kappa f log 1 epsilon right iterations where kappa f is the condition number of the problem our analysis highlights the effect of network and communication weights on the convergence rate through degrees of the nodes the smallest nonzero eigenvalue and operator norm of the communication matrix

distributed least squares solver for network linear equations in this paper we study the problem of finding the least square solutions of over determined linear algebraic equations over networks in a distributed manner each node has access to one of the lin

proof of a conjecture of mceliece regarding the expansion index of the minimal trellis we prove a conjecture of mceliece establishing that for each fixed order of positions of a linear code c the minimal trellis minimizes the quantity e v 1 where e and v stand for the number of edges and vertices in the trellis respectively as a consequence it follows that the minimal trellis uniquely minimizes the total number of operations required for viterbi decoding of a given code moreover we show that these results extend to the general class of rectangular codes namely the class of codes that admit a biproper trellis presentation which includes all group codes and many other useful nonlinear codes

technical note the asymptotic extreme value distribution of the sample minimum of a concave function under linear constraints we show that the minimum value of a sample of feasible points uniformly distributed over a linear constraint set is for concave functions asymptotically weibull distributed with shape parameter equal to the dimension of the feasible region

graph partitioning by eigenvectors let a be the adjacency matrix of a connected graph 9 if z is a column vector we say that a vertex of is positive nonnegative null etc if the corresponding entry of z has that property for z such that az olz we bound the number of components in the subgraph induced by positive vertices for eigenvectors z having a null element we bound the number of components in the graph induced by nonnull vertices finally bounds are established for the number of null elements in an eigenvector for the multiplicity of an eigenvalue and for the magnitudes of the second and last eigenvalues of a general or a bipartite graph

coding with side information for rate constrained consensus average consensus algorithms are protocols to compute the average value of all sensor measurements via near neighbors communications they offer a natural tradeoff between the number of messages exchanged among terminals and the accuracy in the computation most of the models adopted for the message exchange in the literature however neither include explicit rate constraints nor explore the rate distortion tradeoff associated with the algorithm the contribution of our work is in examining the impact of such constraints and in finding strategies to minimize the communication cost in terms of rate the main motivation behind the proposed coding strategies is the observation that consensus algorithms offer the perfect example of a network communication problem where there is an increasing correlation between the data exchanged as the algorithm iterates henceforth it is possible to utilize previously exchanged data and current side information to significantly reduce the demands of quantization bit rate for a certain precision we analyze the case of a network where the links are assumed to be reliable at a constant bit rate we explore the conditions on the quantization noise which lead to a consensus value whose mean squared distance from the initial average is bounded in the case of infinite length vector coding with gaussian states we show that our proposed schemes achieve bounded convergence with vanishing rates as the iteration index tends to infinity

convergence rate analysis of distributed optimization with projected subgradient algorithm abstract in this paper we revisit the consensus based projected subgradient algorithm proposed for a common set constraint we show that the commonly adopted non summable and square summable diminishing step sizes of subgradients can be relaxed to be only non summable if the constrained optimum set is bounded more importantly for a strongly convex aggregate cost with different types of step sizes we provide a systematical analysis to derive the asymptotic upper bound of convergence rates in terms of the optimum residual and select the best step sizes accordingly our result shows that a convergence rate of o 1 k can be achieved with a step size o 1 k

extra an exact first order algorithm for decentralized consensus optimization recently there has been growing interest in solving consensus optimization problems in a multiagent network in this paper we develop a decentralized algorithm for the consensus optimization problem mathrm minimize x in mathbb r p bar f x frac 1 n sum i 1 n f i x which is defined over a connected network of n agents where each function f i is held privately by agent i and encodes the agent s data and objective all the agents shall collaboratively find the minimizer while each agent can only communicate with its neighbors such a computation scheme avoids a data fusion center or long distance communication and offers better load balance to the network this paper proposes a novel decentralized exact first order algorithm abbreviated as extra to solve the consensus optimization problem exact means that it can converge to the exact solution extra uses a fixed large step size which can be determined independently of the network size or topology the local variable of every a

fast distributed gradient methods we study distributed optimization problems when n nodes minimize the sum of their individual costs subject to a common vector variable the costs are convex have lipschitz continuous gradient with constant l and bounded gradient we propose two fast distributed gradient algorithms based on the centralized nesterov gradient algorithm and establish their convergence rates in terms of the per node communications k and the per node gradient evaluations k our first method distributed nesterov gradient achieves rates o logk k and o logk k our second method distributed nesterov gradient with consensus iterations assumes at all nodes knowledge of l and w the second largest singular value of the n n doubly stochastic weight matrix w it achieves rates o 1 k2 and o 1 k2 0 arbitrarily small further we give for both methods explicit dependence of the convergence constants on n and w simulation examples illustrate our findings

distributed heavy ball a generalization and acceleration of first order methods with gradient tracking we study distributed optimization to minimize a sum of smooth and strongly convex functions recent work on this problem uses gradient tracking to achieve linear convergence to the exact global minimizer however a connection among different approaches has been unclear in this paper we first show that many of the existing first order algorithms are related with a simple state transformation at the heart of which lies a recently introduced algorithm known as mathcal ab we then present distributed heavy ball denoted as mathcal ab m that combines mathcal ab with a momentum term and uses nonidentical local step sizes by simultaneously implementing both row and column stochastic weights mathcal ab m removes the conservatism in the related work due to doubly stochastic weights or eigenvector estimation mathcal ab m thus naturally leads to optimization and average consensus over both undirected and directed graphs we show that mathcal ab m has a global r linear rate when the largest step size and momentum parameter are positive and sufficiently small we numerically show that mathcal ab m achieves acceleration particularly when the objective functions are ill conditioned

exponential convergence of distributed primal dual convex optimization algorithm without strong convexity abstract this paper establishes exponential convergence rates for a class of primal dual gradient algorithms in distributed optimization without strong convexity the convergence analysis is based on a carefully constructed lyapunov function by evaluating metric subregularity of the primal dual gradient map we present a general criterion under which the algorithm achieves exponential convergence to facilitate practical applications of this criterion several simplified sufficient conditions are derived we also prove that although these results are developed for the continuous time algorithms they carry over in a parallel manner to the discrete time algorithms constructed by using euler s approximation method

the proximal augmented lagrangian method for nonsmooth composite optimization we study a class of optimization problems in which the objective function is given by the sum of a differentiable but possibly nonconvex component and a nondifferentiable convex regularization term we introduce an auxiliary variable to separate the objective function components and utilize the moreau envelope of the regularization term to derive the proximal augmented lagrangian a continuously differentiable function obtained by constraining the augmented lagrangian to the manifold that corresponds to the explicit minimization over the variable in the nonsmooth term the continuous differentiability of this function with respect to both primal and dual variables allows us to leverage the method of multipliers mm to compute optimal primal dual pairs by solving a sequence of differentiable problems the mm algorithm is applicable to a broader class of problems than proximal gradient methods and it has stronger convergence guarantees and a more refined step size update rules than the alternating direction method of multipliers admm these features make it an attractive option for solving structured optimal control problems we also develop an algorithm based on the primal descent dual ascent gradient method and prove global exponential asymptotic stability when the differentiable component of the objective function is strongly convex and the regularization term is convex finally we identify classes of problems for which the primal dual gradient flow dynamics are convenient for distributed implementation and compare contrast our framework to the existing approaches

non convex distributed optimization we study distributed non convex optimization on a time varying multi agent network each node has access to its own smooth local cost function and the collective goal is to minimize the sum of these functions the perturbed push sum algorithm was previously used for convex distributed optimization we generalize the result obtained for the convex case to the case of non convex functions under some additional technical assumptions on the gradients we prove the convergence of the distributed push sum algorithm to some critical point of the objective function by utilizing perturbations on the update process we show the almost sure convergence of the perturbed dynamics to a local minimum of the global objective function if the objective function has no saddle points our analysis shows that this perturbed procedure converges at a rate of o 1 t

estimation of the minimum of a function using order statistics abstract suppose a is a bounded set in d and f a real function defined on a suppose m min f x x a exists using a random sample x 1 x 2 x n from a uniform distribution over a we construct a confidence interval for m using asymptotic theory our results contain some statistical results valid in general extreme value theory estimation of the main parameter of extreme value distributions

convergence of asynchronous distributed gradient methods over stochastic networks we consider distributed optimization problems in which a number of agents are to seek the global optimum of a sum of cost functions through only local information sharing in this paper we are particularly interested in scenarios where agents are operating asynchronously over stochastic networks subject to random failures most existing algorithms require coordinated and decaying stepsizes to ensure zero gap between the estimated value of each agent and the exact optimum restricting it from asynchronous implementation and resulting in slower convergence results to deal with this issue we develop a new asynchronous distributed gradient method asyndgm based on consensus theory the proposed algorithm not only allows for asynchronous implementation in a completely distributed manner but also most importantly is able to seek the exact optimum even with constant stepsizes we will show that the assumption of boundedness of gradients which is widely used in the literature can be dropped by instead imposing the standard lipschitz continuity condition on gradients moreover we derive an upper bound of stepsize within which the proposed asyndgm can achieve a linear convergence rate for strongly convex functions with lipschitz gradients a canonical example of sensor fusion problems is provided to illustrate the effectiveness of the proposed algorithm

on the sign patterns of the smallest signless laplacian eigenvector abstract let h be a connected bipartite graph whose signless laplacian matrix is q h suppose that the bipartition of h is s t and that x is the eigenvector of the smallest eigenvalue of q h it is well known that x is positive and constant on s and negative and constant on t the resilience of the sign pattern of x under addition of edges into the subgraph induced by either s or t is investigated and a number of cases in which the sign pattern of x persists are described