a two ellipsoid overlap test for on line failure detection the problem of failure detection in dynamic systems is addressed the approach is based on modelling and estimation methods and uses some known statistical tools as well as geometric arguments a test decision is derived which is based on the overlap between the confidence regions associated with two estimates one obtained using online measurements and the other based on a priori information the algorithm proposed here is suitable for on line detection of abrupt changes and failures in the applications where fast on line decisions are of importance the implementation of a necessary and sufficient condition is shown to result in a scalar test which does not call for any optimization procedure and so offers the advantage of low computational expenditure

stochastic primal dual hybrid gradient algorithm with arbitrary sampling and imaging applications we propose a stochastic extension of the primal dual hybrid gradient algorithm studied by chambolle and pock in 2011 to solve saddle point problems that are separable in the dual variable the anal

on the ergodic convergence rates of a first order primal dual algorithm we revisit the proofs of convergence for a first order primal dual algorithm for convex optimization which we have studied a few years ago in particular we prove rates of convergence for a more general version with simpler proofs and more complete results the new results can deal with explicit terms and nonlinear proximity operators in spaces with quite general norms

communication efficient algorithms for decentralized and stochastic optimization we present a new class of decentralized first order methods for nonsmooth and stochastic optimization problems defined over multiagent networks considering that communication is a major bottleneck in decentralized optimization our main goal in this paper is to develop algorithmic frameworks which can significantly reduce the number of inter node communications our major contribution is to present a new class of decentralized primal dual type algorithms namely the decentralized communication sliding dcs methods which can skip the inter node communications while agents solve the primal subproblems iteratively through linearizations of their local objective functions by employing dcs agents can find an epsilon solution both in terms of functional optimality gap and feasibility residual in mathcal o 1 epsilon resp mathcal o 1 sqrt epsilon communication rounds for general convex functions resp strongly convex functions while maintaining the mathcal o 1 epsilon 2 resp mathcal o 1 epsilon bound on the total number of intra node subgradient evaluations we also present a stochastic counterpart for these algorithms denoted by sdcs for solving stochastic optimization problems whose objective function cannot be evaluated exactly in comparison with existing results for decentralized nonsmooth and stochastic optimization we can reduce the total number of inter node communication rounds by orders of magnitude while still maintaining the optimal complexity bounds on intra node stochastic subgradient evaluations the bounds on the stochastic subgradient evaluations are actually comparable to those required for centralized nonsmooth and stochastic optimization under certain conditions on the target accuracy

nonlinear proximal point algorithms using bregman functions with applications to convex programming a bregman function is a strictly convex differentiable function that induces a well behaved distance measure or d function on euclidean space this paper shows that for every bregman function there exists a nonlinear version of the proximal point algorithm and presents an accompanying convergence theory applying this generalization of the proximal point algorithm to convex programming one obtains the d function proximal minimization algorithm of censor and zenios and a wide variety of new multiplier methods these multiplier methods are different from those studied by kort and bertsekas and include nonquadratic variations on the proximal method of multipliers

single image blind deblurring based on the fractional order differential abstract image deblurring is a fundamental problem in image processing when the blur type is unknown the problem becomes more challenging which is called blind deblurring in this paper we propose a novel variational method for single image blind deblurring based on the fractional order differential which can overcome the staircase effect produced by the total variation regularization and alleviate the ringing artifact in deblurring the fractional order gradient fidelity term is added in the cost functional to improve the restoration moreover we make use of the edge detect function with the fractional order gradient to preserve sharp edges and the bregman iteration to reconstruct more structures the primal dual algorithm is developed to solve the proposed model numerical experiments show that our method is able to get the sharp image without the staircase effect and correctly estimate the unknown blur kernel

on the convergence rate improvement of a primal dual splitting algorithm for solving monotone inclusion problems we present two modified versions of the primal dual splitting algorithm relying on forward backward splitting proposed in v tilde mathrm u u adv comput math 38 3 667 681 2013 for solving monotone inclusion problems under strong monotonicity assumptions for some of the operators involved we obtain for the sequences of iterates that approach the solution orders of convergence of mathcal o frac 1 n o 1 n and mathcal o omega n o n for omega in 0 1 0 1 respectively the investigated primal dual algorithms are fully decomposable in the sense that the operators are processed individually at each iteration we also discuss the modified algorithms in the context of convex optimization problems and present numerical experiments in image processing and pattern recognition in cluster analysis

on the optimality of the likelihood ratio test for local sensor decision rules in the presence of nonideal channels distributed detection has been intensively studied in the past in this correspondence we consider the design of local decision rules in the presence of nonideal transmission channels between the sensors and the fusion center under the conditional independence assumption among multiple sensor observations we show that the optimal local decisions that minimize the error probability at the fusion center amount to a likelihood ratio test lrt given a particular constraint on the fusion rule this constraint turns out to be quite general and is easily satisfied for most sensible fusion rules a design example using a parallel sensor fusion structure with binary symmetric channels bscs between local sensors and the fusion center is given to illustrate the usefulness of the result in obtaining optimal thresholds for local sensor observations the study that incorporates the transmission channel in the sensor system design may have potential applications in the emerging field of wireless sensor networks

prox method with rate of convergence o 1 t for variational inequalities with lipschitz continuous monotone operators and smooth convex concave saddle point problems we propose a prox type method with efficiency estimate o epsilon 1 for approximating saddle points of convex concave c 1 1 functions and solutions of variational inequalities with monotone lipschitz continuous operators application examples include matrix games eigenvalue minimization and computing the lovasz capacity number of a graph and these are illustrated by numerical experiments with large scale matrix games and lovasz capacity problems

distributed linearized alternating direction method of multipliers for composite convex consensus optimization given an undirected graph mathcal g mathcal n mathcal e of agents mathcal n lbrace 1 ldots n rbrace connected with edges in mathcal e we study how to compute an optimal decision on which there is consensus among agents and that minimizes the sum of agent specific private convex composite functions lbrace phi i rbrace i in mathcal n where phi i triangleq xi i f i belongs to agent i assuming only agents connected by an edge can communicate we propose a distributed proximal gradient algorithm dpga for consensus optimization over both unweighted and weighted static undirected communication networks in one iteration each agent i computes the prox map of xi i and gradient of f i and this is followed by local communication with neighboring agents we also study its stochastic gradient variant sdpga which can only access to noisy estimates of nabla f i at each agent i this computational model abstracts a number of applications in distributed sensing machine learning and statistical inference we show ergodic convergence in both suboptimality error and consensus violation for the dpga and sdpga with rates mathcal o 1 t and mathcal o 1 sqrt t respectively

on optimal quantization rules for some problems in sequential decentralized detection we consider the design of systems for sequential decentralized detection a problem that entails several interdependent choices the choice of a stopping rule specifying the sample size a global decision function a choice between two competing hypotheses and a set of quantization rules the local decisions on the basis of which the global decision is made this correspondence addresses an open problem of whether in the bayesian formulation of sequential decentralized detection optimal local decision functions can be found within the class of stationary rules we develop an asymptotic approximation to the optimal cost of stationary quantization rules and exploit this approximation to show that stationary quantizers are not optimal in a broad class of settings we also consider the class of blockwise stationary quantizers and show that asymptotically optimal quantizers are likelihood based threshold rules

inertial proximal strictly contractive peaceman rachford splitting method with an indefinite term for convex optimization abstract we consider an inertial proximal strictly contractive peaceman rachford splitting method abbreviated as ipscprsm with an indefinite proximal term for solving convex optimization problems with linear constraints with the aid of variational inequality proximal point method and fundamental inequality we prove global convergence of the proposed method and analyze iteration complexity in the best function value and feasibility residues the experimental results demonstrate the efficiency of the inertial extrapolation step and the indefinite proximal term even compared with the state of the art methods

convergence analysis of a proximal like minimization algorithm using bregman functions an alternative convergence proof of a proximal like minimization algorithm using bregman functions recently proposed by censor and zenios is presented the analysis allows the establishment of a global convergence rate of the algorithm expressed in terms of function values

optimal primal dual methods for a class of saddle point problems we present a novel accelerated primal dual apd method for solving a class of deterministic and stochastic saddle point problems spps the basic idea of this algorithm is to incorporate a multistep acceleration scheme into the primal dual method without smoothing the objective function for deterministic spp the apd method achieves the same optimal rate of convergence as nesterov s smoothing technique our stochastic apd method exhibits an optimal rate of convergence for stochastic spp not only in terms of its dependence on the number of the iteration but also on a variety of problem parameters to the best of our knowledge this is the first time that such an optimal algorithm has been developed for stochastic spp in the literature furthermore for both deterministic and stochastic spp the developed apd algorithms can deal with the situation when the feasible region is unbounded as long as a saddle point exists in the unbounded case we incorporate the modified termination criterion introduced b

on the o 1 n convergence rate of the douglas rachford alternating direction method alternating direction methods adms have been well studied in the literature and they have found many efficient applications in various fields in this note we focus on the douglas rachford adm scheme proposed by glowinski and marrocco and we aim at providing a simple approach to estimating its convergence rate in terms of the iteration number the linearized version of this adm scheme which is known as the split inexact uzawa method in the image processing literature is also discussed

total roto translational variation we consider curvature depending variational models for image regularization such as euler s elastica these models are known to provide strong priors for the continuity of edges and hence have important applications in shape and image processing we consider a lifted convex representation of these models in the roto translation space in this space curvature depending variational energies are represented by means of a convex functional defined on divergence free vector fields the line energies are then easily extended to any scalar function it yields a natural generalization of the total variation to the roto translation space as our main result we show that the proposed convex representation is tight for characteristic functions of smooth shapes we also discuss cases where this representation fails for numerical solution we propose a staggered grid discretization based on an averaged raviart thomas finite elements approximation this discretization is consistent up to minor details with the underlying continuous model the resulting non smooth convex optimization problem is solved using a first order primal dual algorithm we illustrate the results of our numerical algorithm on various problems from shape and image processing

a duality theory for non convex problems in the calculus of variations we present a new duality theory for non convex variational problems under possibly mixed dirichlet and neumann boundary conditions the dual problem reads nicely as a linear programming problem and our main result states that there is no duality gap further we provide necessary and sufficient optimality conditions and we show that our duality principle can be reformulated as a min max result which is quite useful for numerical implementations as an example we illustrate the application of our method to a celebrated free boundary problem the results were announced in bouchitte and fragala c r math acad sci paris 353 4 375 379 2015

rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization this paper presents two classes of decomposition algorithms based on the proximal method of multipliers pmm introduced in the mid 1970s by rockafellar for convex minimization we first show that

robust multicategory support matrix machines we consider the classification problem when the input features are represented as matrices rather than vectors to preserve the intrinsic structures for classification a successful method is the support matrix machine smm in 19 which optimizes an objective function with a hinge loss plus a so called spectral elastic net penalty however the issues of extending smm to multicategory classification still remain moreover in practice it is common to see the training data contaminated by outlying observations which can affect the robustness of existing matrix classification methods in this paper we address these issues by introducing a robust angle based classifier which boils down binary and multicategory problems to a unified framework benefitting from the use of truncated hinge loss functions the proposed classifier achieves certain robustness to outliers the underlying optimization model becomes nonconvex but admits a natural dc difference of two convex functions representation we develop a new and efficient algorithm by incorporating the dc algorithm and primal dual first order methods together the proposed dc algorithm adaptively chooses the accuracy of the subproblem at each iteration while guaranteeing the overall convergence of the algorithm the use of primal dual methods removes a natural complexity of the linear operator in the subproblems and enables us to use the proximal operator of the objective functions and matrix vector operations this advantage allows us to solve large scale problems efficiently theoretical and numerical results indicate that for problems with potential outliers our method can be highly competitive among existing methods

on the equivalence of the primal dual hybrid gradient method and douglas rachford splitting the primal dual hybrid gradient pdhg algorithm proposed by esser zhang and chan and by pock cremers bischof and chambolle is known to include as a special case the douglas rachford splitting algorithm for minimizing the sum of two convex functions we show that conversely the pdhg algorithm can be viewed as a special case of the douglas rachford splitting algorithm

a parallel primal dual splitting method for image restoration we develop a parallel primal dual splitting method to solve large scale image restoration problems which involve the sum of several linear operator coupled nonsmooth but proximable terms with the proposed method the objective function is decomposed into pieces that can be processed individually no inverse operator is involved in our method and the highly parallel structure makes it preferable in distributed computation the convergence is proven and the convergence rate is analyzed besides its equivalence to the relaxed parallel linearized alternating direction method of multipliers pladmm is addressed applications to image restoration problems with compound l1 regularizer and comparisons with state of the art methods are detailed to show the superiority of the proposed method

extremal properties of likelihood ratio quantizers m hypotheses and a random variable y with a different probability distribution under each hypothesis are considered a quantizer is applied to form a quantized random variable gamma y the extreme points of the set of possible probability distributions of gamma y as gamma ranges over all quantizers is characterized optimality properties of likelihood ratio quantizers are established for a very broad class of quantization problems including problems involving the maximization of an ali silvey 1966 distance measure and the neyman pearson variant of the decentralized detection problem

a proximal based decomposition method for convex minimization problems this paper presents a decomposition method for solving convex minimization problems at each iteration the algorithm computes two proximal steps in the dual variables and one proximal step in the primal variables we derive this algorithm from rockafellar s proximal method of multipliers which involves an augmented lagrangian with an additional quadratic proximal term the algorithm preserves the good features of the proximal method of multipliers with the additional advantage that it leads to a decoupling of the constraints and is thus suitable for parallel implementation we allow for computing approximately the proximal minimization steps and we prove that under mild assumptions on the problem s data the method is globally convergent and at a linear rate the method is compared with alternating direction type methods and applied to the particular case of minimizing a convex function over a finite intersection of closed convex sets

entropic proximal mappings with applications to nonlinear programming we introduce a family of new transforms based on imitating the proximal mapping of moreau and the associated moreau yosida proximal approximation of a function the transforms are constructed in terms of the a divergence functional a generalization of the relative entropy and of bregman s measure of distance an analogue of moreau s theorem associated with these entropy like distances is proved we show that the resulting entropic proximal maps share properties similar to the proximal mapping and provide a fairly general framework for constructing approximation and smoothing schemes for optimization problems applications of the results to the construction of generalized augmented lagrangians for nonlinear programs and the minimax problem are presented