metric subregularity and the proximal point method we examine the linear convergence rates of variants of the proximal point method for finding zeros of maximal monotone operators we begin by showing how metric subregularity is sufficient for local linear convergence to a zero of a maximal monotone operator this result is then generalized to obtain convergence rates for the problem of finding a common zero of multiple monotone operators by considering randomized and averaged proximal methods

minimizing total tardiness on one machine is np hard the problem of minimizing the total tardiness for a set of independent jobs on one machine is considered lawler has given a pseudo polynomial time algorithm to solve this problem in spite of extensive research efforts for more than a decade the question of whether it can be solved in polynomial time or it is np hard in the ordinary sense remained open in this paper the problem is shown to be np hard in the ordinary sense

linear and strong convergence of algorithms involving averaged nonexpansive operators abstract we introduce regularity notions for averaged nonexpansive operators combined with regularity notions of their fixed point sets we obtain linear and strong convergence results for quasicyclic cyclic and random iterations new convergence results on the borwein tam method btm and on the cyclically anchored douglas rachford algorithm cadra are also presented finally we provide a numerical comparison of btm cadra and the classical method of cyclic projections for solving convex feasibility problems

adaptation and parameter estimation in systems with unstable target dynamics and nonlinear parametrization in this paper we propose a solution to the problem of adaptive control and parameter estimation in systems with unstable target dynamics models of uncertainties are allowed to be nonlinearly parameterized and required to be smooth and monotonic functions of linear functionals of the parameters the mere assumption of existence of nonlinear operator gains for the target dynamics is sufficient to guarantee that system solutions are bounded reach a neighborhood of the target set and the mismatches between the modeled uncertainties and their compensator converge to zero with respect to parameter convergence a standard persistent excitation condition suffices to ensure that it is exponential when a weaker nonlinear version of persistent excitation is satisfied asymptotic convergence is guaranteed the spectrum of possible applications ranges from tyre road slip control to asynchronous message transmission in spiking neural oscillators

g dominance reference point based dominance for multiobjective metaheuristics abstract one of the main tools for including decision maker dm preferences in the multiobjective optimization mo literature is the use of reference points and achievement scalarizing functions a p wierzbicki the use of reference objectives in multiobjective optimization in g fandel t gal eds multiple criteria decision making theory and application springer verlag new york 1980 pp 469 486 the core idea in these approaches is converting the original mo problem into a single objective optimization problem through the use of a scalarizing function based on a reference point as a result a single efficient point adapted to the dm s preferences is obtained however a single solution can be less interesting than an approximation of the efficient set around this area as stated for example by deb in k deb j sundar n udaya bhaskara rao s chaudhuri reference point based multiobjective optimization using evolutionary algorithms international journal of computational intelligence research 2 3 2006 273 286 in this paper we propose a variation of the concept of pareto dominance called g dominance which is based on the information included in a reference point and designed to be used with any mo evolutionary method or any mo metaheuristic this concept will let us approximate the efficient set around the area of the most preferred point without using any scalarizing function on the other hand we will show how it can be easily used with any mo evolutionary method or any mo metaheuristic just changing the dominance concept and to exemplify its use we will show some results with some state of the art methods and some test problems

on total variation minimization and surface evolution using parametric maximum flows in a recent paper boykov et al lncs vol 3953 pp 409 422 2006 propose an approach for computing curve and surface evolution using a variational approach and the geo cuts method of boykov and kolmogorov international conference on computer vision pp 26 33 2003 we recall in this paper how this is related to well known approaches for mean curvature motion introduced by almgren et al siam journal on control and optimization 31 2 387 438 1993 and luckhaus and sturzenhecker calculus of variations and partial differential equations 3 2 253 271 1995 and show how the corresponding problems can be solved with sub pixel accuracy using parametric maximum flow techniques this provides interesting algorithms for computing crystalline curvature motion possibly with a forcing term

minimizing total tardiness in permutation flowshops we consider the permutation flowshop scheduling problem with the objective of minimizing total tardiness presented are several properties that are used to calculate lower bounds on total tardiness of jobs for a given partial sequence and to identify sequences dominated by others we develop a branch and bound algorithm using these bounds and a dominance rule results of computational experiments on randomly generated problems are reported

adaptive control of linear multivariable systems using dynamic regressor extension and mixing estimators removing the high frequency gain assumptions abstract in this paper it is shown that using the recently introduced dynamic regressor extension and mixing parameter estimation technique it is possible to remove the key assumption of prior knowledge on the high frequency gain imposed in model reference adaptive control of linear time invariant multivariable systems for the case of scalar systems this is tantamount to removing the requirement of known sign of this coefficient the proposed controller is not certainty equivalent in the sense that the parameters generated by the new estimator are not directly applied to the controller but they are modified adding a shifting and a scaling factor under the weak assumption of interval excitation parameter convergence in finite time of the modified parameters is achieved ensuring good robustness of the proposed controller

a modified forward backward splitting method for maximal monotone mappings we consider the forward backward splitting method for finding a zero of the sum of two maximal monotone mappings this method is known to converge when the inverse of the forward mapping is strongly monotone we propose a modification to this method in the spirit of the extragradient method for monotone variational inequalities under which the method converges assuming only the forward mapping is lipschitz continuous on some closed convex subset of its domain the modification entails an additional forward step and a projection step at each iteration applications of the modified method to decomposition in convex programming and monotone variational inequalities are discussed

on adaptive control of nonlinearly parameterized nonlinear systems towards a constructive procedure abstract a new framework to design immersion and invariance adaptive controllers for nonlinearly parameterized nonlinear systems was recently proposed by the authors the key step is the construction of a monotone mapping via a suitable selection of a controller tuning function which has to satisfy some integrability conditions this translates into the need to solve a partial differential equation pde in this paper this result is extended providing some answers to the questions of characterization of monotonizable systems and solvability of the pde first adding to the design a nonlinear dynamic scaling we obviate the need to solve the pde second for the case of factorizable nonlinearities the following results are established i it is shown that the monotonicity condition is satisfied if a linear matrix inequality is feasible ii directly verifiable involutivity conditions that ensure the solution of the pde are presented iii an explicit formula for the required tuning function is given provided the regressor matrix satisfies some rank conditions hence adding a dynamic scaling this yields a constructive solution to the problem

rates of convergence for inexact krasnosel skii mann iterations in banach spaces we study the convergence of an inexact version of the classical krasnosel skii mann iteration for computing fixed points of nonexpansive maps our main result establishes a new metric bound for the fixed point residuals from which we derive their rate of convergence as well as the convergence of the iterates towards a fixed point the results are applied to three variants of the basic iteration infeasible iterations with approximate projections the ishikawa iteration and diagonal krasnosels kii mann schemes the results are also extended to continuous time in order to study the asymptotics of nonautonomous evolution equations governed by nonexpansive operators

convergence rates in forward backward splitting forward backward splitting methods provide a range of approaches to solving large scale optimization problems and variational inequalities in which structures conducive to decomposition can be utilized apart from special cases where the forward step is absent and a version of the proximal point algorithm comes out efforts at evaluating the convergence potential of such methods have so far relied on lipschitz properties and strong monotonicity or inverse strong monotonicity of the mapping involved in the forward step the perspective mainly being that of projection algorithms here convergence is analyzed by a technique that allows properties of the mapping in the backward step to be brought in as well for the first time in such a general setting global and local contraction rates are derived moreover they are derived in a form which makes it possible to determine the optimal step size relative to certain constants associated with the given problem insights are thereby gained into the effects of shifting strong monotonicity between the forward and backward mappings when a splitting is selected

a robust nonlinear position observer for synchronous motors with relaxed excitation conditions abstracta robust nonlinear and globally convergent rotor position observer for surface mounted permanent magnet synchronous motors was recently proposed by the authors the key feature of this observer is that it requires only the knowledge of the motor s resistance and inductance using some particular properties of the mathematical model it is shown that the problem of state observation can be translated into one of estimation of two constant parameters which is carried out with a standard gradient algorithm in this work we propose to replace this estimator with a new one called dynamic regressor extension and mixing which has the following advantages with respect to gradient estimators 1 the stringent persistence of excitation pe condition of the regressor is not necessary to ensure parameter convergence 2 the latter is guaranteed requiring instead a non square integrability condition that has a clear physical meaning in terms of signal energy 3 if the regressor is pe the new observer

convergence rates with inexact non expansive operators in this paper we present a convergence rate analysis for the inexact krasnosel ski mann iteration built from non expansive operators the presented results include two main parts we first establish the global pointwise and ergodic iteration complexity bounds then under a metric sub regularity assumption we establish a local linear convergence for the distance of the iterates to the set of fixed points the obtained results can be applied to analyze the convergence rate of various monotone operator splitting methods in the literature including the forward backward splitting the generalized forward backward the douglas rachford splitting alternating direction method of multipliers and primal dual splitting methods for these methods we also develop easily verifiable termination criteria for finding an approximate solution which can be seen as a generalization of the termination criterion for the classical gradient descent method we finally develop a parallel analysis for the non stationary krasnosel ski mann iteration

an efficient algorithm for image segmentation markov random fields and related problems problems of statistical inference involve the adjustment of sample observations so they fit some a priori rank requirements or order constraints in such problems the objective is to minimize the deviation cost function that depends on the distance between the observed value and the modify value in markov random field problems there is also a pairwise relationship between the objects the objective in markov random field problem is to minimize the sum of the deviation cost function and a penalty function that grows with the distance between the values of related pairs separation function we discuss markov random fields problems in the context of a representative application the image segmentation problem in this problem the goal is to modify color shades assigned to pixels of an image so that the penalty function consisting of one term due to the deviation from the initial color shade and a second term that penalizes differences in assigned values to neighboring pixels is minimized we present here an algorithm that solves the problem in polynomial time when the deviation function is convex and separation function is linear and in strongly polynomial time when the deviation cost function is linear quadratic or piecewise linear convex with few pieces where few means a number exponential in a polynomial function of the number of variables and constraints the complexity of the algorithm for a problem on n pixels or variables m adjacency relations or constraints and range of variable values colors u is o t n m n log u where t n m is the complexity of solving the minimum s t cut problem on a graph with n nodes and m arcs furthermore other algorithms are shown to solve the problem with convex deviation and convex separation in running time o mn log n log nu and the problem with nonconvex deviation and convex separation in running time o t nu mu the nonconvex separation problem is np hard even for fixed value of u for the family of problems with convex deviation functions and linear separation function the algorithm described here runs in polynomial time which is demonstrated to be fastest possible

on the convergence rate of douglas rachford operator splitting method this note provides a simple proof of a worst case convergence rate measured by the iteration complexity for the douglas rachford operator splitting method for finding a root of the sum of two maximal monotone set valued operators the accuracy of an iterate to the solution set is measured by the residual of a characterization of the original problem which is different from conventional measures such as the distance to the solution set

nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems we consider projection algorithms for solving nonconvex feasibility problems in euclidean spaces of special interest are the method of alternating projections ap and the douglas rachford algorithm dr in the case of convex feasibility firm nonexpansiveness of projection mappings is a global property that yields global convergence of ap and for consistent problems dr a notion of local subfirm nonexpansiveness with respect to the intersection is introduced for consistent feasibility problems this together with a coercivity condition that relates to the regularity of the collection of sets at points in the intersection yields local linear convergence of ap for a wide class of nonconvex problems and even local linear convergence of nonconvex instances of the dr algorithm

applications of splitting algorithm to decomposition in convex programming and variational inequalities recently han and lou proposed a highly parallelizable decomposition algorithm for minimizing a strongly convex cost over the intersection of closed convex sets it is shown that their algorithm is in fact a special case of a splitting algorithm analyzed by gabay for finding a zero of the sum of two maximal monotone operators gabay s convergence analysis for the splitting algorithm is sharpened and new applications of this algorithm to variational inequalities convex programming and the solution of linear complementarily problems are proposed for convex programs with a certain separable structure a multiplier method that is closely related to the alternating direction method of multipliers of gabay mercier and of glowinski marrocco but which uses both ordinary and augmented lagrangians is obtained

including preferences into a multiobjective evolutionary algorithm to deal with many objective engineering optimization problems abstract in this paper we introduce a new preference relation based on a reference point approach this relation offers an easy approach to integrate decision maker s preferences into a moea without modifying its basic structure besides finding the optimal solution of the achievement scalarizing function the new preference relation allows the decision maker to find a set of solutions around that optimal solution then a moea equipped with the proposed preference relation can be integrated into an interactive optimization method one of the main advantages of the new method is that setting its parameters is an intuitive task to the decision maker the other advantage is that since our preference relation induces a finer order on vectors of objective space than that achieved by the pareto dominance relation it is appropriate to cope with problems having a high number of objectives we evaluated the proposed preference relation an engineering problem the optimization of an airfoil design with 6 objectives

relative error approximate versions of douglas rachford splitting and special cases of the admm we derive a new approximate version of the alternating direction method of multipliers admm which uses a relative error criterion the new version is somewhat restrictive and allows only one of the two subproblems to be minimized approximately but nevertheless covers commonly encountered special cases the derivation exploits the long established relationship between the admm and both the proximal point algorithm ppa and douglas rachford dr splitting for maximal monotone operators along with a relative error of the ppa due to solodov and svaiter in the course of analysis we also derive a version of dr splitting in which one operator may be evaluated approximately using a relative error criterion we computationally evaluate our method on several classes of test problems and find that it significantly outperforms several alternatives on one problem class

immersion and invariance adaptive control of nonlinearly parameterized nonlinear systems a new framework to design adaptive controllers for nonlinearly parameterized systems is proposed in this technical note the key step is the construction of a monotone mapping which explicitly depends on some of the estimator tuning parameters convexity that is related to monotonicity has been explored by several authors the property is assumed a priori and is useful for the design only in some region of state space which is completed switching the controller in our approach monotonicity can be used in the whole state space and moreover is enforced by the designer effectively becoming a synthesis tool one consequence of these features is that the controller does not rely on state dependent switching in order to dispose of degrees of freedom to render the function monotone we depart from standard adaptive control and adopt instead the recently introduced immersion and invariance approach

convergent dynamics a tribute to boris pavlovich demidovich we review and pay tribute to a result on convergent systems by the russian mathematician boris pavlovich demidovich in a sense demidovich s approach forms a prelude to a field which is now called incremental stability of dynamical systems developments on incremental stability are reviewed from a historical perspective

multiple and bicriteria scheduling a literature survey abstract real life scheduling problems require the decision maker to consider a number of criteria before arriving at any decision a solution which is optimal with respect to a given criterion might be a poor candidate for some other the trade offs involved in considering several different criteria provide useful insights to the decision maker thus considering problems with more than one criterion is more relebant in the context of real life scheduling problems surprisingly research in this important field has been scarce when compared to research in single criterion scheduling in this paper we provide a detailed literature survey of multiple and bicriteria problems in scheduling we also provide a broad classification scheme for scheduling problems

performance enhancement of parameter estimators via dynamic regressor extension and mixing a new procedure to design parameter estimators with enhanced performance is proposed in the technical note for classical linear regression forms it yields a new parameter estimator whose convergence is established without the usual requirement of regressor persistency of excitation the technique is also applied to nonlinear regressions with partially monotonic parameter dependence giving rise again to estimators with enhanced performance simulation results illustrate the advantages of the proposed procedure in both scenarios

a monotone skew splitting model for composite monotone inclusions in duality the principle underlying this paper is the basic observation that the problem of simultaneously solving a large class of composite monotone inclusions and their duals can be reduced to that of finding a zero of the sum of a maximally monotone operator and a linear skew adjoint operator an algorithmic framework is developed for solving this generic problem in a hilbert space setting new primal dual splitting algorithms are derived from this framework for inclusions involving composite monotone operators and convergence results are established these algorithms draw their simplicity and efficacy from the fact that they operate in a fully decomposed fashion in the sense that the monotone operators and the linear transformations involved are activated separately at each iteration comparisons with existing methods are made and applications to composite variational problems are demonstrated

the power of convex relaxation near optimal matrix completion this paper is concerned with the problem of recovering an unknown matrix from a small fraction of its entries this is known as the matrix completion problem and comes up in a great number of applications including the famous netflix prize and other similar questions in collaborative filtering in general accurate recovery of a matrix from a small number of entries is impossible but the knowledge that the unknown matrix has low rank radically changes this premise making the search for solutions meaningful this paper presents optimality results quantifying the minimum number of entries needed to recover a matrix of rank r exactly by any method whatsoever the information theoretic limit more importantly the paper shows that under certain incoherence assumptions on the singular vectors of the matrix recovery is possible by solving a convenient convex program as soon as the number of entries is on the order of the information theoretic limit up to logarithmic factors this convex program simply finds among all matrices consistent with the observed entries that with minimum nuclear norm as an example we show that on the order of nr log n samples are needed to recover a random n x n matrix of rank r by any method and to be sure nuclear norm minimization succeeds as soon as the number of entries is of the form nr polylog n

on dynamic regressor extension and mixing parameter estimators two luenberger observers interpretations dynamic regressor extension and mixing is a new technique for parameter estimation with guaranteed performance improvement with respect to classical gradient or least squares estimators that has proven instrumental in the solution of several open problems in system identification and adaptive control in this brief note we give two interpretations of this parameter estimator in terms of the recent extensions to the cases of nonlinear systems and observation of linear functionals for time varying systems of the classical luenberger s state observers

a parallel multiple reference point approach for multi objective optimization this paper presents a multiple reference point approach for multi objective optimization problems of discrete and combinatorial nature when approximating the pareto frontier multiple reference points can be used instead of traditional techniques these multiple reference points can easily be implemented in a parallel algorithmic framework the reference points can be uniformly distributed within a region that covers the pareto frontier an evolutionary algorithm is based on an achievement scalarizing function that does not impose any restrictions with respect to the location of the reference points in the objective space computational experiments are performed on a bi objective flow shop scheduling problem results quality measures as well as a statistical analysis are reported in the paper

on the complexity of the hybrid proximal extragradient method for the iterates and the ergodic mean in this paper we analyze the iteration complexity of the hybrid proximal extragradient hpe method for finding a zero of a maximal monotone operator recently proposed by solodov and svaiter one of the key points of our analysis is the use of new termination criteria based on the varepsilon enlargement of a maximal monotone operator the advantage of using these termination criteria is that their definition do not depend on the boundedness of the domain of the operator we then show that korpelevich s extragradient method for solving monotone variational inequalities falls in the framework of the hpe method as a consequence using the complexity analysis of the hpe method we obtain new complexity bounds for korpelevich s extragradient method which do not require the feasible set to be bounded as assumed in a recent paper by nemirovski another feature of our analysis is that the derived iteration complexity bounds are proportional to the distance of the initial point to the solution set the hpe framework is also used to obtain the first iteration complexity result for tseng s modified forward backward splitting method for finding a zero of the sum of a monotone lipschitz continuous map with an arbitrary maximal monotone operator whose resolvent is assumed to be easily computable also using the framework of the hpe method we study the complexity of a variant of a newton type extragradient algorithm proposed by solodov and svaiter for finding a zero of a smooth monotone function with lipschitz continuous jacobian

on non ergodic convergence rate of douglas rachford alternating direction method of multipliers this note proposes a novel approach to derive a worst case o 1 k o 1 k convergence rate measured by the iteration complexity in a non ergodic sense for the douglas rachford alternating direction method of multipliers proposed by glowinski and marrocco

distributed algorithms for computing a common fixed point of a group of nonexpansive operators this paper addresses the problem of seeking a common fixed point for a collection of nonexpansive operators over time varying multi agent networks in real hilbert spaces each operator is only privately and approximately known to each individual agent and all agents need to cooperate to solve this problem by local communications over time varying networks to handle this problem inspired by the centralized inexact krasnosel skii mann ikm iteration we propose a distributed algorithm called distributed inexact averaged operator algorithm d iao it is shown that the d iao algorithm can converge weakly to a common fixed point of the family of nonexpansive operators moreover under the linearly regular assumption it is proved that the d iao algorithm converges with a rate o gamma log 16 4k for some constant gamma in 0 1 where k is the iteration number the same converge speed can be ensured by a novel power regularity condition which is strictly weaker than linear regularity to reduce computational complexity another algorithm called d biao is developed and fully analyzed simulation examples are given to support the theoretical res

book review implicit functions and solution mappings a view from variational analysis second edition by a l dontchev and r t rockafellar springer new york 2014

objective space partitioning using conflict information for solving many objective problems we present an algorithm that partitions the objective space based on an analysis of the conflict information obtained from the current pareto front approximation by partitioning the objectives in terms of the conflict among them we aim to separate the multiobjective optimization into several subproblems in such a way that each of them contains the information to preserve as much as possible the structure of the original problem we implement this framework by performing ranking and parent selection independently in each subspace our experimental results show that the proposed conflict based partition strategy outperforms a similar algorithm in a test problem with independent groups of objectives in addition the new strategy achieves a better convergence and distribution than that produced by a strategy that creates subspaces at random in problems in which the degree of conflict among the objectives is significantly different the conflict based strategy presents a better performance

a proximal based decomposition method for convex minimization problems this paper presents a decomposition method for solving convex minimization problems at each iteration the algorithm computes two proximal steps in the dual variables and one proximal step in the primal variables we derive this algorithm from rockafellar s proximal method of multipliers which involves an augmented lagrangian with an additional quadratic proximal term the algorithm preserves the good features of the proximal method of multipliers with the additional advantage that it leads to a decoupling of the constraints and is thus suitable for parallel implementation we allow for computing approximately the proximal minimization steps and we prove that under mild assumptions on the problem s data the method is globally convergent and at a linear rate the method is compared with alternating direction type methods and applied to the particular case of minimizing a convex function over a finite intersection of closed convex sets

alternating projection proximal methods for convex programming and variational inequalities we consider a mixed problem composed in part of finding a zero of a maximal monotone operator and in part of solving a monotone variational inequality problem we propose a solution method for this problem that alternates between a proximal step for the maximal monotone operator part and a projection type step for the monotone variational inequality part and analyze its convergence and rate of convergence this method extends a decomposition method of chen and teboulle math programming 64 1994 pp 81 101 for convex programming and yields as a by product new decomposition methods

on robust parameter estimation in finite time without persistence of excitation the problem of adaptive estimation of constant parameters in the linear regressor model is studied without the hypothesis that regressor is persistently excited first the initial vector estimation problem is transformed to a series of the scalar ones using the method of dynamic regressor extension and mixing second several adaptive estimation algorithms are proposed for the scalar scenario in such a case if the regressor is nullified asymptotically or in finite time then the problem of estimation is also posed on a finite interval of time robustness of the proposed algorithms with respect to measurement noise and exogenous disturbances is analyzed the efficiency of the designed estimators is demonstrated in numeric experiments for academic examples

asymptotically convergent modified recursive least squares with data dependent updating and forgetting factor for systems with bounded noise continual updating of estimates required by most recursive estimation schemes often involves redundant usage of information and may result in system instabilities in the presence of bounded output disturbances an algorithm which eliminates these difficulties is investigated based on a set theoretic assumption the algorithm yields modified least squares estimates with a forgetting factor it updates the estimates selectively depending on whether the observed data contain sufficient information the information evaluation required at each step involves very simple computations in addition the parameter estimates are shown to converge asymptotically at an exponential rate to a region around the true parameter