a class of linear complementarity problems solvable in polynomial time we describe a condition number for the linear complementarity problem lcp which characterizes the degree of difficulty for its solution when a potential reduction algorithm is used consequently we develop a class of lcps solvable in polynomial time the result suggests that the convexity or positive semidefiniteness of the lcp may not be the basic issue that separates lcps solvable and not solvable in polynomial time

an algorithm for a singly constrained class of quadratic programs subject to upper and lower bounds this paper gives an o n algorithm for a singly constrained convex quadratic program using binary search to solve the kuhn tucker system computational results indicate that a randomized version of this algorithm runs in expected linear time and is suitable for practical applications for the nonconvex case ane approximate algorithm is proposed which is based on convex and piecewise linear approximations of the objective function

cost approximation a unified framework of descent algorithms for nonlinear programs this paper describes and analyzes the cost approximation algorithm this class of iterative descent algorithms for nonlinear programs and variational inequalities places a large number of algorithms within a common framework and provides a means for analyzing relationships among seemingly unrelated methods a common property of the methods included in the framework is that their subproblems may be characterized by monotone mappings which replace an additive part of the original cost mapping in an iterative manner alternately a step is taken in the direction obtained in order to reduce the value of a merit function for the original problem the generality of the framework is illustrated through examples and the convergence characteristics of the algorithm are analyzed for applications to nondifferentiable optimization the convergence results are applied to some example methods demonstrating the strength of the analysis compared to existing results

bounds on achievable performance in the identification and adaptive control of time varying systems treats the identification and adaptive control of time varying linear systems for linear systems with a gauss markov parameter process a global lower bound on the mean square error is obtained which is valid for any causal parameter estimator a similar lower bound is obtained for any causal one step ahead predictor these bounds are applied to the adaptive control of time varying systems to obtain a lower bound on closed loop mean square performance for any causal control law for a specific control law mean square stability is established and through simulations it is seen that the performance nearly meets the theoretical lower bound

on the identification property of a projected gradient method the authors study the convergence properties of a projected gradient algorithm for the general problem min f x x in omega where f r n to r is a mapping continuously differentiable on a closed convex set omega subseteq r n the algorithm which requires only one projection per iteration is a special version of the method of projection of the gradient by demyanov and rubinov approximate methods in optimization problems elsevier new york 1970 where the step choice is made according to a scheme similar to the one used by calamai and more math programming 39 1987 pp 93 116 the authors are mainly interested in analysing the identification property of the algorithm for the case where the set omega is a polyhedron that is the ability to identify in a finite number of steps the face in which the final solution lies the convergence results that are shown are very similar to those shown in 6 for the standard projected gradient method

a logarithmic descent direction algorithm for the quadratic knapsack problem abstract the quadratic knapsack problem is an np hard optimization problem with many diverse applications in industrial and management engineering however computational complexities still remain in the quadratic knapsack problem in this study a logarithmic descent direction algorithm is proposed to approximate a solution to the quadratic knapsack problem the proposed algorithm is based on the karush kuhn tucker necessary optimality condition and the damped newton method the convergence of the algorithm is proven and the numerical results indicate its effectiveness

fast primal and dual heuristics for the p median location problem a greedy type heuristic is presented for the p median problem and computational results for problems having up to 400 vertices are shown we describe a dual heuristic procedure for the p median based on erlenkotter s heuristic 1978 for the uncapacitated facility location problem a primal dual heuristic that generates a primal solution based on a good dual solution is also presented for both procedures computational experience relative to the same set of large size test problems is reported finally the results obtained from all procedures are compared with the bounds and corresponding computing times provided by other available heuristics for the p median location problem

algorithms for the continuous nonlinear resource allocation problem new implementations and numerical studies patriksson 2008 provided a then up to date survey on the continuous separable differentiable and convex resource allocation problem with a single resource constraint since the publication of that paper the interest in the problem has grown several new applications have arisen where the problem at hand constitutes a subproblem and several new algorithms have been developed for its efficient solution this paper therefore serves three purposes first it provides an up to date extension of the survey of the literature of the field complementing the survey in patriksson 2008 with more then 20 books and articles second it contributes improvements of some of these algorithms in particular with an improvement of the pegging that is variable fixing process in the relaxation algorithm and an improved means to evaluate subsolutions third it numerically evaluates several relaxation primal and breakpoint dual algorithms incorporating a variety of pegging strategies as well as a quasi newton method our conclusion is that our modification of the relaxation algorithm performs the best at least for problem sizes up to 30 million variables the practical time complexity for the breakpoint and relaxation algorithms is linear

a general descent framework for the monotone variational inequality problem we present a framework for descent algorithms that solve the monotone variational inequality problem vip v which consists in finding a solutionv v satisfyings v t v v 0 for allv v this unified framework includes as special cases some well known iterative methods and equivalent optimization formulations a descent method is developed for an equivalent general optimization formulation and a proof of its convergence is given based on this unified logarithmic framework we show that a variant of the descent method where each subproblem is only solved approximately is globally convergent under certain conditions

an ol n 3 primal interior point algorithm for convex quadratic programming we present a primal interior point method for convex quadratic programming which is based upon a logarithmic barrier function approach this approach generates a sequence of problems each of which is approximately solved by taking a single newton step it is shown that the method requires o sqrt n l iterations and o n3 5l arithmetic operations by using modified newton steps the number of arithmetic operations required by the algorithm can be reduced to o n3l

a survey on the continuous nonlinear resource allocation problem our problem of interest consists of minimizing a separable convex and differentiable function over a convex set defined by bounds on the variables and an explicit constraint described by a separable convex function applications are abundant and vary from equilibrium problems in the engineering and economic sciences through resource allocation and balancing problems in manufacturing statistics military operations research and production and financial economics to subproblems in algorithms for a variety of more complex optimization models this paper surveys the history and applications of the problem as well as algorithmic approaches to its solution the most common techniques are based on finding the optimal value of the lagrange multiplier for the explicit constraint most often through the use of a type of line search procedure we analyze the most relevant references especially regarding their originality and numerical findings summarizing with remarks on possible extensions and future research 2006 elsevier b v all rights reserved

an extension of karmarkar projective algorithm for convex quadratic programming we present an extension of karmarkar s linear programming algorithm for solving a more general group of optimization problems convex quadratic programs this extension is based on the iterated application of the objective augmentation and the projective transformation followed by optimization over an inscribing ellipsoid centered at the current solution it creates a sequence of interior feasible points that converge to the optimal feasible solution in o ln iterations each iteration can be computed in o ln3 arithmetic operations wheren is the number of variables andl is the number of bits in the input in this paper we emphasize its convergence property practical efficiency and relation to the ellipsoid method

competitive facility location model with concave demand abstract we consider a spatial interaction model for locating a set of new facilities that compete for customer demand with each other as well as with some pre existing facilities to capture the market expansion and the market cannibalization effects customer demand is assumed to be a concave non decreasing function of the total utility derived by each customer from the service offered by the facilities the problem is formulated as a non linear knapsack problem for which we develop a novel solution approach based on constructing an efficient piecewise linear approximation scheme for the objective function this allows us to develop exact and optimal solution approaches capable of dealing with relatively large scale instances of the model we also develop a fast heuristic algorithm for which a tight worst case error bound is established

newton s method with a model trust region modification a modified newton method for unconstrained minimization is presented and analyzed the modification is based upon the model trust region approach this report contains a thorough analysis of the locally constrained quadratic minimizations that arise as subproblems in the modified newton iteration several promising alternatives are presented for solving these subproblems in ways that overcome certain theoretical difficulties exposed by this analysis very strong convergence results are presented concerning the minimization algorithm in particular the explicit use of second order information is justified by demonstrating that the iterates converge to a point which satisfies the second order necessary conditions for minimization with the exception of very pathological cases this occurs whenever the algorithm is applied to problems with continuous second partial derivatives

quadratic program based modularity maximization for fuzzy community detection in social networks one of the most important elements of social network analysis is community detection i e finding groups of similar people based on their traits in this paper we present the fuzzy modularity maximization fmm approach for community detection which finds overlapping that is fuzzy communities where appropriate by maximizing a generalized form of newman s modularity the first proposed fmm solution uses a tree based structure to find a globally optimal solution while the second proposed solution uses alternating optimization to efficiently search for a locally optimal solution both of these approaches are based on a proposed algorithm called one step modularity maximization osmm which computes the optimal cluster memberships for one person in the social network we prove that osmm can be formulated as a simplified quadratic knapsack optimization problem which is o n time complexity we then propose a tree based algorithm called fmm find best leaf node fmm fbln which represents sequences of osmm steps in a tree based structure it is proved that fmm fbln finds globally optimal solutions for fmm however the time complexity of fmm fbln is o n d d ge 2 thus it is impractical for most real world networks to combat this inefficiency we propose five heuristic based alternating optimization schemes i e fmm h1 h5 which are all shown to be o n 2 time complexity we compare the results of the fmm h solutions with those of state of the art community detection algorithms multicut spectral fcm msfcm and gals and with those of two fuzzy community detection algorithms called ga and vertex similarity based gradient descent method vsgd on ten real world datasets we conclude that one of the five heuristic algorithms fmm h2 is very competitive with gals and much more effective than msfcm ga and vsgd furthermore all of the fmm h schemes are at least two orders of magnitude faster than gals in run time finally fmm h unlike gals which only produces crisp partitions and msfcm which always finds fuzzy partitions is the only fuzzy community detection algorithm to date that can find the max modularity partition fuzzy or crisp

inverse optimization for linearly constrained convex separable programming problems abstract in this paper we study inverse optimization for linearly constrained convex separable programming problems that have wide applications in industrial and managerial areas for a given feasible point of a convex separable program the inverse optimization is to determine whether the feasible point can be made optimal by adjusting the parameter values in the problem and when the answer is positive find the parameter values that have the smallest adjustments a sufficient and necessary condition is given for a feasible point to be able to become optimal by adjusting parameter values inverse optimization formulations are presented with l 1 and l 2 norms these inverse optimization problems are either linear programming when l 1 norm is used in the formulation or convex quadratic separable programming when l 2 norm is used

lagrangean heuristics for location problems abstract in this paper we present a framework for developing lagrangean heuristics heuristics based upon lagrangean relaxation and subgradient optimisation with respect to location problems computational results are given for four different location problems p median uncapacitated warehouse location capacitated warehouse location and capacitated warehouse location with single source constraints these results indicate that the framework presented in this paper is robust i e it gives good quality solutions for each of these different location problems

an algorithm for convex quadratic programming that requires o n 3 5 l arithmetic operations a new interior point method for minimizing a convex quadratic function over a polytope is developed we show that our method requires o n3 5l arithmetic operations in our algorithm we construct a sequence pz0 pz1 pzk of nested convex sets that shrink towards the set of optimal solution s during iteration k we take a partial newton step to move from an approximate analytic center of pzk 1 to an approximate analytic center of pzk a system of linear equations is solved at each iteration to find the step direction the solution that is available after o ml iterations can be converted to an optimal solution our analysis indicates that inexact solutions to the linear system of equations could be used in implementing this algorithm

the p median problem a survey of metaheuristic approaches the p median problem is one of the basic models in discrete location theory as with most location problems it is classified as np hard and so heuristic methods are usually used to solve it metaheuristics are frameworks for building heuristics in this survey we examine the p median with the aim of providing an overview on advances in solving it using recent procedures based on metaheuristic rules

convex quadratic programming with one constraint and bounded variables in this paper we propose an iterative algorithm for solving a convex quadratic program with one equality constraint and bounded variables at each iteration a separable convex quadratic program with the same constraint set is solved two variants are analyzed one that uses an exact line search and the other a unit step size preliminary testing suggests that this approach is efficient for problems with diagonally dominant matrices

on the solution of concave knapsack problems we consider a version of the knapsack problem which gives rise to a separable concave minimization problem subject to bounds on the variables and one equality constraint we characterize strict local miniimizers of concave minimization problems subject to linear constraints and use this characterization to show that although the problem of determining a global minimizer of the concave knapsack problem is np hard it is possible to determine a local minimizer of this problem with at most o n logn operations and 1 logn evaluations of the function if the function is quadratic this algorithm requires at most o n logn operations

a class of nonlinear nonseparable continuous knapsack and multiple choice knapsack problems this paper considers a general class of continuous nonlinear and nonseparable knapsack problems special cases of which arise in numerous operations and financial contexts we develop important properties of optimal solutions for this problem class based on the properties of a closely related class of linear programs using these properties we provide a solution method that runs in polynomial time in the number of decision variables while also depending on the time required to solve a particular one dimensional optimization problem thus for the many applications in which this one dimensional function is reasonably well behaved e g unimodal the resulting algorithm runs in polynomial time we next develop a related solution approach to a class of continuous nonlinear and nonseparable multiple choice knapsack problems this algorithm runs in polynomial time in both the number of variables and the number of variants per item while again dependent on the complexity of the same one dimensional optimization problem as for the knapsack problem computational testing demonstrates the power of the proposed algorithms over a commercial global optimization software package

on cheap smoothing opportunities in identification of time varying systems in certain applications of nonstationary system identification the model based decisions can be postponed i e executed with a delay this allows one to incorporate into the identification process not only the currently available information but also a number of future data points the resulting estimation schemes which involve smoothing are not causal despite the possible performance improvements the existing smoothing algorithms are seldom used in practice mainly because of their high computational requirements we show that the computationally attractive smoothing procedures can be obtained by means of compensating estimation delays that arise in the standard exponentially weighted least squares least mean squares and kalman filter based parameter trackers

on some properties of quadratic programs with a convex quadratic constraint in this paper we consider the problem of minimizing a possibly nonconvex quadratic function with a quadratic constraint we point out some new properties of the problem in particular in the first part of the paper we show that i given a kkt point that is not a global minimizer it is easy to find a better feasible point ii strict complementarity holds at the local nonglobal minimizer in the second part of this paper we show that the original constrained problem is equivalent to the unconstrained minimization of a piecewise quartic merit function using the unconstrained formulation we give in the nonconvex case a new second order necessary condition for global minimizers in the third part of this paper algorithmic applications of the preceding results are briefly outlined and some preliminary numerical experiments are reported

methods for global concave minimization a bibliographic survey the global concave minimization problem is to find the constrained global minimum of a concave function since such a function may have many local minima finding the global minimum is a computationally difficult problem in this bibliographic survey which includes most of the recent papers on constrained global concave minimization we have attempted to briefly summarize the main ideas in each paper these recent papers include those concerned with large scale global concave minimization and bilinear programming

brief paper on the lower smoothing bound in identification of time varying systems in certain applications of nonstationary system identification the model based decisions can be postponed i e executed with a delay this allows one to incorporate in the identification process not only the currently available information but also a number of future data points the resulting estimation schemes which involve smoothing are not causal assuming that the infinite observation history is available the paper establishes the lower steady state estimation bound for any noncausal estimator applied to a linear system with randomly drifting coefficients under gaussian assumptions this lower bound complements the currently available one which is restricted to causal estimators

partial affine scaling for linearly constrained minimization we propose an interior point method for finding a stationary point of a nonlinear program with linear equality and nonnegativity constraints this method maintains a basis at each iteration and updates the iterate by taking an affine scaling step in the space of nonbasic variables in the general case we propose to choose the basis according to a rule that maximizes the basic components of the iterate in the case where the feasible region is the product of simplices we propose an alternative rule that minimizes the basic components of the cost gradient we analyze the convergence of the method under each rule a key feature of this method is that it can be implemented much like the simplex method

optimal scaling of balls and polyhedra the concern is with solving as linear or convex quadratic programs special cases of the optimal containment and meet problems the optimal containment or meet problem is that of finding the smallest scale of a set for which some translation contains a set or meets each element in a collection of sets respectively these sets are unions or intersections of cells where a cell is either a closed polyhedral convex set or a closed solid ball

an ol n 3 potential reduction algorithm for linear programming we describe a primal dual potential function for linear programming phi x s rho ln x t s sum limits j 1 n ln x j s j where n x is the primal variable ands is the dual slack variable as a result we develop an interior point algorithm seeking reductions in the potential function with rho n sqrt n neither tracing the central path nor using the projective transformation the algorithm converges to the optimal solution set in o sqrt n l iterations and uses o n3l total arithmetic operations we also suggest a practical approach to implementing the algorithm

formulation analysis and numerical study of an optimization based conservative interpolation remap of scalar fields for arbitrary lagrangian eulerian methods we develop and study the high order conservative and monotone optimization based remap obr of a scalar conserved quantity mass between two close meshes with the same connectivity the key idea is to phrase remap as a global inequality constrained optimization problem for mass fluxes between neighboring cells the objective is to minimize the discrepancy between these fluxes and the given high order target mass fluxes subject to constraints that enforce physically motivated bounds on the associated primitive variable density in so doing we separate accuracy considerations handled by the objective functional from the enforcement of physical bounds handled by the constraints the resulting obr formulation is applicable to general unstructured heterogeneous grids under some weak requirements on grid proximity but not on the cell types we prove that the obr algorithm is linearity preserving in one two and three dimensions the paper also examines connections between the obr and the recently proposed flux corrected remap fcr liska et al 1 we show that the fcr solution coincides with the solution of a modified version of obr m obr which has the same objective but a simpler set of box constraints derived by using a worst case scenario because m obr fcr has a smaller feasible set preservation of linearity may be lost and accuracy may suffer for some grid configurations our numerical studies confirm this and show that obr delivers significant increases in robustness and accuracy preliminary efficiency studies of obr reveal that it is only a factor of 2 1 slower than fcr but admits 1 5 times larger time steps

on the continuous quadratic knapsack problem we introduce a new algorithm for the continuous bounded quadratic knapsack problem this algorithm is motivated by the geometry of the problem is based on the iterative solution of a series of simple projection problems and is easy to understand and implement in practice the method compares favorably to other well known algorithms some of which have superior worst case complexity on problem sizes up ton 4000

optimal and suboptimal smoothing algorithms for identification of time varying systems with randomly drifting parameters noncausal estimation algorithms which involve smoothing can be used for off line identification of nonstationary systems since smoothing is based on both past and future data it offers increased accuracy compared to causal tracking estimation schemes incorporating past data only it is shown that efficient smoothing variants of the popular exponentially weighted least squares and kalman filter based parameter trackers can be obtained by means of backward time filtering of the estimates yielded by both algorithms when system parameters drift according to the random walk model and the adaptation gain is sufficiently small the properly tuned two stage kalman filtering smoothing algorithm derived in the paper achieves the cramer rao type lower smoothing bound i e it is the optimal noncausal estimation scheme under the same circumstances performance of the modified exponentially weighted least squares algorithm is often only slightly inferior to that of the kalman filter based smoother

performance analysis of general tracking algorithms a general family of tracking algorithms for linear regression models is studied it includes the familiar least mean square gradient approach recursive least squares and kalman filter based estimators the exact expressions for the quality of the obtained estimates are complicated approximate and easy to use expressions for the covariance matrix of the parameter tracking error are developed these are applicable over the whole time interval including the transient and the approximation error can be explicitly calculated

the nonlinear knapsack problem algorithms and applications abstract we present a survey of algorithms and applications for the nonlinear knapsack problem or the nonlinear resource allocation problem in its most general form the nonlinear knapsack problem will be defined as a nonlinear optimization problem with just one constraint bounds on the variables and in some cases a set of specially structured constraints such as generalized upper bounds gubs this problem is encountered either directly or as a subproblem in a variety of applications including production planning financial modeling stratified sampling and capacity planning in manufacturing health care and computer networks by taking advantage of the special structure of the problem efficient solution methods can be developed problem classes addressed here include continuous and integer problems convex and nonconvex problems separable and nonseparable problems and problems with additional specially structured constraints

local minima for indefinite quadratic knapsack problems we consider the complexity of finding a local minimum for the nonconvex quadratic knapsack problem global minimization for this example of quadratic programming is np hard more and vavasis have investigated the complexity of local minimization for the strictly concave case of qkp here we extend their algorithm to the general indefinite case our main result is an algorithm that computes a local minimum in o n logn 2 steps our approach involves eliminating all but one of the convex variables through parametrization yielding a nondifferentiable problem we use a technique from computational geometry to address the nondifferentiable problem

algorithms for the solution of quadratic knapsack problems we consider the problem of minimizing a quadratic function with a knapsack constraint quadratic knapsack problems have numerous applications including the least distance problem quadratic programming defined on the convex hull of a set of points and the maximum clique problem we propose and analyze three algorithms for solving quadratic knapsack problems two algorithms are based on recently developed interior point methods the first solves convex quadratic problems and the second computes a stationary point for the indefinite case for both computational results on a variety of test problems are presented the third algorithm based on simplicial partitioning and convex underestimating functions can be used to compute the global optimum of indefinite quadratic knapsack problems

quasi newton updates with bounds we develop a quasi newton method which preserves known bounds on the jacobian matrix we show that this update can be computed with the same amount of work as competitive methods in particular we prove that the number of operations required to obtain this update is proportional to the number of nonzeros in the sparsity pattern of the jacobian matrix the method is also shown to share the local convergence properties of broyden s and schubert s method

on the solution of multidimensional convex separable continuous knapsack problem with bounded variables abstract a minimization problem with a convex separable objective function subject to linear equality constraints and box constraints bounds on the variables is considered necessary and sufficient optimality condition is proved for a feasible solution to be an optimal solution to this problem primal dual analysis is also included examples of some convex separable objective functions for the considered problem are presented