exterior point algorithms for nearest points and convex quadratic programs we consider the problem of finding the nearest point by euclidean distance in a simplicial cone to a given point and develop an exterior penalty algorithm for it each iteration in the algorithm consists of a single newton step following a reduction in the value of the penalty parameter proofs of convergence of the algorithm are given various other versions of exterior penalty algorithms for nearest point problems in nonsimplicial polyhedral cones and for convex quadratic programs all based on a single descent step following a reduction in the value of the penalty parameter per iteration are discussed the performance of these algorithms in large scale computational experiments is very encouraging it shows that the number of iterations grows very slowly if at all with the dimension of the problem

inertia controlling methods for general quadratic programming active set quadratic programming qp methods use a working set to define the search direction and multiplier estimates in the method proposed by fletcher in 1971 and in several subsequent mathematically equivalent methods the working set is chosen to control the inertia of the reduced hessian which is never permitted to have more than one nonpositive eigenvalue such methods will be called inertia controlling this paper presents an overview of a generic inertia controlling qp method including the equations satisfied by the search direction when the reduced hessian is positive definite singular and indefinite recurrence relations are derived that define the search direction and lagrange multiplier vector through equations related to the karush kuhn tucker system discussion is included of connections with inertia controlling methods that maintain an explicit factorization of the reduced hessian matrix

a finite continuation algorithm for bound constrained quadratic programming the dual of the strictly convex quadratic programming problem with unit bounds is posed as a linear ell 1 minimization problem with quadratic terms a smooth approximation to the linear ell 1 function is used to obtain a parametric family of piecewise quadratic approximation problems the unique path generated by the minimizers of these problems yields the solution to the original problem for finite values of the approximation parameter thus a finite continuation algorithm is designed results of extensive computational experiments are reported

the approximation method for two stage fuzzy random programming with recourse in this paper a new class of fuzzy random optimization problem called two stage fuzzy random programming or fuzzy random programming with recourse frpr problem is first presented then its deterministic equivalent programming problem is characterized because the frpr problems include fuzzy random variable parameters with an infinite support they are inherently infinite dimensional optimization problems that can rarely be solved directly therefore an approximation approach to the fuzzy random variables with infinite supports by finitely supported ones is proposed which results in finite dimensional frpr problems after that this paper is devoted to establishing the conditions under which the objective value optimal objective value and minimizers of such finite dimensional frpr problem can be shown to converge to the objective value respectively optimal objective value and minimizers of the original infinite dimensional frpr problem

on affine scaling algorithms for nonconvex quadratic programming we investigate the use of interior algorithms especially the affine scaling algorithm to solve nonconvex indefinite or negative definite quadratic programming qp problems although the nonconvex qp with a polytope constraint is a hard problem we show that the problem with an ellipsoidal constraint is easy when the hard qp is solved by successively solving the easy qp the sequence of points monotonically converge to a feasible point satisfying both the first and the second order optimality conditions

simplified newton type adaptive estimation algorithms a new adaptive estimation algorithm is presented it is the result of a combination of the lms and the fast newton transversal filters fntf class the main characteristic of the proposed algorithm is its improved convergence rate as compared to lms for cases where it is known that lms behaves poorly this improved characteristic is achieved in expense of a slight increase in the computational complexity while the overall algorithmic structure is very simple lms type the proposed algorithm seems also to compare relatively well against rls and fntf

an interior newton method for quadratic programming quadratic programming represents an extremely important class of optimization problem in this paper we propose a new interior approach for the general quadratic programming problem we establish that our new method is globally and quadratically convergent published alternative interior approaches do not share such strong convergence properties for the nonconvex case we also report on the results of preliminary numerical experiments the results indicate that the proposed method has considerable prcatival potential

a fast newton multichannel algorithm for decision feedback equalization fast newton transversal filters fntf are a new class of adaptive algorithms unifying nlms and fast rls this is achieved by allowing the algorithm to exploit a priori information about the associated predictors order this note extends fntf to the multichannel case and demonstrates the applicability of the resulting algorithm to decision feedback equalization

on long step path following and sumt for linear and quadratic programming we consider a long step barrier algorithm for the minimization of a convex quadratic objective subject to linear inequality constraints the algorithm is a dual version of a method developed by anstreicher et al algorithmica 10 1993 pp 365 382 and requires o nl or o sqrt n l iterations to solve a problem with n constraints depending on how the barrier parameter is reduced as a corollary of our analysis we demonstrate that the classical sumt algorithm exactly as implemented in 1968 solves linear and quadratic programs in o sqrt n l log l iterations with proper initialization and choice of parameters

a null space method for computing the search direction in the general inertia controlling method for dense quadratic programming the inertia controlling strategy in active set methods consists of choosing the working set so that the reduced hessian never has more than one non positive eigenvalue usually this strategy has been implemented by permitting to delete constraints only at stationary points in a general inertia controlling method constraints may be deleted at non stationary points a null space method for dense quadratic programming is presented in which only one triangular system has to be solved at each iteration for computing the search direction this method takes advantage of previously developed recurrence formulas for updating the search direction when the working set changes

fast newton transversal filters a new class of adaptive estimation algorithms a class of adaptive algorithms for the estimation of fir finite impulse response transversal filters is presented the main characteristic of this class is the fast computation of the gain vector needed for the adaptation of the transversal filters the method for deriving these algorithms is based on the assumption that the input signal is autoregressive of order m where m can be much smaller than the order of the filter to be estimated under this assumption the covariance matrix of the input signal is estimated by extending in a min max way the m order sample covariance matrix this estimate can be regarded as a generalization of the diagonal covariance matrix used in lms and leads to an efficient computation of the gain needed for the adaptation the new class of algorithms contains the lms and the fast versions of ls as special cases the complexity changes linearly with m starting from the complexity of the lms for m 0 and ending at the complexity of the fast versions of ls

a numerically stable dual method for solving strictly convex quadratic programs an efficient and numerically stable dual algorithm for positive definite quadratic programming is described which takes advantage of the fact that the unconstrained minimum of the objective function can be used as a starting point its implementation utilizes the cholesky and qr factorizations and procedures for updating them the performance of the dual algorithm is compared against that of primal algorithms when used to solve randomly generated test problems and quadratic programs generated in the course of solving nonlinear programming problems by a successive quadratic programming code the principal motivation for the development of the algorithm these computational results indicate that the dual algorithm is superior to primal algorithms when a primal feasible point is not readily available the algorithm is also compared theoretically to the modified simplex type dual methods of lemke and van de panne and whinston and it is illustrated by a numerical example

fast recursive least squares transversal filters for adaptive filtering fast transversal filter ftf implementations of recursive least squares rls adaptive filtering algorithms are presented in this paper substantial improvements in transient behavior in comparison to stochastic gradient or lms adaptive algorithms are efficiently achieved by the presented algorithms the true not approximate solution of the rls problem is always obtained by the ftf algorithms even during the critical initialization period first n iterations of the adaptive filter this true solution is recursively calculated at a relatively modest increase in computational requirements in comparison to stochastic gradient algorithms factor of 1 6 to 3 5 depending upon application additionally the fast transversal filter algorithms are shown to offer substantial reductions in computational requirements relative to existing fast rls algorithms such as the fast kalman algorithms of morf ljung and falconer 1976 and the fast ladder lattice algorithms of morf and lee 1977 1981 they are further shown to attain steady state unnormalized or improve upon first n initialization steps the very low computational requirements of the efficient rls solutions of carayannis manolakis and kalouptsidis 1983 finally several efficient procedures are presented by which to ensure the numerical stability of the transversal filter algorithms including the incorporation of soft constraints into the performance criteria internal bounding and rescuing procedures and dynamic range increasing square root normalized variations of the transversal filters

a weighted gram schmidt method for convex quadratic programming range space methods for convex quadratic programming improve in efficiency as the number of constraints active at the solution decreases in this paper we describe a range space method based upon updating a weighted gram schmidt factorization of the constraints in the active set the updating methods described are applicable to both primal and dual quadratic programming algorithms that use an active set strategy

a new penalty function algorithm for convex quadratic programming in this paper we develop an exterior point algorithm for convex quadratic programming using a penalty function approach each iteration in the algorithm consists of a single newton step followed by a reduction in the value of the penalty parameter the points generated by the algorithm follow an exterior path that we define convergence of the algorithm is established the proposed algorithm was motivated by the work of al sultan and murty on nearest point problems a special quadratic program a preliminary implementation of the algorithm produced encouraging results in particular the algorithm requires a small and almost constant number of iterations to solve the small to medium size problems tested

a dual active set algorithm for positive semi definite quadratic programming because of the many important applications of quadratic programming fast and efficient methods for solving quadratic programming problems are valued goldfarb and idnani 1983 describe one such method well known to be efficient and numerically stable the goldfarb and idnani method suffers only from the restriction that in its original form it cannot be applied to problems which are positive semi definite rather than positive definite in this paper we present a generalization of the goldfarb and idnani method to the positive semi definite case and prove finite termination of the generalized algorithm in our generalization we preserve the spirit of the goldfarb and idnani method and extend their numerically stable implementation in a natural way

weak convergence and local stability properties of fixed step size recursive algorithms a recursive equation that subsumes several common adaptive filtering algorithms is analyzed for general stochastic inputs and disturbances by relating the motion of the parameter estimate errors to the behavior of an unforced deterministic ordinary differential equation ode the odes describing the motion of several common adaptive filters are examined in some simple settings including the least mean square lms algorithm and all three of its signed variants the signed regressor the signed error and the sign sign algorithms stability and instability results are presented in terms of the eigenvalues of a correlation like matrix this generalizes known results for lms signed regressor lms and signed error lms and gives new stability criteria for the sign sign algorithm the ability of the algorithms to track moving parameterizations can be analyzed in a similar manner by relating the time varying system to a forced ode the asymptotic distribution about the forced ode is an ornstein uhlenbeck process the properties of which can be described in a straightforward manner

bound constrained quadratic programming via piecewise quadratic functions the smallest eigenvalue of a symmetric positive definite matrix and is solved by newton iteration with line search the paper describes the algorithm and its implementation including estimation of 1 how to get a good starting point for the iteration and up and downdating of cholesky factorization results of extensive testing and comparison with other methods for constrained qp are given

constraint deletion strategy in the inertia controlling quadratic programming method abstract the inertia controlling strategy in active set methods consists of choosing the working set so that the reduced hessian never has more than one non positive eigenvalue usually this strategy has been implemented by permitting to delete constraints only at stationary points this paper concerns the general inertia controlling quadratic programming method in which constraints may be deleted at non stationary points we consider the determination of the search direction when the reduced hessian is positive definite positive semidefinite and singular and indefinite or negative definite recurrence formulas are presented to update the search direction and multiplier estimates when the working set changes

stabilizing the fast kalman algorithms the fast kalman algorithms are stabilized by introducing a quantity that measures the accumulation of the roundoff errors this quantity is used to correct the variables of the algorithm at every time step the correction is defined as the solution of a specific minimization problem the resulting algorithm still has the nice complexity properties of the original algorithm linear in the number of parameters to be estimated but it has a much more stable behavior

a new algorithm for solving strictly convex quadratic programs we reformulate convex quadratic programs with simple bound constraints and strictly convex quadratic programs as problems of unconstrained minimization of convex quadratic splines therefore any algorithm for finding a minimizer of a convex quadratic spline can be used to solve these quadratic programming problems in this paper we propose a newton method to find a minimizer of a convex quadratic spline derived from the unconstrained reformulation of a strictly convex quadratic programming problem the newton method is a natural mixture of a descent method and an active set method moreover it is an iterative method yet it terminates in finite operations in exact arithmetic