adaptive control of first order nonlinear systems with reduced knowledge of the plant parameters this paper presents an adaptive control strategy for a class of first order nonlinear systems of the form x spl dot spl theta sub 1 sup t f x spl theta sub 2 sup t g x where g x is a smooth function whereas f x satisfies sectoricity conditions spl theta sub 1 and spl theta sub 2 are constant parameter vectors it is assumed that the system remains controllable for all values of x but the sign of spl theta sub 2 sup t g x x is unknown the proposed adaptive scheme extends ideas previously presented the authors 1992 where the term premultiplying the input was supposed to be constant the standard least squares estimates of spl theta sub 2 are modified using a hysteresis type switching algorithm that enables us to conclude on existence uniqueness boundedness and convergence of the solutions of the adaptive closed loop system

improved complexity using higher order correctors for primal dual dikin affine scaling in this paper we show that the primal dual dikin affine scaling algorithm for linear programming of jansen roos and terlaky enhances an asymptotical o sqrt n l complexity by using corrector steps we also show that the result remains valid when the method is applied to positive semi definite linear complementarity problems

discrete time adaptive iterative learning control for high order nonlinear systems with unknown control directions an adaptive iterative learning control method is proposed for a class of nonlinear strict feedback discrete time systems with random initial conditions and iteration varying desired trajectories an n step ahead predictor approach is employed to estimate the future states in the control design discrete nussbaum gain method is utilised to deal with the lack of a priori knowledge of control directions the proposed control algorithm guarantees the boundedness of all the signals in the controlled system the tracking error converges to zero asymptotically along the iterative learning axis except for beginning states affected by random initial conditions the effectiveness of the proposed control scheme is verified through numerical simulation

on the superlinear convergence of interior point algorithms for a general class of problems in this paper the authors extend the q superlinear convergence theory recently developed by zhang tapia and dennis for a class of interior point linear programming algorithms to similar interior point algorithms for quadratic programming and for linear complementarily problems this unified approach consists of viewing all these algorithms as a damped newton method applied to perturbations of a general problem a set of sufficient conditions for these algorithms to achieve q superlinear convergence is established the key ingredients consist of asymptotically taking the step to the boundary of the positive orthant and letting the centering parameter approach zero at a specific rate the construction of algorithms that have both the global property of polynomiality and the local property of superlinear convergence will be the subject of further research

a survey on iterative learning control for nonlinear systems in this article we review the recent advances in iterative learning control ilc for nonlinear dynamic systems in the research field of ilc two categories of system nonlinearities are considered namely the global lipschitz continuous glc functions and local lipschitz continuous llc functions ilc for glc systems is widely studied and analysed using contraction mapping approach and the focus of recent exploration moves to application problems though a number of theoretical issues remain open ilc for llc systems is currently a hot area and the recent research focuses on ilc design and analysis by means of lyapunov approach the objectives of this article are to introduce recent development and advances in nonlinear ilc schemes highlight their effectiveness and limitations as well as discuss the directions for further exploration of nonlinear ilc

splitting dense columns in sparse linear systems abstract we consider systems of equations of the form aa t x b where a is a sparse matrix having a small number of columns which are much denser than the other columns these dense columns in a cause aat to be very or even completely dense which greatly limits the effectiveness of sparse matrix techniques for directly solving the above system of equations in the literature on interior point methods for linear programming the usual technique for dealing with this problem is to split a into a sparse part s and a dense part d a s d and to solve systems involving aat in terms of the solution of systems involving sst using either conjugate gradient techniques or the sherman morrison woodbury formula this approach has the difficulty that sst is often rank deficient even when aat has full rank in this paper we propose an alternative method which avoids the rank deficiency problem and at the same time allows for the effective use of sparse matrix techniques the resulting algorithm is both efficient and robust

on the number of iterations of karmarkar s algorithm for linear programming karmarkar s algorithm for linear programming was published in 1984 and it is highly important to both theory and practice on the practical side some of its variants have been found to be far more efficient than the simplex method on a wide range of very large calculations while its polynomial time properties are fundamental to research on complexity these properties depend on the fact that each iteration reduces a potential function by an amount that is bounded away from zero the bound being independent of all the coefficients that occur it follows that under mild conditions on the initial vector of variables the number of iterations that are needed to achieve a prescribed accuracy in the final value of the linear objective function is at most a multiple ofn wheren is the number of inequality constraints by considering a simple example that allowsn to be arbitrarily large we deduce analytically that the magnitude of this complexity bound is correct specifically we prove that the solution of the example by karmarkar s original algorithm can require aboutn 20 iterations further we find that the algorithm makes changes to the variables that are closely related to the steps of the simplex method

an implementation of karmarkar s algorithm for linear programming this paper describes the implementation of power series dual affine scaling variants of karmarkar s algorithm for linear programming based on a continuous version of karmarkar s algorithm two variants resulting from first and second order approximations of the continuous trajectory are implemented and tested linear programs are expressed in an inequality form which allows for the inexact computation of the algorithm s direction of improvement resulting in a significant computational advantage implementation issues particular to this family of algorithms such as treatment of dense columns are discussed the code is tested on several standard linear programming problems and compares favorably with the simplex codeminos 4 0

properties of the log barrier function on degenerate nonlinear programs we examine the sequence of local minimizers of the log barrier function for a nonlinear program near a solution at which second order sufficient conditions and the mangasarian fromovitz constraint qualification are satisfied but the active constraint gradients are not necessarily linearly independent when a strict complementarity condition is satisfied we show uniqueness of the local minimizer of the barrier function in the vicinity of the nonlinear program solution and we obtain a semiexplicit characterization of this point when strict complementarity does not hold we obtain several other interesting characterizations in particular an estimate of the distance between the minimizers of the barrier function and the nonlinear program in terms of the barrier parameter and a result about the direction of approach of the sequence of minimizers of the barrier function to the nonlinear programming solution

solving combinatorial optimization problems using karmakar s algorithm we describe a cutting plane algorithm for solving combinatorial optimization problems the primal projective standard form variant of karmarkar s algorithm for linear programming is applied to the duals of a sequence of linear programming relaxations of the combinatorial optimization problem

continuous robust control design for nonlinear uncertain systems without a priori knowledge of control direction in this paper a robust control scheme is proposed for a class of nonlinear systems that have not only additive nonlinear uncertainties but also unknown multiplicative signs these signs are called control directions since they represent effectively the direction of motion under any given control except for the unknown control directions the class of systems satisfy the generalized matching conditions nonlinear robust control is designed to identify on line unknown control directions and to guarantee global stability of uniform ultimate boundedness without the knowledge of nonlinear dynamics except their size bounding functions it is also shown that the proposed robust control can be made continuous through utilizing the so called shifting laws that change smoothly and accordingly the signs of robust controls and that no matter what time constants and gains of the shifting laws are global stability is always ensured the analysis and design is done using lyapunov s direct method

a delta mywe algorithm for parameter estimation of noisy ar processes we develop a delta operator based modified yule walker equation algorithm mywe for parameter estimation of a noisy autoregressive ar process the methodology in developing this new algorithm is similar to the previous works on pure ar processes computer simulation results are given to show the improvement of performance in estimating ar parameters in white noise over the q operator mywe algorithm

the convergence of an interior point method using modified search directions in final iterations abstract we provide an asymptotic analysis of a primal dual algorithm for linear programming that uses modified search directions in the final iterations the algorithm determines the search directions by solving the normal equations using the preconditioned conjugate gradient algorithm small dual slack variables are slightly perturbed in the later stage of the interior point algorithm to obtain better conditioned systems without interfering with convergence the modification and its motivation are discussed and a convergence analysis of the resulting algorithm is presented the analysis shows the iterates of the modified system converge to the solution of the karush kuhn tucker optimality system associated with the lagrangian of the logarithmic barrier subproblem the global convergence of the interior point method is thus established

effects of finite precision arithmetic on interior point methods for nonlinear programming we show that the effects of finite precision arithmetic in forming and solving the linear system that arises at each iteration of primal dual interior point algorithms for nonlinear programming are benign provided that the iterates satisfy centrality and feasibility conditions of the type usually associated with path following methods when we replace the standard assumption that the active constraint gradients are independent by the weaker mangasarian fromovitz constraint qualification rapid convergence usually is attainable even when cancellation and roundoff errors occur during the calculations in deriving our main results we prove a key technical result about the size of the exact primal dual step this result can be used to modify existing analysis of primal dual interior point methods for convex programming making it possible to extend the superlinear local convergence results to the nonconvex case

quadratic convergence in a primal dual method we show that the mizuno todd ye o nl iteration predictor corrector primal dual interior point algorithm for linear programming is quadratically convergent our proof does not assume that the problems be nondegenerate we do not assume that the iterate generated by the algorithm be convergent an assumption common to all previous asymptotic convergence analysis

an iterative learning control law for dynamical systems abstract this paper is concerned with an iterative learning control law which enables us to find a control input that generates the desired output exactly over a finite time interval through the repetition of trials we derive a sufficient condition for nonlinear systems to achieve the desired output by the iterative learning control based on this result we show that the direct transmission term of the plant plays a crucial role in the error convergence of the learning process further we identify the class of plants to which the learning control law is applicable

maximum likelihood estimation of delta domain model parameters from noisy output signals fast sampling is desirable to describe signal transmission through wide bandwidth systems the delta operator provides an ideal discrete time modeling description for such fast sampled systems however the estimation of delta domain model parameters is usually biased by directly applying the delta transformations to a sampled signal corrupted by additive measurement noise this problem is solved here by expectation maximization where the delta transformations of the true signal are estimated and then used to obtain the model parameters the method is demonstrated on a numerical example to improve on the accuracy of using a shift operator approach when the sample rate is fast

computational results of an interior point algorithm for large scale linear programming this paper gives computational results for an efficient implementation of a variant of dual projective algorithm for linear programming the implementation uses the preconditioned conjugate gradient method for computing projections our computational experience reported in this paper indicates that this algorithm has potential as an alternative for solving very large lps in which the direct methods fail due to memory and cpu time requirements the conjugate gradient algorithm was able to find very accurate directions even when the system was ill conditioned the paper also discusses a new mathematical technique called the reciprocal estimates for estimating the primal variables we have conducted extensive computational experiments on problems representative of large classes of applications of current interest we have also chosen instances of the problems of future potential interest which could not be solved in the past due to the weakness of the prior solution methods but which represent a large class of new applications the hypergraph model is such an example comparison of our implementation with minos 5 1 shows that our implementation is orders of magnitude faster than minos 5 1 for these problems

regularization techniques in interior point methods regularization techniques i e modifications on the diagonal elements of the scaling matrix are considered to be important methods in interior point implementations so far regularization in interior point methods has been described for linear programming problems in which case the scaling matrix is diagonal it was shown that by regularization free variables can be handled in a numerically stable way by avoiding column splitting that makes the set of optimal solutions unbounded regularization also proved to be efficient for increasing the numerical stability of the computations during the solutions of ill posed linear programming problems in this paper we study the factorization of the augmented system arising in interior point methods in our investigation we generalize the methods developed and used in linear programming to the case when the scaling matrix is positive semidefinite but not diagonal we show that regularization techniques may be applied beyond the linear programming case

a levinson type algorithm for modeling fast sampled data the standard discrete time autoregressive model is poorly suited for modeling series obtained by sampling continuous time processes at fairly rapid rates large computational errors can occur when the levinson algorithm is used to estimate the parameters of this model because the toeplitz covariance matrix is ill suited for inversion an alternative model is developed based on an incremental difference operator rather than the shift operator it is shown that as the sampling period goes to zero unlike the standard autoregressive parameters the coefficients of this model converge to certain parameters that depend directly on the statistics of the continuous time process a levinson type algorithm for efficiently estimating the parameters of this model is derived numerical examples are given to show that when the sampling interval is small this algorithm is considerably less sensitivity to arithmetic roundoff errors than the levinson algorithm

lattice filters for rls estimation of a delta operator based model a lattice filter algorithm is derived for the recursive least squares rls estimation of the parameters of the input output model of a plant based on the delta operator formulation the lattice derivation presented allows a numerically efficient implementation particularly for large orders the order recursive property of the lattices is ideally suited for order determination techniques and applications where the order of the plant may change during operation an example concerning parameter identification for a fourth order plant with measurement noise is given

on implementing mehrotra s predictor corrector interior point method for linear programming mehrotra tech report 90 03 department of industrial engineering and management sciences northwestern university evanston il 1990 recently described a predictor corrector variant of the primal dual interior point algorithm for linear programming this paper describes a full implementation of this algorithm with extensions for solving problems with free variables and problems with bounds on primal variables computational results on the netlib test set are given to show that this new method almost always improves the performance of the primal dual algorithm and that the improvement increases dramatically as the size and complexity of the problem increases a numerical instability in using schur complements to remove dense columns is identified and a numerical remedy is given

product form cholesky factorization in interior point methods for second order cone programming second order cone programming socp problems are typically solved by interior point methods as in linear programming lp interior point methods can in theory solve socps in polynomial time and can in practice exploit sparsity in the problem data specifically when cones of large dimension are present the density that results in the normal equations that are solved at each iteration can be remedied in a manner similar to the treatment of dense columns in an lp here we propose a product form cholesky factorization pfcf approach and show that it is more numerically stable than the alternative sherman morrison woodbury approach we derive several pfcf variants and compare their theoretical perfomance finally we prove that the elements of l in the cholesky factorizations ldlt that arise in interior point methods for socp are uniformly bounded as the duality gap tends to zero as long as the iterates remain is some conic neighborhood of the cental path

primal dual methods for linear programming many interior point methods for linear programming are based on the properties of the logarithmic barrier function after a preliminary discussion of the convergence of the primal projected newton barrier method three types of barrier method are analyzed these methods may be categorized as primal dual and primal dual and may be derived from the application of newton s method to different variants of the same system of nonlinear equations a fourth variant of the same equations leads to a new primal dual method in each of the methods discussed convergence is demonstrated without the need for a nondegeneracy assumption or a transformation that makes the provision of a feasible point trivial in particular convergence is established for a primal dual algorithm that allows a different step in the primal and dual variables and does not require primal and dual feasibility finally a new method for treating free variables is proposed

modified cholesky factorizations in interior point algorithms for linear programming we investigate a modified cholesky algorithm typical of those used in most interior point codes for linear programming cholesky based interior point codes are popular for three reasons their implementation requires only minimal changes to standard sparse cholesky algorithms allowing us to take full advantage of software written by specialists in that area they tend to be more efficient than competing approaches that use alternative factorizations and they perform robustly on most practical problems yielding good interior point steps even when the coefficient matrix of the main linear system to be solved for the step components is ill conditioned we investigate this surprisingly robust performance by using analytical tools from matrix perturbation theory and error analysis illustrating our results with computational experiments finally we point out the potential limitations of this approach

modelling and identification of non linear deterministic systems in the delta domain this paper provides a formulation for using the delta operator in the modelling of non linear systems it is shown that a unique representation of a deterministic non linear auto regressive with exogenous input narx model can be obtained for polynomial basis functions using the delta operator and expressions are derived to convert between the shift and delta domain a delta narx model is applied to the identification of a test problem a van der pol oscillator a comparison is made with the standard shift operator non linear model and it is demonstrated that the delta domain approach improves the numerical properties of structure detection leads to a parsimonious description and provides a model that is closely linked to the continuous time non linear system in terms of both parameters and structure

technical communique limit or limit superior observations on the convergence of some iterative learning control schemes in this paper we survey the convergence properties of a number of iterative learning control schemes from the past decade we observe a common technical error that appeared in these papers point out how the results should be corrected and finally consider the significance of these corrections with regard to the effectiveness of the schemes concerned

a structural view of asymptotic convergence speed of adaptive iir filtering algorithms i infinite precision implementation for pt i see ibid vol 41 no 4 p 1493 1517 1993 finite precision fp implementation is the ultimately inevitable reality of all adaptive filters including adaptive infinite impulse response iir filters this paper continues to examine the asymptotic convergence speed of adaptive iir filters of various structures and algorithms including the simple constant gain type and the newton type but under fp implementation a stochastic differential equation sde approach is used in the analysis such an approach not only greatly simplifies the fp analysis which is traditionally very involved algebraically but it also provides valuable information about the first order as well as the second order moments that the latter are not available using the ordinary differential equation ode approach the asymptotic convergence speed as well as the convergent values of the pertinent moments of fp errors are examined in terms of unknown system pole zero locations the adverse effects of lightly damped low frequency ldlf poles resulting from fast sampling on the local transient and convergent behavior of various structures and algorithms are analyzed and compared the new results agree with the existing ones when reduced to the finite impulse response fir case in particular the explosive behavior of pertinent error variances of newton type iir algorithms when the forgetting factor spl lambda 1 is also concluded computer simulation verifies the predicted theoretical results

a note on the computation of an orthonormal basis for the null space of a matrix a highly regarded method to obtain an orthonormal basis z for the null space of a matrix a t is the qr decomposition of a where q is the product of householder matrices in several optimization contexts a x varies continuously with x and it is desirable the z x vary continuously also in this note we demonstrate that the standard implementation of the qr decomposition does not yield an orthonormal basis z x whose elements vary continuously with x we suggest three possible remedies

kernel independent component analysis we present a class of algorithms for independent component analysis ica which use contrast functions based on canonical correlations in a reproducing kernel hilbert space on the one hand we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence on the other hand building on recent developments in kernel methods we show that these criteria and their derivatives can be computed efficiently minimizing these criteria leads to flexible and robust algorithms for ica we illustrate with simulations involving a wide variety of source distributions showing that our algorithms outperform many of the presently known algorithms

minimizing the peak to average power ratio of ofdm signals using convex optimization the main disadvantage of orthogonal frequency division multiplexing ofdm is the high time domain peak to average power ratio par that limits transmitter power efficiency assuming no changes in receiver structure the transmitter can only reduce par by distorting the data carriers or by adding power to the free carriers this paper shows that the ofdm signal with globally minimum par subject to constraints on the allowed constellation error and the free carrier power can be efficiently computed using convex optimization simulation results are presented for the 802 11a g wireless local area network standard the globally minimum par subject to the constellation error constraint ranges from 0 7 db for 6 mbps binary phase shift keying to 4 1 db for 54 mbps 64 qam tradeoff analysis shows that the free carrier power can be drastically reduced by backing off from this globally minimum par by less than 1 db a customized interior point method ipm is derived for solving the ofdm optimization problem the ipm reaches the desired tradeoff between par and free carrier power within two iterations where the main computational complexity per iteration is four fast fourier transforms plus the solution of a linear system of equations the customized ipm is about 100 times faster than a general purpose optimization algorithm

efficient svm training using low rank kernel representations svm training is a convex optimization problem which scales with the training set size rather than the feature space dimension while this is usually considered to be a desired quality in large scale problems it may cause training to be impractical the common techniques to handle this difficulty basically build a solution by solving a sequence of small scale subproblems our current effort is concentrated on the rank of the kernel matrix as a source for further enhancement of the training procedure we first show that for a low rank kernel matrix it is possible to design a better interior point method ipm in terms of storage requirements as well as computational complexity we then suggest an efficient use of a known factorization technique to approximate a given kernel matrix by a low rank matrix which in turn will be used to feed the optimizer finally we derive an upper bound on the change in the objective function value based on the approximation error and the number of active constraints support vectors this bound is general in the sense that it holds regardless of the approximation method

on the convergence of the iteration sequence in primal dual interior point methods recently numerous research efforts most of them concerned with superlinear convergence of the duality gap sequence to zero in the kojima mizuno yoshise primal dual interior point method for linear programming have as a primary assumption the convergence of the iteration sequence yet except for the case of nondegeneracy uniqueness of solution the convergence of the iteration sequence has been an important open question now for some time in this work we demonstrate that for general problems under slightly stronger assumptions than those needed for superlinear convergence of the duality gap sequence except of course the assumption that the iteration sequence converges the iteration sequence converges hence we have not only established convergence of the iteration sequence for an important class of problems but have demonstrated that the assumption that the iteration sequence converges is redundant in many of the above mentioned works

a universal adaptive stabilizer for a class of nonlinear systems abstract an adaptive stabilizer is described which is universal for a class of single input nonlinear systems modelled as a controlled differential inclusion on r n the stabilizer has a discontinous feedback structure and incorporates a gain of nussbaum type the analysis draws on an extension to differential inclusions of lasalle s principle for ordinary differential equations

a primal dual method for large scale image reconstruction in emission tomography in emission tomography images can be reconstructed from a set of measured projections using a maximum likelihood ml criterion in this paper we present a primal dual algorithm for large scale three dimensional image reconstruction the primal dual method is specialized to the ml reconstruction problem the reconstruction problem is extremely large in several of our data sets the hessian of the objective function is the product of a 1 4 million by 63 million matrix and its scaled transpose as such we consider only approaches that are suitable for large scale parallel computation we apply a stabilization technique to the system of equations for computing the primal direction and demonstrate the need for stabilization when approximately solving the system using an early terminated conjugate gradient iteration we demonstrate that the primal dual method for this problem converges faster than the logarithmic barrier method and considerably faster than the expectation maximization algorithm the use of extrapolation in conjunction with the primal dual method further reduces the overall computation required to achieve convergence

on modified factorizations for large scale linearly constrained optimization we consider the algebraic issues concerning the solution of general large scale linearly constrained nonlinear optimization problems particular attention is given to suitable methods for solving the linear systems that occur at each iteration of such methods the main issue addressed is how to ensure that a quadratic model of the objective function is positive definite in the null space of the constraints while neither adversely affecting the convergence of newton s method nor incurring a significant computational overhead numerical evidence to support the theoretical developments is provided

an augmented lagrangian interior point method using directions of negative curvature we describe an efficient implementation of an interior point algorithm for non convex problems that uses directions of negative curvature these directions should ensure convergence to second order kkt points and improve the computational efficiency of the procedure some relevant aspects of the implementation are the strategy to combine a direction of negative curvature and a modified newton direction and the conditions to ensure feasibility of the iterates with respect to the simple bounds the use of multivariate barrier and penalty parameters is also discussed as well as the update rules for these parameters we analyze the convergence of the procedure both the linesearch and the update rule for the barrier parameter behave appropriately as the main goal of the paper is the practical usage of negative curvature a set of numerical results on small test problems is presented based on these results the relevance of using directions of negative curvature is discussed

adaptive control of a simple nonlinear system without a priori information on the plant parameters the authors present an adaptive control scheme for nonlinear systems of the form x c sup t f x b u where f x is lipschitz c is a constant vector and b is a constant scalar the control scheme achieves asymptotical model matching without a priori knowledge of the sign of the b gain the adaptive scheme is free from singularities in the sense that the estimate of b entering in the denominator of the control law is bounded away from zero the singularity has been overcome through a suitable modification of the parameter estimates which is based on standard least squares covariance matrix properties

avoiding numerical cancellation in the interior point method for solving semidefinite programs the matrix variables in a primal dual pair of semidefinite programs are getting increasingly ill conditioned as they approach a complementary solution multiplying the primal matrix variable with a vector from the eigenspace of the non basic part will therefore result in heavy numerical cancellation this effect is amplified by the scaling operation in interior point methods a complete example illustrates these numerical issues in order to avoid numerical problems in interior point methods we propose to maintain the matrix variables in a cholesky form we discuss how the factors of the v space cholesky form can be updated after a main iteration of the interior point method with nesterov todd scaling an analogue for second order cone programming is also developed numerical results demonstrate the success of this approach

brief paper a universal iterative learning stabilizer for a class of mimo systems design of iterative learning control ilc often requires some prior knowledge about a system s control matrix in some applications such as uncalibrated visual servoing this kind of knowledge may be unavailable so that a stable learning control cannot always be achieved in this paper a universal ilc is proposed for a class of multi input multi output mimo uncertain nonlinear systems with no prior knowledge about the system control gain matrix it consists of a gain matrix selector from the unmixing set and a learned compensator in a form of the positive definite discrete matrix kernel corresponding to rough gain matrix probing and refined uncertainty compensating respectively asymptotic convergence for a trajectory tracking within a finite time interval is achieved through repetitive tracking simulations and experiments of uncalibrated visual servoing are carried out in order to verify the validity of the proposed control method

optimizing over three dimensional subspaces in an interior point method for linear programming abstract interior point algorithms for solving linear programming problems are considered a three dimensional method is developed that at each iteration solves a subproblem based on minimizing the cost function on low dimensional cross sections of the feasible region the generators for the three dimensional subproblem include the dual affine search direction and two higher order search directions one of the higher order directions is a third order correction to the newton recentering direction and the other is a correction to the dual affine direction that is motivated by the use of rank one updates of the second derivative information numerical results are presented for this method that indicate a nearly 20 reduction in cpu time compared to our best dual affine implementation

a delta least squares lattice algorithm for fast sampling most shift operator based adaptive algorithms exhibit poor numerical behavior when the input discrete time process is obtained from a continuous time process by fast sampling this includes the shift operator based least squares lattice algorithm we develop a delta least squares lattice algorithm this algorithm has a low computational complexity compared with the delta levinson rls algorithm and shows better numerical properties compared with the shift least squares lattice algorithm under fast sampling computer simulations show that the new algorithm also outperforms an existing delta least squares lattice algorithm

convergence properties of an augmented lagrangian algorithm for optimization with a combination of general equality and linear constraints we consider the global and local convergence properties of a class of augmented lagrangian methods for solving nonlinear programming problems in these methods linear and more general constraints are handled in different ways the general constraints are combined with the objective function in an augmented lagrangian the iteration consists of solving a sequence of subproblems in each subproblem the augmented lagrangian is approximately minimized in the region defined by the linear constraints a subproblem is terminated as soon as a stopping condition is satisfied the stopping rules that we consider here encompass practical tests used in several existing packages for linearly constrained optimization our algorithm also allows different penalty parameters to be associated with disjoint subsets of the general constraints in this paper we analyze the convergence of the sequence of iterates generated by such an algorithm and prove global and fast linear convergence as well as show that potentially troublesome penalty parameters remain bounded away from zero

affine scaling for linear programs with free variables the affine scaling modification of karmarkar s algorithm is extended to solve problems with free variables this extended primal algorithm is used to prove two important results first the geometrically elegant feasibility algorithm proposed by chandru and kochar is the same algorithm as the one obtained by appending a single column of residuals to the constraint matrix second the dual algorithm as first described by adler et al is the same as the extended primal algorithm applied to the dual

solution of and bounding in a linearly constrained optimization problem with convex polyhedral objective function a dual method is presented to solve a linearly constrained optimization problem with convex polyhedral objective function along with a fast bounding technique for the optimum value the method can be used to solve problems obtained from lps where some of the constraints are not required to be exactly satisfied but are penalized by piecewise linear functions which are added to the objective function of the original problem the method generalizes an earlier solution technique developed by prekopa 1990 applications to stochastic programming are also presented

universal stabilisation design for uncertain nonlinear system with sign switching unknown control direction a universal robust control approach is proposed to stabilise a class of nonlinear systems the system contains both nonlinear dynamics uncertainty and an unknown control direction which is the multiplier of the control term being different from most previous studies the unknown control direction is allowed to switch its sign in this article a new nussbaum gain technique is designed and integrated with robust controller to tackle the sign switching unknown control direction it is proven that the proposed control approach can yield asymptotic stabilisation and guarantee the boundedness of the closed loop signals in addition these results can be readily extended to tracking problems under a different assumption

brief parameter identification with derivative shift operator parametrization a way of parametrizing linear systems using an operator which combines the two commonly used operators the shift q and the delta d operators is proposed it is shown that the proposed parametrization improves the numerical property of parameter identification process by using the proposed operator the condition number of the information matrix in the identification algorithm is significantly lower than that using the q or d operator it is also shown that accurate and fast converging parameter identification can be achieved by using the new operator at fast sampling rates especially for systems with relatively wide bandwidth the numerical superiority of the new parametrization is demonstrated by simulation results

j lossless and extended j lossless factorizations approach for domain h control the paper addresses problems of numerical conditioning of the discrete time h x control design based on j lossless factorizations of the chain scattering representation of a plant it is demonstrated that for a sufficiently small sampling period forward shift operator techniques may become ill conditioned and numerical robustness and reliability of design procedures can be significantly improved by utilizing a so called i operator form of the source problem state space models of all i domain controllers are given the solutions are obtained by considering two coupled i domain algebraic riccati equations that can be solved by applying a generalized eigenproblem formulation numerical conditioning of these solutions can effectively be evaluated by using a suitable relative condition number an extended j lossless factorization for generalized plants with zeros on the stability boundary circle is also given a so called zero compensator technique adapted to the i domain models is utilized to cancel suc

a roundoff error analysis of the lms adaptive algorithm the steady state output error of the least mean square lms adaptive algorithm due to the finite precision arithmetic of a digital processor is analyzed it is found to consist of three terms 1 the error due to the input data quantization 2 the error due to the rounding of the arithmetic operations in calculating the filter s output and 3 the error due to the deviation of the filter s coefficients from the values they take when infinite precision arithmetic is used the last term is of paricular interest because its mean squared value is inversely proportional to the adaptation step size both fixed and floating point arithmetics are examined and the expressions for the final mean square error are found to be similar the relation between the quantization error and the error that occurs when adaptation possibly ceases due to quantization is also investigated

a product form cholesky factorization method for handling dense columns in interior point methods for linear programming cholesky factorization has become the method of choice for solving the symmetric system of linear equations arising in interior point methods ipms for linear programming lp its main advantages being numerical stability and efficiency for sparse systems however in the presence of dense columns there is dramatic loss of efficiency a typical approach to remedy this problem is to apply the sherman morrison woodbury smw update formula to the dense part this approach while being very efficient is not numerically stable here we propose using product form cholesky factorization to handle dense columns the proposed approach is essentially as stable as the original cholesky factorization and nearly as efficient as the smw approach we demonstrate these properties both theoretically and computationally a key part of our theoretical analysis is the proof that the elements of the cholesky factors of the matrices that arise in ipms for lp are uniformly bounded as the duality gap converges to zero

a composite energy function based learning control approach for nonlinear systems with time varying parametric uncertainties a new learning control approach is developed in this note to address a class of nonlinear systems with time varying parametric uncertainties the concept of composite energy function cef which provides the system information along both time and learning repetition horizons is introduced in the analysis of learning control cef consists of two parts the first part is a standard lyapunov function which is used to access system behavior along time horizon during each learning cycle the second part is an l sup 2 norm of parametric learning errors which reflects the variation of the system status when the control system is updated on the basis of learning cycles the proposed learning control algorithm achieves asymptotical convergence along a learning repetition horizon at the same time the boundedness and pointwise convergence of the tracking error along time horizon is guaranteed the proposed learning control strategy is applicable to quite general classes of nonlinear systems without requiring the global lipschitz continuity condition and zero relative degree condition

ill conditioning and computational error in interior methods for nonlinear programming ill conditioning has long been regarded as a plague on interior methods but its damaging effects have rarely been documented in fact implementors of interior methods who ignore warnings about the dire consequences of ill conditioning usually manage to compute accurate solutions we offer some insight into this seeming contradiction by analyzing ill conditioning within a primal dual method in which the full usually well conditioned primal dual matrix is transformed to a condensed inherently ill conditioned matrix mpd we show that ill conditioning in the exact condensed matrix closely resembles that known for the primal barrier hessian and then examine the influence of cancellation in the computed constraints using the structure of mpd various bounds are obtained on the absolute accuracy of the computed primal dual steps without cancellation the portion of the computed x step in the small space of mpd a subspace close to the null space of the jacobian of the active constraints has an absolute error bound comparable to machine precision and its large space component has a much smaller error bound with cancellation the usual case the absolute error bounds for both the small and large space components of the computed x step are comparable to machine precision in either case the absolute error bound for the computed multiplier steps associated with active constraints is comparable to machine precision the computed multiplier steps for inactive constraints although converging to zero retain approximately full relative precision because of errors in forming the right hand side the absolute error in the computed solution of the full well conditioned primal dual system is shown to be comparable to machine precision thus under quite general conditions ill conditioning in mpd does not noticeably impair the accuracy of the computed primal dual steps a similar analysis applies to search directions obtained by direct solution of the primal newton equations

the cholesky factorization in interior point methods the paper concerns the cholesky factorization of symmetric positive definite matrices arising in interior point methods our investigation is based on a property of the cholesky factorization which interprets small diagonal values during factorization as degeneracy in the scaled optimization problem a practical scaling independent technique based on the above property is developed for the modified cholesky factorization of interior point methods this technique increases the robustness of cholesky factorizations performed during interior point iterations when the optimization problem is degenerate our investigations show also the limitations of interior point methods with the recent implementation technology and floating point arithmetic standard we present numerical results on degenerate linear programming problems of netlib

design of a multichannel two dimensional delta domain lattice filter for noise removal in this paper a multichannel form of the two dimensional 2 d delta domain lattice filter has been designed motivated by the success of 2 d delta domain lattice filters delta domain concepts are generalized to the multichannel case by a good choice of the transformation matrix that relates the q domain and delta domain filters the channels can be handled independently which makes the algorithm simple and attractive however it is capable of handling multichannel problems even if the noise is correlated between channels resulting in extremely low snr values the proposed algorithm provides the ability of handling multichannel image frames and colored images in the so called delta domain that is known to be always better than the traditional q domain in handling ill conditioned problems on line image frames carrying gaussian and multiplicative noise simultaneously on different channels as well as color images are taken into consideration and successful results are obtained in terms of noise removal

stabilization of mehrotra s primal dual algorithm and its implementation abstract in this paper we apply a stabilization procedure proposed by kovacevic vujcic and asic to the mehrotra s primal dual interior point algorithm for linear programming transformations of the dual problem corresponding to the stabilization procedure are considered the stabilization procedure and mehrotra s algorithm are implemented in the package mathematica a number of highly degenerate test examples are used to compare the modified mehrotra s method with respect to the original one and the codes pcx lipsol and mosek we also provide numerical results on some examples from the netlib test set

universal repetitive learning control for nonparametric uncertainty and unknown state dependent control direction matrix we propose a continuous universal repetitive learning control to track periodic trajectory for a class of nonlinear dynamical systems with nonparametric uncertainty and unknown state dependent control direction matrix the proposed controller is an integration of high gain feedback repetitive learning and nussbaum gain matrix selector the control signal is always continuous thus it avoids the potential chattering effect caused by discontinuity asymptotic convergence of the tracking error is achieved by the controller and the control performance is illustrated by simulation although the proposed method is derived for input state systems it can be readily extended to multi input multi output systems under appropriate assumption

on a wide region of centers and primal dual interior point algorithms for linear programming in the adaptive step primal dual interior point method for linear programming polynomial algorithms are obtained by computing newton directions towards targets on the central path and restricting the iterates to a neighborhood of this central path in this paper the adaptive step methodology is extended by considering targets in a certain central region which contains the usual central path and subsequently generating iterates in a neighborhood of this region the size of the central region can vary from the central path to the whole feasible region by choosing a certain parameter an o nl iteration bound is obtained under mild conditions on the choice of the target points in particular we leave plenty of room for experimentation with search directions the practical performance of the new primal dual interior point method is measured on part of the netlib test set for various sizes of the central region

delta levinson and schur type rls algorithms for adaptive signal processing in this paper we develop delta operator based levinson and schur type on line rls algorithms such algorithms have the potential of improved numerical behavior for ill conditioned input data these new algorithms are obtained by a unified transformation on the existing q operator based ones we first show that the conventional lattice structure can be naturally derived when the backward delta operator is used with this operator levinson and schur algorithms for the stationary stochastic model in q domain can easily be transformed into the delta domain then same transformation will be applied to the q domain on line levinson and schur type rls algorithms to obtain the delta domain counterparts their normalized versions as well as a systolic array architecture implementing the new delta schur rls algorithm are proposed extension to the equal length multichannel case is also given computer simulations show the expected numerical advantages of the delta based algorithms for fast sampled data in real time over the q domain ones under finite precision implementation

ilc for networked nonlinear systems with unknown control direction through random lossy channel abstract the iterative learning control is constructed for the discrete time networked nonlinear systems with random measurement losses and unknown control direction which have not been studied simultaneously in literature differing from the conventional bernoulli random variable model the random packet loss is modeled by an arbitrary stochastic sequence with bounded length requirement which is a new model of realistic packet losses a novel regulating approach based on truncations is introduced to make the proposed algorithm find the correct control direction adaptively and then guarantee the almost sure convergence property therefore this paper has three major innovations compared with reported studies namely the stochastic sequence model of packet loss the novel control direction regulation method and the almost sure convergence property of the proposed algorithm an illustrative example shows the effectiveness of the proposed approach

brief iterative learning control with initial rectifying action this paper addresses the initial shift problem in iterative learning control with system relative degree the tracking error caused by nonzero initial shift is detected when applying a conventional learning algorithm finite initial rectifying action is introduced in the learning algorithm and is shown effective in the improvement of tracking performance in particular robustness with respect to variable initial shifts the uniform convergence of the output trajectory to a desired one jointed smoothly with a specified transient trajectory from the starting position is ensured in the presence of fixed initial shift

robust maximum likelihood estimation of multivariable dynamic systems this paper examines the problem of estimating linear time invariant state space system models in particular it addresses the parametrization and numerical robustness concerns that arise in the multivariable case these difficulties are well recognised in the literature resulting for example in extensive study of subspace based techniques as well as recent interest in data driven local co ordinate approaches to gradient search solutions the paper here proposes a different strategy that employs the expectation maximisation em technique the consequence is an algorithm that is iterative with associated likelihood values that are locally convergent to stationary points of the gaussian likelihood function furthermore theoretical and empirical evidence presented here establishes additional attractive properties such as numerical robustness avoidance of difficult parametrization choices the ability to naturally and easily estimate non zero initial conditions and moderate computational cost moreover since the methods here are maximum likelihood based they have associated known and asymptotically optimal statistical properties

formulating two stage stochastic programs for interior point methods this paper describes an approach for modeling two stage stochastic programs that yields a form suitable for interior point algorithms a staircase constraint structure is created by replacing first stage variables with sparse split variables in conjunction with side constraints dense columns are thereby eliminated the resulting model is larger than traditional stochastic programs but computational savings are substantial over a tenfold improvement for the problems tested a series of experiments with stochastic networks drawn from financial planning demonstrates the attained efficiencies comparisons with minos and the dual block angular stochastic programming model are provided as benchmarks the split variable approach is applicable to general two stage stochastic programs and other dual block angular models

on finding a vertex solution using interior point methods abstract an approach is proposed to generate a vertex solution while using a primal dual interior point method to solve linear programs a controlled random perturbation is made to the cost vector a method to identify the active constraints at the vertex to which the solutions are converging is given this basic method is further refined to save computational effort the proposed approach is tested by using a variation of the primal dual interior point method our method is developed by taking a predictor corrector approach in practice this method takes considerably fewer iterations to solve linear programs than methods described by choi monma and shanno lustig marsten and shanno and domich boggs donaldson and witzgall computational results on problems from the netlib test set are reported to test our approach for finding vertex solutions these results show that one perturbation is enough to force the solutions to converge to a vertex the results indicate that the proposed approach is insensitive to the number of degenerate variables the results also indicate that the effort required to generate a vertex solution is comparable to that required to solve the problem using an interior point method

an iterative learning controller with initial state learning in iterative learning control ilc a common assumption is that the initial states in each repetitive operation should be inside a given ball centred at the desired initial states which may be unknown this assumption is critical to the stability analysis and the size of the ball will directly affect the final output trajectory tracking errors in this paper this assumption is removed by using an initial state learning scheme together with the traditional d type ilc updating law both linear and nonlinear time varying uncertain systems are investigated uniform bounds for the final tracking errors are obtained and these bounds are only dependent on the system uncertainties and disturbances yet independent of the initial errors furthermore the desired initial states can be identified through learning iterations

on properties of information matrices of delta operator based adaptive signal processing algorithms in this paper we study the properties of information matrices of the difference operator or the so called delta spl delta operator based algorithms for adaptive signal processing we show that the conditioning of a transformed information matrix in the spl delta domain is always better than that of the original information matrix in the conventional q domain for ill conditioned problems the results obtained in this paper give an explanation of the advantages of using delta operator algorithms for adaptive signal processing that have been developed the analysis in this paper also helps to clarify the problem about the effect of spl delta in the delta operator algorithms and justify the use of sampling interval as spl delta in most cases when the ill conditioning is caused by fast sampling of continuous time systems

technical communique iterative learning control design without a priori knowledge of the control direction in this work we explore the possibility of designing a new iterative learning control scheme for systems without a priori knowledge of the control direction by incorporating a nussbaum type function a new learning control mechanism is constructed with both differential and difference updating laws the new learning control mechanism can warrant a l t 2 convergence of the tracking error sequence along the iteration axis in the presence of time varying parametric uncertainties and local lipschitz nonlinearities

a polynomial time primal dual affine scaling algorithm for linear and convex quadratic programming and its power series extension we describe an algorithm for linear and convex quadratic programming problems that uses power series approximation of the weighted barrier path that passes through the current iterate in order to find the next iterate if r 1 is the order of approximation used we show that our algorithm has time complexity o n 1 1 r l 1 1 r iterations and o n3 n2r arithmetic operations per iteration where n is the dimension of the problem and l is the size of the input data when r 1 we show that the algorithm can be interpreted as an affine scaling algorithm in the primal dual setup

the interior point revolution in optimization history recent developments and lasting consequences interior methods are a pervasive feature of the optimization landscape today but it was not always so although interior point techniques primarily in the form of barrier methods were widely used during the 1960s for problems with nonlinear constraints their use for the fundamental problem of linear programming was unthinkable because of the total dominance of the simplex method during the 1970s barrier methods were superseded nearly to the point of oblivion by newly emerging and seemingly more efficient alternatives such as augmented lagrangian and sequential quadratic programming methods by the early 1980s barrier methods were almost universally regarded as a closed chapter in the history of optimization this picture changed dramatically in 1984 when narendra karmarkar announced a fast polynomial time interior method for linear programming in 1985 a formal connection was established between his method and classical barrier methods since then interior methods have continued to transform both the theory and practice of constrained optimization we present a condensed unavoidably incomplete look at classical material and recent research about interior methods

a dantzig wolfe like variant of karmarkar s interior point linear programming algorithm we show that a variant of karmarkar s projective algorithm for linear programming can be viewed as following the approach of dantzig wolfe decomposition at each iteration the current primal feasible solution generates prices which are used to form a simple subproblem the solution to the subproblem is then incorporated into the current feasible solution with a suitable choice of stepsize a constant reduction in potential function is achieved at each iteration we also use our analysis to motivate a new primal simplex pivot rule that is closely related to rules used by e klotz and l schrage

exploiting special structure in a primal dual path following algorithm a primal dual path following algorithm that applies directly to a linear program of the form min ctx ax b hx u x 0 x n is presented this algorithm explicitly handles upper bounds generalized upper bounds variable upper bounds and block diagonal structure we also show how the structure of time staged problems and network flow problems can be exploited especially on a parallel computer finally using our algorithm we obtain a complexity bound of o sqrt n ds2 log nk fortransportation problems withs origins d destinations s

conditioning of lms algorithms with fast sampling the lms algorithm is very commonly used in signal processing its convergence properties depend primarily on the step size chosen and the condition number of an information matrix associated with the system in most applications today the lms uses a regression vector based on the shift operator including the ubiquitous tapped delay line we demonstrate that generically when fast sampling is employed these regression vectors lead to poorly conditioned lms by comparison delta operator based regression vectors lend with rapid sampling to improved condition numbers hence to better performance

barrier function based model predictive control a new formulation of nonlinear model predictive control mpc is developed by including a weighted barrier function in the control objective while the barrier ensures that inequality constraints are strictly satisfied it also provides a smooth transition between points in the interior and those on the boundary of the constraint set in addition the resulting optimisation problem to be solved at each control step is effectively unconstrained and thus amenable to elegant optimisation techniques the barrier must satisfy certain conditions in order that the state converges to the origin and we show how to construct such a barrier conventional mpc may be seen as a limiting case of the new class as the barrier weighting itself approaches zero we pay particular attention to the novel approach of fixing the weighting parameter to some positive value possibly large and observe that this provides a degree of controller caution near constraint boundaries we construct an ellipsoidal invariant set by exploiting the geometry of self concordant functions and show nominal closed loop stability for this class of controllers under full state feedback

on scaled projections and pseudoinverses abstract let x be a matrix of full column rank and let d be a diagonal matrix with positive diagonal elements the weighted pseudoinverse defined by x d x t dx 1 x t d and the associated oblique projection p d xx d arise in many applications in this paper we show that the norms of both matrices are bounded by numbers that are independent of d

computational experience with a primal dual interior point method for linear programming abstract a new comprehensive implementation of a primal dual algorithm for linear programming is described it allows for easy handling of simple bounds on the primal variables and incorporates free variables which have not previously been included in a primal dual implementation we discuss in detail a variety of computational issues concerning the primal dual implementation and barrier methods for linear programming in general we show that in a certain way lustig s method for obtaining feasibility is equivalent to newton s method this demonstrates that the method is in some sense the natural way to reduce infeasibility the role of the barrier parameter in computational practice is studied in detail numerical results are given for the entire expanded netlib test set for the basic algorithm and its variants as well as version 5 3 of minos

hopdm version 2 12 a fast lp solver based on a primal dual interior point method hopdm is an implementation of the primal dual interior point method for solving large scale linear programming lp problems hopdm stands for higher order primal dual method its newest version 2 12 of may 1995 is a robust and efficient code that compares favourably with today s commercial lp solvers

consistent independent component analysis and prewhitening we study the statistical merits of two techniques used in the literature of independent component analysis ica first we analyze the characteristic function based ica method chfica and study its statistical properties such as consistency spl radic n consistency and robustness against small additive noise second we study the validity of prewhitening a preprocessing technique used by many ica algorithms as applied to the chfica method in particular we establish the surprising effectiveness of this technique even when some components have heavy tails and others do not a fast new algorithm implementing the prewhitened chfica method is also provided

some properties of the hessian of the logarithmic barrier function more than twenty years ago murray and lootsma showed that hessian matrices of the logarithmic barrier function become increasingly ill conditioned at points on the barrier trajectory as the solution is approached this paper explores some further characteristics of the barrier hessian we first show that except in two special cases the barrier hessian is ill conditioned in an entire region near the solution at points in a more restricted region including the barrier trajectory itself this ill conditioning displays a special structure connected with subspaces defined by the jacobian of the active constraints we then indicate how a cholesky factorization with diagonal pivoting can be used to detect numerical rank deficiency in the barrier hessian and to provide information about the underlying subspaces without making an explicit prediction of the active constraints using this subspace information a close approximation to the newton direction can be calculated by solving linear systems whose condition reflects that of the original problem