on sizing and shifting the bfgs update within the sized broyden family of secant updates in a numerical study contreras and tapia demonstrated the value of selectively sizing the bfgs secant update a conclusion of the current study is that there is value in following a sizing of the bfgs update with a shifting of the bfgs update our motivation for such a strategy comes from the known fact that the bfgs update is prone to producing updates with large eigenvalues hence sizing may contribute to near singularity and shifting can help compensate for this deficiency we follow contreras and tapia in our form of selective sizing our form of shifting can be viewed as switching from the bfgs update to an alternative member of the broyden family of updates our shift is obtained by minimizing a weighted form of the byrd nocedal measure over a sized broyden family of secant updates numerically our most effective shift is the one that gives the member of the class which is closest to steepest descent in the byrd nocedal measure

a symmetric rank one method based on extra updating techniques for unconstrained optimization abstract in this paper we present a new symmetric rank one sr1 method for the solution of unconstrained optimization problems the proposed method involves an algorithm in which the usual sr1 hessian is updated a number of times in a way to be specified in some iterations to improve the performance of the hessian approximation in particular we discuss how to consider a criterion for indicating at each iteration whether it is necessary to employ extra updates however it is well known that there are some theoretical difficulties when applying the sr1 update even for a current positive definite hessian approximation it is possible that the sr1 update may not be defined or the sr1 update may not preserve positive definiteness at some iterations we then employ a restarting procedure that guarantees that updated matrices will be well defined while preserving positive definiteness of updates numerical results support these theoretical considerations they show that the implementation of the sr1 method using extra updating techniques improves the performance of the sr1 method substantially for a number of test problems from the literature

adaptive non orthogonal mfsk m ary frequency shift keying mfsk using one or more orthogonal tones per symbol has been in use in non coherent systems for quite some time we present an alternate system for non coherent systems using non orthogonal tones spread non uniformly throughout the bandwidth which outperforms the equivalent orthogonal tone system we demonstrate that from a theoretical capacity point of view the same bit rate could be achieved by the proposed system with as much as 2 5 db lower signal to noise ratio snr for some practical configurations we demonstrate as much as 2 db of gain we present the performance with maximum likelihood ml receivers as well as least squares ls receivers as well as lower complexity receivers based on compressed sensing cs we also demonstrate that such a non orthogonal system may be used to increase the spectral efficiency of the overall system by adapting the spectral efficiency in response to the link snr such an adaptive non orthogonal mfsk system requires estimation of the snr or e s n 0 and we present both the cramer rao lower bound crlb as well as maximum likelihood ml estimators which attain the crlb for both data aided da snr estimation as well as non data aided nda snr estimation

global convergence of a cass of quasi newton methods on convex problems we study the global convergence properties of the restricted broyden class of quasi newton methods when applied to a convex objective function we assume that the line search satisfies a standard sufficient decrease condition and that the initial hessian approximation is any positive definite matrix we show global and superlinear convergence for this class of methods except for dfp this generalizes powell s well known result for the bfgs method the analysis gives us insight into the properties of these algorithms in particular it shows that dfp lacks a very desirable self correcting property possessed by bfgs

a tool for the analysis of quasi newton methods with application to unconstrained minimization the bfgs update formula is shown to have an important property that is inde pendent of the algorithmic context of the update and that is relevant to both constrained and unconstrained optimization the bfgs method for unconstrained optimization using a variety of line searches including backtracking is shown to be globally and superlinearly convergent on uniformly convex problems the analysis is particularly simple due to the use of some new tools introduced in this paper

variable metric methods for unconstrainted optimization and nonlinear least squares variable metric or quasi newton methods are well known and commonly used in connection with unconstrained optimization since they have good theoretical and practical convergence properties although these methods were originally developed for small and moderate size dense problems their modifications based either on sparse partitioned or limited memory updates are very efficient on large scale sparse problems very significant applications of these methods also appear in nonlinear least squares approximation and nonsmooth optimization in this contribution we give an extensive review of variable metric methods and their use in various optimization fields

an improved multi step gradient type method for large scale optimization in this paper we propose an improved multi step diagonal updating method for large scale unconstrained optimization our approach is based on constructing a new gradient type method by means of interpolating curves we measure the distances required to parameterize the interpolating polynomials via a norm defined by a positive definite matrix by developing on implicit updating approach we can obtain an improved version of hessian approximation in diagonal matrix form while avoiding the computational expenses of actually calculating the improved version of the approximation matrix the effectiveness of our proposed method is evaluated by means of computational comparison with the bb method and its variants we show that our method is globally convergent and only requires o n memory allocations

on the performance of a new symmetric rank one method with restart for solving unconstrained optimization problems quasi newton qn methods are generally held to be the most efficient minimization methods for solving unconstrained optimization problems among the qn methods symmetric rank one sr1 is one of the very competitive formulas in the present paper we propose a new sr1 method the new technique attempts to improve the quality of the sr1 hessian by employing the scaling of the identity in a certain sense however since at some iterations these updates might be singular indefinite or undefined this paper proposes an updates criterion based on the eigenvalues of the sr1 update to measure this quality hence the new method is employed only to improve the approximation of the sr1 hessian it is shown that the numerical results support the theoretical considerations for the usefulness of this criterion and show that the proposed method improves the performance of the sr1 update substantially

scaled memoryless symmetric rank one method for large scale optimization this paper concerns the memoryless quasi newton method that is precisely the quasi newton method for which the approximation to the inverse of hessian at each step is updated from the identity matrix hence its search direction can be computed without the storage of matrices in this paper a scaled memoryless symmetric rank one sr1 method for solving large scale unconstrained optimization problems is developed the basic idea is to incorporate the sr1 update within the framework of the memoryless quasi newton method however it is well known that the sr1 update may not preserve positive definiteness even when updated from a positive definite matrix therefore we propose the memoryless sr1 method which is updated from a positive scaled of the identity where the scaling factor is derived in such a way that positive definiteness of the updating matrices are preserved and at the same time improves the condition of the scaled memoryless sr1 update under very mild conditions it is shown that for strictly convex objective functions the method is globally convergent with a linear rate of convergence numerical results show that the optimally scaled memoryless sr1 method is very encouraging

fuzzy wavelet neural network models for prediction and identification of dynamical systems this paper presents fuzzy wavelet neural network fwnn models for prediction and identification of nonlinear dynamical systems the proposed fwnn models are obtained from the traditional takagi sugeno kang fuzzy system by replacing the then part of fuzzy rules with wavelet basis functions that have the ability to localize both in time and frequency domains the first and last model use summation and multiplication of dilated and translated versions of single dimensional wavelet basis functions respectively and in the second model then parts of the rules consist of radial function of wavelets gaussian type of activation functions are used in if part of the fuzzy rules a fast gradient based training algorithm i e the broyden fletcher goldfarb shanno method is used to find the optimal values for unknown parameters of the fwnn models simulation examples are also given to compare the effectiveness of the models with the other known methods in the literature according to simulation results we see that the proposed fwnn models have impressive generalization ability

an analogue of a hardy littlewood fejer inequality for upper triangular trace class operators let h denote a separable infinite dimensional hilbert space we shall consider operators that is bounded linear transformations on an operator t is said to be in the schatten class cp if t is compact and if the eigenvalues of t t t 2 are in 1 p 0 p oo the norm in cp is the i p norm of these eigenvalues the class c 2 is called the hilbert schmidt class and c 1 is called the trace class by c o we denote the class of all compact operators for information on these classes see for example 23 4 chap xi 8 chap iii 19 13 although there is an obvious analogy with the sequence spaces i p there is a deeper analogy with the function spaces l p on the unit circle however the containment relations are reversed from the l p case namely cp c cr for p r this analogy serves as a source for new conjectures in operator theory motivated by this we establish an operator analgue of a coefficient inequality for the function space h 1 first we survey some known results in order to emphasize the nature and depth of the analogy fix an orthonormal basis en n l 2 in h and let en span e i en each operator t on has a matrix representation

on the behavior of broyden s class of quasi newton methods this paper analyzes algorithms from the broyden class of quasi newton methods for nonlinear unconstrained optimization this class depends on a parameter phi k for which the choices phi k 0 and phi k 1 give the well known bfgs and dfp methods this paper examines algorithms that allow for negative values of the parameter phi k it shows that severe restrictions have to be imposed on the selection of phi k to guarantee q superlinear convergence it is argued that negative values of phi k are desirable and conditions on phi k that guarantee superlinear convergence are given however practical algorithms that preserve the excellent properties of the bfgs method are not easy to design

factorization in analytic operator algebras abstract a constructive and unified approach is used to obtain the upper lower factorization of positive operators and the outer function factorization of positive operator valued functions on the circle for a projection nest e it is shown that every positive operator admits a canonical factorization c a a with a an outer operator if and only if e is well ordered with new methods we generalize the inner outer factorizations obtained by arveson for nests of order type z and the riesz factorization due to shields for trace class triangular operators weak factorization is obtained in noncommutative h1 spaces associated with general nest subalgebras of a semifinite factor characterizations of a nehari type are given for the associated hankel forms and hankel operators

a broyden class of quasi newton methods for riemannian optimization this paper develops and analyzes a generalization of the broyden class of quasi newton methods to the problem of minimizing a smooth objective function f on a riemannian manifold a condition on vector transport and retraction that guarantees convergence and facilitates efficient computation is derived experimental evidence is presented demonstrating the value of the extension to the riemannian broyden class through superior performance for some problems compared to existing riemannian bfgs methods in particular those that depend on differentiated retraction

global analysis of the dennis wolkowicz least change secant algorithm this paper explores the global convergence of the least change secant method proposed by dennis and wolkowicz siam j numer anal 30 1993 pp 1291 1314 this method is a member of the broyden family but it doesn t necessarily belong to the broyden convex family furthermore it can be very close to the dfp method we will prove that this method with the wolfe line search on uniformly convex objective functions is globally convergent

snr and noise variance estimation for mimo systems accurate signal to noise ratio snr and noise variance estimation are extremely important aspects of receiver design in multiple input multiple output mimo systems typically these parameters are estimated using known pilot training symbols however significant improvements may be made by using both the known pilot symbols as well as the unknown data symbols in this paper we address snr and noise variance estimation of mimo systems for both a data aided da model a non data aided nda model as well as a mixed model that uses known and unknown data symbols the cramer rao lower bound crlb and modified cramer rao lower bound mcrlb for mimo snr and mimo noise variance estimation are determined for digital constellations such as bpsk qpsk 8psk and 16qam maximum likelihood estimators are derived in closed form for the da model for the nda model closed form approximations are derived in addition to iterative expectation maximization em algorithm based estimators all of which are demonstrated to perform very close to the crlb

a new variational result for quasi newton formulae the recent measure function of byrd and nocedal siam j numer anal 26 1989 pp 727 739 is considered and simple proofs of some of its properties are given it is then shown that the bfgs and dfp formulae satisfy a least change property with respect to this new measure

the least prior deviation quasi newton update we propose a new choice for the parameter in the broyden class and derive and discuss properties of the resulting self complementary quasi newton update our derivation uses a variational principle that minimizes the extent to which the quasi newton relation is violated on a prior step we discuss the merits of the variational principle used here vis a vis the other principle in common use which minimizes deviation from the current hessian or hessian inverse approximation in an appropriate frobenius matrix norm one notable advantage of our principle is an inherent symmetry that results in the same update being obtained regardless of whether the hessian matrix or the inverse hessian matrix is updated

a partitioned psb method for partially separable unconstrained optimization problems in this paper we propose a partitioned psb method for solving partially separable unconstrained optimization problems by using a projection technique we construct a sufficient descent direction under appropriate conditions we show that the partitioned psb method with projected direction is globally and superlinearly convergent for uniformly convex problems in particular the unit step length is accepted after finitely many iterations finally some numerical results are presented which show that the partitioned psb method is effective and competitive

a double parameter scaled bfgs method for unconstrained optimization abstract a double parameter scaled bfgs method for unconstrained optimization is presented in this method the first two terms of the known bfgs update formula are scaled with a positive parameter while the third one is scaled with another positive parameter these parameters are selected in such a way as to improve the eigenvalues structure of the bfgs update the parameter scaling the first two terms of the bfgs update is determined by clustering the eigenvalues of the scaled bfgs matrix on the other hand the parameter scaling the third term is determined as a preconditioner to the hessian of the minimizing function combined with the minimization of the conjugacy condition from conjugate gradient methods under the inexact wolfe line search the global convergence of the double parameter scaled bfgs method is proved in very general conditions without assuming the convexity of the minimizing function using 80 unconstrained optimization test functions with a medium number of variables the preliminary numerical experiments show that this double parameter scaled bfgs method is more efficient than the standard bfgs update or than some other scaled bfgs methods

subset selection in noise based on diversity measure minimization we develop robust methods for subset selection based on the minimization of diversity measures a bayesian framework is used to account for noise in the data and a maximum a posteriori map estimation procedure leads to an iterative procedure which is a regularized version of the focal underdetermined system solver focuss algorithm the convergence of the regularized focuss algorithm is established and it is shown that the stable fixed points of the algorithm are sparse we investigate three different criteria for choosing the regularization parameter quality of fit sparsity criterion l curve the l curve method as applied to the problem of subset selection is found not to be robust and we propose a novel modified l curve procedure that solves this problem each of the regularized focuss algorithms is evaluated through simulation of a detection problem and the results are compared with those obtained using a sequential forward selection algorithm termed orthogonal matching pursuit omp in each case the regularized focuss algorithm is shown to be superior to the omp in noisy environments

properties and numerical performance of quasi newton methods with modified quasi newton equations quasi newton qn equation plays a core role in contemporary nonlinear optimization the usual qn equation employs only the gradients but ignores the available function value information in this paper we derive a class of modified qn equations with a vector parameter which use both available gradient and function value information the modified quasi newton methods maintain most properties of the usual quasi newton methods meanwhile they achieve a higher order accuracy in approximating the second order curvature of the problem functions than the usual ones do numerical experiments are reported which support the theoretical analyses and show the advantages of the modified qn methods over the usual ones

spectral scaling quasi newton methods with updates from the one parameter of the broyden family in this paper based on the spectral scaling secant condition w y cheng d h li spectral scaling bfgs method journal of optimization theory and applications 146 2010 305 319 we propose spectral scaling one parameter broyden family methods which allow for negative values of the parameter we show that the proposed methods possess some good properties such as quadratic termination property and single step convergence rate not inferior to that of the steepest descent method when minimizing an n dimensional quadratic function under appropriate conditions we establish the global convergence of the proposed methods for uniformly convex functions numerical results from problems in the cute test set show that the proposed methods are promising

on solving large scale limited memory quasi newton equations abstract we consider the problem of solving linear systems of equations arising with limited memory members of the restricted broyden class of updates and the symmetric rank one sr1 update in this paper we propose a new approach based on a practical implementation of the compact representation for the inverse of these limited memory matrices numerical results suggest that the proposed method compares favorably in speed and accuracy to other algorithms and is competitive with several update specific methods available to only a few members of the broyden class of updates using the proposed approach has an additional benefit the condition number of the system matrix can be computed efficiently

descent directions of quasi newton methods for symmetric nonlinear equations in general when a quasi newton method is applied to solve a system of nonlinear equations the quasi newton direction is not necessarily a descent direction for the norm function in this paper we show that when applied to solve symmetric nonlinear equations a quasi newton method with positive definite iterative matrices may generate descent directions for the norm function on the basis of a gauss newton based bfgs method d h li and m fukushima siam j numer anal 37 1999 pp 152 172 we develop a norm descent bfgs method for solving symmetric nonlinear equations under mild conditions we establish the global and superlinear convergence of the method the proposed method shares some favorable properties of the bfgs method for solving unconstrained optimization problems a the generated sequence of the quasi newton matrices is positive definite b the generated sequence of iterates is norm descent c a global convergence theorem is established without nonsingularity assumption on the jacobian preliminary numerical results are reported which positively support the method

gradient based optimization for poroelastic and viscoelastic mr elastography we describe an efficient gradient computation for solving inverse problems arising in magnetic resonance elastography mre the algorithm can be considered as a generalized adjoint method based on a lagrangian formulation one requirement for the classic adjoint method is assurance of the self adjoint property of the stiffness matrix in the elasticity problem in this paper we show this property is no longer a necessary condition in our algorithm but the computational performance can be as efficient as the classic method which involves only two forward solutions and is independent of the number of parameters to be estimated the algorithm is developed and implemented in material property reconstructions using poroelastic and viscoelastic modeling various gradient and hessian based optimization techniques have been tested on simulation phantom and in vivo brain data the numerical results show the feasibility and the efficiency of the proposed scheme for gradient calculation

an all inclusive efficient region of updates for least change secant methods least change secant methods for function minimization depend on finding a good symmetric positive definite update to approximate the hessian this update contains new curvature information while simultaneously preserving as much as possible the built up information from the previous update updates are generally derived using measures of least change based on some function of the eigenvalues of the scaled hessian a new approach for finding good least change updates is the multicriteria problem of byrd which uses the deviation from unity of the n eigenvalues of the scaled update as measures of least change the efficient multicriteria optimal class for this problem is the broyden class on the good side of the symmetric rank one sr1 update called the broyden efficient class this paper uses the framework of multicriteria optimization and the eigenvalues of the scaled sized and inverse scaled updates to study the question of what is a good update in particular it is shown that the basic

statistical quasi newton a new look at least change a new method for quasi newton minimization outperforms bfgs by combining least change updates of the hessian with step sizes estimated from a wishart model of uncertainty the hessian update is in the broyden family but uses a negative parameter outside the convex range that is usually regarded as the safe zone for broyden updates although full newton steps based on this update tend to be too long excellent performance is obtained with shorter steps estimated from the wishart model in numerical comparisons to bfgs the new statistical quasi newton sqn algorithm typically converges with about 25 fewer iterations functions and gradient evaluations on the top 1 3 hardest unconstrained problems in the cute library typical improvement on the 1 3 easiest problems is about 5 the framework used to derive sqn provides a simple way to understand differences among various broyden updates such as bfgs and dfp and shows that these methods do not preserve accuracy of the hessian in a certain sense while the new method does in fact bfgs dfp and all other updates with nonnegative broyden parameters tend to inflate hessian estimates and this accounts for their observed propensity to correct eigenvalues that are too small more readily than eigenvalues that are too large numerical results on three new test functions validate these conclusions

sizing and least change secant methods oren and luenberger introduced in 1974 a strategy for replacing hessian approximations by their scalar multiples and then performing quasi newton updates generally least change secant updates such as the bfgs or dfp updates oren and luenberger management sci 20 1974 pp 845 862 in this paper the function omega a left frac operatorname trace a n operatorname det a 1 n right is shown to be a measure of change with a direct connection to the oren luenberger strategy this measure is interesting because it is related to the ell 2 condition number but it takes all the eigenvalues of a into account rather than just the extremes if the class of possible updates is restricted to the broyden class i e scalar premultiples are not allowed then the optimal update depends on the dimension of the problem it may or may not be in the convex class but it becomes the bfgs update as the dimension increases this seems to be yet another explanation for why

improved hessian approximation with modified secant equations for symmetric rank one method symmetric rank one sr1 is one of the competitive formulas among the quasi newton qn methods in this paper we propose some modified sr1 updates based on the modified secant equations which use both gradient and function information furthermore to avoid the loss of positive definiteness and zero denominators of the new sr1 updates we apply a restart procedure to this update three new algorithms are given to improve the hessian approximation with modified secant equations for the sr1 method numerical results show that the proposed algorithms are very encouraging and the advantage of the proposed algorithms over the standard sr1 and bfgs updates is clearly observed

solving hankel matrix approximation problem using semidefinite programming positive semidefinite hankel matrices arise in many important applications some of their properties may be lost due to rounding or truncation errors incurred during evaluation the problem is to find the nearest matrix to a given matrix to retrieve these properties the problem is converted into a semidefinite programming problem as well as a problem comprising a semidefined program and second order cone problem the duality and optimality conditions are obtained and the primal dual algorithm is outlined explicit expressions for a diagonal preconditioned and crossover criteria have been presented computational results are presented a possibility for further improvement is indicated

measures for symmetric rank one updates measures of deviation of a symmetric positive definite matrix from the identity are derived they give rise to symmetric rank one sr1 type updates the measures are motivated by considering the volume of the symmetric difference of the two ellipsoids which arise from the current and updated quadratic models in quasi newton methods the measure defined by the problem maximize the determinant subject to a bound of 1 on the largest eigenvalue yields the sr1 update the measure a 1 a det a 1 n yields the optimally conditioned sized symmetric rank one updates the volume considerations also suggest a correction for the initial stepsize for these sized updates it is then shown that the optimal updates as well as the oren luenberger self scaling updates are all optimal updates for the measure the l2 condition number moreover all four sized updates result in the same largest and smallest scaled eigenvalue and corresponding eigenvector in fact the inversesized bfgs is the mean of the

analysis of a self scaling quasi newton method we study the self scaling bfgs method of oren and luenberger 1974 for solving unconstrained optimization problems for general convex functions we prove that the method is globally convergent with inexact line searches we also show that the directions generated by the self scaling bfgs method approach newton s direction asymptotically this would ensure superlinear convergence if in addition the search directions were well scaled but we show that this is not always the case we find that the method has a major drawback to achieve superlinear convergence it may be necessary to evaluate the function twice per iteration even very near the solution an example is constructed to show that the step sizes required to achieve a superlinear rate converge to 2 and 0 5 alternately

generalizing inverse compositional and esm image alignment inverse compositional ic image alignment baker and matthews in int j comput vis 56 3 221 255 2004 uses the symmetry between the roles of the fixed and moving images for faster processing however it requires implementation of compositional optimizer update steps the ic approach can be viewed as an efficient way of computing the similarity measure derivative relative to the fixed image warp parameters since the mapping between the fixed and moving warp parameters is continuous and differentiable this derivative can be converted into the moving warp space using the chain rule this avoids the need for compositional update steps our generalization also allows the efficient second order method esm malis in proceedings of the 2004 ieee international conference on robotics and automation icra04 pp 1843 1848 2004 benhimane and malis in ieee rsj international conference on intelligent robots and systems 2004 malis and benhimane in robot auton syst 52 1 39 52 2005 to be applied to general parameterizations of the transformation experiments using multiple similarity measures and optimizers show that our generalized ic method equals or exceeds the performance of the original ic approach the generalized esm approach is more reliable than the classic approach as it increases the capture radius of the optimization

the quasi cauchy relation and diagonal updating the quasi cauchy qc relation is the weak quasi newton relation of dennis and wolkowicz siam j numer anal 30 1993 pp 1291 1314 with the added restriction that full matrices are replaced by diagonal matrices this relation is justified and explored and in particular two basic variational techniques for updating diagonal matrices that satisfy it are formulated for purposes of illustration a numerical experiment is described where a diagonal updated matrix with hereditary positive definiteness is used to precondition cauchy s steepest descent direction the resulting qc algorithm is shown to be significantly accelerated in the concluding section the following topics are briefly discussed additional variational principles use of diagonal updates within other optimization algorithms together with some further numerical experience summarized in an appendix and an interesting connection between qc diagonal updating and trust region techniques

a globally convergent bfgs method with nonmonotone line search for non convex minimization in this paper we propose a modified bfgs broyden fletcher goldfarb shanno method with nonmonotone line search for unconstrained optimization under some mild conditions we show that the method is globally convergent without a convexity assumption on the objective function we also report some preliminary numerical results to show the efficiency of the proposed method

peak to average power ratio in high order ofdm the problem of peak to average power ratio papr of high order orthogonal frequency division modulation ofdm is considered using results on level crossing of random processes an upper bound on the probability that the papr of an ofdm signal will exceed a given value is derived numerical computations are used to show that this bound is tight for low pass ofdm systems the central limit theorem is used to find an asymptotic expression for the bound when the number of carriers n grows to infinity the central limit theorem is also used to find an asymptotic expression for another bound that is based on the envelope of the ofdm signal and is tighter for bandpass systems it is shown that effectively the papr grows as 2lnn and not linearly with n and by developing a lower bound on the probability that the papr of an ofdm signal will exceed a given value it is shown that asymptotically most ofdm symbols have a papr close to 2lnn some approaches to coping with the papr problem are discussed in light of the obtained results

robust pole assignment for synthesizing feedback control systems using recurrent neural networks this paper presents a neurodynamic optimization approach to robust pole assignment for synthesizing linear control systems via state and output feedback the problem is formulated as a pseudoconvex optimization problem with robustness measure i e the spectral condition number as the objective function and linear matrix equality constraints for exact pole assignment two coupled recurrent neural networks are applied for solving the formulated problem in real time in contrast to existing approaches the exponential convergence of the proposed neurodynamics to global optimal solutions can be guaranteed even with lower model complexity in terms of the number of variables simulation results of the proposed neurodynamic approach for 11 benchmark problems are reported to demonstrate its superiority

quasi newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization in this paper we propose a hybrid ode based quasi newton qn method for unconstrained optimization problems which combines the idea of low order implicit runge kutta rk techniques for gradient systems with the qn type updates of the jacobian matrix such as the symmetric rank one sr1 update the main idea of this approach is to associate a qn matrix to approximate numerically the jacobian matrix in the gradient system fundamentally this is a gradient system based on the first order optimality conditions of the optimization problem to further extend the methods for solving large scale problems a feature incorporated to the proposed methods is that a limited memory setting for matrix vector multiplications is used thus avoiding the computational and storage issues when computing jacobian information under suitable assumptions global convergence of the proposed method is proved practical insights on the effectiveness of these approaches on a set of test functions are given by a numerical comparison with that of the limited memory bfgs algorithm l bfgs and conjugate gradient algorithm cg

on simulation and optimization of macroeconometric models abstract this paper develops adapted methods for solving and optimizing deterministic nonlinear discrete dynamic macroeconometric models we give computational results concerning a belgian model of about 500 nonlinear equations per time period which leads us to a large scale minimization problem with several thousands of nonlinear constraints we use new optimization techniques based on partially known quasi newton hessian matrix updates in a reduced gradient framework we also discuss different techniques for dealing with bound constraints arising form different model formulations

the performance of trellis coded nonorthogonal noncoherent fsk in noise and jamming the performance of trellis coded fsk modulation with noncoherent detection is presented as applicable to a frequency hopped spread spectrum communication system or a conventional fsk system nonorthogonal tone spacing is considered as a parameter to control bandwidth occupancy of the hopped signal and to allow an increased number of users soft decision decoding with a maximum metric decoder is employed the metrics considered are the energy metric with and without perfect jammer state information as well as a practical self normalized metric that compensates for jamming or interference performance analyses for full and partial band and both noise and multitone jammings are presented for nonorthogonal tone spacing the signals are correlated and so chernoff bounds on the pairwise error probability for energy detectors with correlated signals are presented and evaluated these bounds supplement earlier performance estimates for trellis coded fsk to assess the tightness of the bounds simulations have been carried out with results that come within 1 to 3 db of the bounds system performance is evaluated for 2 4 and 8 state trellis codes with tone spacings of 1 3t 1 2t 2 3t and 1 t hz

a bregman extension of quasi newton updates ii analysis of robustness properties in part i of this series of articles we introduced the information geometric framework of quasi newton methods and gave an extension of hessian update formulas based on the bregman divergence the purpose of this article is to investigate the convergence and robustness properties of extended hessian update formulas fletcher has studied a variational problem which derives the approximate hessian update formula of the quasi newton methods we point out that the variational problem is identical to optimization of the kullback leibler divergence which is a discrepancy measure between two probability distributions then we introduce the bregman divergence as an extension of the kullback leibler divergence and derive extended quasi newton update formulas based on the variational problem with the bregman divergence the proposed update formulas belong to a class of self scaling quasi newton methods we study the convergence property of the proposed quasi newton method moreover we apply tools in the robust statistics to analyze the robustness properties of hessian update formulas against numerical rounding errors or a shift of tuning parameters included in line search methods of the step length as the main contribution of this paper we present that the influence of perturbations in the line search is bounded only for the standard bfgs formula for the hessian approximation numerical studies are conducted to verify the usefulness of the tools borrowed from the robust statistics

a globally convergent bfgs method for nonlinear monotone equations without any merit functions r since 1965 there has been significant progress in the theoretical study on quasi newton methods for solving nonlinear equations especially in the local convergence analysis however the study on global convergence of quasi newton methods is relatively fewer especially for the bfgs method to ensure global convergence some merit function such as the squared norm merit function is typically used in this paper we propose an algorithm for solving nonlinear monotone equations which combines the bfgs method and the hyperplane projection method we also prove that the proposed bfgs method converges globally if the equation is monotone and lipschitz continuous without differentiability requirement on the equation which makes it possible to solve some nonsmooth equations an attractive property of the proposed method is that its global convergence is independent of any merit function we also report some numerical results to show efficiency of the proposed method

new quasi newton methods via higher order tensor models many researches attempt to improve the efficiency of the usual quasi newton qn methods by accelerating the performance of the algorithm without causing more storage demand they aim to employ more available information from the function values and gradient to approximate the curvature of the objective function in this paper we derive a new qn method of this type using a fourth order tensor model and show that it is superior with respect to the prior modification of wei et al 2006 4 convergence analysis gives the local convergence property of this method and numerical results show the advantage of the modified qn method

factorization problems for nests factorization methods and characterizations of the universal factorization property let m be a von neumann algebra and let be a nest in m we consider the problem of factoring an invertible element s of m as s wa where w is a unitary element in m and both a and a 1 are elements of m which also belong to alg it is known that this is not always possible however using a variant of the lu decomposition for matrices we show that if is an injective nest and s is an invertible operator then there exists an isometry w such that both s 1w and w s belong to alg we characterize when an invertible operator factors with respect to an injective nest we also prove a result which simultaneously generalizes results of arveson and gohberg and krein finally we give a number of characterizations of those nests within a factor such that every invertible operator factors with respect to the nest

a family of hybrid conjugate gradient methods for unconstrained optimization conjugate gradient methods are an important class of methods for unconstrained optimization especially for large scale problems recently they have been much studied this paper proposes a three parameter family of hybrid conjugate gradient methods two important features of the family are that i it can avoid the propensity of small steps namely if a small step is generated away from the solution point the next search direction will be close to the negative gradient direction and ii its descent property and global convergence are likely to be achieved provided that the line search satisfies the wolfe conditions some numerical results with the family are also presented

quasi newton bundle type methods for nondifferentiable convex optimization in this paper we provide implementable methods for solving nondifferentiable convex optimization problems a typical method minimizes an approximate moreau yosida regularization using a quasi newton technique with inexact function and gradient values which are generated by a finite inner bundle algorithm for a bfgs bundle type method global and superlinear convergence results for the outer iteration sequence are obtained

a matrix free quasi newton method for solving large scale nonlinear systems one of the widely used methods for solving a nonlinear system of equations is the quasi newton method the basic idea underlining this type of method is to approximate the solution of newton s equation by means of approximating the jacobian matrix via quasi newton update application of quasi newton methods for large scale problems requires in principle vast computational resource to form and store an approximation to the jacobian matrix of the underlying problem hence this paper proposes an approximation for newton step based on the update of approximation requiring a computational effort similar to that of matrix free settings it is made possible by approximating the jacobian into a diagonal matrix using the least change secant updating strategy commonly employed in the development of quasi newton methods under suitable assumptions local convergence of the proposed method is proved for nonsingular systems numerical experiments on popular test problems confirm the effectiveness of the approach in comparison with newton s chord newton s and broyden s methods

nest algebras and similarity transformations in recent years the theory of algebras of operators on hilbert space has been stimulated by developments in the theory of quasitriangularity andersen 1 has shown that up to unitary equivalence there is only one continuous quasitrianguilar algebra we use this to provide the following answer to a question posed by j r ringrose approximately 20 years ago similar continuous nests on separable hilbert space can fail to be unitarily equivalent theorem 2 2 a consequence is the existence of a nonhyperintransitive compact operator corollary 2 3 which answers a question of kadison and singer 12 and of gohberg and krein 11 we extend our initial theorem to show that arbitrary continuous nests are similar theorem 2 10 and that every maximal nest is similar to a multiplicity one nest theorem 2 11 a consequence is that every compact operator is similar to a hyperintransitive compact operator corollary 2 12 the similarity transformation can be induced by an arbitrarily small compact perturbation of a unitary operator the methods of section 2 apply only to the continuous parts of nests for general results the atomic core part must be dealt with in section 3 different methods are developed for this purpose again utilizing andersen s results these are used in section 4 to prove that a complete nest x admits an arveson factorization for every positive invertible operator if and only if x is countable as a family of subspaces theorem 4 7 a consequence is that if x is an uncountable nest with atomic core then some similarity transformation of x has a continuous part this could not be deduced from section 2 these methods also yield a weak factorization result which concludes the paper it should be noted that a negative resolution to the ringrose question was conjectured in recent years by several mathematicians including w arveson and j ringrose also many of the results presented in this paper were announced in an a m s bulletin article 16 finally we wish to thank the referee for suggesting that the original manuscript could be condensed and improved

on the behaviour of a combined extra updating self scaling bfgs method this paper attempts to combine the best features of certain extra updating bfgs method and self scaling bfgs method it describes an algorithm similar to the bfgs method except that extra self scaling updates are employed at some iterations the bfgs hessian is scaled and updated a number of times depending on the information of the first order derivatives the results of numerical tests on certain sets of standard problems are reported it is shown that the new algorithm is very competitive with the bfgs method in several cases

approximate and exact completion problems for euclidean distance matrices using semidefinite programming abstract a partial pre distance matrix a is a matrix with zero diagonal and with certain elements fixed to given nonnegative values the other elements are considered free the euclidean distance matrix completion problem chooses nonnegative values for the free elements in order to obtain a euclidean distance matrix edm the nearest or approximate euclidean distance matrix problem is to find a euclidean distance matrix edm that is nearest in the frobenius norm to the matrix a when the free variables are discounted in this paper we introduce two algorithms one for the exact completion problem and one for the approximate completion problem both use a reformulation of edm into a semidefinite programming problem sdp the first algorithm is based on an implicit equation for the completion that for many instances provides an explicit solution the other algorithm is based on primal dual interior point methods that exploit the structure and sparsity included are results on maps that arise that keep the edm and sdp cones invariant we briefly discuss numerical tests

modifying the bfgs update by a new column scaling technique letb be a positive definite symmetric approximation to the second derivative matrix of the objective function of a minimization calculation we study modifications of the bfgs method that apply a scaling technique to the columns of a conjugate direction matrixz satisfyingz t bz i for a simple scaling technique similar to the ones considered by powell 1987 and 1989 we show that due to a two iteration cycle linear convergence can occur when the method is applied to a quadratic function and wolfe s line search is employed although for exact line searches quadratic termination can be proved we then suggest a different scaling technique that prevents certain columns thought to contain important curvature information from being scaled for this algorithm we prove global and superlinear convergence and demonstrate the superiority of our method over the bfgs formula on a range of illconditioned optimization problems moreover we present an implementation of our algorithm that requires only 3n 2 o n multiplications per iteration

a class of one parameter conjugate gradient methods this paper proposes a class of one parameter conjugate gradient methods which can be regarded as some kinds of convex combinations of some modified form of prp and hs methods the scalar k has the form of k k 1 k the convergence of the given methods is analyzed by some unified tools which show the global convergence of the proposed methods numerical experiments with the cute collections show that the proposed methods are promising

global convergence of online limited memory bfgs global convergence of an online stochastic limited memory version of the broyden fletcher goldfarb shanno bfgs quasi newton method for solving optimization problems with stochastic objectives that arise in large scale machine learning is established lower and upper bounds on the hessian eigenvalues of the sample functions are shown to suffice to guarantee that the curvature approximation matrices have bounded determinants and traces which in turn permits establishing convergence to optimal arguments with probability 1 experimental evaluation on a search engine advertising problem showcase reductions in convergence time relative to stochastic gradient descent algorithms

how bad are the bfgs and dfp methods when the objective function is quadratic we study the use of the bfgs and dfp algorithms with step lengths of one for minimizing quadratic functions of only two variables the updating formulae in this case imply nonlinear three term recurrence relations between the eigenvalues of consecutive second derivative approximations which are analysed in order to explain some gross inefficiencies that can occur specifically the bfgs algorithm may require more than 10 iterations to achieve the first decimal place of accuracy while the performance of the dfp method is far worse the results help to explain why the dfp method is often less suitable than the bfgs algorithm for general unconstrained optimization calculations and they show that quadratic functions provide much information about efficiency when the current vector of variables is too far from the solution for an asymptotic convergence analysis

a reduced hessian method for large scale constrained optimization we propose a quasi newton algorithm for solving large optimization problems with nonlinear equality constraints it is designed for problems with few degrees of freedom and is motivated by the need to use sparse matrix factorizations the algorithm incorporates a correction vector that approximates the cross term z t wy p y in order to estimate the curvature in both the range and null spaces of the constraints the algorithm can be considered to be in some sense a practical implementation of an algorithm of coleman and conn we give conditions under which local and superlinear convergence is obtained