on averaging and representation properties of the bfgs and related secant updates in this paper we present several representation theorems and averaging theorems for members of the difference class of secant updates introduced by brodlie et al j inst math appl 11 73 82 1973 major contributions are that the integral form of the mean value theorem leads to a proof that the bfgs update is pointwise the infinite average of all the updates on the one dimension manifold in the dennis class that connects the dfp secant update to the greenstadt update and that it can be expressed as the pointwise average of these latter two updates analogous results hold for all secant updates that belong to the difference class these results contribute new understanding of the structural properties of the highly popular bfgs secant update and other updates from the difference class

on the discovery of the good broyden method this short note traces the events that led to the unsymmetric rank one formula known as the good broyden update 5 6 which is widely used within derivative free mathematical software for solving a system of nonlinear equations

ergodic actions of group extensions on von neumann algebras on considere des actions automorphes sur des algebres de von neumann d un groupe localement compact e donne comme une extension topologique o a e g o ou a est abelien compact et second denombrable on cherche a decrire et a classer des actions ergodiques de e quand g est fini

rational recursion operators for integrable differential difference equations in this paper we introduce the concept of prehamiltonian pairs of difference operators demonstrate their connections with nijenhuis operators and give a criteria for the existence of weakly nonlocal inverse recursion operators for differential difference equations we begin with a rigorous setup of the problem in terms of the skew field of rational pseudo difference operators over a difference field with a zero characteristic subfield of constants and the principal ideal ring of matrix rational pseudo difference operators in particular we give a criteria for a rational operator to be weakly nonlocal a difference operator is called prehamiltonian if its image is a lie subalgebra with respect to the lie bracket on the difference field two prehamiltonian operators form a prehamiltonian pair if any linear combination of them is prehamiltonian then we show that a prehamiltonian pair naturally leads to a nijenhuis operator and a nijenhuis operator can be represented in terms of a prehamiltonian pair this provides a systematic method to check whether a rational operator is nijenhuis as an application we construct a prehamiltonian pair and thus a nijenhuis recursion operator for the differential difference equation recently discovered by adler and postnikov the nijenhuis operator obtained is not weakly nonlocal we prove that it generates an infinite hierarchy of local commuting symmetries we also illustrate our theory on the well known examples including the toda the ablowitz ladik and the kaup newell differential difference equations

differential difference equations associated with the fractional lax operators we study integrable hierarchies associated with spectral problems of the form p q where p and q are difference operators the corresponding nonlinear differential difference equations can be viewed as inhomogeneous generalizations of the bogoyavlensky type lattices while the latter turn into the korteweg de vries equation under the continuous limit the lattices under consideration provide discrete analogs of the sawada kotera and kaup kupershmidt equations the r matrix formulation and several of the simplest explicit solutions are presented

why nonlocal recursion operators produce local symmetries new results and applications it is well known that integrable hierarchies in 1 1 dimensions are local while the recursion operators that generate these hierarchies usually contain nonlocal terms we resolve this apparent discrepancy by providing simple and universal sufficient conditions for a nonlocal recursion operator in 1 1 dimensions to generate a hierarchy of local symmetries these conditions are satisfied by virtually all recursion operators known today and are much easier to verify than those found in earlier work we also give explicit formulae for the nonlocal parts of higher recursion hamiltonian and symplectic operators of integrable systems in 1 1 dimensions using these two results we prove under some natural assumptions the maltsev novikov conjecture stating that higher hamiltonian symplectic and recursion operators of integrable systems in 1 1 dimensions are weakly nonlocal i e the coefficients of these operators are local and these operators contain at most one integration operator in each term

on the local systems hamiltonian in the weakly non local poission brackets we study in this work the important class of non local poisson brackets pb which we call weakly non local they appeared recently in some investigations in soliton theory however there was no theory of such brackets except for very special first order case even in this case the theory was not developed enough in particular we introduce the physical forms and find casimirs momentum and canonical forms for the most important hydrodynamic type pb of that kind and their dependence on the boundary conditions

integrable odes on associative algebras in this paper we give definitions of basic concepts such as symmetries first integrals hamiltonian and recursion operators suitable for ordinary differential equations on associative algebras and in particular for matrix differential equations we choose existence of hierarchies of first integrals and or symmetries as a criterion for integrability and justify it by examples using our componentless approach we have solved a number of classification problems for integrable equations on free associative algebras also in the simplest case we have listed all possible hamiltonian operators of low order

mastersymmetries angle variables and recursion operator of the relativistic toda lattice conserved quantities bi hamiltonian formulation and recursive structure of the relativistic toda lattice rt are obtained in an algorithmic way without making use of the lax representation furthermore for the multisoliton solutions the gradients of the angle variables are described in terms of mastersymmetries a new hierarchy of completely integrable systems is discovered which turns out to correspond to the negative of the hierarchy of rt thus it is shown that the full algebra of time dependent symmetry group generators for each member of the rt hierarchy is isomorphic to the algebra of first order differential operators with laurent polynomials as coefficients the surprising phenomenon is revealed that the members of the rt hierarchy are connected to their negative counterparts by explicit backlund transformation

a derivative free optimization algorithm based on conditional moments in this paper we propose a derivative free optimization algorithm based on conditional moments for finding the maximizer of an objective function the proposed algorithm does not require calculation or approximation of any order derivative of the objective function the step size in iteration is determined adaptively according to the local geometrical feature of the objective function and a pre specified quantity representing the desired precision the theoretical properties including convergence of the method are presented numerical experiments comparing with the newton quasi newton and trust region methods are given to illustrate the effectiveness of the algorithm

an optimal control theory for nonlinear optimization abstract the karush kuhn tucker conditions for a given nonlinear programming problem are generated as the transversality conditions of an optimal control problem the directional derivatives of the objective and constraint functions supply the vector fields for the optimal control problem with the search vector as the control variable zero hamiltonian trajectories along the steepest descent of control lyapunov functions provide optimal optimization algorithms the optimality of the algorithm also depends upon the choice of a metric for the finite dimensional control space many well known algorithms such as newton s method the first order lagrangian method the steepest descent method and richardson s method to name a few are derived by minimizing the lie derivative of a quadratic control lyapunov function merit functions in optimization may also be generated using the concept of a control lyapunov function these results suggest that optimal control principles hold the potential for a unified theory for optimization

lenard scheme for two dimensional periodic volterra chain we prove that for compatible weakly nonlocal hamiltonian and symplectic operators hierarchies of infinitely many commuting local symmetries and conservation laws can be generated under some easily verified conditions no matter whether the generating nijenhuis operators are weakly nonlocal or not we construct a recursion operator of the two dimensional periodic volterra chain from its lax representation and prove that it is a nijenhuis operator furthermore we show that this system is a generalized bi hamiltonian system rather surprisingly the product of its weakly nonlocal hamiltonian and symplectic operators gives rise to the square of the recursion operator

on the convergence of pattern search algorithms we introduce an abstract definition of pattern search methods for solving nonlinear unconstrained optimization problems our definition unifies an important collection of optimization methods that neither compute nor explicitly approximate derivatives we exploit our characterization of pattern search methods to establish a global convergence theory that does not enforce a notion of sufficient decrease our analysis is possible because the iterates of a pattern search method lie on a scaled translated integer lattice this allows us to relax the classical requirements on the acceptance of the step at the expense of stronger conditions on the form of the step and still guarantee global convergence

direct search methods then and now we discuss direct search methods for unconstrained optimization we give a modern perspective on this classical family of derivative free algorithms focusing on the development of direct search methods during their golden age from 1960 to 1971 we discuss how direct search methods are characterized by the absence of the construction of a model of the objective we then consider a number of the classical direct search methods and discuss what research in the intervening years has uncovered about these algorithms in particular while the original direct search methods were consciously based on straightforward heuristics more recent analysis has shown that in most but not all cases these heuristics actually suffice to ensure global convergence of at least one subsequence of the sequence of iterates to a first order stationary point of the objective function

compact abelian group actions on injective factors we classify compact abelian group actions on injective type iii factors up to conjugacy which completes the final step of classification of compact abelian group actions on injective factors

on the integrability of homogeneous scalar evolution equations abstract we determine the existence of infinitely many symmetries for equations of the formut uk f u uk 1 when they are homogeneous with respect to the scalinguk k with 0 algorithms are given to determine whether a system has a symmetry also independent oftandx if it has one generalized symmetry we prove it has infinitely many and these can be found using recursion operators or master symmetries the method of proof uses the symbolic method and results from diophantine approximation theory we list the 10 integrable hierarchies the methods can in principle be applied to the 0 case as we illustrate for one example with 0 which seems to be new in principle they can also be used for systems of evolution equations but so far this has only been demonstrated for one class of examples

karmarkar s linear programming algorithm and newton s method this paper describes a full dimensional version of karmarkar s linear programming algorithm theprojective scaling algorithm which is defined for any linear program in n having a bounded full dimensional polytope of feasible solutions if such a linear program hasm inequality constraints then it is equivalent under an injective affine mappingj n m to karmarkar s original algorithm for a linear program in m havingm n equality constraints andm inequality constraints karmarkar s original algorithm minimizes a potential functiong x and the projective scaling algorithm is equivalent to that version of karmarkar s algorithm whose step size minimizes the potential function in the step direction

integrable systems and their recursion operators in this paper we discuss the structure of recursion operators we show that recursion operators of evolution equations have a nonlocal part that is determined by symmetries and cosymmetries this enables us to compute recursion operators more systematically under certain conditions which hold for all examples known to us nijenhuis operators are well defined i e they give rise to hierarchies of infinitely many commuting symmetries of the operator moreover the nonlocal part of a nijenhuis operator contains the candidates of roots and coroots

on construction of recursion operators from lax representation in this work we develop a general procedure for constructing the recursion operators for nonlinear integrable equations admitting lax representation several new examples are given in particular we find the recursion operators for some kdv type systems of integrable equations

compact abelian prime actions on von neumann algebras we classify the compact abelian actions on semifinite injective von neumann algebras with factor fixed point algebra the method uses that the nonzero eigenspaces of such an action contain unitaries which give rise to a classifying invariant

optimization by direct search new perspectives on some classical and modern methods direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because they lacked coherent mathematical analysis nonetheless users remained loyal to these methods most of which were easy to program some of which were reliable in the past fifteen years these methods have seen a revival due in part to the appearance of mathematical analysis as well as to interest in parallel and distributed com puting this review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited our focus then turns to a broad class of methods for which we provide a unifying framework that lends itself to a variety of convergence results the underlying principles allow general ization to handle bound constraints and linear constraints we also discuss extensions to problems with nonlinear constraints

parallel variable metric algorithms for unconstrained optimization we classify straeter s ideas for parallel unconstrained optimization and apply them to huang s class of updating formulas straeter s rank one updating formula appears to be the only parallel extension within huang s class with the property of quadratic termination we also develop a parallel extension of broyden s 1965 rank one updating formula and prove quadratic termination finally we present numerical results obtained by testing the algorithms on several standard example problems

on the use of function values in unconstrained optimisation abstract by the use of a nonlinear model for the gradient of the objective function along a chosen direction we show how information available via values of the objective function may be efficiently utilised in an optimisation method of quasi newton type numerical experiments indicate that computational gains are possible by such means

using function values in multi step quasi newton methods in previous work the authors 1993 1994 developed the concept of multi step quasi newton methods based on the use of interpolating polynomials determined by data from the m most recent steps different methods for parametrizing these polynomials were studied by the authors 1993 and several methods were shown empirically to yield substantial gains over the standard one step bfgs method for unconstrained optimization in this paper we will consider the issue of how to incorporate function value information within the framework of such multi step methods this is achieved in the case of two step methods through the use of a carefully chosen rational form to interpolate the three most recent iterates the results of numerical experiments on the new methods are reported

on the conditioning of the hessian approximation in quasi newton methods abstract a bound on the possible deterioration in the condition number of the inverse hessian approximation is derived for updating formulae of bfgs type we construct a nonlinear model for the gradient of the objective function along a chosen ray in the variable space the model has a free parameter which is determined by minimising the condition number bound numerical tests indicate that using such a device on the first iteration can lead to an improvement in the performance of several algorithms of quasi newton type

a survey of conjugate gradient algorithms for solution of extreme eigen problems of a symmetric matrix a survey of various conjugate gradient cg algorithms is presented for the minimum maximum eigen problems of a fixed symmetric matrix the cg algorithms are compared to a commonly used conventional method found in imsl it is concluded that the cg algorithms are more flexible and efficient than some of the conventional methods used in adaptive spectrum analysis and signal processing

statistical quasi newton a new look at least change a new method for quasi newton minimization outperforms bfgs by combining least change updates of the hessian with step sizes estimated from a wishart model of uncertainty the hessian update is in the broyden family but uses a negative parameter outside the convex range that is usually regarded as the safe zone for broyden updates although full newton steps based on this update tend to be too long excellent performance is obtained with shorter steps estimated from the wishart model in numerical comparisons to bfgs the new statistical quasi newton sqn algorithm typically converges with about 25 fewer iterations functions and gradient evaluations on the top 1 3 hardest unconstrained problems in the cute library typical improvement on the 1 3 easiest problems is about 5 the framework used to derive sqn provides a simple way to understand differences among various broyden updates such as bfgs and dfp and shows that these methods do not preserve accuracy of the hessian in a certain sense while the new method does in fact bfgs dfp and all other updates with nonnegative broyden parameters tend to inflate hessian estimates and this accounts for their observed propensity to correct eigenvalues that are too small more readily than eigenvalues that are too large numerical results on three new test functions validate these conclusions

on the use of curvature estimates in quasi newtonian methods abstract in previous work ford and ghandhari 1989 considered the use of nonlinear gradient models for quasi newton methods by means of such models values of the objective function in addition to values of the gradient were utilised in updating formulae numerical experiments provided evidence that such an approach could yield gains in terms of the evaluations required but at the cost of the solution of a nonlinear equation at each iteration in a subsequent paper ford and ghandhari to appear have developed alternative models which enable the parameter to be determined explicitly in this paper we shall consider further models of the general type introduced in the latter paper which determine the parameter explicitly by employing estimates of the curvature of the objective function thus following a technique first introduced by biggs 1971 1973 we conclude by presenting the results of numerical tests on the new methods

multi step quasi newton methods for optimization abstract quasi newton methods update at each iteration the existing hessian approximation or its inverse by means of data deriving from the step just completed we show how multi step methods employing in addition data from previous iterations may be constructed by means of interpolating polynomials leading to a generalization of the secant or quasi newton equation the issue of positive definiteness in the hessian approximation is addressed and shown to depend on a generalized version of the condition which is required to hold in the original single step methods the results of extensive numerical experimentation indicate strongly that computational advantages can accrue from such an approach by comparison with single step methods particularly as the dimension of the problem increases

a variable metric algorithm for unconstrained minimization without evaluation of derivatives this paper presents a modification of the bfgs method for unconstrained minimization that avoids computation of derivatives the gradients are approximated by the aid of differences of function values these approximations are calculated in such a way that a complete convergence proof can be given the presented algorithm is implementable no exact line search is required it is shown that if the objective function is convex and some usually required conditions hold the algorithm converges to a solution if the hessian matrix of the objective function is positive definite and satisfies a lipschitz condition in a neighbourhood of the solution then the rate of convergence is superlinear

a nonlinear model for function value multistep methods abstract we develop a framework employing scaling functions for the construction of multistep quasi newton methods for unconstrained optimization these methods utilize values of the objective function they are constructed via interpolants of the m 1 most recent iterates gradient evaluations and possess a free parameter which introduces an additional degree of flexibility this permits the interpolating functions to assimilate information in the form of function values which is readily available at each iteration motivated by previous experience 1 with the use of function values in multistep methods we investigate the incorporation of this information in the construction of the hessian approximation at each iteration in an attempt to accelerate convergence we concentrate on a specific example from the general family of methods corresponding to a particular choice of the scaling function and from it derive three new algorithms the relative numerical performance of these methods is assessed and the most successful of them is then compared with the standard bfgs method and with an earlier algorithm utilizing function values also developed by the authors 1

nonmonotone bfgs trained recurrent neural networks for temporal sequence processing in this paper we propose a nonmonotone approach to recurrent neural networks training for temporal sequence processing applications this approach allows learning performance to deteriorate in some iterations nevertheless the network s performance is improved over time a self scaling bfgs is equipped with an adaptive nonmonotone technique that employs approximations of the lipschitz constant and is tested on a set of sequence processing problems simulation results show that the proposed algorithm outperforms the bfgs as well as other methods previously applied to these sequences providing an effective modification that is capable of training recurrent networks of various architectures

variable metric method for minimization this is a method for determining numerically local minima of differentiable functions of several variables in the process of locating each minimum a matrix which characterizes the behavior of the function about the minimum is determined for a region in which the function depends quadratically on the variables no more than n iterations are required where n is the number of variables by suitable choice of starting values and without modification of the procedure linear constraints can be imposed upon the variables

symmetries conserved quantities and hierarchies for some lattice systems with soliton structure basic invariants such as conserved quantities symmetries mastersymmetries and recursion operators are explicitly constructed for the following nonlinear lattice systems the modified korteweg de vries lattice the ablowitz ladik lattice the brusci ragnisco lattice the ragnisco tu lattice and some cases of the class of integrable systems introduced by bogoyavlensky the algorithmic basis for obtaining these quantities is described and the interrelation between the underlying mastersymmetry approach and the lax pair analysis is discussed by explicit presentation of the higher order members of the corresponding hierarchies new completely integrable lattice flows are found for all systems multi hamiltonian formulations are given

symplectic structures their backlund transformations and hereditary symmetries abstract it is shown that compatible symplectic structures lead in a natural way to hereditary symmetries we recall that a hereditary symmetry is an operator valued function which immediately yields a hierarchy of evolution equations each having infinitely many commuting symmetries all generated by this hereditary symmetry furthermore this hereditary symmetry usually describes completely the soliton structure and the conservation laws of these equations this result then provide us with a method for constructing hereditary symmetries and hence exactly solvable evolution equations in addition we show how symplectic structures transform under backlund transformations this leads to a method for generating a whole class of symplectic structures from a given one several examples and applications are given illustrating the above results also the connection of our results with those of gelfand and dikii and of magri is briefly pointed out

integrable discretizations of derivative nonlinear schrodinger equations we propose integrable discretizations of derivative nonlinear schrodinger dnls equations such as the kaup newell equation the chen lee liu equation and the gerdjikov ivanov equation by constructing lax pairs the discrete dnls systems admit the reduction of complex conjugation between two dependent variables and possess bi hamiltonian structure through transformations of variables and reductions we obtain novel integrable discretizations of the nonlinear schrodinger nls modified kdv mkdv mixed nls matrix nls matrix kdv matrix mkdv coupled nls coupled hirota coupled sasa satsuma and burgers equations we also discuss integrable discretizations of the sine gordon equation the massive thirring model and their generalizations

quasi newton updates in abstract vector spaces we derive least change updates of linear mappings that approximate derivatives of functions defined on abstract vector spaces these correspond to well known matrix updates our approach has the value of demonstrating the minimal assumptions necessary to derive particular updates and hence their robustness in addition the choice of one or other update is indicated by the presence of natural inner or scalar products in the vector spaces all spaces considered are finite dimensional real vector spaces

a new scheme of integrability for bi hamiltonian pde we develop a new method for constructing integrable hamiltonian hierarchies of lax type equations which combines the fractional powers technique of gelfand and dickey and the classical hamiltonian reduction technique of drinfeld and sokolov the method is based on the notion of an adler type matrix pseudodifferential operator and the notion of a generalized quasideterminant we also introduce the notion of a dispersionless adler type series which is applied to the study of dispersionless hamiltonian equations non commutative hamiltonian equations are discussed in this framework as well

hamiltonian structure of discrete soliton systems we describe an approach for investigating the hamiltonian structures of the lattice isospectral evolution equations associated with a general discrete spectral problem by using the so called implicit representations of the isospectral flows we demonstrate the existence of the recursion operator l which is a strong and hereditary symmetry of the flows it is then proven that every equation in the isospectral hierarchy possesses the hamiltonian structure if l has a skew symmetric factorization and the first equation ut k 0 in the hierarchy satisfies some simple condition we obtain related properties such as the implectic symplectic factorization of l the liouville complete integrability and the multi hamiltonian structures of the isospectral hierarchy four examples are given

reminiscences on the development of the variational approach to davidon s variable metric method in the mid 1960 s davidon s method was brought to the author s attention by m j d powell one of its earliest proponents its great efficacy in solving a rather difficult computational problem in which the author was involved led to an attempt to find a best updating formula best seemed to suggest least in the sense of some norm to further the stability of the method this led to the idea of minimizing a generalized quadratic frobenius norm with the quasi newton and symmetry constraints on the updates several interesting formulas were derived including the davidon fletcher powell formula as shown by goldfarb this approach was extended to the derivation of updates requiring no derivatives and to broyden like updates for the solution of simultaneous nonlinear equations attempts were made to derive minimum norm corrections in product form updates with an eye to preserving positive definiteness in the course of this attempt it was discovered that the dfp formula could be written as a product leading to some interesting theoretical developments finally a linearized product form update was developed which was competitive with the best update bfgs of that time

faster independent component analysis by preconditioning with hessian approximations independent component analysis ica is a technique for unsupervised exploration of multichannel data that is widely used in observational sciences in its classic form ica relies on modeling the data as linear mixtures of nongaussian independent sources the maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data this problem has been addressed by resorting to quasi newton methods which rely on sparse approximations of the hessian of the log likelihood however those approximations are not accurate when the ica model does not hold exactly as is often the case for real datasets we propose a new algorithm dubbed picard which makes use of sparse approximate hessians only as a preconditioner to the l bfgs algorithm refining the hessian approximation from a memory of the past iterates extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique especially on real data

relativistic toda systems we present and study poincare invariant generalizations of the galilei invariant toda systems the classical nonperiodic systems are solved by means of an explicit action angle transformation