on averaging and representation properties of the bfgs and related secant updates in this paper we present several representation theorems and averaging theorems for members of the difference class of secant updates introduced by brodlie et al j inst math appl 11 73 82 1973 major contributions are that the integral form of the mean value theorem leads to a proof that the bfgs update is pointwise the infinite average of all the updates on the one dimension manifold in the dennis class that connects the dfp secant update to the greenstadt update and that it can be expressed as the pointwise average of these latter two updates analogous results hold for all secant updates that belong to the difference class these results contribute new understanding of the structural properties of the highly popular bfgs secant update and other updates from the difference class

collet eckmann and holder we prove that collet eckmann condition for rational functions which requires exponential expansion only along the critical orbits yields the holder regularity of fatou components this implies geometric regularity of julia sets with non hyperbolic and critically recurrent dynamics in particular polynomial collet eckmann julia sets are locally connected if connected and their hausdorff dimension is strictly less than 2 the same is true for rational collet eckmann julia sets with at least one non empty fully invariant fatou component

a derivative free optimization algorithm based on conditional moments in this paper we propose a derivative free optimization algorithm based on conditional moments for finding the maximizer of an objective function the proposed algorithm does not require calculation or approximation of any order derivative of the objective function the step size in iteration is determined adaptively according to the local geometrical feature of the objective function and a pre specified quantity representing the desired precision the theoretical properties including convergence of the method are presented numerical experiments comparing with the newton quasi newton and trust region methods are given to illustrate the effectiveness of the algorithm

an optimal control theory for nonlinear optimization abstract the karush kuhn tucker conditions for a given nonlinear programming problem are generated as the transversality conditions of an optimal control problem the directional derivatives of the objective and constraint functions supply the vector fields for the optimal control problem with the search vector as the control variable zero hamiltonian trajectories along the steepest descent of control lyapunov functions provide optimal optimization algorithms the optimality of the algorithm also depends upon the choice of a metric for the finite dimensional control space many well known algorithms such as newton s method the first order lagrangian method the steepest descent method and richardson s method to name a few are derived by minimizing the lie derivative of a quadratic control lyapunov function merit functions in optimization may also be generated using the concept of a control lyapunov function these results suggest that optimal control principles hold the potential for a unified theory for optimization

rational misiurewicz maps are rare we show that the set of misiurewicz maps has lebesgue measure zero in the space of rational functions for any fixed degree d 2

bifurcation currents in holomorphic dynamics on p k potential theory has been introduced in one dimensional rational dynamics by brolin and tortrat 4 29 but does not play a central role there in higher dimension however as the classical tools are not any longer efficient pluri potential theory has revealed itself to be essential the fundamental works of hubbard papadopol fornaess sibony bedfordsmillie briend duval see 26 for precise references enlight the remarkable efficency of pluri potential theory in holomorphic dynamics on pk or ck it is therefore tempting to study the parameter spaces in a similar way more precisely one would like to relate the bifurcations of an holomorphic family f x of endomorphisms of pk to the powers of a certain current on the parameter space x let us recall that in dimension k 1 a bifurcation is said to occur at some point 0 x if the julia set of f does not move continuously around 0 the famous work of mane sadsullivan 16 which is based on the lemma and the fatou cremer sullivan classification relates the bifurcations with the instability of the critical orbits it also establishes that the bifurcations concentrate on the complement of an open dense subset of x for the quadratic family z2 x c the bifurcation locus is precisely the boundary of the mandelbrot set

on properties not inherited by monoids from their schutzenberger groups we give an example of a monoid with finitely many left and right ideals all of whose schutzenberger groups are presentable by finite complete rewriting systems and so each have finite derivation type but such that the monoid itself does not have finite derivation type and therefore does not admit a presentation by a finite complete rewriting system the example also serves as a counterexample to several other natural questions regarding complete rewriting systems and finite derivation type specifically it allows us to construct two finitely generated monoids m and n with isometric cayley graphs where n has finite derivation type respectively admits a presentation by a finite complete rewriting system but m does not this contrasts with the case of finitely generated groups for which finite derivation type is known to be a quasi isometry invariant the same example is also used to show that neither of these two properties is preserved under finite green index extensions

on the convergence of pattern search algorithms we introduce an abstract definition of pattern search methods for solving nonlinear unconstrained optimization problems our definition unifies an important collection of optimization methods that neither compute nor explicitly approximate derivatives we exploit our characterization of pattern search methods to establish a global convergence theory that does not enforce a notion of sufficient decrease our analysis is possible because the iterates of a pattern search method lie on a scaled translated integer lattice this allows us to relax the classical requirements on the acceptance of the step at the expense of stronger conditions on the form of the step and still guarantee global convergence

syntactic and rees indices of subsemigroups abstract we define two different notions of index for subsemigroups of semigroups the right syntactic index and the rees index we investigate the relationships between them and with the group index in particular we show that the syntactic index is a generalisation of both the group index and the rees index we use this fact to prove further similarities between the group index and the rees index it turns out however that very few of the nice properties of the group index are inherited by the syntactic index

equivalence and topological invariance of conditions for non uniform hyperbolicity in the iteration of rational maps we show equivalence of several standard conditions for non uniform hyperbolicity of complex rational functions including the topological collet eckmann condition tce uniform hyperbolicity on periodic orbits exponential shrinking of components of pre images of small discs backward collet eckmann condition at one point positivity of the infimum of lyapunov exponents of finite invariant measures on the julia set the condition tce is stated in purely topological terms so we conclude that all these conditions are invariant under topological conjugacy for rational maps with one critical point in julia set all the conditions above are equivalent to the usual collet eckmann and backward collet eckmann conditions thus the latter ones are invariant by topological conjugacy in the unicritical setting we also prove that neither part of this stronger statement is valid in the multicritical case

karmarkar s linear programming algorithm and newton s method this paper describes a full dimensional version of karmarkar s linear programming algorithm theprojective scaling algorithm which is defined for any linear program in n having a bounded full dimensional polytope of feasible solutions if such a linear program hasm inequality constraints then it is equivalent under an injective affine mappingj n m to karmarkar s original algorithm for a linear program in m havingm n equality constraints andm inequality constraints karmarkar s original algorithm minimizes a potential functiong x and the projective scaling algorithm is equivalent to that version of karmarkar s algorithm whose step size minimizes the potential function in the step direction

the collet eckmann condition for rational functions on the riemann sphere we show that the set of collet eckmann maps has positive lebesgue measure in the space of rational maps on the riemann sphere for any fixed degree d 2

distribution of rational maps with a preperiodic critical point let f lambda be a family of rational maps of a fixed degree with a marked critical point c lambda under a natural assumption we first prove that the hypersurfaces of parameters for which c lambda is periodic converge as a sequence of positive closed 1 1 currents to the bifurcation current attached to c and defined by demarco we then turn our attention to the parameter space of polynomials of a fixed degree d by intersecting the d 1 currents attached to each critical point of a polynomial bassaneli and berteloot obtained a positive measure mu rm bif of finite mass which is supported on the connectedness locus they showed that its support is included in the closure of the set of parameters admitting d 1 neutral cycles we show that the support of this measure is precisely the closure of the set of strictly critically finite polynomials i e of misiurewicz points

on iterations of 1 ax 2 on 1 1 on etudie des iterations de f x a 1 ax u 2 1 x 1 ou a est un parametre dans l intervalle 0 2 on etablit la non existence de cycles attractifs on definit une partition on donne une estimation uniforme de la a derivee

collet eckmann and the bifurcation measure the moduli space mathcal m d of degree d ge 2 rational maps can naturally be endowed with a measure mu text bif detecting maximal bifurcations called the bifurcation measure we prove that the support of the bifurcation measure mu text bif has positive lebesgue measure to do so we establish a general sufficient condition for the conjugacy class of a rational map to belong to the support of mu text bif and we exhibit a large set of collet eckmann rational maps which satisfy this condition as a consequence we get a set of collet eckmann rational maps of positive lebesgue measure which are approximated by hyperbolic rational maps

green index and finiteness conditions for semigroups let s be a semigroup and let t be a subsemigroup of s then t acts on s by left and by right multiplication if the complement s t has finitely many strong orbits by both these actions we say that t has finite green index in s this notion of finite index encompasses subgroups of finite index in groups and also subsemigroups of finite rees index complement therefore the question of s and t inheriting various finiteness conditions from each other arises in this paper we consider and resolve this question for the following finiteness conditions finiteness residual finiteness local finiteness periodicity having finitely many right ideals and having finitely many idempotents

similarity between the mandelbrot set and julia sets the mandelbrot setm is self similar about any misiurewicz pointc in the sense that if we examine a neighborhood ofc inm with a very powerful microscope and then increase the magnification by a carefully chosen factor the picture will be unchanged except for a rotation the corresponding julia setj c is also self similar in the same sense with the same magnification factor moreover the two setsm andj c are similar in the sense that if we use a very powerful microscope to look atm andj c both focused atc the structures we see look like very much the same

on the use of function values in unconstrained optimisation abstract by the use of a nonlinear model for the gradient of the objective function along a chosen direction we show how information available via values of the objective function may be efficiently utilised in an optimisation method of quasi newton type numerical experiments indicate that computational gains are possible by such means

using function values in multi step quasi newton methods in previous work the authors 1993 1994 developed the concept of multi step quasi newton methods based on the use of interpolating polynomials determined by data from the m most recent steps different methods for parametrizing these polynomials were studied by the authors 1993 and several methods were shown empirically to yield substantial gains over the standard one step bfgs method for unconstrained optimization in this paper we will consider the issue of how to incorporate function value information within the framework of such multi step methods this is achieved in the case of two step methods through the use of a carefully chosen rational form to interpolate the three most recent iterates the results of numerical experiments on the new methods are reported

positive lyapunov exponents in families of one dimensional dynamical systems we give a generalization of jakobson s theorem ja and a precise estimate for the density of parameters corresponding to systems with absolutely continuous invariant measures

on the conditioning of the hessian approximation in quasi newton methods abstract a bound on the possible deterioration in the condition number of the inverse hessian approximation is derived for updating formulae of bfgs type we construct a nonlinear model for the gradient of the objective function along a chosen ray in the variable space the model has a free parameter which is determined by minimising the condition number bound numerical tests indicate that using such a device on the first iteration can lead to an improvement in the performance of several algorithms of quasi newton type

on residual finiteness of monoids their schutzenberger groups and associated actions abstract in this paper we discuss connections between the following properties rfm residual finiteness of a monoid m rfsg residual finiteness of schutzenberger groups of m and rfrl residual finiteness of the natural actions of m on its green s r and l classes the general question is whether rfm implies rfsg and or rfrl and vice versa we consider these questions in all the possible combinations of the following situations m is an arbitrary monoid m is an arbitrary regular monoid every j class of m has finitely many r and l classes m has finitely many left and right ideals in each case we obtain complete answers which are summarised in a table

the hausdorff dimension of the boundary of the mandelbrot set and julia sets it is shown that the boundary of the mandelbrot set m has hausdorff dimension two and that for a generic c e am the julia set of z i z2 c also has hausdorff dimension two the proof is based on the study of the bifurcation of parabolic periodic points

statistical quasi newton a new look at least change a new method for quasi newton minimization outperforms bfgs by combining least change updates of the hessian with step sizes estimated from a wishart model of uncertainty the hessian update is in the broyden family but uses a negative parameter outside the convex range that is usually regarded as the safe zone for broyden updates although full newton steps based on this update tend to be too long excellent performance is obtained with shorter steps estimated from the wishart model in numerical comparisons to bfgs the new statistical quasi newton sqn algorithm typically converges with about 25 fewer iterations functions and gradient evaluations on the top 1 3 hardest unconstrained problems in the cute library typical improvement on the 1 3 easiest problems is about 5 the framework used to derive sqn provides a simple way to understand differences among various broyden updates such as bfgs and dfp and shows that these methods do not preserve accuracy of the hessian in a certain sense while the new method does in fact bfgs dfp and all other updates with nonnegative broyden parameters tend to inflate hessian estimates and this accounts for their observed propensity to correct eigenvalues that are too small more readily than eigenvalues that are too large numerical results on three new test functions validate these conclusions

on the use of curvature estimates in quasi newtonian methods abstract in previous work ford and ghandhari 1989 considered the use of nonlinear gradient models for quasi newton methods by means of such models values of the objective function in addition to values of the gradient were utilised in updating formulae numerical experiments provided evidence that such an approach could yield gains in terms of the evaluations required but at the cost of the solution of a nonlinear equation at each iteration in a subsequent paper ford and ghandhari to appear have developed alternative models which enable the parameter to be determined explicitly in this paper we shall consider further models of the general type introduced in the latter paper which determine the parameter explicitly by employing estimates of the curvature of the objective function thus following a technique first introduced by biggs 1971 1973 we conclude by presenting the results of numerical tests on the new methods

multi step quasi newton methods for optimization abstract quasi newton methods update at each iteration the existing hessian approximation or its inverse by means of data deriving from the step just completed we show how multi step methods employing in addition data from previous iterations may be constructed by means of interpolating polynomials leading to a generalization of the secant or quasi newton equation the issue of positive definiteness in the hessian approximation is addressed and shown to depend on a generalized version of the condition which is required to hold in the original single step methods the results of extensive numerical experimentation indicate strongly that computational advantages can accrue from such an approach by comparison with single step methods particularly as the dimension of the problem increases

a nonlinear model for function value multistep methods abstract we develop a framework employing scaling functions for the construction of multistep quasi newton methods for unconstrained optimization these methods utilize values of the objective function they are constructed via interpolants of the m 1 most recent iterates gradient evaluations and possess a free parameter which introduces an additional degree of flexibility this permits the interpolating functions to assimilate information in the form of function values which is readily available at each iteration motivated by previous experience 1 with the use of function values in multistep methods we investigate the incorporation of this information in the construction of the hessian approximation at each iteration in an attempt to accelerate convergence we concentrate on a specific example from the general family of methods corresponding to a particular choice of the scaling function and from it derive three new algorithms the relative numerical performance of these methods is assessed and the most successful of them is then compared with the standard bfgs method and with an earlier algorithm utilizing function values also developed by the authors 1

nonmonotone bfgs trained recurrent neural networks for temporal sequence processing in this paper we propose a nonmonotone approach to recurrent neural networks training for temporal sequence processing applications this approach allows learning performance to deteriorate in some iterations nevertheless the network s performance is improved over time a self scaling bfgs is equipped with an adaptive nonmonotone technique that employs approximations of the lipschitz constant and is tested on a set of sequence processing problems simulation results show that the proposed algorithm outperforms the bfgs as well as other methods previously applied to these sequences providing an effective modification that is capable of training recurrent networks of various architectures

variable metric method for minimization this is a method for determining numerically local minima of differentiable functions of several variables in the process of locating each minimum a matrix which characterizes the behavior of the function about the minimum is determined for a region in which the function depends quadratically on the variables no more than n iterations are required where n is the number of variables by suitable choice of starting values and without modification of the procedure linear constraints can be imposed upon the variables

reminiscences on the development of the variational approach to davidon s variable metric method in the mid 1960 s davidon s method was brought to the author s attention by m j d powell one of its earliest proponents its great efficacy in solving a rather difficult computational problem in which the author was involved led to an attempt to find a best updating formula best seemed to suggest least in the sense of some norm to further the stability of the method this led to the idea of minimizing a generalized quadratic frobenius norm with the quasi newton and symmetry constraints on the updates several interesting formulas were derived including the davidon fletcher powell formula as shown by goldfarb this approach was extended to the derivation of updates requiring no derivatives and to broyden like updates for the solution of simultaneous nonlinear equations attempts were made to derive minimum norm corrections in product form updates with an eye to preserving positive definiteness in the course of this attempt it was discovered that the dfp formula could be written as a product leading to some interesting theoretical developments finally a linearized product form update was developed which was competitive with the best update bfgs of that time

faster independent component analysis by preconditioning with hessian approximations independent component analysis ica is a technique for unsupervised exploration of multichannel data that is widely used in observational sciences in its classic form ica relies on modeling the data as linear mixtures of nongaussian independent sources the maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data this problem has been addressed by resorting to quasi newton methods which rely on sparse approximations of the hessian of the log likelihood however those approximations are not accurate when the ica model does not hold exactly as is often the case for real datasets we propose a new algorithm dubbed picard which makes use of sparse approximate hessians only as a preconditioner to the l bfgs algorithm refining the hessian approximation from a memory of the past iterates extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique especially on real data