a modified broyden like quasi newton method for nonlinear equations abstract in this paper we introduce a modified broyden like quasi newton method for solving nonlinear equations which is an acceleration version of the broyden like method presented by li and fukushima 2000 we show that the proposed method has global and superlinear convergence properties we report some numerical results to show its efficiency

the wasserstein distance and approximation theorems by an extension of the idea of the multivariate quantile transform we obtain an explicit formula for the wasserstein distance between multivariate distributions in certain cases for the general case we use a modification of the definition of the wasserstein distance and determine optimal markov constructions we give some applications to the problem of approximation of stochastic processes by simpler ones as e g weakly dependent processes by independent sequences and finally determine the optimal martingale approximation to a given sequence of random variables the doob decomposition gives only the one step optimal approximation

on the global convergence of the bfgs method for nonconvex unconstrained optimization problems this paper is concerned with the open problem of whether the bfgs method with inexact line search converges globally when applied to nonconvex unconstrained optimization problems we propose a cautious bfgs update and prove that the method with either a wolfe type or an armijo type line search converges globally if the function to be minimized has lipschitz continuous gradients

global convergence of a cass of quasi newton methods on convex problems we study the global convergence properties of the restricted broyden class of quasi newton methods when applied to a convex objective function we assume that the line search satisfies a standard sufficient decrease condition and that the initial hessian approximation is any positive definite matrix we show global and superlinear convergence for this class of methods except for dfp this generalizes powell s well known result for the bfgs method the analysis gives us insight into the properties of these algorithms in particular it shows that dfp lacks a very desirable self correcting property possessed by bfgs

nature of singularities for analyticity domains of invariant curves we present theoretical arguments based on infinite dimensional bifurcation theory and numerical evidence based on nonperturbative methods that the boundaries of analyticity of invariant curves can be described as an accumulation of branch points which are typically of order 2 we show how this fact would explain previous numerical results of several authors and how it suggests more efficient numerical algorithms which we implement

a tool for the analysis of quasi newton methods with application to unconstrained minimization the bfgs update formula is shown to have an important property that is inde pendent of the algorithmic context of the update and that is relevant to both constrained and unconstrained optimization the bfgs method for unconstrained optimization using a variety of line searches including backtracking is shown to be globally and superlinearly convergent on uniformly convex problems the analysis is particularly simple due to the use of some new tools introduced in this paper

minimization for conditional simulation relationship to optimal transport in this paper we consider the problem of generating independent samples from a conditional distribution when independent samples from the prior distribution are available although there are exact methods for sampling from the posterior e g markov chain monte carlo or acceptance rejection these methods tend to be computationally demanding when evaluation of the likelihood function is expensive as it is for most geoscience applications as an alternative in this paper we discuss deterministic mappings of variables distributed according to the prior to variables distributed according to the posterior although any deterministic mappings might be equally useful we will focus our discussion on a class of algorithms that obtain implicit mappings by minimization of a cost function that includes measures of data mismatch and model variable mismatch algorithms of this type include quasi linear estimation randomized maximum likelihood perturbed observation ensemble kalman filter and ensemble of perturbed analyses 4d var when the prior pdf is gaussian and the observation operators are linear we show that these minimization based simulation methods solve an optimal transport problem with a nonstandard cost function when the observation operators are nonlinear however the mapping of variables from the prior to the posterior obtained from those methods is only approximate errors arise from neglect of the jacobian determinant of the transformation and from the possibility of discontinuous mappings

the equivalence of strict convexity and injectivity of the gradient in bounded level sets it is shown that lipschitzian functions are strictly convex if and only if their generalized gradients are disjoint at distinct interior points of a given bounded level set

extremal problems on the set of nonnegative definite matrices abstract an optimization problem of minimizing a real valued function of certain elements of a symmetric matrix subject to this matrix being nonnegative definite is considered optimality conditions are proposed the duality result of olkin and pukelsheim 1982 is extended to a wide class of such problems applications are discussed

the convergence of variable metric matrices in unconstrained optimization it is proved that if the dfp or bfgs algorithm with step lengths of one is applied to a functionf x that has a lipschitz continuous second derivative and if the calculated vectors of variables converge to a point at which f is zero and 2f is positive definite then the sequence of variable metric matrices also converges the limit of this sequence is identified in the case whenf x is a strictly convex quadratic function

on a generalization of cyclic monotonicity and distances among random vectors abstract we generalize an investigation of distances among random vectors begun by olkin and pukelsheim and continued by olkin and rachev

a quasi newton method for estimating the parameter in a nonlinear hyperbolic system a quasi newton method qnm in infinite dimensional spaces for estimating parameters involved in distributed parameter systems is presented in this paper next the global convergence of a sequence generated by the algorithm qnm is also proved we apply the algorithm qnm to an identification problem of a distributed parameter system governed by a nonlinear hyperbolic partial differential equation

superlinear convergence of symmetric huang s class of methods in this paper the problem of minimizing the functionalf d r n r is considered typical assumptions onf are assumed a class of quasi newton methods namely huang s class of methods is used for finding an optimal solution of this problem a new theorem connected with this class is presented by means of this theorem some convergence results known up till now only for the methods which satisfy quasi newton condition are extended that is the results of superlinear convergence of variable metric methods in the cases of exact and asymptotically exact minimization and the so called direct prediction case this theorem allows to interpretate one of the parameters as the scaling parameter

notes on the wasserstein metric in hilbert spaces let x y be a pair of hilbert valued random variables for which the wasserstein distance between the marginal distributions is reached we prove that the mapping omega rightarrow x omega y omega is increasing in a certain sense moreover if y satisfies a nondegeneration condition we can take x t y with t monotone in the sense of zarantarello we apply these results to obtain a proof of the central limit theorem clt in hilbert spaces which does not make use of the clt for real valued random variables

singularities of periodic orbits near invariant curves abstract we show numerically for standard like maps how the singularities in the complex parameter of the function which conjugates the map to a rotation of rational period behave when the period goes to an irrational number furthermore we propose a numerical method to extrapolate the radius of convergence of the series parametrizing the solution of periodic orbits the results are compared with analyses performed by pade approximants greene s method root criterion and the prediction by renormalization theory

on the smoothness of convex envelopes we examine differentiability properties of the convex envelope conv e of a given function e rn cc cc in terms of properties of e it is shown that c1 as well as optimal c1 a regularity results 0 a 1 can be obtained under general conditions

superlinear convergence of broyden s bounded θ class of methods in this paper the so called broyden s bounded class of methods is considered it contains as a subclass broyden s restricted class of methods in which the updating matrices retain symmetry and positive definiteness these iteration methods are used for solving unconstrained minimization problems of the following form f hat x min x in r n f x it is assumed that the step size coefficient k 1 in each iteration and the functionalf rn r1 satisfies the standard assumptions viz f is twice continuously differentiable and the hessian matrix is uniformly positive definite and bounded there exist constantsm m 0 such that m y 2 y m parallel y parallel 2 leqslant left langle y f hat x left y right rangle right leqslant m parallel y parallel 2 for ally rn and satisfies a lipschitz like condition at the optimal point hat x the gradient vanishes at hat x under these assumptions the local convergence of broyden s methods is proved furthermore the q superlinear convergence is shown

the global convergence of partitioned bfgs on problems with convex decompositions and lipschitzian gradients the main purpose of this paper is the extension of powell s 1976 global convergence result to the partitioned bfgs method introduced by griewank and toint 1982 even in the unpartitioned case the original result is strengthened because the search directions need not be computed exactly and the gradient is only required to be lipschitzian rather than differentiable using the functional of byrd and nocedal 1987 a strong form of r superlinear convergence is obtained if the element functions are uniformly convex and their gradients are strictly differentiable at the minimizerx only in order to deal with the possibility of singular functions we utilize a damping of the bfgs update that becomes inactive if the problem turns out to be regular nearx

optimal conditioning and convergence in rank one quasi newton updates we show that the optimally conditioned update proposed by spedicato and greenstadt for solving nonlinear equations is obtainable in closed form however unlike its rank two counterpart designed for minimization conditioning in the rank one case is not necessarily a good measure of closeness we propose updates that compromise the two properties local and superlinear convergence is proved for a class of updates

a globally and superlinearly convergent gauss newton based bfgs method for symmetric nonlinear equations in this paper we present a gauss newton based bfgs method for solving symmetric nonlinear equations which contain as a special case an unconstrained optimization problem a saddle point problem and an equality constrained optimization problem a suitable line search is proposed with which the presented bfgs method exhibits an approximate norm descent property under appropriate conditions global convergence and superlinear convergence of the method are established the numerical results show that the proposed method is successful

iterative solution of integral equations by a quasi newton method abstract when broyden s type methods are applied to the solution of integral equations they can be specialized in such a way to generate in the course of the iteration low rank degenerate kernel equations we describe the features of this approach

sequential quadratic programming sqp for optimal control in direct numerical simulation of turbulent flow the current work focuses on the development and application of an efficient algorithm for optimization of three dimensional turbulent flows simulated using direct numerical simulation dns or large eddy simulations and further characterized by large dimensional optimization parameter spaces the optimization algorithm is based on sequential quadratic programming sqp in combination with a damped formulation of the limited memory bfgs method the latter is suitable for solving large scale constrained optimization problems whose hessian matrices cannot be computed and stored at a reasonable cost we combine the algorithm with a line search merit function based on an l 1 norm to enforce the convergence from any remote point it is first shown that the proposed form of the damped l bfgs algorithm is suitable for solving equality constrained rosenbrock type functions then we apply the algorithm to an optimal control test problem that consists of finding the optimal initial perturbations to a turbulent temporal mixing layer such that mixing is improved at the end of a simulation time horizon t the controls are further subject to a non linear equality constraint on the total control energy dnss are used to resolve all turbulent scales of motion and a continuous adjoint formulation is employed to calculate the gradient of the cost functionals we compare the convergence speed of the sqp l bfgs algorithm to a conventional non linear conjugate gradient method i e the current standard in dns based optimal control and find that the sqp algorithm is more than an order of magnitude faster than the conjugate gradient method

second order methods for optimal control of time dependent fluid flow second order methods for open loop optimal control problems governed by the two dimensional instationary navier stokes equations are investigated optimality systems based on a lagrangian formulation and adjoint equations are derived the newton and quasi newton methods as well as various variants of sqp methods are developed for applications to optimal flow control and their complexity in terms of system solves is discussed local convergence and rate of convergence are proved a numerical example illustrates the feasibility of solving optimal control problems for two dimensional instationary navier stokes equations by second order numerical methods in a standard workstation environment

a partitioned psb method for partially separable unconstrained optimization problems in this paper we propose a partitioned psb method for solving partially separable unconstrained optimization problems by using a projection technique we construct a sufficient descent direction under appropriate conditions we show that the partitioned psb method with projected direction is globally and superlinearly convergent for uniformly convex problems in particular the unit step length is accepted after finitely many iterations finally some numerical results are presented which show that the partitioned psb method is effective and competitive

geometric methods for spectral analysis this paper explores a geometric framework for modeling nonstationary but slowly varying time series based on the assumption that short windowed power spectra capture their spectral character and that energy transference in the frequency domain has a physical significance the framework relies on certain notions of transportation distance and their respective geodesics to model possible nonparametric changes in the power spectral density with respect to time we discuss the relevance of this framework to applications in spectral tracking spectral averaging and speech morphing

on the convergence of the modified levenberg marquardt method with a nonmonotone second order armijo type line search recently fan j fan the modified levenberg marquardt method for nonlinear equations with cubic convergence math comput 81 2012 447 466 proposed a modified levenberg marquardt mlm method for nonlinear equations using a trust region technique global and cubic convergence of the mlm method is proved by fan 2012 12 under the local error bound condition which is weaker than nonsingularity the purpose of the paper is to investigate the convergence properties of the mlm method with a line search technique since the search direction of the mlm method may be not a descent direction standard line searches can not be used directly in this paper we propose a nonmonotone second order armijo line search which guarantees the global convergence of the mlm method moreover we prove that the unit step will be always accepted finally then cubic convergence of the mlm method is preserved under the local error bound condition some preliminary numerical results are also reported

a schur analysis approach to minimum distance problems abstract some minimum distance problems in hilbert space are to be transformed into associated matrix optimization problems which is their turn are studied using schur parametrization of nonnegative hermitian block matrices this enables a clear geometrical insight into the structure of the feasible region and a complete description of the set of all solutions

superlinear convergence of affine scaling interior point newton methods for infinite dimensional nonlinear problems with pointwise bounds we develop and analyze a superlinearly convergent affine scaling interior point newton method for infinite dimensional problems with pointwise bounds in lp space the problem formulation is motivated by optimal control problems with lp controls and pointwise control constraints the finite dimensional convergence theory by coleman and li siam j optim 6 1996 pp 418 445 makes essential use of the equivalence of norms and the exact identifiability of the active constraints close to an optimizer with strict complementarity since these features are not available in our infinite dimensional framework algorithmic changes are necessary to ensure fast local convergence the main building block is a newton like iteration for an affine scaling formulation of the kkt condition we demonstrate in an example that a stepsize rule to obtain an interior iterate may require very small stepsizes even arbitrarily close to a nondegenerate solution using a pointwise projection instead we prove superlinear convergence under a weak strict complementarity condition and convergence with mbox q rate 1 under a slightly stronger condition if a smoothing step is available we discuss how the algorithm can be embedded in the class of globally convergent trust region interior point methods recently developed by m heinkenschloss and the authors numerical results for the control of a heating process confirm our theoretical findings

local and superlinear convergence of quasi newton methods based on modified secant conditions for solving unconstrained minimization problems quasi newton methods are popular iterative methods the secant condition which employs only the gradient information is imposed on these methods several researchers paid attention to other secant conditions to get a better approximation of the hessian matrix of the objective function recently zhang et al new quasi newton equation and related methods for unconstrained optimization j optim theory appl 102 1999 147 167 and zhang and xu properties and numerical performance of quasi newton methods with modified quasi newton equations j comput appl math 137 2001 269 278 proposed the modified secant condition which uses both gradient and function value information in order to get a higher order accuracy in approximating the second curvature of the objective function they showed the local and q superlinear convergence property of the bfgs like and dfp like updates based on their proposed secant condition in this paper we incorporate one parameter into this secant condition to smoothly switch the standard secant condition and the secant condition of zhang et al we consider a modified broyden family which includes the bfgs like and the dfp like updates proposed by zhang et al we prove the local and q superlinear convergence of our method

a pde sensitivity equation method for optimal aerodynamic design the use of gradient based optimization algorithms in inverse design is well established as a practical approach to aerodynamic design a typical procedure uses a simulation scheme to evaluate the objective function from the approximate states and its gradient then passes this information to an optimization algorithm once the simulation scheme cfd flow solver has been selected and used to provide approximate function evaluations there are several possible approaches to the problem of computing gradients one popular method is to differentiate the simulation scheme and compute design sensitivities that are then used to obtain gradients although this black box approach has many advantages in shape optimization problems one must compute mesh sensitivities in order to compute the design sensitivity in this paper we present an alternative approach using the pde sensitivity equation to develop algorithms for computing gradients this approach has the advantage that mesh sensitivities need not be computed moreover when it is possible to use the cfd scheme for both the forward problem and the sensitivity equation then there are computational advantages an apparent disadvantage of this approach is that it does not always produce consistent derivatives however for a proper combination of discretization schemes one can showasymptotic consistencyunder mesh refinement which is often sufficient to guarantee convergence of the optimal design algorithm in particular we show that when asymptotically consistent schemes are combined with a trust region optimization algorithm the resulting optimal design method converges we denote this approach as thesensitivity equation method the sensitivity equation method is presented convergence results are given and the approach is illustrated on two optimal design problems involving shocks

natural boundaries for area preserving twist maps we consider kam invariant curves for generalizations of the standard map of the form x y x y y f x wheref x is an odd trigonometric polynomial we study numerically their analytic properties by a pade approximant method applied to the function which conjugates the dynamics to a rotation in the complex plane natural boundaries of different shapes are found in the complex plane the analyticity region appears to be a strip bounded by a natural boundary whose width tends linearly to 0 as tends to the critical value

a double parameter scaled bfgs method for unconstrained optimization abstract a double parameter scaled bfgs method for unconstrained optimization is presented in this method the first two terms of the known bfgs update formula are scaled with a positive parameter while the third one is scaled with another positive parameter these parameters are selected in such a way as to improve the eigenvalues structure of the bfgs update the parameter scaling the first two terms of the bfgs update is determined by clustering the eigenvalues of the scaled bfgs matrix on the other hand the parameter scaling the third term is determined as a preconditioner to the hessian of the minimizing function combined with the minimization of the conjugacy condition from conjugate gradient methods under the inexact wolfe line search the global convergence of the double parameter scaled bfgs method is proved in very general conditions without assuming the convexity of the minimizing function using 80 unconstrained optimization test functions with a medium number of variables the preliminary numerical experiments show that this double parameter scaled bfgs method is more efficient than the standard bfgs update or than some other scaled bfgs methods

a quasi newton method for solving fixed point problems in hilbert spaces we study the convergence properties of a projective variant of newton s method for the approximation of fixed points of completely continuous operators in hilbert spaces we then discuss applications to nonlinear integral equations and we produce some numerical examples

computation of domains of analyticity for some perturbative expansions of mechanics abstract we compute the domain of analyticity of some perturbative expansions for invariant circles appearing in mechanics we use pade approximants for the perturbative expansions and introduce methods to ascertain their domain of convergence we also use non perturbative methods based on direct computation of the invariant circles and in analogy with greene s criterion approximation by circles with rational rotation we find that the domains computed by all the methods agree within the limits of accuracy we also study rigorously the nature of the singularities when the frequency is rational

a modified bfgs type quasi newton method with line search for symmetric nonlinear equations problems abstract in this paper using approximate gradient of the norm square metric function we present an inexact mbfgs method with line search for solving symmetric nonlinear equations which is a generalization of the mbfgs method proposed by li and fukushima 2001 for solving smooth unconstrained optimization the used approximate gradient of the proposed method only requires computing two residuals at every iteration and is easy to obtain by utilizing symmetric structure of the system we establish global convergence of the proposed method and report some numerical results

a theoretical and experimental study of the symmetric rank one update this paper first discusses computational experience using the sri update in conventional line search and trust region algorithms for unconstrained optimization the experiments show that the sri is very competitive with the widely used bfgs method they also indicate two interesting features the final hessian approximations produced by the sri method are not generally appreciably better than those produced by the bfgs and the sequences of steps produced by the sri do not usually seem to have the uniform linear independence property that is assumed in recent convergence analysis this paper presents a new analysis that shows that the sri method with a line search is n 1 step q superlinearly convergent without the assumption of linearly independent iterates this analysis assumes that the hessian approximations are positive definite and bounded asymptotically which from computational experience are reasonable assumptions

the convergence of matrices generated by rank 2 methods from the restricted β class of broyden it is shown that the matricesb k generated by any method from the restricted class of broyden converge if the method is applied to the unconstrained minimization of a functionf c 2 r n with lipschitz continuous 2 f x and if the method is such that it generates vectorsx k converging sufficiently fast sum limits k x k x infty to a local minimumx off with positive definite 2 f x this result not only holds for constant step sizes k 1 in each iterationx k x k 1 x k k b k 1 f x k of these methods but also for step sizes determined by asymptotically exact line searches the paper generalizes corresponding results of ge ren pu and powell 6 for the dfp and bfgs methods used in conjunction with step sizes k 1

shape of analyticity domains of lindstedt series the standard map the analyticity domains of the lindstedt series for the standard map are studied numerically using pade approximants to model their natural boundaries we show that if the rotation number is a diophantine number close to a rational value p q then the radius of convergence of the lindstedt series becomes smaller than the critical threshold for the corresponding kol mogorov arnol d moser curve and the natural boundary on the plane of the complexified perturbative parameter acquires a flower like shape with 2q petals

spectral residual method without gradient information for solving large scale nonlinear systems of equations a fully derivative free spectral residual method for solving largescale nonlinear systems of equations is presented it uses in a systematic way the residual vector as a search direction a spectral steplength that produces a nonmonotone process and a globalization strategy that allows for this nonmonotone behavior the global convergence analysis of the combined scheme is presented an extensive set of numerical experiments that indicate that the new combination is competitive and frequently better than well known newton krylov methods for largescale problems is also presented

periodic and quasi periodic orbits for the standard map we consider both periodic and quasi periodic solutions for the standard map and we study the corresponding conjugating functions i e the functions conjugating the motions to trivial rotations we compare the invariant curves with rotation numbers satisfying the bryuno condition and the sequences of periodic orbits with rotation numbers given by their convergents n p n q n we prove the following results for n 1 for rotation numbers n n we study the radius of convergence of the conjugating functions and we find lower bounds on them which tend to a limit which is a lower bound on the corresponding quantity for 2 the periodic orbits consist of points which are more and more close to the invariant curve with rotation number 3 such orbits lie on analytical curves which tend uniformly to the invariant curve

a pointwise quasi newton method for integral equations for fully nonlinear integral equations conventional quasi newton methods such as broyden s method may not possess good convergence properties in this paper we present a new method that gives superlinear convergence for such integral equations and provides for savings over newton s method in the cost of solution of the linear equation for the quasi newton step

convergence of quasi newton matrices generated by the symmetric rank one update quasi newton algorithms for unconstrained nonlinear minimization generate a sequence of matrices that can be considered as approximations of the objective function second derivatives this paper gives conditions under which these approximations can be proved to converge globally to the true hessian matrix in the case where the symmetric rank one update formula is used the rate of convergence is also examined and proven to be improving with the rate of convergence of the underlying iterates the theory is confirmed by some numerical experiments that also show the convergence of the hessian approximations to be substantially slower for other known quasi newton formulae

on the convergence of a modified regularized newton method for convex optimization with singular solutions in this paper we propose a modified regularized newton method for convex minimization problems whose hessian matrices may be singular the proposed method is proved to converge globally if the gradient and hessian of the objective function are lipschitz continuous under the local error bound condition we first show that the method converges quadratically which implies that x k x is equivalent to dist x k x where x is the solution set and x k x x then we in turn prove the cubic convergence of the proposed method under the same local error bound condition which is weaker than nonsingularity

optimal shrinkage of eigenvalues in the spiked covariance model we show that in a common high dimensional covariance model the choice of loss function has a profound effect on optimal estimation in an asymptotic framework based on the spiked covariance model and use of orthogonally invariant estimators we show that optimal estimation of the population covariance matrix boils down to design of an optimal shrinker that acts elementwise on the sample eigenvalues indeed to each loss function there corresponds a unique admissible eigenvalue shrinker dominating all other shrinkers the shape of the optimal shrinker is determined by the choice of loss function and crucially by inconsistency of both eigenvalues and eigenvectors of the sample covariance matrix details of these phenomena and closed form formulas for the optimal eigenvalue shrinkers are worked out for a menagerie of 26 loss functions for covariance estimation found in the literature including the stein entropy divergence frechet bhattacharya matusita frobenius norm operator norm nuclear norm and condition number losses

a renormalization approach to invariant circles in area preserving maps abstract kadanoff and shenker introduced a renormalisation approach to invariant circles in area preserving maps this paper makes more precise the connection between invariant circles and the renormalisation operator restricting attention to noble rotation numbers the stability of a simple fixed point of the renormalisation is analysed corresponding to a linear twist map it is found to be essentially attracting so that noble circles persist under perturbation giving a new view on kam theory shenker and kadanoff found evidence for another fixed point corresponding to a map with a non smooth noble circle further evidence is given in this paper it has essentially only one unstable direction and its stable manifold is believed to give the boundary of the set of twist maps with a noble circle finally noble circles are shown to be locally most robust in an important sense

a note on the superlinear convergence of gmres in the paper it is shown how the rate of convergence of the gmres method for solving a linear operator equation lambda i k u f in a hilbert space is related to the degree of compactness of k measured by the products of its singular values

the local convergence of broyden like methods on lipschitzian problems in hilbert spaces the classical local convergence analysis for broyden s method requires a finite number of real variables holder continuity of the jacobian and a rather accurate initial approximation b 0 here these three conditions are relaxed to allow for lipschitz continuous hilbert space operators that can be reasonably well approximated by a linear operator b that is close to b 0 up to an arbitrary compact perturbation to achieve local and r linear convergence broyden s method is modified by a very mild line search condition in case of a continuously differentiable system in euclidean space we obtain r superlinear convergence without restricting b 0 or requiring the jacobian to be holder continuous

convergence properties of minimization algorithms for convex constraints using a structured trust region in this paper we present a class of trust region algorithms for minimization problems within convex feasible regions in which the structure of the problem is explicitly used in the definition of the trust region this development is intended to reflect the possibility that some parts of the problem may be more accurately modelled than others a common occurrence in large scale nonlinear applications after describing the structured trust region mechanism we prove global convergence for all algorithms in our class

a convex optimization approach for computing correlated choice probabilities with many alternatives a popular discrete choice model that incorporates correlation information is the multinomial probit mnp model where the random utilities of the alternatives are chosen from a multivariate normal distribution computing the choice probabilities is challenging in the mnp model when the number of alternatives is large mishra et al ieee transactions on automatic control 2012 have proposed a semidefinite optimization approach to compute choice probabilities for the distribution of the random utilities that maximizes expected agent utility given only the mean variance and covariance information their model is referred to as the cross moment cmm model computing the choice probabilities with many alternatives is challenging in the cmm model since one needs to solve large scale semidefinite programs we develop a simpler formulation as a representative agent model by maximizing over the choice probabilities in the unit simplex where the objective function is the sum of the expected utilities and a strongly concave perturbation function by characterizing the perturbation function for the cmm model and its gradient we develop a simple first order gradient method with inexact line search to compute choice probabilities we establish local linear convergence of this algorithm under mild assumptions on the choice probabilities an implication of our results is that inverting the choice probabilities to compute the mean utilities is straightforward given any positive definite covariance matrix numerical experiments show that this method can compute choice probabilities for a large number of alternatives within a reasonable amount of time while explicitly capturing the correlation information comparisons with simulation methods for mnp and semidefinite programming methods for cmm indicate the efficacy of the method

a modified two steps levenberg marquardt method for nonlinear equations the modified levenberg marquardt mlm method to solve nonlinear equations was introduced by fan 2012 this method uses an addition of the levenberg marquardt step and an approximate lm step as the trial step at every iteration using a trust region technique the global and cubic convergence of the mlm method under the local error bound condition is proved fan 2012 recently fan proposed an accelerated mlm algorithm by using a line search strategy to generate a modified lm step and showed that the convergence rate of the algorithm is min 1 2 3 which results the cubic convergence for 1 fan 2014 in this paper by introducing an adaptive lm parameter for amlm algorithm we propose an efficient amlm algorithm the cubic convergence of the new algorithm is presented while numerical experiments show the new algorithm is promising

the distance between two random vectors with given dispersion matrices for two p dimensional random vectors x and y with dispersion matrices 11 and 22 respectively we determine that covariance matrix 0 of x and y that minimizes the l2 distance between x and y there is a dual to this problem that is of interest in another context

fresh look into the design and computation of optimal output feedback controls for linear multivariable systems this paper presents as its main result a new and efficient method for the design of optimal dynamic compensators for linear multivariable control systems the discussion starts with a fresh look into the design of optimal constant gain output feedback in this context three new ideas are introduced that prove their usefulness particularly in the dynamic compensator case 1 deterministic measurement disturbances are included in the formulation of the underlying optimization problem 2 the relationship between a deterministic and a stochastic optimal control problem is shown 3 by proper modifications of the quadratic cost function the effort involved in the numerical solution of the optimal output feedback problem is considerably reduced furthermore a canonical form is introduced for the dynamic compensator and an algorithm is developed for the numerical computation of its free parameters it is based on quasi newtonian gradient methods and an efficient line search procedure some features of the

a globally convergent bfgs method for nonlinear monotone equations without any merit functions r since 1965 there has been significant progress in the theoretical study on quasi newton methods for solving nonlinear equations especially in the local convergence analysis however the study on global convergence of quasi newton methods is relatively fewer especially for the bfgs method to ensure global convergence some merit function such as the squared norm merit function is typically used in this paper we propose an algorithm for solving nonlinear monotone equations which combines the bfgs method and the hyperplane projection method we also prove that the proposed bfgs method converges globally if the equation is monotone and lipschitz continuous without differentiability requirement on the equation which makes it possible to solve some nonsmooth equations an attractive property of the proposed method is that its global convergence is independent of any merit function we also report some numerical results to show efficiency of the proposed method

the approximation of generalized turning points newline by krylov subspace methods certain types of singular solutions of nonlinear parameter dependent operator equations were characterized by griewank and reddien 5 6 as regular solutions of suitable augmented systems for their numerical approximation an approach based on the use of krylov subspaces is here presented the application to boundary value problems is illustrated by numerical examples

a prp based residual method for large scale monotone nonlinear equations this paper presents a derivative free residual method for solving large scale monotone nonlinear equations which may be nonsmooth the method is constructed by replacing the gradients of the unmodified polak ribiere polyak prp nonlinear conjugate gradient method with the residuals and combining the hyperplane projection technique under suitable conditions we show that the proposed method converges globally in the sense that the whole iterative sequence converges to a solution of the problem some numerical results are reported to show its efficiency

a convexity principle for interacting gases a new set of inequalities is introduced based on a novel but natural interpolation between borel probability measures on r d using these estimates in lieu of convexity or rearrangement inequalities the existence and uniqueness problems are solved for a family of attracting gas models in these models the gas interacts with itself through a force which increases with distance and is governed by an equation of state p p relating pressure to density p d 1 d is assumed non decreasing for a d dimensional gas by showing that the internal and potential energies for the system are convex functions of the interpolation parameter an energy minimizing stateunique up to translationis proven to exist the concavity established for t pd q as a function of t 0 1 generalizes the brunnminkowski inequality from sets to measures 1997 academic press

on the existence of convex decompositions of partially separable functions the concept of a partially separable functionf developed in 4 is generalized to include all functionsf that can be expressed as a finite sum of element functionsf i whose hessians have nontrivial nullspacesn i such functions can be efficiently minimized by the partitioned variable metric methods described in 5 provided that each element functionf i is convex if this condition is not satisfied we attempt toconvexify the given decomposition by shifting quadratic terms among the originalf i such that the resulting modified element functions are at least locally convex to avoid tests on the numerical value of the hessian we study the totally convex case where all locally convexf with the separability structuren i 1 have a convex decomposition it is shown that total convexity only depends on the associated linear conditions on the hessian matrix in the sparse case when eachn i is spanned by cartesian basis vectors it is shown that a sparsity pattern corresponds to a totally convex structure if and only if it allows a permuted ldlt factorization without fill in

a rigorous partial justification of greene s criterion we prove several theorems that lend support to greene s criterion for the existence or not of invariant circles in twist maps in particular we show that some of the implications of the criterion are correct when the aubry mather sets are smooth invariant circles or uniformly hyperbolic we also suggest a simple modification that can work in the case that the aubry mather sets have nonzero lyapunov exponents the latter is based on a closing lemma for sets with nonzero lyapunov exponents which may have several other applications

partitioned variable metric updates for large structured optimization problems this paper presents a minimization method based on the idea of partitioned updating of the hessian matrix in the case where the objective function can be decomposed in a sum of convex element functions this situation occurs in a large class of practical problems including nonlinear finite elements calculations some theoretical and algorithmic properties of the update are discussed and encouraging numerical results are presented

broyden s method in hilbert space broyden s method is formulated for the solution of nonlinear operator equations in hilbert spaces the algorithm is proven to be well defined and a linear rate of convergence is shown under an additional assumption on the initial approximation for the derivative we prove the superlinear rate of convergence

a modified bfgs method and its global convergence in nonconvex minimization in this paper we propose a modification of the bfgs method for unconstrained optimization a remarkable feature of the proposed method is that it possesses a global convergence property even without convexity assumption on the objective function under certain conditions we also establish superlinear convergence of the method

linear models based on noisy data and the frisch scheme we address the problem of identifying linear relations among variables based on noisy measurements this is a central question in the search for structure in large data sets often a key assumption is that measurement errors in each variable are independent this basic formulation has its roots in the work of charles spearman in 1904 and of ragnar frisch in the 1930s various topics such as errors in variables factor analysis and instrumental variables all refer to alternative viewpoints on this problem and on ways to account for the anticipated way that noise enters the data in the present paper we begin by describing certain fundamental contributions by the founders of the field and provide alternative modern proofs to certain key results we then go on to consider a modern viewpoint and novel numerical techniques to the problem the central theme is expressed by the frisch kalman dictum which calls for identifying a noise contribution that allows a maximal number of simultaneous linear relations a

numerical calculation of domains of analyticity for perturbation theories in the presence of small divisors we study numerically the complex domains of validity for kam theory in generalized standard mappings we compare methods based on pade approximants and methods based on the study of periodic orbits

a quasi newton method for elliptic boundary value problems for some boundary value problems for equations of the form nabla 2 u f x u nabla u 0 with linear boundary conditions we give a convergence analysis for a quasi newton method that converges superlinearly in the c 1 norm this scheme for infinite dimensional problems becomes a method for the finite dimensional problems that are produced by discretizing the differential equation this method is a generalization of a method proposed by hart and soul for discretizations of two point boundary value problems