on the bridge between combinatorial optimization and nonlinear optimization a family of semidefinite bounds for 0 1 quadratic problems leading to quasi newton methods this article presents a family of semidefinite programming bounds obtained by lagrangian duality for 0 1 quadratic optimization problems with linear or quadratic constraints these bounds have useful computational properties they have a good ratio of tightness to computing time they can be optimized by a quasi newton method and their final tightness level is controlled by a real parameter these properties are illustrated on three standard combinatorial optimization problems unconstrained 0 1 quadratic optimization heaviest k subgraph and graph bisection

a numerical study of the limited memory bfgs method and the truncated newton method for large scale optimization this paper examines the numerical performances of two methods for large scale optimization a limited memory quasi newton method l bfgs and a discrete truncated newton method tn various ways of classifying test problems are discussed in order to better understand the types of problems that each algorithm solves well the l bfgs and tn methods are also compared with the polak ribiere conjugate gradient method

identification of boundary conditions in a nonlinear shallow water flow abstract in this paper we present sufficient smoothness results on the solution of a nonlinear shallow water problem in order to obtain an existence result for the associated adjoint problem we introduce a necessary but not sufficient existence condition for the functional minimum the control is made on the velocity boundary condition in considering observations at isolated points moreover we give a set of numerical results obtained in a real situation

variable metric methods for unconstrainted optimization and nonlinear least squares variable metric or quasi newton methods are well known and commonly used in connection with unconstrained optimization since they have good theoretical and practical convergence properties although these methods were originally developed for small and moderate size dense problems their modifications based either on sparse partitioned or limited memory updates are very efficient on large scale sparse problems very significant applications of these methods also appear in nonlinear least squares approximation and nonsmooth optimization in this contribution we give an extensive review of variable metric methods and their use in various optimization fields

global convergence of a cass of quasi newton methods on convex problems we study the global convergence properties of the restricted broyden class of quasi newton methods when applied to a convex objective function we assume that the line search satisfies a standard sufficient decrease condition and that the initial hessian approximation is any positive definite matrix we show global and superlinear convergence for this class of methods except for dfp this generalizes powell s well known result for the bfgs method the analysis gives us insight into the properties of these algorithms in particular it shows that dfp lacks a very desirable self correcting property possessed by bfgs

gradient constitutive relations numerical aspects and application to gradient damage abstract the description of localisation and structural failure through local strain softening constitutive laws leads to ill posed problems to retrieve a well posed problem the material state is described through the gradients of internal variables in addition to the strain and the internal variables themselves under some assumptions it is shown that these gradient laws can be cast into a variational formulation after time discretisation the evolution of the internal variable field is characterised by a minimum principle a specific algorithm is proposed to perform the minimisation in the context of finite elements finally it is applied to a gradient brittle damage model

some numerical experiments with variable storage quasi newton algorithms this paper describes some numerical experiments with variable storage quasi newton methods for the optimization of some large scale models coming from fluid mechanics and molecular biology in addition to assessing these kinds of methods in real life situations we compare an algorithm of a buckley with a proposal by j nocedal the latter seems generally superior provided that careful attention is given to some nontrivial implementation aspects which concern the general question of properly initializing a quasi newton matrix in this context we find it appropriate to use a diagonal matrix generated by an update of the identity matrix so as to fit the rayleigh ellipsoid of the local hessian in the direction of the change in the gradient also a variational derivation of some rank one and rank two updates in hilbert spaces is given

a hessian based method for uncertainty quantification in global ocean state estimation national science foundation u s collaboration in mathematical geosciences grant 0934404

conjugate gradient methods less dependent on conjugacy this paper reviews some recent conjugate gradient methods for nonlinear minimization and outlines some promising directions for further development

the method of successive affine reduction for nonlinear minimization the traditional development of conjugate gradient cg methods emphasizes notions of conjugacy and the minimization of quadratic functions the associated theory of conjugate direction methods strictly a branch of numerical linear algebra is both elegant and useful for obtaining insight into algorithms for nonlinear minimization nevertheless it is preferable that favorable behavior on a quadratic be a consquence of a more general approach one which fits in more naturally with newton and variable metric methods we give new cg algorithms along these lines and discuss some of their properties along with some numerical supporting evidence

stochastic quasi newton molecular simulations we report a new and efficient factorized algorithm for the determination of the adaptive compound mobility matrix b in a stochastic quasi newton method s qn that does not require additional potential evaluations for one dimensional and two dimensional test systems we previously showed that s qn gives rise to efficient configurational space sampling with good thermodynamic consistency c d chau g j a sevink and j g e m fraaije j chem phys 128 244110 2008 potential applications of s qn are quite ambitious and include structure optimization analysis of correlations and automated extraction of cooperative modes however the potential can only be fully exploited if the computational and memory requirements of the original algorithm are significantly reduced in this paper we consider a factorized mobility matrix b jj t and focus on the nontrivial fundamentals of an efficient algorithm for updating the noise multiplier j the new algorithm requires o n 2 multiplications per time step instead of the o n 3 multiplications in the original scheme due to choleski decomposition in a recursive form the update scheme circumvents matrix storage and enables limited memory implementation in the spirit of the well known limited memory broyden fletcher goldfarb shanno l bfgs method allowing for a further reduction of the computational effort to o n we analyze in detail the performance of the factorized fsu and limited memory l fsu algorithms in terms of convergence and multiscale sampling for an elementary but relevant system that involves multiple time and length scales finally we use this analysis to formulate conditions for the simulation of the complex high dimensional potential energy landscapes of interest

diagnosis and impacts of non gaussianity of innovations in data assimilation most of the atmospheric and oceanic data assimilation da schemes rely on the best linear unbiased estimator blue which is sub optimal if errors of assimilated data are non gaussian thus calling for a full bayesian data assimilation this paper contributes to the study of the non gaussianity of errors in the observational space possible sources of non gaussianity range from the inherent statistical skewness and positiveness of some physical observables e g moisture chemical species the nonlinearity both of the data assimilation models and of the observation operators among others deviations from gaussianity can be justified from a priori hypotheses or inferred from statistical diagnostics of innovations observation minus background leading to consistency relationships between the error statistics from samples of observations and backgrounds as well as their specified error variances we evaluate some measures of the innovation non gaussianity such as the skewness kurtosis and negentropy under the assumption of additive errors and by relating statistical moments from both data errors and innovations we identify potential sources of the innovation non gaussianity these sources range from 1 univariate error non gaussianity 2 nonlinear correlations between errors 3 spatio temporal variability of error variances heteroscedasticity and 4 multiplicative noise observational and background errors are often assumed independent this leads to variance dependent bounds for the skewness and the kurtosis of errors from innovation statistics we assess the potential da impact of some scenarios of non gaussian errors this impact is measured through the mean square difference between the blue and the minimum variance unbiased estimator mvue obtained with univariate observations and background estimates in order to accomplish this we compute maximum entropy probability density functions pdfs of the errors constrained by the first four order moments these pdfs are then used to compute the bayesian posterior pdf and the mvue the referred impact is studied for a large range of statistical moments being higher for skewed innovations and growing in average with the skewness of data errors specially if the skewnesses have the same sign an application has been performed to the quality accepted ecmwf innovations of brightness temperatures of a set of high resolution infrared sounder hirs channels in this context the mvue has led in some extreme cases to a potential reduction of 20 60 of the posterior error variance as compared to the blue specially for extreme values of the innovations

a algorithm for convex minimization for convex minimization we introduce an algorithm based on space decomposition the method uses a bundle subroutine to generate a sequence of approximate proximal points when a primal dual track leading to a solution and zero subgradient pair exists these points approximate the primal track points and give the algorithm s or corrector steps the subroutine also approximates dual track points that are gradients needed for the method s newton predictor steps with the inclusion of a simple line search the resulting algorithm is proved to be globally convergent the convergence is superlinear if the primal dual track points and the objective s hessian are approximated well enough

four dimensional variational data assimilation for doppler radar wind data all forecast models whether they represent the state of the weather the spread of a disease or levels of economic activity contain unknown parameters these parameters may be the model s initial conditions its boundary conditions or other tunable parameters which have to be determined four dimensional variational data assimilation 4d var is a method of estimating this set of parameters by optimizing the fit between the solution of the model and a set of observations which the model is meant to predict although the method of 4d var described in this paper is not restricted to any particular system the application described here has a numerical weather prediction nwp model at its core and the parameters to be determined are the initial conditions of the model the purpose of this paper is to give a review covering assimilation of doppler radar wind data into a nwp model some associated problems such as sensitivity to small variations in the initial conditions or due to small changes in the background variables and biases due to nonlinearity are also studied

boundary control of burgers equation a numerical approach we describe a methodology for solving boundary control problems for the viscous burgers equation the aim is to identify boundary forcing in order to ensure the best fit between data and model results by minimizing a functional which measures model and data discrepancies a continuous variational formulation involving the adjoint technique is used and its counterpart discretized version is obtained with a matrix approach as a guideline a particular discretization of the nonlinear term of the equation is performed in order to insure for the gradient of the functional to be minimized a discretized expression which can be directly deduced from its continuous counterpart numerical experiments validate the proposed optimization algorithm

on diagonally preconditioning the 2 step bfgs method with accumulated steps for linearly constrained nonlinear programming abstract we present an algorithm for very large scale linearly constrained nonlinear programming lcnp based on a limited storage quasi newton method in large scale programming solving the reduced newton equation at each iteration can be expensive and may not be justified when far from a local solution besides the amount of storage required by the reduced hessian matrix and even the computing time for its quasi newton approximation may be prohibitive an alternative based on the reduced truncated newton methodology that has proved to be satisfactory for large scale problems is not recommended for very large scale problems since it requires an additional gradient evaluation and the solving of two systems of linear equations per each minor iteration we recommend a 2 step bfgs approximation of the inverse of the reduced hessian matrix that does not require to store any matrix since the product matrix vector is the vector to be approximated it uses the reduced gradient and information from two previous iterations and the so termed restart iteration a diagonal direct bfgs preconditioning is used

the barzilai and borwein gradient method for the large scale unconstrained minimization problem the barzilai and borwein gradient method for the solution of large scale unconstrained minimization problems is considered this method requires few storage locations and very inexpensive computations furthermore it does not guarantee descent in the objective function and no line search is required recently the global convergence for the convex quadratic case has been established however for the nonquadratic case the method needs to be incorporated in a globalization scheme in this work a nonmonotone line search strategy that guarantees global convergence is combined with the barzilai and borwein method this strategy is based on the nonmonotone line search technique proposed by grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 numerical results to compare the behavior of this method with recent implementations of the conjugate gradient method are presented these results indicate that the global barzilai and borwein method may allow some significant reduction in the number of line searches and also in the number of gradient evaluations

kullback proximal algorithms for maximum likelihood estimation accelerated algorithms for maximum likelihood image reconstruction are essential for emerging applications such as three dimensional 3 d tomography dynamic tomographic imaging and other high dimensional inverse problems in this paper we introduce and analyze a class of fast and stable sequential optimization methods for computing maximum likelihood estimates and study its convergence properties these methods are based on a proximal point algorithm implemented with the kullback liebler kl divergence between posterior densities of the complete data as a proximal penalty function when the proximal relaxation parameter is set to unity one obtains the classical expectation maximization em algorithm for a decreasing sequence of relaxation parameters relaxed versions of em are obtained which can have much faster asymptotic convergence without sacrifice of monotonicity we present an implementation of the algorithm using more s 1983 trust region update strategy for illustration the method is applied to a nonquadratic inverse problem with poisson distributed data

conjugate gradient methods using quasi newton updates with inexact line searches abstract conjugate gradient methods are conjugate direction or gradient deflection methods which lie somewhere between the method of steepest descent and newton s method their principal advantage is that they do not require the storage of any matrices as in newton s method or as in quasi newton methods and they are designed to converge faster than the method of steepest descent unlike quasi newton or variable metric methods these are fixed metric methods in which the search direction at each iteration is based on an approximation to the inverse hessian constructed by updating a fixed symmetric positive definite matrix typically the identity matrix the resulting approximation is usually not symmetric although some variants force symmetry and hence derive memoryless quasi newton methods in this paper we present a scaled modified version of the conjugate gradient method suggested by perry which employs the quasi newton condition rather than conjugacy under inexact line searches in order to derive the search directions the analysis is extended to the memoryless quasi newton modification of this method as suggested by shanno computational experience on standard test problems indicates that the proposed method along with beale and powell s restarts improves upon existing conjugate gradient strategies

a fast nonrigid image registration with constraints on the jacobian using large scale constrained optimization this paper presents a new nonrigid monomodality image registration algorithm based on b splines the deformation is described by a cubic b spline field and found by minimizing the energy between a reference image and a deformed version of a floating image to penalize noninvertible transformation we propose two different constraints on the jacobian of the transformation and its derivatives the problem is modeled by an inequality constrained optimization problem which is efficiently solved by a combination of the multipliers method and the l bfgs algorithm to handle the large number of variables and constraints of the registration of 3 d images numerical experiments are presented on magnetic resonance images using synthetic deformations and atlas based segmentation

compact dynamical model of brain activity a compact physiologically based mean field formulation of brain dynamics is proposed to model observed brain activity and electroencephalographic eeg signals in contrast to existing formulations which are more detailed and complicated our model is described by a single second order delay differential equation that encapsulates salient aspects of the physiology the model captures essential features of activity mediated by fast corticocortical connections and delayed feedbacks via extracortical pathways and external stimuli in the linear regime these features can be simply expressed by three coefficients derived from the properties of these physiological pathways and explicit nonlinear approximations are also derived this compact model successfully reproduces the main features of experimental eeg s and the predictions of previous models including resonance peaks in eeg spectra and nonlinear dynamics as an illustration key features of the dynamics of epileptic seizures are shown to be reproduced by the model due to its compact form the model will facilitate insight into nonlinear brain dynamics via standard nonlinear techniques and will guide analysis and investigation of more complex models it is thus a useful tool for analyzing complex brain activity especially when it exhibits low dimensional dynamics

integrals containing confluent hypergeometric functions with applications to perturbed singular potentials we show that many integrals containing products of confluent hypergeometric functions follow directly from one single integral that has a very simple formula in terms of appell s double series f2 we present some techniques for computing such series applications requiring the matrix elements of singular potentials and the perturbed kratzer potential are presented

identification of an optimal derivatives approximation by variational data assimilation variational data assimilation technique applied to identification of optimal approximations of derivatives near boundary is discussed in frames of one dimensional wave equation simplicity of the equation and of its numerical scheme allows us to discuss in detail as the development of the adjoint model and assimilation results it is shown what kind of errors can be corrected by this control and how these errors are corrected this study is carried out in view of using this control to identify optimal numerical schemes in coastal regions of ocean models

space decomposition minimization method for large scale minimization problems abstract this paper introduces a set of new algorithms called the space decomposition minimization sdm algorithms that decomposes the minimization problem into subproblems if the decomposed space subproblems are not coupled to each other they can be solved independently with any convergent algorithm otherwise iterative algorithms presented in this paper can be used furthermore if the design space is further decomposed into one dimensional decomposed spaces the solution can be found directly using one dimensional search methods a hybrid algorithm that yields the benefits of the sdm algorithm and the conjugate gradient method is also given an example that demonstrates application of sdm algorithm to the learning of a single layer perceptron neural network is presented and five large scale numerical problems are used to test the sdm algorithms the results obtained are compared with results from the conjugate gradient method

global convergence properties of conjugate gradient methods for optimization this paper explores the convergence of nonlinear conjugate gradient methods without restarts and with practical line searches the analysis covers two classes of methods that are globally convergent on smooth nonconvex functions some properties of the fletcher reeves method play an important role in the first family whereas the second family shares an important property with the polak ribiere method numerical experiments are presented

propagation and stability of waves of electrical activity in the cerebral cortex nonlinear equations are introduced to model the behavior of the waves of cortical electrical activity that are responsible for signals observed in electroencephalography these equations incorporate nonlinearities axonal and dendritic lags excitatory and inhibitory neuronal populations and the two dimensional nature of the cortex while rendering nonlinear features far more tractable than previous formulations both analytically and numerically the model equations are first used to calculate steady state levels of cortical activity for various levels of stimulation dispersion equations for linear waves are then derived analytically and an analytic expression is found for the linear stability boundary beyond which a seizure will occur the effects of boundary conditions in determining global eigenmodes are also studied in various geometries and the corresponding eigenfrequencies are found numerical results confirm the analytic ones which are also found to reproduce existing results in the relevant limits thereby elucidating the limits of validity of previous approximations

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

qn like variable storage conjugate gradients both conjugate gradient and quasi newton methods are quite successful at minimizing smooth nonlinear functions of several variables and each has its advantages in particular conjugate gradient methods require much less storage to implement than a quasi newton code and therefore find application when storage limitations occur they are however slower so there have recently been attempts to combine cg and qn algorithms so as to obtain an algorithm with good convergence properties and low storage requirements one such method is the code conmin due to shanno and phua it has proven quite successful but it has one limitation it has no middle ground in that it either operates as a quasi newton code using o n 2 storage locations or as a conjugate gradient code using 7n locations but it cannot take advantage of the not unusual situation where more than 7n locations are available but a quasi newton code requires an excessive amount of storage

a posteriori error estimators for the stokes equations we present two a posteriori error estimators for the mini element discretization of the stokes equations one is based on a suitable evaluation of the residual of the finite element solution the other one is based on the solution of suitable local stokes problems involving the residual of the finite element solution both estimators are globally upper and locally lower bounds for the error of the finite element discretization numerical examples show their efficiency both in estimating the error and in controlling an automatic self adaptive mesh refinement process the methods presented here can easily be generalized to the navier stokes equations and to other discretization schemes

unconstrained optimization of real functions in complex variables nonlinear optimization problems in complex variables are frequently encountered in applied mathematics and engineering applications such as control theory signal processing and electrical engineering optimization of these problems often requires a first or second order approximation of the objective function to generate a new step or descent direction however such methods cannot be applied to real functions of complex variables because they are necessarily nonanalytic in their argument i e the taylor series expansion in their argument alone does not exist to overcome this problem the objective function is usually redefined as a function of the real and imaginary parts of its complex argument so that standard optimization methods can be applied however this approach may needlessly disguise any inherent structure present in the derivatives of such complex problems although little known it is possible to construct an expansion of the objective function in its original complex variables by noti

some reduction and transformation formulas for the appell hypergeometric function f2 abstract an integral representation of the appell series f 2 1 2 1 2 x y is used here to obtain several finite sum expansions in terms of the less cumbersome hypergeometric functions 2f1 and 3f2 in the case when the parameters 1 2 1 and 2 are all positive integers some of our results may be seen as a generalization of the finite sum expansions of the appell series f 1 obtained by cuyt et al j comput appl math 5 1999 213 219 other important consequences as well as potential applications of our results are discussed and a listing of some useful reduction and transformation formulas is provided

a numerical method for incompressible viscous flow simulation we describe a numerical scheme for computing time dependent solutions of the incompressible navier stokes equations in the primitive variable formulation this scheme uses finite elements for the space discretization and operator splitting techniques for the time discretization the resulting discrete equations are solved using specialized nonlinear optimization algorithms that are computationally efficient and have modest storage requirements the basic numerical kernel is the preconditioned conjugate gradient method for symmetric positive definite sparse matrix systems which can be efficiently implemented on the architectures of vector and parallel processing supercomputers

reduced storage quasi newton trust region approaches to function optimization in this paper we consider several algorithms for reducing the storage when using a quasi newton method in a dogleg trust region setting for minimizing functions of many variables secant methods require o n2 locations to store an approximate hessian and o n2 operations per iteration when minimizing a function of n variables this storage requirement becomes impractical when n becomes large our algorithms use a bfgs update and require kn storage and 4kn o k2 operations per iteration but they may require more iterations than the standard trust region techniques typically k is between 10 and 100 our dogleg trust region strategies involve expressions with matrix products with both the inverse of this hessian and with the hessian itself our techniques for updating expressions for the hessian and its inverse can be used to improve the performance of line search limited memory algorithms

globally convergent limited memory bundle method for large scale nonsmooth optimization many practical optimization problems involve nonsmooth that is not necessarily differentiable functions of thousands of variables in the paper haarala miettinen makela optimization methods and software 19 2004 pp 673 692 we have described an efficient method for large scale nonsmooth optimization in this paper we introduce a new variant of this method and prove its global convergence for locally lipschitz continuous objective functions which are not necessarily differentiable or convex in addition we give some encouraging results from numerical experiments

a survey of quasi newton methods with reduced storage a family of non linear solution methods is investigated based on limited memory quasi newton updates depending upon the number of updates and the updating formula a number of solution schemes may be constructed such as the conjugate and secant newton methods the conventional and modified newton raphson methods and a variety of quasi newton updates under the proposed implementation the preconditioned truncated lanczos method is used for the solution of the linearized problem in each non linear iteration the complete factorization of the stiffness matrix is avoided and large scale problems can be solved efficiently both in terms of computing time and storage the non linear iterative scheme is properly modified to account for loading variation inside the increment in order to be able to trace post critical equilibrium paths

induction heating processes optimization a general optimal control approach a general automatic optimization procedure coupled to a finite element induction heating process simulation has been developed the mathematical model and the numerical methods are presented along with results validating the model the first part of this paper presents the direct induction heating mathematical model the related main numerical choices and especially the ultra weak coupling procedure the general optimization problem is then presented with the full detailed transposition of the ultra weak coupling procedure to the adjoint problem numerical results provided at the end prove the efficiency and robustness of the adjoint model in optimizing induction heating processes

analogues of dixon s and powell s theorems for unconstrained minimization with inexact line searches dixon s theorem math programming 2 1972 pp 383 387 states that all variable metric methods in the broyden class develop identical iterates when line searches are exact powell s theorem rep tp 495 aere harwell england 1972 is a variant on this which states that under similar conditions the hessian approximation developed by a bfgs update at any step is independent of the updates used at earlier steps by modifying the way in which search directions are defined we show how to remove the restrictive assumption on line searches in these two theorems we show also that the bfgs algorithm modified in this way is equivalent to the three term recurrence ttr method on quadratic functions algorithmic implications are discussed and the results of some numerical experimentation are reported

the quasi cauchy relation and diagonal updating the quasi cauchy qc relation is the weak quasi newton relation of dennis and wolkowicz siam j numer anal 30 1993 pp 1291 1314 with the added restriction that full matrices are replaced by diagonal matrices this relation is justified and explored and in particular two basic variational techniques for updating diagonal matrices that satisfy it are formulated for purposes of illustration a numerical experiment is described where a diagonal updated matrix with hereditary positive definiteness is used to precondition cauchy s steepest descent direction the resulting qc algorithm is shown to be significantly accelerated in the concluding section the following topics are briefly discussed additional variational principles use of diagonal updates within other optimization algorithms together with some further numerical experience summarized in an appendix and an interesting connection between qc diagonal updating and trust region techniques

controlling limit cycle behaviors of brain activity the limit cycles of brain activity are studied using a compact continuum model that reproduces the main features of electroencephalographic signals including bifurcations of fixed points and limit cycles in seizures frequencies and amplitudes are predicted analytically and related to physiology gaussian stimuli yield two distinct evoked responses in the linearly stable zone consistent with experiment limit cycles can be initiated or suppressed by control signals or stimuli

a finite sum representation of the appell series f 1 a b b c x y abstract we use picard s integral representation of the appell series f 1 a b b c x y for re a 0 re c a 0 to obtain a finite sum algebraic representation of f1 in the case when a b b and c are positive integers with c a the series converges for x y and we show that f 1 a b b c x y has two overlaying singularities at each of the points x 1 and y 1 one polar and one logarithmic in nature when a b b c n with c a

numerical experience with limited memory quasi newton and truncated newton methods computational experience with several limited memory quasi newton and truncated newton methods for unconstrained nonlinear optimization is described comparative tests were conducted on a well known test library j j more b s gaxbow and k e hillstrom acm trans math software 7 1981 pp 17 41 on several synthetic problems allowing control of the clustering of eigenvalues in the hessian spectrum and on some large scale problems in oceanography and meteorology the results indicate that among the tested limited memory quasi newton methods the l bfgs method d c liu and j nocedal math programming 45 1989 pp 503 528 has the best overall performance for the problems examined the numerical performance of two truncated newton methods differing in the inner loop solution for the search vector is competitive with that of l bfgs

migration based traveltime waveform inversion of 2 d simple structures a synthetic example migration based traveltime mbtt formulation provides algorithms for automatically determining background velocities from full waveform surface seismic reflection data using local optimization methods in particular it addresses the difficulty of the nonconvexity of the least squares data misfit function the method consists of parameterizing the reflectivity in the time domain through a migration step and providing a multiscale representation for the smooth background velocity we present an implementation of the mbtt approach for a 2 d finite difference fd full wave acoustic model numerical analysis on a 2 d synthetic example shows the ability of the method to find much more reliable estimates of both long and short wavelengths of the velocity than the classical least squares approach even when starting from very poor initial guesses this enlargement of the domain of attraction for the global minima of the least squares misfit has a price each evaluation of the new objective function requires besides the usual fd full wave forward modeling an additional full wave prestack migration hence the fd implementation of the mbtt approach presented in this paper is expected to provide a useful tool for the inversion of data sets of moderate size

a family of variable metric proximal methods we consider conceptual optimization methods combining two ideas the moreau yosida regularization in convex analysis and quasi newton approximations of smooth functions we outline several approaches based on this combination and establish their global convergence then we study theoretically the local convergence properties of one of these approaches which uses quasi newton updates of the objective function itself also we obtain a globally and superlinearly convergent bfgs proximal method at each step of our study we single out the assumptions that are useful to derive the result concerned

on the realization of the wolfe conditions in reduced quasi newton methods for equality constrained optimization this paper describes a reduced quasi newton method for solving equality constrained optimization problems a major difficulty encountered by this type of algorithm is the design of a consistent technique for maintaining the positive definiteness of the matrices approximating the reduced hessian of the lagrangian a new approach is proposed in this paper the idea is to search for the next iterate along a piecewise linear path the path is designed so that some generalized wolfe conditions can be satisfied these conditions allow the algorithm to sustain the positive definiteness of the matrices from iteration to iteration by a mechanism that has turned out to be efficient in unconstrained optimization

2 d and 3 d interpretation of electrical tomography measurements part 2 the inverse problem the interpretation of borehole to borehole electrical measurements requires solving an inverse problem for a given class of model geometries the usual approach to an inverse problem includes a model dependent task i e forward modeling and a generic task i e an optimization scheme we have developed an optimization algorithm using a nonlinear inversion technique this algorithm allows recovery of a possible resistivity distribution in an investigated zone between two boreholes or in the vicinity of them this resistivity distribution is defined as a set of 2 d or 3 d volumes of constant resistivity the inversion procedure minimizes a least squares term plus a damping term this latter term seeks to minimize the roughness of the solution an improved form of this smoothness term may enhance the spatial resolution of the resistivity image assuming that the resistivity contrast is known a priori this reconstruction algorithm has been tested for both 2 d and 3 d geometries these inversion tests we

newton like methods with limited storage for the solution of elasto viscoplasticity problems quasi newton and secant newton methods have been successful in solving problems of elasto plasticity in this paper these methods are applied to elasto viscoplastic analyses in an algorithm coupling the global solution for displacement increments with local newton raphson stress iteration at each sampling point the familiar rank two dfp and bfgs methods are generalized to the broyden family of algorithms and this generalization is also used to produce a secant newton algorithm corresponding to hoshino s method consideration is given to the most effective strategy when only a limited number of quasi newton iterates may be stored overwriting re start or continuation in secant newton form results are presented for a tunnelling problem using both plane strain and axisymmetric formulations

numerical simulation of tridimensional electromagnetic shaping of liquid metals we describe a numerical method to compute free surfaces in electromagnetic shaping and levitation of liquid metals we use an energetic variational formulation and optimization techniques to compute a critical point the surfaces are represented by piecewise linear finite elements each step of the algorithm requires solving an elliptic boundary value problem in the exterior of the intermediate surfaces this is done by using an integral representation on these surfaces

an efficient algorithm for large scale nonlinear programming problems with simple bounds on the variables in this paper we present a new conjugate gradient algorithm for nonlinear programming problems with simple bounds on the variables the method is an extension of our conjugate gradient algorithm described in r pytlak ima j numer anal 14 1994 pp 443 460 the simple constraints on the variables are treated by a projection under mild assumptions the method is globally convergent the algorithm has the property whereby many constraints can leave or enter an active set of constraints at one iteration if the strict complementarity condition is satisfied the active set at a solution is identified in a finite number of iterations the algorithm has been tested on problems with more than 10 000 variables

delayed feedback control of collective synchrony an approach to suppression of pathological brain rhythms we suggest a method for suppression of synchrony in a globally coupled oscillator network based on the time delayed feedback via the mean field having in mind possible applications for suppression of pathological rhythms in neural ensembles we present numerical results for different models of coupled bursting neurons a theory is developed based on the consideration of the synchronization transition as a hopf bifurcation

termination and equivalence results for conjugate gradient algorithms this paper considers the problem of minimizing a functionf x when linear equality constraints onx could be present and when an algorithm is to be used which may employ both conjugate gradient and quasi newton steps the conjugate gradient algorithm is examined when a preconditioner is used which may be only positive semi definite quite a general theorem is given about the equivalence between conjugate gradient steps and quasi newton steps which includes several previously published results as special cases some results are also given for the case where some line searches are inexact

automatic preconditioning by limited memory quasi newton updating this paper proposes a preconditioner for the conjugate gradient method cg that is designed for solving systems of equations ax bi with different right hand side vectors or for solving a sequence of slowly varying systems ak x bk the preconditioner has the form of a limited memory quasi newton matrix and is generated using information from the cg iteration the automatic preconditioner does not require explicit knowledge of the coefficient matrix a and is therefore suitable for problems where only products of a times a vector can be computed numerical experiments indicate that the preconditioner has most to offer when these matrix vector products are expensive to compute and when low accuracy in the solution is required the effectiveness of the preconditioner is tested within a hessian free newton method for optimization and by solving certain linear systems arising in finite element models

adjoint based acoustic inversion for the physical characterization of a shallow water environment recently the concept of adjoint modeling has been introduced in shallow water acoustics for solving inverse problems analytical adjoints have been derived for normal modes and for both the standard parabolic equation and claerbout s wide angle approximation wape this paper proposes the application of a semiautomatic adjoint approach that has been successfully applied in the past for multidimensional variational data assimilation in meteorological and climate modeling starting from a modular graph representation of the underlying forward model a programming tool facilitates the generation and coding of both the tangent linear and the adjoint models the potential of this numerical adjoint approach for the physical characterization of a shallow water environment is illustrated with two applications for geoacoustic inversion and ocean acoustic tomography using claerbout s wape in combination with nonlocal boundary conditions furthermore the adjoint optimization is extended to multiple frequencies and

some remarks on conjugate gradient methods without line search abstract the conjugate gradient method is widely used in unconstrained optimization especially in case of large scale problems however the line search in the conjugate gradient method is sometimes very difficult or prohibitively expensive in sun and zhang j sun j p zhang global convergence of conjugate gradient methods without line search annals of operations research 103 2001 161 173 it is shown that by taking a fixed steplength k defined by the formula k g k t d k d k t q k d k the conjugate gradient method is globally convergent for several popular choices of k without line search in the simplest case all qk could be identity matrices however it would not even guarantee the descent property in this paper we study some methods to select qk which are based on the amount of descent and are superior to taking qk i the unit matrix

the newton and cauchy perspectives on computational nonlinear optimization the methods of newton and cauchy provide complementary perspectives on computational nonlinear optimization and a study of the interplay between these two classical approaches sheds new light on other more recent methods for minimizing a nonlinear function a coherent picture newton cauchy framework emerges at the root level which relates the basic methods of the subject to one another and reveals some promising directions for further algorithmic development by way of illustration a particular algorithm motivated by the framework is formulated and studied numerically

limited memory interior point bundle method for large inequality constrained nonsmooth minimization abstract many practical optimization problems involve nonsmooth that is not necessarily differentiable functions of hundreds or thousands of variables with various constraints in this paper we describe a new efficient adaptive limited memory interior point bundle method for large possible nonconvex nonsmooth inequality constrained optimization the method is a hybrid of the nonsmooth variable metric bundle method and the smooth limited memory variable metric method and the constraint handling is based on the primal dual feasible direction interior point approach the preliminary numerical experiments to be presented confirm the effectiveness of the method

solving k cluster problems to optimality with semidefinite programming this paper deals with the computation of exact solutions of a classical np hard problem in combinatorial optimization the k cluster problem this problem consists in finding a heaviest subgraph with k nodes in an edge weighted graph we present a branch and bound algorithm that applies a novel bounding procedure based on recent semidefinite programming techniques we use new semidefinite bounds that are less tight than the standard semidefinite bounds but cheaper to get the experiments show that this approach is competitive with the best existing ones