the primal dual active set strategy as a semismooth newton method this paper addresses complementarity problems motivated by constrained optimal control problems it is shown that the primal dual active set strategy which is known to be extremely efficient for this class of problems and a specific semismooth newton method lead to identical algorithms the notion of slant differentiability is recalled and it is argued that the max function is slantly differentiable in lp spaces when appropriately combined with a two norm concept this leads to new local convergence results of the primal dual active set strategy global unconditional convergence results are obtained by means of appropriate merit functions

stability in the presence of degeneracy and error estimation given an approximation to a local minimizer to a nonlinear optimization problem and to associated multipliers we obtain a tight error estimate in terms of the violation of the first order conditions our results apply to degenerate optimization problems where independence of the active constraint gradients and strict complementarity can be violated

modifying sqp for degenerate problems most local convergence analyses of the sequential quadratic programming sqp algorithm for nonlinear programming make strong assumptions about the solution namely that the active constraint gradients are linearly independent and that there are no weakly active constraints in this paper we establish a framework for variants of sqp that retain the characteristic superlinear convergence rate even when these assumptions are relaxed proving general convergence results and placing some recently proposed sqp variants in this framework we discuss the reasons for which implementations of sqp often continue to exhibit good local convergence behavior even when the assumptions commonly made in the analysis are violated finally we describe a new algorithm that formalizes and extends standard sqp implementation techniques and we prove convergence results for this method also

on the r superlinear convergence of the kkt residuals generated by the augmented lagrangian method for convex composite conic programming due to the possible lack of primal dual type error bounds it was not clear whether the karush kuhn tucker kkt residuals of the sequence generated by the augmented lagrangian method alm for solving convex composite conic programming cccp problems converge superlinearly in this paper we resolve this issue by establishing the r superlinear convergence of the kkt residuals generated by the alm under only a mild quadratic growth condition on the dual of cccp with easy to implement stopping criteria for the augmented lagrangian subproblems this discovery may help to explain the good numerical performance of several recently developed semismooth newton cg based alm solvers for linear and convex quadratic semidefinite programming

newton type methods for optimization problems without constraint qualifications we consider equality constrained optimization problems where a given solution may not satisfy any constraint qualification but satisfies the standard second order sufficient condition for optimality based on local identification of the rank of the constraints degeneracy via the singular value decomposition we derive a modified primal dual optimality system whose solution is locally unique nondegenerate and thus can be found by standard newton type techniques using identification of active constraints we further extend our approach to mixed equality and inequality constrained problems and to mathematical programs with complementarity constraints mpcc in particular for mpcc we obtain a local algorithm with quadratic convergence under the second order sufficient condition only without any constraint qualifications not even the special mpcc constraint qualifications

binary n step markov chains and long range correlated systems a theory of systems with long range correlations based on the consideration of binary n step markov chains is developed in our model the conditional probability that the i th symbol in the chain equals zero or unity is a linear function of the number of unities among the preceding n symbols the correlation and distribution functions as well as the variance of number of symbols in the words of arbitrary length l are obtained analytically and numerically if the persistent correlations are not extremely strong the variance is shown to be nonlinearly dependent on l a self similarity of the studied stochastic process is revealed the applicability of the developed theory to the coarse grained written and dna texts is discussed

an adaptive finite element moreau yosida based solver for a coupled cahn hilliard navier stokes system an adaptive a posteriori error estimator based finite element method for the numerical solution of a coupled cahn hilliard navier stokes system with a double obstacle homogenous free interfacial energy density is proposed a semi implicit euler scheme for the time integration is applied which results in a system coupling a quasi stokes or oseen type problem for the fluid flow to a variational inequality for the concentration and the chemical potential according to the cahn hilliard model 16 a moreau yosida regularization is employed which relaxes the constraints contained in the variational inequality and thus enables semi smooth newton solvers with locally superlinear convergence in function space moreover upon discretization this yields a mesh independent method for a fixed relaxation parameter for the finite dimensional approximation of the concentration and the chemical potential piecewise linear and globally continuous finite elements are used and for the numerical approximation of the fluid velocity taylor hood finite elements are employed the paper ends by a report on numerical examples showing the efficiency of the new method

a practical relative error criterion for augmented lagrangians this paper develops a new error criterion for the approximate minimization of augmented lagrangian subproblems this criterion is practical since it is readily testable given only a gradient or subgradient of the augmented lagrangian it is also relative in the sense of relative error criteria for proximal point algorithms in particular it uses a single relative tolerance parameter rather than a summable parameter sequence our analysis first describes an abstract version of the criterion within rockafellar s general parametric convex duality framework and proves a global convergence result for the resulting algorithm specializing this algorithm to a standard formulation of convex programming produces a version of the classical augmented lagrangian method with a novel inexact solution condition for the subproblems finally we present computational results drawn from the cute test set including many nonconvex problems indicating that the approach works well in practice

stabilized sequential quadratic programming for optimization and a stabilized newton type method for variational problems the stabilized version of the sequential quadratic programming algorithm ssqp had been developed in order to achieve fast convergence despite possible degeneracy of constraints of optimization problems when the lagrange multipliers associated to a solution are not unique superlinear convergence of ssqp had been previously established under the strong second order sufficient condition for optimality without any constraint qualification assumptions we prove a stronger superlinear convergence result than the above assuming the usual second order sufficient condition only in addition our analysis is carried out in the more general setting of variational problems for which we introduce a natural extension of ssqp techniques in the process we also obtain a new error bound for karush kuhn tucker systems for variational problems that holds under an appropriate second order condition

primal dual interior point methods for pde constrained optimization this paper provides a detailed analysis of a primal dual interior point method for pde constrained optimization considered are optimal control problems with control constraints in l p it is shown that the developed primal dual interior point method converges globally and locally superlinearly not only the easier l setting is analyzed but also a more involved l q analysis q is presented in l the set of feasible controls contains interior points and the frechet differentiability of the perturbed optimality system can be shown in the l q setting which is highly relevant for pde constrained optimization these nice properties are no longer available nevertheless a convergence analysis is developed using refined techniques in parti cular two norm techniques and a smoothing step are required the l q analysis with smoothing step yields global linear and local superlinear convergence whereas the l analysis without smoothing step yields only global linear convergence

smoothing methods and semismooth methods for nondifferentiable operator equations we consider superlinearly convergent analogues of newton methods for nondifferentiable operator equations in function spaces the superlinear convergence analysis of semismooth methods for nondifferentiable equations described by a locally lipschitzian operator in rn is based on rademacher s theorem which does not hold in function spaces we introduce a concept of slant differentiability and use it to study superlinear convergence of smoothing methods and semismooth methods in a unified framework we show that a function is slantly differentiable at a point if and only if it is lipschitz continuous at that point an application to the dirichlet problems for a simple class of nonsmooth elliptic partial differential equations is discussed

an lp newton method nonsmooth equations kkt systems and nonisolated solutions we define a new newton type method for the solution of constrained systems of equations and analyze in detail its properties under suitable conditions that do not include differentiability or local uniqueness of solutions the method converges locally quadratically to a solution of the system thus filling an important gap in the existing theory the new algorithm improves on known methods and when particularized to kkt systems derived from optimality conditions for constrained optimization or variational inequalities it has theoretical advantages even over methods specifically designed to solve such systems

uniform convergence and mesh independence of newton s method for discretized variational problems in an abstract framework we study local convergence properties of newton s method for a sequence of generalized equations which models a discretized variational inequality we identify conditions under which the method is locally quadratically convergent uniformly in the discretization moreover we show that the distance between the newton sequence for the continuous problem and the newton sequence for the discretized problem is bounded by the norm of a residual as an application we present mesh independence results for an optimal control problem with control constraints

several path following methods for a class of gradient constrained variational inequalities path following splitting and semismooth newton methods for solving a class of problems related to elasto plastic material deformations are proposed analyzed and tested numerically while the splitting techniques result in alternating minimization schemes which are typically linearly convergent the proposed moreau yosida regularization based semismooth newton technique and an associated lifting step yield local superlinear convergence in function space the lifting step accounts for the fact that the operator associated with the linear system in the newton iteration need not be boundedly invertible uniformly along the iterates for devising an efficient update strategy for the path following parameter regularity properties of the path are studied and utilized within an inexact path following scheme for all approaches the paper ends by a report on numerical tests of the different approaches

method for generating long range correlations for large systems we propose a new method to generate a sequence of random numbers with long range power law correlations that overcomes known difficulties associated with large systems the new method presents an improvement on the commonly used methods we apply the algorithm to generate enhanced diffusion isotropic and anisotropic self affine surfaces and isotropic and anisotropic correlated percolation

boundary element methods for variational inequalities in this paper we present a priori error estimates for the galerkin solution of variational inequalities which are formulated in fractional sobolev trace spaces i e in widetilde h 1 2 gamma h 1 2 in addition to error estimates in the energy norm we also provide by applying the aubin nitsche trick for variational inequalities error estimates in lower order sobolev spaces including l 2 gamma l 2 the resulting discrete variational inequality is solved by using a semi smooth newton method which is equivalent to an active set strategy a numerical example is given which confirms the theoretical results

local behavior of an iterative framework for generalized equations with nonisolated solutions an iterative framework for solving generalized equations with nonisolated solutions is presented for generalized equations with the structure where is a multifunction and f is single valued the framework covers methods that at each step solve subproblems of the type the multifunction approximates f around s besides a condition on the quality of this approximation two other basic assumptions are employed to show q superlinear or q quadratic convergence of the iterates to a solution a key assumption is the upper lipschitz continuity of the solution set map of the perturbed generalized equation moreover the solvability of the subproblems is required conditions that ensure these assumptions are discussed in general and by means of several applications they include monotone mixed complementarity problems karush kuhn tucker systems arising from nonlinear programs and nonlinear equations particular results deal with error bounds and upper lipschitz continuity properties for these problems

cyclization of a polymer first passage problem for a non markovian process we discuss a closure of a polymer chain released from a state with given end to end distance considering the situation as a first passage problem for a non markovian process we express the survival probability through the three time distribution of the end to end distance and evaluate it numerically by solving a volterra equation we also show how the wilemski fixmann approximation emerges within this scheme and discuss its geometrical nature

a mesh independence principle for operator equations and their discretizations the mesh independence principle asserts that when newton s method is applied to a nonlinear equation between some banach spaces as well as to some finite dimensional discretization of that equation then the behavior of the discretized process is asymptotically the same as that for the original iteration and as a consequence the number of steps required by the two processes to converge to within a given tolerance is essentially the same so far this result has been proved only for certain classes of boundary value problems in this paper a proof is presented for a general class of operator equations and discretizations it covers the earlier results and extends them well beyond the cases that have been considered before

a schwarz domain decomposition method with gradient projection for optimal control governed by elliptic partial differential equations a b s t r a c t a domain decomposition method ddm is presented to solve the distributed optimal control problem the optimal control problem essentially couples an elliptic partial differential equation with respect to the state variable and a variational inequality with respect to the constrained control variable the proposed algorithm called sa gp algorithm consists of two iterative stages in the inner loops the schwarz alternating method sa is applied to solve the state and co state variables and in the outer loops the gradient projection algorithm gp is adopted to obtain the control variable convergence of iterations depends on both the outer and the inner loops which are coupled and affected by each other in the classical iteration algorithms a given tolerance would be reached after sufficiently many iteration steps but more iterations lead to huge computational cost for solving constrained optimal control problems most of the computational cost is used to solve pdes in this paper a proposed iterative number independent of the tolerance is used in the inner loops so as to save a lot of computational cost the convergence rate of l 2 error of control variable is derived also the analysis on how to choose the proposed iteration number in the inner loops is given some numerical experiments are performed to verify the theoretical results

an interior point algorithm with inexact step computation in function space for state constrained optimal control we consider an interior point method in function space for pde constrained optimal control problems with state constraints our emphasis is on the construction and analysis of an algorithm that integrates a newton path following method with adaptive grid refinement this is done in the framework of inexact newton methods in function space where the discretization error of each newton step is controlled by adaptive grid refinement in the innermost loop this allows to perform most of the required newton steps on coarse grids such that the overall computational time is dominated by the last few steps for this purpose we propose an a posteriori error estimator for a problem suited norm

an algorithm for degenerate nonlinear programming with rapid local convergence this paper describes and analyzes an algorithmic framework for solving nonlinear programming problems in which strict complementarity conditions and constraint qualifications are not necessarily satisfied at a solution the framework is constructed from three main algorithmic ingredients the first is any conventional method for nonlinear programming that produces estimates of the lagrange multipliers at each iteration the second is a technique for estimating the set of active constraint indices the third is a stabilized lagrange newton algorithm with rapid local convergence properties results concerning rapid local convergence and global convergence of the proposed framework are proved the approach improves on existing approaches in that less restrictive assumptions are needed for convergence and or the computational workload at each iteration is lower

correlation properties of the random linear high order markov chains abstract the aim of this paper is to study the correlation properties of random sequences with additive linear conditional probability distribution function cpdf and elaborate a reliable tool for their generation it is supposed that the state space of the sequence under examination belongs to a finite set of real numbers the cpdf is assumed to be additive and linear with respect to the values of the random variable we derive the equations that relate the correlation functions of the sequence to the memory function coefficients which determine the cpdf the obtained analytical solutions for the equations connecting the memory and correlation functions are compared with the results of numerical simulation examples of possible correlation scenarios in the high order additive linear chains are given

superlinear convergence of affine scaling interior point newton methods for infinite dimensional nonlinear problems with pointwise bounds we develop and analyze a superlinearly convergent affine scaling interior point newton method for infinite dimensional problems with pointwise bounds in lp space the problem formulation is motivated by optimal control problems with lp controls and pointwise control constraints the finite dimensional convergence theory by coleman and li siam j optim 6 1996 pp 418 445 makes essential use of the equivalence of norms and the exact identifiability of the active constraints close to an optimizer with strict complementarity since these features are not available in our infinite dimensional framework algorithmic changes are necessary to ensure fast local convergence the main building block is a newton like iteration for an affine scaling formulation of the kkt condition we demonstrate in an example that a stepsize rule to obtain an interior iterate may require very small stepsizes even arbitrarily close to a nondegenerate solution using a pointwise projection instead we prove superlinear convergence under a weak strict complementarity condition and convergence with mbox q rate 1 under a slightly stronger condition if a smoothing step is available we discuss how the algorithm can be embedded in the class of globally convergent trust region interior point methods recently developed by m heinkenschloss and the authors numerical results for the control of a heating process confirm our theoretical findings

local convergence of exact and inexact augmented lagrangian methods under the second order sufficient optimality condition we establish local convergence and rate of convergence of the classical augmented lagrangian algorithm under the sole assumption that the dual starting point is close to a multiplier satisfying the second order sufficient optimality condition in particular no constraint qualifications of any kind are needed previous literature on the subject required in addition the linear independence constraint qualification and either the strict complementarity assumption or a stronger version of the second order sufficient condition that said the classical results allow the initial multiplier estimate to be far from the optimal one at the expense of proportionally increasing the threshold value for the penalty parameters although our primary goal is to avoid constraint qualifications if the stronger assumptions are introduced then starting points far from the optimal multiplier are allowed within our analysis as well using only the second order sufficient optimality condition for penalty parameters large

perturbed proximal primal dual algorithm for nonconvex nonsmooth optimization in this paper we propose a perturbed proximal primal dual algorithm pprox pda for an important class of linearly constrained optimization problems whose objective is the sum of smooth possibly nonconvex and convex possibly nonsmooth functions this family of problems can be used to model many statistical and engineering applications such as high dimensional subspace estimation and the distributed machine learning the proposed method is of the uzawa type in which a primal gradient descent step is performed followed by an approximate dual gradient ascent step one distinctive feature of the proposed algorithm is that the primal and dual steps are both perturbed appropriately using past iterates so that a number of asymptotic convergence and rate of convergence results to first order stationary solutions can be obtained finally we conduct extensive numerical experiments to validate the effectiveness of the proposed algorithm

convergence properties of an augmented lagrangian algorithm for optimization with a combination of general equality and linear constraints we consider the global and local convergence properties of a class of augmented lagrangian methods for solving nonlinear programming problems in these methods linear and more general constraints are handled in different ways the general constraints are combined with the objective function in an augmented lagrangian the iteration consists of solving a sequence of subproblems in each subproblem the augmented lagrangian is approximately minimized in the region defined by the linear constraints a subproblem is terminated as soon as a stopping condition is satisfied the stopping rules that we consider here encompass practical tests used in several existing packages for linearly constrained optimization our algorithm also allows different penalty parameters to be associated with disjoint subsets of the general constraints in this paper we analyze the convergence of the sequence of iterates generated by such an algorithm and prove global and fast linear convergence as well as show that potentially troublesome penalty parameters remain bounded away from zero

a globally convergent stabilized sqp method sequential quadratic programming sqp methods are a popular class of methods for nonlinearly constrained optimization they are particularly effective for solving a sequence of related problems such as those arising in mixed integer nonlinear programming and the optimization of functions subject to differential equation constraints recently there has been considerable interest in the formulation of stabilized sqp methods which are specifically designed to handle degenerate optimization problems existing stabilized sqp methods are essentially local in the sense that both the formulation and analysis focus on the properties of the methods in a neighborhood of a solution a new sqp method is proposed that has favorable global convergence properties yet under suitable assumptions is equivalent to a variant of the conventional stabilized sqp method in the neighborhood of a solution the method combines a primal dual generalized augmented lagrangian function with a flexible line search to obtain a seque

on the cost of solving augmented lagrangian subproblems at each iteration of the augmented lagrangian algorithm a nonlinear subproblem is being solved the number of inner iterations of some any method needed to obtain a solution of the subproblem or even a suitable approximate stationary point is in principle unknown in this paper we show that to compute an approximate stationary point sufficient to guarantee local superlinear convergence of the augmented lagrangian iterations it is enough to solve two quadratic programming problems or two linear systems in the equality constrained case in other words two inner newtonian iterations are sufficient to the best of our knowledge such results are not available even under the strongest assumptions of second order sufficiency strict complementarity and the linear independence constraint qualification our analysis is performed under second order sufficiency only which is the weakest assumption for obtaining local convergence and rate of convergence of outer iterations of the augmented lagrangian algorithm the structure of the quadratic problems in question is related to the stabilized sequential quadratic programming and to second order corrections

combined symmetric bem and semi smooth newton method for the contact problems in linear elasticity of yukawa type the semi smooth newton method and the boundary element method are developed and analyzed for the solution of 2 d signorini contact problems in linear elasticity of yukawa type first we consider the contact problem with tresca friction this leads to a constrained non differentiable minimization problem where the solvability is in general problematic but by utilizing the fenchel duality theory the dual formulation in terms of contact stresses turns out to be a quadratic optimization problem with a smooth functional the regularization of the dual problem motivated by the augmented lagrangian is suitable for the application of the generalized newton method applying the boundary integral equation method the problem is reduced to the boundary curve the corresponding boundary integral equations are approximated by using a galerkin method with the help of b splines on the boundary curve bem this yields an algebraic system of linear equations with dense stiffness matrix but which is symmetric the symmetry property of the stiffness matrix enables the application of efficient iterative solution strategies for the linear systems at each newton step second the methods are carried over to the coulomb friction problem by means of a fixed point approach in particular the analysis of the algorithm is presented and some numerical examples which show a remarkable efficiency and reliability of the semi smooth newton method are given

a note on upper lipschitz stability error bounds and critical multipliers for lipschitz continuous kkt systems we prove a new local upper lipschitz stability result and the associated local error bound for solutions of parametric karush kuhn tucker systems corresponding to variational problems with lipschitzian base mappings and constraints possessing lipschitzian derivatives and without any constraint qualifications this property is equivalent to the appropriately extended to this nonsmooth setting notion of noncriticality of the lagrange multiplier associated to the primal solution which is weaker than second order sufficiency all this extends several results previously known only for optimization problems with twice differentiable data or assuming some constraint qualifications in addition our results are obtained in the more general variational setting

on attraction of linearly constrained lagrangian methods and of stabilized and quasi newton sqp methods to critical multipliers it has been previously demonstrated that in the case when a lagrange multiplier associated to a given solution is not unique newton iterations e g those of sequential quadratic programming sqp have a tendency to converge to special multipliers called critical multipliers when such critical multipliers exist this fact is of importance because critical multipliers violate the second order sufficient optimality conditions and this was shown to be the reason for slow convergence typically observed for problems with degenerate constraints convergence to noncritical multipliers results in superlinear rate despite degeneracy some theoretical and numerical validation of this phenomenon can be found in izmailov and solodov comput optim appl 42 231 264 2009 math program 117 271 304 2009 however previous studies concerned the basic forms of newton iterations the question remained whether the attraction phenomenon still persists for relevant modifications as well as in professional implementations in this paper we answer this question in the affirmative by presenting numerical results for the well known minos and snopt software packages applied to a collection of degenerate problems we also extend previous theoretical considerations to the linearly constrained lagrangian methods and to the quasi newton sqp on which minos and snopt are based experiments also show that in the stabilized version of sqp the attraction phenomenon still exists but appears less persistent

long range correlation properties of coding and noncoding dna sequences genbank analysis an open question in computational molecular biology is whether long range correlations are present in both coding and noncoding dna or only in the latter to answer this question we consider all 33301 coding and all 29453 noncoding eukaryotic sequences each of length larger than 512 base pairs bp in the present release of the genbank to dtermine whether there is any statistically significant distinction in their long range correlation properties standard fast fourier transform fft analysis indicates that coding sequences have practically no correlations in the range from 10 bp to 100 bp spectral exponent beta 0 00 0 04 where the uncertainty is two standard deviations in contrast for noncoding sequences the average value of the spectral exponent beta is positive 0 16 0 05 which unambiguously shows the presence of long range correlations we also separately analyze the 874 coding and the 1157 noncoding sequences that have more than 4096 bp and find a larger region of power law behavior we calculate the probability that these two data sets coding and noncoding were drawn from the same distribution and we find that it is less than 10 10 we obtain independent confirmation of these findings using the method of detrended fluctuation analysis dfa which is designed to treat sequences with statistical heterogeneity such as dna s known mosaic structure patchiness arising from the nonstationarity of nucleotide concentration the near perfect agreement between the two independent analysis methods fft and dfa increases the confidence in the reliability of our conclusion

on the convergence of augmented lagrangian methods for constrained global optimization in this paper we present new convergence properties of the primal dual method based on four types of augmented lagrangian functions in the context of constrained global optimization convergence to a global optimal solution is first established for a basic primal dual scheme under standard conditions we then prove this convergence property for a modified augmented lagrangian method using a safeguarding strategy without appealing to the boundedness assumption of the multiplier sequence we further show that under the same weaker conditions the convergence to a global optimal solution can still be achieved by either modifying the multiplier updating rule or normalizing the multipliers in augmented lagrangian methods

global minimization using an augmented lagrangian method with variable lower level constraints a novel global optimization method based on an augmented lagrangian framework is introduced for continuous constrained nonlinear optimization problems at each outer iteration k the method requires the varepsilon k global minimization of the augmented lagrangian with simple constraints where varepsilon k to varepsilon global convergence to an varepsilon global minimizer of the original problem is proved the subproblems are solved using the bb method numerical experiments are presented

abstract newtonian frameworks and their applications we unify and extend some newtonian iterative frameworks developed earlier in the literature which results in a collection of convenient tools for local convergence analysis of various algorithms under various sets of assumptions including strong metric regularity semistability or upper lipschitz stability the latter allowing for nonisolated solutions these abstract schemes are further applied for deriving sharp local convergence results for some constrained optimization algorithms under reduced smoothness hypotheses specifically we consider applications to the augmented lagrangian method and to the linearly constrained lagrangian method for problems with lipschitzian derivatives but possibly without second derivatives and our local convergence analysis for these methods improves all the existing theories of this kind

convergence of newton s method for singular smooth and nonsmooth equations using adaptive outer inverses we present a local convergence analysis of generalized newton methods for singular smooth and nonsmooth operator equations using adaptive constructs of outer inverses we prove that for a solution x of f x 0 there exists a ball s s x r r 0 such that for any starting point x 0 in s the method converges to a solution bar x in s of gamma f x 0 where gamma is a bounded linear operator that depends on the frechet derivative of f at x 0 or on a generalized jacobian of f at x 0 point bar x may be different from x when x is not an isolated solution moreover we prove that the convergence is quadratic if the operator is smooth and superlinear if the operator is locally lipschitz these results are sharp in the sense that they reduce in the case of an invertible derivative or generalized derivative to earlier theorems with no additional assumptions the results are illustrated by a system of smooth equations and a system of nonsmooth equations each of which is equivalent to a nonlinear complementarity problem

convergence analysis of some algorithms for solving nonsmooth equations this paper presents convergence analysis of some algorithms for solving systems of nonlinear equations defined by locally lipschitzian functions for the directional derivative based and the generalized jacobian based newton methods both the iterates and the corresponding function values are locally superlinearly convergent globally a limiting point of the iterate sequence generated by the damped directional derivative based newton method is a zero of the system if and only if the iterate sequence converges to this point and the stepsize eventually becomes one provided that the system is strongly bd regular and semismooth at this point in this case the convergence is superlinear a general attraction theorem is presented which can be applied to two algorithms proposed by han pang and rangaraj a hybrid method which is both globally convergent in the sense of finding a stationary point of the norm function of the system and locally quadratically convergent is also presented

on attraction of newton type iterates to multipliers violating second order sufficiency conditions assuming that the primal part of the sequence generated by a newton type e g sqp method applied to an equality constrained problem converges to a solution where the constraints are degenerate we investigate whether the dual part of the sequence is attracted by those lagrange multipliers which satisfy second order sufficient condition sosc for optimality or by those multipliers which violate it this question is relevant at least for two reasons one is speed of convergence of standard methods the other is applicability of some recently proposed approaches for handling degenerate constraints we show that for the class of damped newton methods convergence of the dual sequence to multipliers satisfying sosc is unlikely to occur we support our findings by numerical experiments we also suggest a simple auxiliary procedure for computing multiplier estimates which does not have this undesirable property finally some consequences for the case of mixed equality and inequality constraints are discussed

competition between two kinds of correlations in literary texts a theory of additive markov chains with long range memory is used to describe the correlation properties of coarse grained literary texts the complex structure of the correlations in the texts is revealed anti persistent correlations at small distances l approximately 300 define this non trivial structure for some concrete examples of literary texts the memory functions are obtained and their power law behavior at long distances is disclosed this property is shown to be a cause of self similarity of texts with respect to the decimation procedure

high order discretization and multigrid solution of elliptic nonlinear constrained optimal control problems high mixed order finite difference discretization of optimality systems arising from elliptic nonlinear constrained optimal control problems are discussed for the solution of these systems an efficient and robust multigrid algorithm is presented theoretical and experimental results show the advantages of higher order discretization and demonstrate that the proposed multigrid scheme is able to solve efficiently constrained optimal control problems also in the limit case of bang bang control

asymptotic mesh independence of newton galerkin methods via a refined mysovskii theorem the paper presents a theoretical characterization of the often observed asymptotic mesh independence of newton s method which means that newton s method applied to discretized operator equations behaves essentially the same for all sufficiently fine discretizations unlike previous theoretical approaches the theory presented here does not need any uniform lipschitz assumptions which may be hard to verify in realistic problems or in some cases may not hold at all moreover this approach leads to a significantly simpler presentation the new results are obtained by means of a new refined newton mysovskii theorem in affine invariant formulation this theorem will be of interest in a wider context because it gives both existence and uniqueness of the solution and quadratic convergence for sufficiently good starting points attention is restricted to galerkin approximations even though similar results should hold for finite difference methods but corresponding proofs would certainly be more technical

mesh independence of the generalized davidson algorithm abstract we give conditions under which the generalized davidson algorithm for eigenvalue computations is mesh independent in this case mesh independence means that the iteration statistics residual norms convergence rates for example of a sequence of discretizations of a problem in a banach space converge the statistics for the infinite dimensional problem we illustrate the result with several numerical examples

correlation function inadequacy in random sequence entropy measures considering symbolic and numerical random sequences in the framework of the additive markov chain approach we establish a relation between their correlation functions and conditional entropies we express the entropy by means of the two point probability distribution functions and then evaluate the entropy for the numerical random chain in terms of the correlation function we show that such approximation gives a satisfactory result only for special types of random sequences in general case the conditional entropy of numerical sequences obtained in the two point distribution function approach is lower we derive the conditional entropy of the additive markov chain as a sum of the kullback leibler mutual information and give an example of random sequence with the exactly zero correlation function and the nonzero correlations

mesh independence of semismooth newton methods for lavrentiev regularized state constrained nonlinear optimal control problems a class of nonlinear elliptic optimal control problems with mixed control state constraints arising e g in lavrentiev type regularized state constrained optimal control is considered based on its first order necessary optimality conditions a semismooth newton method is proposed and its fast local convergence in function space as well as a mesh independence principle for appropriate discretizations are proved the paper ends by a numerical verification of the theoretical results including a study of the algorithm in the case of vanishing lavrentiev parameter the latter process is realized numerically by a combination of a nested iteration concept and an extrapolation technique for the state with respect to the lavrentiev parameter

a duality based path following semismooth newton method for elasto plastic contact problems a fenchel dualization scheme for the one step time discretized contact problem of quasi static elasto plasticity with combined kinematic isotropic hardening is considered the associated path is induced by a coupled moreau yosida tikhonov regularization of the dual problem the sequence of solutions to the regularized problems is shown to converge strongly to the optimal displacement stress strain triple of the original elasto plastic contact problem in the space continuous setting this property relies on the density of the intersection of certain convex sets which is shown as well it is also argued that the mappings associated with the resulting problems are newton or slantly differentiable consequently each regularized subsystem can be solved mesh independently at a local superlinear rate of convergence for efficiency purposes an inexact path following approach is proposed and a numerical validation of the theoretical results is given

the augmented lagrangian method for equality and inequality constraints in hilbert spaces in this paper we consider an augmented lagrangian method for the minimization of a nonlinear functional in the presence of an equality constraint whose image space is in a hilbert space an inequality constraint whose image space is finite dimensional and an affine inequality constraint whose image space is in an infinite dimensional hilbert space we obtain local convergence of this method without imposing strict complementarity conditions when the equality as well as the inequality constraint with finite dimensional image space are augmented to the author s knowledge this result even generalizes the convergence results which are known when all spaces are finite dimensional

lagrange multipliers and optimality lagrange multipliers used to be viewed as auxiliary variables introduced in a problem of constrained minimization in order to write first order optimality conditions formally as a system of equations modern applications with their emphasis on numerical methods and more complicated side conditions than equations have demanded deeper understanding of the concept and how it fits into a larger theoretical picture a major line of research has been the nonsmooth geometry of one sided tangent and normal vectors to the set of points satisfying the given constraints another has been the game theoretic role of multiplier vectors as solutions to a dual problem interpretations as generalized derivatives of the optimal value with respect to problem parameters have also been explored lagrange multipliers are now being seen as arising from a general rule for the subdifferentiation of a nonsmooth objective function which allows black and white constraints to be replaced by penalty expressions this paper traces su

a sequential homotopy method for mathematical programming problems we propose a sequential homotopy method for the solution of mathematical programming problems formulated in abstract hilbert spaces under the guignard constraint qualification the method is equivalent to performing projected backward euler timestepping on a projected gradient antigradient flow of the augmented lagrangian the projected backward euler equations can be interpreted as the necessary optimality conditions of a primal dual proximal regularization of the original problem the regularized problems are always feasible satisfy a strong constraint qualification guaranteeing uniqueness of lagrange multipliers yield unique primal solutions provided that the stepsize is sufficiently small and can be solved by a continuation in the stepsize we show that equilibria of the projected gradient antigradient flow and critical points of the optimization problem are identical provide sufficient conditions for the existence of global flow solutions and show that critical points with emanating descent curves cannot be asymptotically stable equilibria of the projected gradient antigradient flow practically eradicating convergence to saddle points and maxima the sequential homotopy method can be used to globalize any locally convergent optimization method that can be used in a homotopy framework we demonstrate its efficiency for a class of highly nonlinear and badly conditioned control constrained elliptic optimal control problems with a semismooth newton approach for the regularized subproblems

sharp primal superlinear convergence results for some newtonian methods for constrained optimization as is well known q superlinear or q quadratic convergence of the primal dual sequence generated by an optimization algorithm does not in general imply q superlinear convergence of the primal part primal convergence however is often of particular interest for the sequential quadratic programming sqp algorithm local primal dual quadratic convergence can be established under the assumptions of uniqueness of the lagrange multiplier associated to the solution and the second order sufficient condition at the same time previous primal q superlinear convergence results for sqp required strengthening of the first assumption to the linear independence constraint qualification in this paper we show that this strengthening of assumptions is actually not necessary specifically we show that once primal dual convergence is assumed or already established for primal superlinear rate one needs only a certain error bound estimate this error bound holds for example under the second order sufficient condition which is needed for primal dual local analysis in any case moreover in some situations even second order sufficiency can be relaxed to the weaker assumption that the multiplier in question is noncritical our study is performed for a rather general perturbed sqp framework which covers in addition to sqp and quasi newton sqp some other algorithms as well for example as a byproduct we obtain primal q superlinear convergence results for the linearly constrained augmented lagrangian methods for which no primal q superlinear rate of convergence results were previously available another application of the general framework is sequential quadratically constrained quadratic programming methods finally we discuss some difficulties with proving primal superlinear convergence for the stabilized version of sqp

karush kuhn tucker systems regularity conditions error bounds and a class of newton type methods we consider optimality systems of karush kuhn tucker kkt type which arise for example as primal dual conditions characterizing solutions of optimization problems or variational inequalities in particular we discuss error bounds and newton type methods for such systems an exhaustive comparison of various regularity conditions which arise in this context is given we obtain a new error bound under an assumption which we show to be strictly weaker than assumptions previously used for kkt systems such as quasi regularity or semistability equivalently the r0 property error bounds are useful among oth er things for identifying active constraints and developing efficient local algorithms we propose a family of local newton type algorithms this family contains some known active set newton methods as well as some new methods regularity conditions required for local superlinear convergence compare favorably with convergence conditions of nonsmooth newton methods and sequential quadratic programming methods

a mesh independence result for semismooth newton methods for a class of semismooth operator equations a mesh independence result for generalized newton methods is established the main result states that the continuous and the discrete newton process when initialized properly converge q linearly with the same rate the problem class considered in the paper includes mcp function based reformulations of first order conditions of a class of control constrained optimal control problems for partial differential equations for which a numerical validation of the theoretical results is given

stabilized sqp revisited the stabilized version of the sequential quadratic programming algorithm ssqp had been developed in order to achieve superlinear convergence in situations when the lagrange multipliers associated to a solution are not unique within the framework of fischer math program 94 91 124 2002 the key to local superlinear convergence of ssqp are the following two properties upper lipschitzian behavior of solutions of the karush kuhn tucker kkt system under canonical perturbations and local solvability of ssqp subproblems with the associated primal dual step being of the order of the distance from the current iterate to the solution set of the unperturbed kkt system according to fernandez and solodov math program 125 47 73 2010 both of these properties are ensured by the second order sufficient optimality condition sosc without any constraint qualification assumptions in this paper we state precise relationships between the upper lipschitzian property of solutions of kkt systems error bounds for kkt systems the notion of critical lagrange multipliers a subclass of multipliers that violate sosc in a very special way the second order necessary condition for optimality and solvability of ssqp subproblems moreover for the problem with equality constraints only we prove superlinear convergence of ssqp under the assumption that the dual starting point is close to a noncritical multiplier since noncritical multipliers include all those satisfying sosc but are not limited to them we believe this gives the first superlinear convergence result for any newtonian method for constrained optimization under assumptions that do not include any constraint qualifications and are weaker than sosc in the general case when inequality constraints are present we show that such a relaxation of assumptions is not possible we also consider applying ssqp to the problem where inequality constraints are reformulated into equalities using slack variables and discuss the assumptions needed for convergence in this approach we conclude with consequences for local regularization methods proposed in izmailov and solodov siam j optim 16 210 228 2004 wright siam j optim 15 673 676 2005 in particular we show that these methods are still locally superlinearly convergent under the noncritical multiplier assumption weaker than sosc employed originally

on the accurate identification of active constraints we consider nonlinear programs with inequality constraints and we focus on the problem of identifying those constraints which will be active at an isolated local solution the correct identification of active constraints is important from both a theoretical and a practical point of view such an identification removes the combinatorial aspect of the problem and locally reduces the inequality constrained minimization problem to an equality constrained problem which can be more easily dealt with we present a new technique which identifies active constraints in a neighborhood of a solution and which requires neither complementary slackness nor uniqueness of the multipliers we also present extensions to variational inequalities and numerical examples illustrating the identification technique

a quasi newton strategy for the ssqp method for variational inequality and optimization problems the quasi newton strategy presented in this paper preserves one of the most important features of the stabilized sequential quadratic programming method the local convergence without constraint qualifications assumptions it is known that the primal dual sequence converges quadratically assuming only the second order sufficient condition in this work we show that if the matrices are updated by performing a minimization of a bregman distance which includes the classic updates the quasi newton version of the method converges superlinearly without introducing further assumptions also we show that even for an unbounded lagrange multipliers set the generated matrices satisfies a bounded deterioration property and the dennis more condition

symbolic stochastic dynamical systems viewed as binary n step markov chains a theory of systems with long range correlations based on the consideration of binary n step markov chains is developed in the model the conditional probability that the ith symbol in the chain equals zero or unity is a linear function of the number of unities among the preceding n symbols the correlation and distribution functions as well as the variance of the number of symbols in the words of arbitrary length l are obtained analytically and numerically a self similarity of the studied stochastic process is revealed and the similarity group transformation of the chain parameters is presented the diffusion fokker planck equation governing the distribution function of the l words is explored if the persistent correlations are not extremely strong the distribution function is shown to be the gaussian with the variance being nonlinearly dependent on l the applicability of the developed theory to the coarse grained written and dna texts is discussed

a nonsmooth version of newton s method newton s method for solving a nonlinear equation of several variables is extended to a nonsmooth case by using the generalized jacobian instead of the derivative this extension includes the b derivative version of newton s method as a special case convergence theorems are proved under the condition of semismoothness it is shown that the gradient function of the augmented lagrangian forc2 nonlinear programming is semismooth thus the extended newton s method can be used in the augmented lagrangian method for solving nonlinear programs

a globally convergent lp newton method we develop a globally convergent algorithm based on the lp newton method which has been recently proposed for solving constrained equations possibly nonsmooth and possibly with nonisolated solutions the new algorithm makes use of linesearch for the natural merit function and preserves the strong local convergence properties of the original lp newton scheme we also present computational experiments on a set of generalized nash equilibrium problems and a comparison of our approach with the previous hybrid globalization employing the potential reduction method

mesh independence of the gradient projection method for optimal control problems the gradient projection method and its application to optimal control have been analyzed with regard to the rate of convergence quite extensively here another aspect of this method is considered in finite dimension the set of active indices is identified after finitely many iterations under mild nondegeneracy assumptions however it is clear that this property is restricted to the finite dimensional case in this paper for example a sequence of discretized optimal control problems is considered and it is observed that the number of steps to identify all active indices increases with the refinement of the discretization a result analogous to the finite dimensional result is valid in this situation if identification of active indices is understood in the correct light this paper shows that if a different termination criterion is imposed then the number of necessary steps for termination is indeed mesh independent numerical observations illustrating this result are reported for various examples fr

mesh independence for an augmented lagrangian sqp method in hilbert spaces an augmented lagrangian sqp algorithm for optimal control of differential equations in hilbert spaces is analyzed this algorithm has second order convergence rate provided that a second order sufficient optimality condition is satisfied the internal approximation of this method is investigated and convergence results are presented a mesh independence principle for the augmented lagrangian sqp method is proved which asserts that asymptotically the infinite dimensional algorithm and finite dimensional discretizations have the same convergence property more precisely for sufficiently small mesh size there is at most a difference of one iteration step between the number of steps required by the infinite dimensional method and its discretization to converge within a given tolerance varepsilon 0 the theoretical results are demonstrated by two optimal control problems for the burgers equation

fast solution of cahn hilliard variational inequalities using implicit time discretization and finite elements we consider the efficient solution of the cahn hilliard variational inequality using an implicit time discretization which is formulated as an optimal control problem with pointwise constraints on the control by applying a semi smooth newton method combined with a moreau yosida regularization technique for handling the control constraints we show superlinear convergence in function space at the heart of this method lies the solution of large and sparse linear systems for which we propose the use of preconditioned krylov subspace solvers using an effective schur complement approximation numerical results illustrate the competitiveness of this approach

improved local convergence results for augmented lagrangian methods in varvec c mathbf 2 c2 cone reducible constrained optimization this paper deals with a class of cone reducible constrained optimization problems which encompasses nonlinear programming semidefinite programming second order cone programming and any combination thereof using the second order sufficient condition and a strict version of the robinson constraint qualification we provide a semi local error bound which generalizes known results from the literature moreover under the same assumptions we prove that an augmented lagrangian method is locally convergent with rate proportional to 1 rho k where rho k is the penalty parameter and that rho k remains bounded