Stationarity and Convergence in Reduce-or-Retreat Minimization [electronic resource] /
Levy, Adam B.author.
Stationarity and Convergence in Reduce-or-Retreat Minimization presents and analyzes a unifying framework for a wide variety of numerical methods in optimization. The author’s “reduce-or-retreat” framework is a conceptual method-outline that covers any method whose iterations choose between reducing the objective in some way at a trial point, or retreating to a closer set of trial points. The alignment of various derivative-based methods within the same framework encourages the construction of new methods, and inspires new theoretical developments as companions to results from across traditional divides. The text illustrates the former by developing two generalizations of classic derivative-based methods which accommodate non-smooth objectives, and the latter by analyzing these two methods in detail along with a pattern-search method and the famous Nelder-Mead method.In addition to providing a bridge for theory through the “reduce-or-retreat” framework, this monograph extends and broadens the traditional convergence analyses in several ways. Levy develops a generalized notion of approaching stationarity which applies to non-smooth objectives, and explores the roles of the descent and non-degeneracy conditions in establishing this property. The traditional analysis is broadened by considering “situational” convergence of different elements computed at each iteration of a reduce-or-retreat method. The “reduce-or-retreat” framework described in this text covers specialized minimization methods, some general methods for minimization and a direct search method, while providing convergence analysis which complements and expands existing results.  .
Description based upon print version of record.
Stationarity and Convergence in Reduce-or-Retreat Minimization; Preface; Contents; Chapter1 A Framework for Reduce-or-Retreat Minimization; 1.1 Iteration-Generated Elements; 1.2 Reduction, Retreat, and Termination; 1.3 Framework Outline; 1.4 Descent; 1.5 Lower-Diminishing Target-Gaps Γk; 1.6 Approaching Stationarity; 1.6.1 Lipschitz df; 1.6.2 Full-Sequence Stationarity; 1.7 Stationarity-Inducing Target-Gaps; Chapter2 Particular Reduce-or-Retreat Methods; 2.1 Generalized Line-Search Method; 2.1.1 Method Outline; 2.1.2 Descent; 2.1.3 Lower-Diminishing Target-Gaps; 2.1.4 Approaching Stationarity
2.2 Generalized Trust-Region Method2.2.1 Method Outline; 2.2.2 Descent; 2.2.3 Lower-Diminishing Target-Gaps; 2.2.4 Approaching Stationarity; 2.3 General Pattern-Search Method; 2.3.1 Method Outline; 2.3.2 Descent; 2.3.3 Lower-Diminishing Target-Gaps; 2.3.4 Approaching Stationarity; 2.4 Nelder-Mead Method; 2.4.1 Method Outline; 2.4.2 Lower-Diminishing Target-Gaps; 2.4.3 Approaching Stationarity; 2.5 Incorporating a Degeneracy Check; 2.5.1 Non-degeneracy Conditions; Chapter3 Scenario Analysis; 3.1 Convergence of Trial-Sizes Δk; 3.2 Convergence of Iterate-Sets Xk
3.2.1 Iterate-Diameter and Iterate-Radius3.2.2 f-Stability; References
Includes bibliographical references (p. 55).
Stationarity and Convergence in Reduce-or-Retreat Minimization presents and analyzes a unifying framework for a wide variety of numerical methods in optimization. The author’s “reduce-or-retreat” framework is a conceptual method-outline that covers any method whose iterations choose between reducing the objective in some way at a trial point, or retreating to a closer set of trial points. The alignment of various derivative-based methods within the same framework encourages the construction of new methods, and inspires new theoretical developments as companions to results from across traditional divides. The text illustrates the former by developing two generalizations of classic derivative-based methods which accommodate non-smooth objectives, and the latter by analyzing these two methods in detail along with a pattern-search method and the famous Nelder-Mead method.In addition to providing a bridge for theory through the “reduce-or-retreat” framework, this monograph extends and broadens the traditional convergence analyses in several ways. Levy develops a generalized notion of approaching stationarity which applies to non-smooth objectives, and explores the roles of the descent and non-degeneracy conditions in establishing this property. The traditional analysis is broadened by considering “situational” convergence of different elements computed at each iteration of a reduce-or-retreat method. The “reduce-or-retreat” framework described in this text covers specialized minimization methods, some general methods for minimization and a direct search method, while providing convergence analysis which complements and expands existing results.  .
Algorithms.
Mathematical optimization.
Computer mathematics.
Probabilities.
Calculus of variations.
Algorithms.
Optimization.
Computational Mathematics and Numerical Analysis.
Probability Theory and Stochastic Processes.
Calculus of Variations and Optimal Control; Optimization.