SIGEST
Venue: SIAM Review (2005)

Donald Knuth, the renowned computer scientist and mathematician, commented in a recent interview [D. E. Knuth, "All questions answered," Notices of the American Mathematical Society, 29 (2002), pp.~318--324] that "...my main conclusion after spending ten years of my life on the TeX project is that software is hard. It's harder than anything else I've ever had to do." Everyone who has written software for general-purpose consumption will agree with these statements. Software is the most frequent transformation through which applied mathematics becomes useful. For all but the simplest problems, software is hard because it must not only correctly implement one or more theoretical algorithms, but also cope gracefully with finite-precision arithmetic, poor scaling, problems that fail to satisfy the properties required for the theory to apply, and the vagaries of users who ignore or knowingly violate guidelines specified by the author. Writing software is hard, and writing well about software is equally hard. However, this issue's SIGEST paper, "SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization," by Philip Gill, Walter Murray, and Michael Saunders, proves that it is possible to write a clear, interesting, and lively paper about a complicated algorithm and the associated software. SNOPT represents a long accumulation of varied theoretical advances and hard-earned experiences with real-world problems, and also exemplifies Knuth's assertion. Solving nonlinearly constrained optimization problems via a sequence of quadratic programming (QP) subproblems was first proposed more than 40 years ago, and a 1977 paper by a prominent optimization researcher stated that a sequential quadratic programming (SQP) method could be "programmed in an afternoon." However, this early optimism about the ease of SQP software development turned out to be unjustified for many reasons, in part because of the special nature of the QP subproblems and the importance of fine details in the linear algebra calculations, particularly for large-scale problems. The SNOPT paper includes a concise overview not just of SQP methods, but also of the nonobvious decisions to be made and the tricky problems to be solved. For example, should the software attempt to determine when the specified nonlinear constraints are likely to be infeasible (i.e., there is no point that satisfies them)? If so, given that there is no general procedure guaranteed to detect infeasibility, what can be done that is efficient yet has a high probability of correctness? The list of this and other questions that must be answered when solving constrained optimization problems is very long, and the paper provides the thoughtful answers embodied in SNOPT. The world of constrained optimization software---including the development of methods and the collection of test problems---has noticeably changed since the SNOPT paper first appeared in the SIAM Journal on Optimization in 2002 (volume 12, pp. 979--1006). The authors have accordingly updated the paper's discussion of alternative software (notably, several interior-point methods). In addition, they have expended considerable effort (and machine cycles!) to apply the latest version of SNOPT to the most recent versions of the two most popular sets of test problems, whose largest instances have increased significantly in dimension since 2002. We are grateful to the authors for giving SIAM Review readers a paper that explains the details of the theory and implementation of SQP methods, along with strategies for addressing problems arising along the way that are provably impossible, but that users want to solve anyway.