4 Local Descent
Up to this point, we have focused on optimization involving a single design
variable. This chapter introduces a general approach to optimization involving
multivariate functions, or functions with more than one variable. The focus of thischapter is on how to use local models to incrementally improve a design point
until some convergence criterion is met. We begin by discussing methods that,at each iteration, choose a descent direction based on a local model and thenchoose a step size. We then discuss methods that restrict the step to be within aregion where the local model is believed to be valid. This chapter concludes witha discussion of convergence conditions. The next two chapters will discuss how
to use first- and second-order models built from gradient or Hessian information.

4.1

Descent Direction Iteration

A common approach to optimization is to incrementally improve a design point
x by taking a step that minimizes the objective value based on a local model. The
local model may be obtained, for example, from a first- or second-order Taylor
approximation. Optimization algorithms that follow this general approach arereferred to as descent direction methods. They start with a design point x(1) andthen generate a sequence of points, sometimes called iterates, to converge to a
local minimum.1
The iterative descent direction procedure involves the following steps:
1. Check whether x(k) satisfies the termination conditions. If it does, terminate;
otherwise proceed to the next step.
2. Determine the descent direction d(k) using local information such as the gradientor Hessian. Some algorithms assume kd(k) k = 1, but others do not.

The choice of x(1) can affect thesuccess of the algorithm in finding a minimum. Domain knowledge is often used to choose a reasonable value. When that is notavailable, we can search over thedesign space using the techniquesthat will be covered in chapter 13.

1

54 c ha p te r 4 . l oc a l d e sc e n t

3. Determine the step size or learning rate α(k) . Some algorithms attempt to
optimize the step size so that the step maximally decreases f .2
4. Compute the next design point according to:x( k +1) ← x( k ) + α ( k ) d( k )(4.1)

There are many different optimization methods, each with their own ways ofdetermining α and d.

We use step size to refer to themagnitude of the overall step. Obtaining a new iterate using equation (4.1) with a step size α(k) implies that the descent direction d(k)has unit length. We use learning
rate to refer to a scalar multiple
used on a descent direction vector which does not necessarily haveunit length.

2

4.2 Line Search
For the moment, assume we have chosen a descent direction d, perhaps using oneof the methods discussed in one of the subsequent chapters. We need to choose
the step factor α to obtain our next design point. One approach is to use line search,which selects the step factor that minimizes the one-dimensional function:
minimize f (x + αd)
α

(4.2)

Line search is a univariate optimization problem, which was covered in chapter 3.
We can apply the univariate optimization method of our choice.3 To inform thesearch, we can use the derivative of the line search objective, which is simply the
directional derivative along d at x + αd. Line search is demonstrated in example 4.1
and implemented in algorithm 4.1.
function line_search(f, x, d)objective = α -> f(x + α*d)a, b = bracket_minimum(objective)
α = minimize(objective, a, b)
return x + α*d
end

One disadvantage of conducting a line search at each step is the computational
cost of optimizing α to a high degree of precision. Instead, it is common to quickly
find a reasonable value and then move on, selecting x(k+1) , and then picking a
new direction d(k+1) .

3
The Brent-Dekker method, mentioned in the previous chapter, isa commonly used univariate optimization method. It combines
the robustness of the bisection
method with the speed of the secant method.

Algorithm 4.1. A method for conducting a line search, which finds
the optimal step factor along adescent direction d from design
point x to minimize function f.
The minimize function can be implemented using a univariate optimization algorithm such as the
Brent-Dekker method.

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

4 . 3. a p p rox i ma te l i n e se a rc h

55

Some algorithms use a fixed step factor. Large steps will tend to result in fasterconvergence but risk overshooting the minimum. Smaller steps tend to be morestable but can result in slower convergence. A fixed step factor α is sometimes
referred to as a learning rate.
Another method is to use a decaying step factor:α ( k ) = α (1) γ k −1for γ ∈ (0, 1](4.3)

Decaying step factors are especially popular when minimizing noisy objectivefunctions,4 and are commonly used in machine learning applications.

Example 4.1. Line search used tominimize a function along a descent direction.
line search objective

Consider conducting a line search on f ( x1 , x2 , x3 ) = sin( x1 x2 ) + exp( x2 +x3 ) − x3 from x = [1, 2, 3] in the direction d = [0, −1, −1]. The corresponding
optimization problem is:

We will discuss optimization in
the presence of noise and otherforms of uncertainty in chapter 17.

4minimize sin((1 + 0α)(2 − α)) + exp((2 − α) + (3 − α)) − (3 − α)αwhich simplifies to:minimize sin(2 − α) + exp(5 − 2α) + α − 3
α

150
100
50
α∗

0
0

The minimum is at α ≈ 3.127 with x ≈ [1, −1.126, −0.126].

24α

4.3 Approximate Line Search
It is often more computationally efficient to perform more iterations of a descentmethod than to do exact line search at each iteration, especially if the function
and derivative calculations are expensive. Many of the methods discussed so far
can benefit from using approximate line search to find a suitable step size with asmall number of evaluations. Since descent methods must descend, a step size α
may be suitable if it causes a decrease in the objective function value. However, avariety of other conditions may be enforced to encourage faster convergence.
The condition for sufficient decrease5 requires that the step size cause a sufficientdecrease in the objective function value:f (x(k+1) ) ≤ f (x(k) ) + βα∇d(k) f (x(k) )This condition is sometimes referred to as the Armijo condition.
5

(4.4)

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

56 c ha p te r 4 . l oc a l d e sc e n t

with β ∈ [0, 1] often set to β = 1 × 10−4 . Figure 4.1 illustrates this condition. If
β = 0, then any decrease is acceptable. If β = 1, then the decrease has to be atleast as much as what would be predicted by a first-order approximation.
f (x) + β α ∇yf (x) +d f (x)α∇d f (x)

Figure 4.1. The sufficient decrease
condition, the first Wolfe condition,
can always be satisfied by a sufficiently small step size along a descent direction.

f (x + αd)
sufficient decrease0α

If d is a valid descent direction, then there must exist a sufficiently small step
size that satisfies the sufficient decrease condition. We can thus start with a large
step size and decrease it by a constant reduction factor until the sufficient decrease
condition is satisfied. This algorithm is known as backtracking line search6 becauseof how it backtracks along the descent direction. Backtracking line search is shownin figure 4.2 and implemented in algorithm 4.2. We walk through the procedure
in example 4.2.
function backtracking_line_search(f, ∇f, x, d, α; p=0.5, β=1e-4)
y, g = f(x), ∇f(x)while f(x + α*d) > y + β*α*(g⋅d)α *= pend
return α
end

The first condition is insufficient to guarantee convergence to a local minimum.
Very small step sizes will satisfy the first condition but can prematurely converge.
Backtracking line search avoids premature convergence by accepting the largest
satisfactory step size obtained by sequential downscaling and is guaranteed toconverge to a local minimum.
Another condition, called the curvature condition, requires the directional derivative at the next iterate to be shallower:∇ d( k ) f ( x( k + 1) ) ≥ σ ∇ d( k ) f ( x( k ) )Also known as Armijo line search,
L. Armijo, “Minimization of Functions Having Lipschitz Continuous First Partial Derivatives,” Pacific Journal of Mathematics, vol. 16,
no. 1, pp. 1–3, 1966.

6

Algorithm 4.2. The backtracking
line search algorithm, which takes
objective function f, its gradient∇f, the current design point x, a
descent direction d, and the maximum step size α. We can optionallyspecify the reduction factor p andthe first Wolfe condition parameterβ.
Note that the cdot character ⋅aliases to the dot function such thata⋅b is equivalent to dot(a,b). The
symbol can be created by typing\cdot and hitting tab.

(4.5)

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

4 . 3. a p p rox i ma te l i n e se a rc h 57

4

3

5

Figure 4.2. Backtracking line
search used on the Rosenbrockfunction (appendix B.6). The blacklines show the seven iterations
taken by the descent method andthe red lines show the points considered during each line search.

7

6
x2

8

2

1
x1

where σ controls how shallow the next directional derivative must be. Figures 4.3
and 4.4 illustrate this condition. It is common to set β < σ < 1 with σ = 0.1 whenapproximate linear search is used with the conjugate gradient method and to 0.9
when used with Newton’s method.7
An alternative to the curvature condition is the strong curvature condition, which
is a more restrictive criterion in that the slope is also required not to be too positive:|∇d(k) f (x(k+1) )| ≤ −σ∇d(k) f (x(k) )The conjugate gradient method is
introduced in section 5.2, and Newton’s method is introduced in section 6.1.
7

(4.6)

Figure 4.5 illustrates this condition.
Together, the sufficient decrease condition and the curvature condition form
the Wolfe conditions. The sufficient decrease condition is often called the first Wolfe
condition and the curvature condition is called the second Wolfe condition. The
sufficient decrease condition with the strong curvature condition form the strong
Wolfe conditions.
Satisfying the strong Wolfe conditions requires a more complicated algorithm,
strong backtracking line search (algorithm 4.3).8 The method operates in two phases.
The first phase, the bracketing phase, tests successively larger step sizes to bracket
an interval [α(k−1) , α(k) ] guaranteed to contain step lengths satisfying the Wolfe
conditions.

J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. Springer,
2006.
8© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

58 c ha p te r 4 . l oc a l d e sc e n t

Consider approximate line search on f ( x1 , x2 ) = x12 + x1 x2 + x22 fromx = [1, 2] in the direction d = [−1, −1], using a maximum step size of 10,
a reduction factor of 0.5, a first Wolfe condition parameter β = 1 × 10−4 , anda second Wolfe condition parameter σ = 0.9.
We check whether the maximum step size satisfies the first Wolfe condition,
where the gradient at x is g = [4, 5]:

Example 4.2. An example of backtracking line search, an approximate line search method.

f (x + αd) ≤ f (x) + βα(g⊤ d)f ([1, 2] + 10 · [−1, −1]) ≤ 7 + 1 × 10−4 · 10 · [4, 5]⊤ [−1, −1]
217 ≤ 6.991

It is not satisfied.
The step size is multiplied by 0.5 to obtain 5, and the first Wolfe condition
is checked again:f ([1, 2] + 5 · [−1, −1]) ≤ 7 + 1 × 10−4 · 5 · [4, 5]⊤ [−1, −1]
37 ≤ 6.996

It is not satisfied.
The step size is multiplied by 0.5 to obtain 2.5, and the first Wolfe condition
is checked again:f ([1, 2] + 2.5 · [−1, −1]) ≤ 7 + 1 × 10−4 · 2.5 · [4, 5]⊤ [−1, −1]
3.25 ≤ 6.998

The first Wolfe condition is satisfied.
The candidate design point x′ = x + αd = [−1.5, −0.5] is checked against
the second Wolfe condition:∇ d f (x′ ) ≥ σ ∇ d f (x )[−3.5, −2.5]⊤ [−1, −1] ≥ σ [4, 5]⊤ [−1, −1]
6 ≥ −8.1

The second Wolfe condition is satisfied.
Approximate line search terminates with x = [−1.5, −0.5].

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

4 . 3. a p p rox i ma te l i n e se a rc h 59yFigure 4.3. The curvature condition, the second Wolfe condition, is necessary to ensurethat second-order function approximations have positive curvature,
thereby having a unique global
minimum.

minimum of second-order approximation0αf (x) +
yf (x) +

Figure 4.4. Regions where the curvature condition is satisfied.

ασ∇

α∇

d f (x)d f (x)reduced curvature0αf (x) +
yf (x) +

Figure 4.5. Regions where thestrong curvature condition is satisfied.

ασ∇

α∇

d f (x)d f (x)0α© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

60c ha p te r 4 . l oc a l d e sc e n t

An interval guaranteed to contain step lengths satisfying the Wolfe conditions
is found when one of the following conditions hold:f (x + αd) ≥ f (x)f (x + αd) > f (x) + βα∇d f (x)∇ f (x + αd) ≥ 0

y = f (x)yf (x) +βα∇d f (x)

(4.7)
(4.8)
(4.9)

f (x + αd) ≥ f (x)f (x + αd) > f (x) + βα∇d f (x)∇ f (x + αd) ≥ 0
Wolfe conditions satisfied

0αFigure 4.6. The interval [0, α] isguaranteed to bracket an interval
containing a step length satisfying
the strong Wolfe conditions when
any of these three conditions is
true.

Satisfying equation (4.8) is equivalent to violating the first Wolfe condition,
thereby ensuring that shrinking the step length will guarantee a satisfactory step
length. Similarly, equation (4.7) and equation (4.9) guarantee that the descent
step has overshot a local minimum, and the region between must therefore containa satisfactory step length.
Figure 4.6 shows where each bracketing condition is true for an example line
search. The figure shows bracket intervals [0, α], whereas advanced backtracking
line search successively increases the step length to obtain a bracketing interval[ α ( k −1) , α ( k ) ] .
In the zoom phase, we shrink the interval to find a step size satisfying thestrong Wolfe conditions. The shrinking can be done using the bisection method(section 3.7), updating the interval boundaries according to the same interval
conditions. This process is shown in figure 4.7.

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

4 . 4 . tru st re g i on me thod s 61

f (x + αd) ≥ f (x)f (x + αd) > f (x) + βα∇d f (x)∇ f (x + αd) ≥ 0

α (1)
α (2)
y

α (4)
α (3)

Figure 4.7. The first phase of strong
backtracking line search, indicatedwith black open circles, is used tobracket an interval. In this case, thecondition ∇ f (x + α(4) d) ≥ 0 is
triggered, causing the bracketinginterval to be [α(3) , α(4) ]. Then thezoom phase, indicated by the red
open circle, shrinks the bracketedregion until a suitable step length
is found.

α (5)α4.4 Trust Region Methods
Descent methods can place too much trust in their first- or second-order information, which can result in excessively large steps or premature convergence. A
trust region9 is the local area of the design space where the local model is believed
to be reliable. A trust region method, or restricted step method, maintains a local
model of the trust region that both limits the step taken by traditional line searchand predicts the improvement associated with taking the step. If the improvement closely matches the predicted value, the trust region is expanded. If the
improvement deviates from the predicted value, the trust region is contracted.10
Figure 4.8 shows a design point centered within a circular trust region.
Trust region methods first choose the maximum step size and then the step
direction, which is in contrast with line search methods that first choose a step
direction and then optimize the step size. A trust region approach finds thenext step by minimizing a model of the objective function fˆ over a trust region
centered on the current design point x. An example of fˆ is a second-order Taylor
approximation (see appendix C.2). The radius of the trust region, δ, is expanded
and contracted based on how well the model predicts function evaluations. The
next design point x′ is obtained by solving:
minimizefˆ(x′ )

subject tox − x′ ≤ δx′(4.10)

where the trust region is defined by the positive radius δ and a vector norm.11
The equation above is a constrained optimization problem, which is covered inchapter 10.

K. Levenberg, “A Method for
the Solution of Certain Non-Linear
Problems in Least Squares,” Quarterly of Applied Mathematics, vol. 2,
no. 2, pp. 164–168, 1944.

9

A recent review of trust region methods is provided by Y. X.
Yuan, “Recent Advances in Trust
Region Algorithms,” Mathematical Programming, vol. 151, no. 1,
pp. 249–281, 2015.
10

There are a variety of efficientmethods for solving equation (4.10) efficiently. For an
overview of the trust region
method applied to quadraticmodels, see D. C. Sorensen,“Newton’s Method with a Model
Trust Region Modification,” SIAM
Journal on Numerical Analysis,
vol. 19, no. 2, pp. 409–426, 1982.
11© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

62c ha p te r 4 . l oc a l d e sc e n tfunction strong_backtracking(f, ∇, x, d; α=1, β=1e-4, σ=0.1)
y0, g0, y_prev, α_prev = f(x), ∇(x)⋅d, NaN, 0αlo, αhi = NaN, NaN# bracket phase
while true
y = f(x + α*d)if y > y0 + β*α*g0 || (!isnan(y_prev) && y ≥ y_prev)
αlo, αhi = α_prev, α
break
end
g = ∇(x + α*d)⋅d
if abs(g) ≤ -σ*g0
return αelseif g ≥ 0αlo, αhi = α, α_prev
break
end
y_prev, α_prev, α = y, α, 2α
end

end

Algorithm 4.3. Strong backtracking approximate line search forsatisfying the strong Wolfe conditions. It takes as input the objectivefunction f, the gradient function∇, the design point x and direction
d from which line search is conducted, an initial step size α, and
the Wolfe condition parameters
β and σ. The algorithm’s bracket
phase first brackets an interval containing a step size that satisfies thestrong Wolfe conditions. It then reduces this bracketed interval in thezoom phase until a suitable step
size is found. We interpolate withbisection, but other schemes can beused.

# zoom phase
ylo = f(x + αlo*d)while true
α = (αlo + αhi)/2
y = f(x + α*d)if y > y0 + β*α*g0 || y ≥ ylo
αhi = αelse
g = ∇(x + α*d)⋅d
if abs(g) ≤ -σ*g0
return αelseif g*(αhi - αlo) ≥ 0
αhi = αloend
αlo = αend
end

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

4 . 5 . te rmi n a ti on c on d i ti on s 63

Figure 4.8. Trust region methods
constrain the next step to lie within
a local region. The trusted region
is expanded and contracted basedon the predictive performance ofmodels of the objective function.

The trust region radius δ is expanded or contracted based on the local model’s
predictive performance. Trust region methods compare the predicted improvement ∆ypred = f (x) − fˆ(x′ ) to the actual improvement ∆yact = f (x) − f (x′ ):η=f (x ) − f (x′ )actual improvement=
predicted improvement
f (x) − fˆ(x′ )(4.11)The ratio η is close to 1 when the predicted step size matches the actual step
size. If the ratio is too small, such as below a threshold η1 , then the improvement
is considered sufficiently less than expected, and the trust region radius is scaled
down by a factor γ1 < 1. If the ratio is sufficiently large, such as above a thresholdη2 , then our prediction is considered accurate, and the trust region radius isscaled up by a factor γ2 > 1. Algorithm 4.4 provides an implementation andfigure 4.9 visualizes the optimization procedure. Example 4.3 shows how toconstruct noncircular trust regions.

4.5 Termination Conditions
There are four common termination conditions for descent direction methods:• Maximum iterations. We may want to terminate when the number of iterations
k exceeds some threshold kmax . Alternatively, we might want to terminate oncea maximum amount of elapsed time is exceeded.
k > kmax(4.12)© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

64

c ha p te r 4 . l oc a l d e sc e n tfunction trust_region_descent(f, ∇f, H, x, k_max;
η1=0.25, η2=0.5, γ1=0.5, γ2=2.0, δ=1.0)
y = f(x)for k in 1 : k_max
x′, y′ = solve_trust_region_subproblem(∇f, H, x, δ)η = (y - f(x′)) / (y - y′) # improvement ratio
if η < η1
δ *= γ1 # scale down trust region
else
x, y = x′, y′ # accept new point
if η > η2
δ *= γ2 # scale up trust region
end
end
end
return x
end

Algorithm 4.4. The trust region
descent method, where f is theobjective function, ∇f produces
the derivative, H produces the
Hessian, x is an initial design
point, and k_max is the number of
iterations. The optional parameters
η1 and η2 determine when thetrust region radius δ is increased
or decreased, and γ1 and γ2
control the magnitude of thechange. An implementation forsolve_trust_region_subproblem

must be provided that solves
equation (4.10). We have provided
an example implementationthat uses a second-order Taylor
approximation about x0 with acircular trust region.

using Convex, SCS
function solve_trust_region_subproblem(∇f, H, x0, δ)
x = Variable(length(x0))
p = minimize(∇f(x0)⋅(x-x0) + quadform(x-x0, H(x0))/2)
p.constraints += norm(x-x0) <= δsolve!(p, SCS.Optimizer)
return (x.value, p.optval)end© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

4 . 5 . te rmi n a ti on c on d i ti on s 65

x2

x(10)

Figure 4.9. Trust region optimization used on the Rosenbrock function (appendix B.6).

x(1)
x1

Trust regions need not be circular. In some cases, certain directions may havehigher trust than others.
A norm can be constructed to produce elliptical regions:kx − x0 kE = (x − x0 )⊤ E(x − x0 )Example 4.3. Trust region optimization need not use circulartrust regions. Additional detail isprovided by J. Nocedal and S. J.
Wright, “Trust-Region Methods,”
in Numerical Optimization. Springer,
2006, pp. 66–100.

with kx − x0 kE ≤ 1 where E is a symmetric matrix that defines the ellipse.

The ellipse matrix E can be updated with each descent iteration, which
can involve more complicated adjustments than scaling the trusted region.

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

66

c ha p te r 4 . l oc a l d e sc e n t• Absolute improvement. This termination condition looks at the change in thefunction value over subsequent steps. If the change is smaller than a given
threshold, it will terminate:f (x( k ) ) − f (x( k +1) ) < ǫ a(4.13)• Relative improvement. This termination condition also looks at the change infunction value but uses the step factor relative to the current function value:f (x(k) ) − f (x(k+1) ) < ǫr | f (x(k) )|(4.14)• Gradient magnitude. We can also terminate based on the magnitude of thegradient:k∇ f (x(k+1) )k < ǫg
(4.15)
In cases where multiple local minima are likely to exist, it can be beneficial toincorporate random restarts after our termination conditions are met where we
restart our local descent method from randomly selected initial points.

4.6 Summary• Descent direction methods incrementally descend toward a local optimum.
• Univariate optimization can be applied during line search.
• Approximate line search can be used to identify appropriate descent step sizes.
• Trust region methods constrain the step to lie within a local region that expands
or contracts based on predictive accuracy.
• Termination conditions for descent methods can be based on criteria such as
the change in the objective function value or magnitude of the gradient.

4.7 Exercises
Exercise 4.1. Why is it important to have more than one termination condition?

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com

4 . 7. e x e rc i se s 67

Exercise 4.2. The first Wolfe condition requiresf (x(k) + αd(k) ) ≤ f (x(k) ) + βα∇d(k) f (x(k) )(4.16)

What is the maximum step length α that satisfies this condition, given that f (x) =5 + x12 + x22 , x(k) = [−1, −1], d = [1, 0], and β = 10−4 ?

© 2019 Massachusetts Institute of Technology, shared under a Creative Commons CC-BY-NC-ND license.
2022-05-22 00:25:57-07:00, revision 47fd495, comments to bugs@algorithmsbook.com