Learning the Step-size Policy for the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno Algorithm
Authors: Lucas N. Egidio
Anders Hansson
Bo Wahlberg
Venue:  (2021)
Keywords: Unconstrained optimization
Step-size policy
L-BFGS
Learned optimizers

Abstract: We consider the problem of how to learn a step-size policy for the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm. This is a limited computational memory quasi-Newton method widely used for deterministic unconstrained optimization but currently avoided in large-scale problems for requiring step sizes to be provided at each iteration. Existing methodologies for the step size selection for L-BFGS use heuristic tuning of design parameters and massive re-evaluations of the objective function and gradient to find appropriate step-lengths. We propose a neural network architecture with local information of the current iterate as the input. The step-length policy is learned from data of similar optimization problems, avoids additional evaluations of the objective function, and guarantees that the output step remains inside a pre-defined interval. The corresponding training procedure is formulated as a stochastic optimization problem using the backpropagation through time algorithm.  The performance of the proposed method is evaluated on the training of classifiers for the MNIST database for handwritten digits and for CIFAR-10. The results show that the proposed algorithm outperforms heuristically tuned optimizers such as ADAM, RMSprop,  L-BFGS with a backtracking line search and L-BFGS with a constant step size. The numerical results also show that a learned policy can be used as a warm-start to train new policies for different problems after a few additional training steps, highlighting its potential use in multiple large-scale optimization problems.

Under review as a conference paper at ICLR 2021
LEARNING THE STEP-SIZE POLICY FOR THE LIMITED- MEMORY BROVEEN-EEETCIRR-GLDDERRRS-ASAAT ALGORITHM
Anonymous authors Paper under double-bling review
ABSTRACT
We consider the problem of how learn a step-size policy for the Limited- Memory Seaeenr-leeccee-ooffff--------ae ited computational memory quasi-Newton method widely used for deterministic unconstrained optimization but currently avoided in large-scale problems for re- quiring step sizes be provided at each iteration. Existing methodologies for the step size selection for L-BFGS use heuristic tuning of design parameters and massive re-evaluations of the objective function and gradient to find appropriate step-lengths. We propose neural network architecture with local information of the current iterate as the input. The step-length policy is learned from data of simi- lar optimization problems, avoids additional evaluations of the objective function, and guarantees that the output step remains inside pre-defined interval. The cor- responding training procedure is formulated as stochastic optimization problem using the backpropagation through time algorithm. The performance of the pro- posed method is evaluated on the training of classifiers for the MNIST database for handwritten digits and for CIFAR-10. 10. The results show that the proposed al- gorithm outperforms heuristically tuned optimizers such as ADAM, RMSprop. L-BFGS with backtracking line search and L-BFGS with constant step size. The numerical results also show that learned policy can be used warm-start to train new policies for different problems after a few additional training steps, highlighting its potential use in multiple large-scale optimization problems.
(L-BFGS) algorithm. This is a lim-
INTRODUCTION
Consider the unconstrained optimization problem
minimize f(x)
(1)
where f R" + R is objective function that is differentiable for all E IR", with being the number of decision variables forming æ. Let V (xx) be the gradient f(x) evaluated some
Co E Ra A general quasi-Newton algorithm for solving this problem iterates
#b++
t Hage
(2)
for an initial Ty E R" until a given stop criterion is met. At the k-th iteration, gk = V fax))) the gradient, Hg is positive-definite matrix satisfying the secant equation (Nocedal and Wright, 2006, 137) and l is the step size.
In this paper, develop a policy that learns suitably determine step sizes te when the product Seeeen-iieechee-GGGlrrr--------
H&g is calculated by the Limited-Memory rithm (Liu and Nocedal, 1989). The main contributions of the paper are:
(L-BFGS) algo-
1. We propose a neural network architecture defining this policy taking as input local information of the current iterate. In contrast with more standard strategies, this policy is tuning- free and avoids re-evaluations of the objective function and gradients at each step. The training procedure is formulated as a stochastic optimization problem and can be performed by easily applying backpropagation through time (TBPTT).

---

Under review as a conference paper at ICLR 2021
2. Training classifiers in the MNIST database (LeCun et al., 1998), our approach is competitive against heuristically tuned optimization procedures. Our tests show that the proposed policy is not only able to outperform competitors such as ADAM and RMSprop in wall-clock time and optimal/final value, but also performs better than L-BFGS with backtracking line searches, which is the gold standard, and with constant step sizes, which is the baseline.
3. According to subsequent experiments on CIFAR-10 (Krizhevsky et al., 2009), the proposed pol- icy can generalize to different classes of problems after a few additional training steps on exam- ples from these classes. This indicates that learning may be transferable between distinct types of tasks, allowing to explore transfer learning strategies.
This result is a step towards the development of optimization methods that frees the designer from tuning control parameters as it will be motivated in Section 2. The remaining parts of this paper organized as follows: Section 3 presents the classical L--FGS algorithm and discuss some method- ologies to determine step sizes; Section 4 contains the architecture for the proposed policy and also discussions on how it was implemented: Section 5 describes the training procedure; and, finally, Section 6 presents experiments using classifiers to operate on MNIST and CIFAR-I0 databases. The notation is mainly standard. Scalars are plain lower-case letters, vectors are bold lower-case, and matrices are bold upper-case. The clip function is defined as clip; (y) min (u, max y)).
2 MOTIVATION
Most algorithms used in artificial intelligence and statistics are based on optimization theory, which has widely collaborated for the success of machine learning applications the last decades. How- ever, this wwo-way bridge seems not to be currently leveraging its full potential in the other sense, that is, to learn how to automate optimization procedures. Indeed, performing satisfactory opti- mization, or solving learning problems, still relies upon the appropriate tuning of parameters of the chosen algorithm, which are often grouped with other syper-parameee the existence of several methodologies to obtain good values for these parameters (Bengio, 2000; Bergstra et al., 2011; Bergstra and Bengio, 2012; Snoek et al., 2015; Daniel et al., 2016; Dong et al., 2018), the search for tuning- funing-free algorithms that perform better than heuristically designed ones is of great interest among practitioner and theoreticians. Indeed, besides the cenerally-desirall convergence, the ready-to-use nature of such algorithms allows the user focus his attention on other problem-level hyper-pane parameters while the optimization procedure is automatically performed, resulting in better time and effort allocation. As recent advancements of machine learning have helped automatize the solution of numberless problems, optimization theory should equally benefit from these, balaneing the bridge flows.
of the learning task. Despite
faster
From a wider viewpoint, most optimization problem requires the user to select an algorithm and tune it to some extent. Although intuition and knowledge about the problem can speed-up these processes, trial-and-error methodologies are often employed which can be a time-consuming and inefficient task. With that in mind, the concept of Learned optimizers has been gathering attention in the last few years and, basically, refers to optimization policies and routines that were learned by looking at instances of optimization problems, here called tasks. This idea was introduced by Li and Malik (2016) and Andrychowicz et al. (2016) building upon previous results of "leanning to learn" or "meta-learning" (Thrun and Pratt, 1 1998; Hochreiter et al., 2001). In the former, the authors presented an optimization policy based on neural network trained by reinforcement learning and taking as input the history of gradient vectors at previous iterations. The latter adopts a long short-term memory (LSTM) to achieve a similar task, but the learning is done by truncated backpropagation through time after unrolling the proposed optimizer for a certain number of steps. Subsequently, it was shown in Metz et al. (2019) how multilayer perceptrons (MLP). adequately trained using a combined gradient estimation method, can perform faster in wall-clock time compared to current algorithms of choice. Also within this scenario, Xu et al. (2019) a reinforcement learning- methodology to auto-learn an adaptive learning rate is presented. Following this same fashion, in this present paper, instead of completely learning an optimizer from data, we propose a mixture of these ideas into a classical optimization procedure. Thus, the resulting optimizer, composed by a combination of L-BFGS and the proposed policy, will be learned in a constrained domain that assures valuable mathematical properties. The idea is to leverage both frameworks, inheriting the theoretical aspects assured by optimization theory while learning a policy to rule out the dend-n of parameters.
2

---

Under review as a conference paper at ICLR 2021
Algorithm 1: L-BFGS algorithm
Input: 1i+1 T;; y: Di+1 D: and 1/(0f yi) for all i E k m.... ........-. k
and current gradient gk:
Result: update direction d -4
x 9k: 2 for i = I.... m do
7 r 77:
8 for = k m. ...... do
a; t --:8 q:
9
3 p.y r;
at= q o;yi:
r r + 8: -3);
end
end
2 k yk 11/y1//////t 1 11
T
dg --;;
3 L-BFGS ALGORITHM
The L-BFGS algorithm was originally presented in Liu and Nocedal (1989) and is here transcribed into Algorithm 1. It is a quasi-Newton method derived from the BFGS algorithm (Nocedal and Wright, 2006) lowering space complexity from quadratic to linear in the problem dimension at the expense of precision. This algorithm calculates descending direction in the search space taking into account an estimate of the inverse hessian matrix of f(z), given by Hg. This matrix is not explicitly constructed but rather the product d, of it and Ok. which have to be stored. This property makes it often the algorithm of choice for large-scale deterministic non-linear optimization problems. If (ax) is convex in æ, this algorithm is guaranteed to provide a descending update direction, but the same does not apply for non-conved objective functions. However, simple way to circumvent this is by removing iterations in lines 2 and 8 of Algorithm such that Ps S 0 (Nocedal and Wright, 2006, p. 537), which is used in this paper.
H is obtained from the past values
A matter of great relevance within this scope is how to choose an appropriate step size t2 to apply the update rule Eq. (2). To the best of our knowledge, there does not seem to exist a consensus on how to choose l in general way for non-convex objective functions. The scaling factor y in lines 6-7 of Algorithm is known to assure that the step size 4 = is accepted most iterations the convex optimization context, but not always. We will refer to constant step-size policy that outputs t,- as the baseline L-BFGS. However, line search (LS) procedure is often combined with L-BFGS to assure its convergence. Ideally, this should be performed by solving 44 arg min f tdl:) but this exact approach is often too expensive to be adopted, motivating the use of inexact ones. An example is the bucktracking line search (BTLS), which takes initial length 4. for the step size and shrinks it repeatedly until the so-called sufficient decrease Wolfe Condition f(x + (xdx) S [**) + 64 4. d2 is fulfilled, where e₁ E (0, 1) is control parameter to be tuned. Another parameter that has to be designed is the contraction factor C2 E (0. 1) that shrinks the step size, i.c., t + C2t2, see Nocedal and Wright (2006, p. 37). This method assures convergence to local- minima at the cost of re-evaluating the objective function several times per iteration. This is price that the user is, in some cases, willing to pay, but for large-dimensional problems this procedure likely to become the bottle-neck of the optimization task. is important to highlight that the method to presented may also apply to other optimization algorithms that deeply rely on line searches to perform well. However, this paper focus on L-BFGS it is often the algorithm of choice in large-scale deterministic optimization.
In the context of stochastic optimization, many modified versions of Algorithm 1 together with methodologies for choosing & are available (Moritz et al., 2016; Zhou et al., 2017; Bollapragada et al., 2018; Wills and Schön, 2019), but for sake of simplicity, our work will deal exclusively with deterministic non-linenn optimization problems.
4 LEARNED POLICY FOR SELECTING STEP SIZES
Recalling the definition of Sk and ys in Algorithm 1, our policy is defined as tx
π (d Sk-1, $ :: 0) and selects an adequate step size for L-BFGS but neither relying on any parameter tuning nor requiring additional evaluations of the objective function. Instead, its parameters that are represented by 0 should be learned from data. Let us, from now on, de-
3

---

Under review as a conference paper at ICLR 2021
r-: 0)
Linear Layer ( 18
B) RR
d2. Dk. 8k- 1.-0-1
dotIn(-)
uo
P(-
7V
exp (+)
1/
Linear Layer 2 1 RR
Figure 1: Neural network architecture for the proposed policy to generate step sizes f.--
fine this policy combined with Algorithm as the L-BFGS-T approach. The architecture of the policy π (dd,, Ih. 8k-1, 3k-1: 0) is shown in Fig. To allow the policy to be independent from the problem size n, only the inner products between its inputs are used. These values de- fine uo dotln(dk, gk., 8k-1, 1k-1) where dotin(-) returns the component-wise application of f(x) = In(min((z,)) e)) to the elements of X [dx 9k 8k-1 yk-1] I |'[d, [dæ gt 8 yis- i] but with the superdiagonal entries having their signs reversed. We have chosen = 10-8 to avoid imaginary- valued entries.
The vector ug is the input to two parallel input layers, which are fully connected linear layers that transport information in Ug another vector space R (in our tests, we adopted nu = 6). Their outputs, as usual, are defined as i = Wo uo + bos and U2 = Wo2 + bo2- The logarithm operation was adopted to let the linear layers evaluate products and divisions between powers of the inputs by simply summing and subtracting them. Moreover, as the output is positive, working in the logarithmic vector space allows us to use a wider range of numerical values. Subsequently, let us to calculate the scalar projection of un onto Uy and clip the result to some interval [Tm. TM. yielding the log-step size
define the normalized vectors ū = ui /|lu2| and ú2 u2/ /
The clip TM (u? ū1)
p (u, U2)
Finally, the selected step size is obtained as 4 = e To geometrically interpret this, we sketch three different scenarios in Fig. 2. The dashed lines represent orthogonal axes spanned by some arbitrary ú2 and the gray strip represents the interval [Tm: TM] along the direction of ú2 whence ты should be taken. When the Linear Layer 1 maps uo into u1, the scalar projection of ūj onto ú2 is beyond the maximal TM, so Tı is clipped to it. In the same way, for on the step size will be the minimal one ia = eTt whereas for the intermediate üj we have т E (Tm, TMx ). The two layers, jointly trained, should learn how to position ūn and ūy in the lifted space to represent important directional information of de and gs by looking similar optimization tasks, being thus able produced suitable step sizes. This approach is powerful enough to capture in- teresting mathematical local properties of this prob- lem. As an instance, it could calculate cos O
frmm fark
- g//g/ di dugl ge. where de is the angle formed be- tween d2 and the steepest descend direction -9k. by let- ting parameters 0 = (Wo1, bo1. Wo2, 6o2) and limits Tm and Tyr be given as described in Appendix A. Also, x 9 defines the so-called curvature condition (No- cedal and Wright, 2006, p. 137) and the occurrence of small-valued g gb and $k-1k-1 with - < 0 implies that the iterate wa approximates a local maximum or saddle point, as this inequality indicates that at least one eigenvalue of the hessian matrix is negative at æg.
22
Figure 2: Geometric representation of the scalar projection and clip procedures for 3 cases.
Indeed, the considered inner products forming uo also employed in many procedures for determining step sizes, for example, in the sufficient decrease Wolfe condition for backtracking line search, which makes our policy comparable to them in the sense that π (1; 0) does not require additional information to operate.
However, notice that the clip function is not suitable for training given that it is non-differentiable and gradients cannot be backpropagated through it. Fortunately, the clip operation (3) can be cast as
4

---

Under review as a conference paper at ICLR 2021
convex optimization problem
Tk arg min ||22T ui TERR
(4)
s.t. Tm ST S TM
(5)
allowing TI to be calculated by a convex optimization layer, defined here as a CVX Layer, (Agrawal al., 2019). This last layer can output the solution problem. For the special case where a solution is not differentiable with respect to the input (e.g..
parameter-dependdntt convex optimization
in our case when an inequality constraint is active), the automatic differentiation procedure delivers an heuristic quantity that can be employed as a gradient. The use of a CVX Layer is therefore convenient for training our policy but, on the other hand, using Eq. (3) in its place when applying the already-trained policy significantly speeds up the step-size evaluation, compared to solving (4).
important to remark that this policy is defined as independent from both the memory length m of Algorithm 1 and the problem dimension n. Additionally, the lower and upper limits for the log-step size are Tm and TM, respectively, and can also be learned. In this work, however, we chose Tm -3 and TM 0, letting t E [0.0497. 1]. This interval is comprehensive enough to let our method be compared in a fair way to backtracking line searches. Moreover, when we allowed TM to be learned in our tests it converged to values that were very close to TM 0, indicating that I was already an adequate upper limit for the step size.
TRAINING THE POLICY
The L-BFGS- procedure can be trained by truncated backpropagation through time (TBPTT), in a similar way to Andrychowicz et al. (2016). From this point on, training the optimizer is referred to as the outer optimization problem whereas an instance of a task in the form of (I) called the inner optimization problem. Therefore, this outer problem is defined as
minimize F(0)= RR ES-T (Ex wx fxx((x))
(6)
s.t. Fk++1 = Tg + n (d;, 9k, 8k-1; yk f)de
(7)
where de is given by Algorithm 1, E is the truncated horizon over which optimization steps unrolled, We, h = 1,.... K are weight scalars, herein considered wi = and T is some set of tasks formed by inner objective functions f(x) to be optimized. In (6), the innermost expected value is approximated by sampling tasks within training set Trrain, one a time, and unrolling the optimization for K inner steps for some random 2g with i.i.d. components. One outer optimization step consists of, performing K inner steps, computing gradient for the outer optimization problem ((F and updating 6, in our case, by ADADELTA with learning rate equals I (Zeiler, 2012). To assure that different orders of magnitude of x are seen during this training, we set the initial point for the next outer step to be the last iterate from the previous one, i.e., Do +- Kx and perform another whole outer optimization step. This is repeated for T outer steps or until ||g < 10- 10 when new random æo is then sampled. Backpropagation to calculate (oF happens through all operations with exception of the inner gradient evaluation gs. which is considered an external input. Double floating boating-point precision is used to assure accurate results in the comparisons.
6 EXAMPLE: TRAINING CLASSIFIER ON MNIST
All tests were carried out with the aid of PyTorch (Paszke et al.. 2019) to backpropagate gradients and of cvxpy Layers (Agrawal et al., 2019) to implement the CVX layers. They were run on an Intel Xeon Gold 6132 equipped with an NVidia V100. First, we carried out tests on convex opti- mization problems, namely quadratic functions and logistic regression problems, but no significant difference was noticed and these were, therefore, omitted.
In this example, we trained an MLP with T4 = hidden layer with n.y = 20 units and sigmoid activation functions to classify images of digits in MNIST database (LeCun et al., 1998). This model is adapted from one example in Andrychowicz et al. (2016). We used a bull-batch gradient at every iteration, even though stochastic optimization is generally the most common strategy employed in similar tasks. Nevertheless, our main interest in this example is not the classification problem itself but rather to analyze the optimization problem and how our deterministic algorithm performs it.
5

---

Under review as a conference paper at ICLR 2021
10"1
[(ma) Thelk 1
f( 'Dhll
[(aa) Talk
10-3
10
20"
10
10
L-00C5-v
0.0
0.
0.6
0.1
0.6
-2
0.0
012
04 time
0.6
0.0
0.22
3.4
0.6
time
time
Figure 3: Objective function f(xx)) at the current iterate with respect to wall-clock time for all optimizers and 3 selected tasks (top) and correspondent step sizes for x(, 0) and BTLS (bottom).
The 16,280 parameters defining an MLP are concatenated æ and f(x) is the associated cross-entropy loss function for a given set of images. This is known to be a non-convex optimization problem, mainly because of the presence of non-linear activation functions. A training set of tasks Trrain was constructed by randomly grouping images in MNIST training set into 60 batches of N 1,000 images. For each these batches one initial condition Zn was sampled, which altogether compose Ttrain with 60 tasks. The policy (:; was trained for 50 epochs, K 50 and T 8, and its performance was compared to other methods, namely, ADAM, RMSProp. L-BFGS with a BTLS and the bascline L-BFGS (referred to L-BFGS-B). For rumning Algorithm I, we selected The learning rates ADAM and RMSProp were heuristically tuned to yield fast convergence by exhaustive search within the set {i 102 i {1,3), j {-3,.... -1]] and the values 0.03 and 0.01 were used, respectively. The BTLS parameters с₁ and 02 were searched in the set (0.25, 0.5, 0.75)² and C₁ 0.25, 02 0.5 were chosen, associated the best results, i.e., fastest convergence on tasks in Terain- The initial step size for the BTLS was t4 1. The following comparisons were performed images in the MNIST test set split into 10 batches of N = 1, 1,000 images, and 100 random samples of Zg were generated for each batch, resulting 1,000 tasks. The optimization was performed for K = 800 steps or until Iggl| < 10-5, The first 3 samples for each optimizer were considered "warm-up" runs and, therefore, were discarded to avoid having time measurement affected by any initial overhead.
5.
test set of tasks Treat built similarly to Terain but considering all
The objective function value for three selected tasks is shown in the upper plots of Fig. 3 along with the correspondent selected step sizes by (f: 0) and the BTLS, the bottom ones. For Task 1, L-BFGS-JT was successful in attaining lower values for ff fxx when compared to the other algorithms. For some tasks, such Task 2, poorer performance was noticed when compared to the other L-- BEGS approaches and, for some other tasks as Task 3, Adam and R.MSprop are more successful than the others. This suggests that none of these methods outperforms the others in a general case. Notice that in these figures, the spikes in the curves associated with L-BFGS-T and with the baseline L-BFGS-B represent steps at which 4 (:: 0) and & 1, respectively, were step sizes that provided decrease. Results for other tasks are presented in Appendix B.
For each individual task, the first instant of time t, which the optimization procedure attained some precision-bases stop criteria ||gx|] < for different values of & was computed for all four optimization procedures, and comparison between our methodology and others is shown in Fig. 4. These plots compare algorithms two-by-two and the way to interpret them is by observing that point above the blue line represents a task for which the algorithm associated to the x axes reached the precision criterion faster, and vice-versa. If such ε does not exist, we define i, two subplots, on the right and on the top, are use to represent tasks for which the given precision was reached by one of the algorithms but not by the other. Tasks for which the criterion was not reached by both algorithms are not displayed. Notice that better precision values were reached by approach when compared to ADAM and RMSProp but similar performance obtained
and the

---

Under review as a conference paper at ICLR 2021
Ia]
[o]] <le43
Ia
Ia.
16 Klo44
[od <lo44
Io <I:04
clo94
lal <l643
<le40
la
1-280005
1-201
1-B9
ADAM
RMi8proo
LAHESS-BTTS
Figure 4: Time b (in seconds) when the step criterion | e was reached for all tasks in Trest compared for different optimizers and (m,, nu)) (1,
Table 1: Percentages of wins (W) and ties (T) of L-BFGS-T ग trained for (n,, n.) (1,20) 20) against other algorithms on tasks in Trest with respect to which one attained the stop criterion for different (ni, ) (m4, ny) = (1, 20)
[ < first
(nu, na) = (2, 400)
(nu, R2)) = (4, 800)
Competitor
E
W (%)
T (%)
E
W (%)
T (%)
W (%)
T (%)
10 10-3 3
0
0
10 5
4.7
10
0
0
ADAM
10-4
95.8
4.2
10-6
5.8
10 6
0.1
0
10
90.9
9.1
10-7
10.5
10
0.2
0.1
10
5.5
10
100
10
92.7
7.3
RMSProp
10-1
95.8
4.2
10-6
100
10
81.0
19.0
10-5
90.9
9.1
10-7
100
10-7
67.2
32.8
10-3
32.1
10
5.7
10
9.2
1.2
L-BFGS-B
10 10-4
49.3
0.5
10 10-6
5.5
10
8.5
2.1
10-5
54.9
3.3
10-7
5.5
10-7
7.6
3.0
10-3 3
40.8
10-5
1.0
10-5
0
0
L-BFGS-BTLS
10-4 55.6
0.6
10-6
1.0
10-6
0
0
10-5
59.0
2.5
10-7
1.2
10-7
0
0
when compared the heuristically designed backtracking line search method, which is the gold standard.
Additionally, Table 1 presents the percentage of times that L-BFGS-7 reached the defined precision before other methods, characterizing a "win"" and that both methods reached the precision at the exact time (which is very unlikely) have not reached this precision after K 800 inner steps, denoting "tie". To investigate whether our policy is able to generalize and perform well on higher-dimension problems, we also present these results for (r4, n.) equals to (2, 400) and (4, 800), characterizing problems of size n = 637, 600 and = 128, 317, 600 respectively. Different values of were considered as smaller values for ||gk] were reached for these two last cases.
) For ) (1, 20), which contains problems of the same dimension as those seen during train-
ing, L-BFGS-r clearly outperforms RMSProp and ADAM whereas it presents slightly faster con- vergence than L-BFGS-B and L-BFGG-BBTH BTLS for smaller s. Unfortunately. for higher dimension problems, the proposed policy did not achieve the same level of performance as the problem was trained for.
In spite of that, given the non-convexity of this problem, is also important to observe what were the minimum values obtained for /f f(x) by each algorithm. As the proposed policy does not assure de- creasing step size at each iteration, instead the final value f(xx) we looked at f. := min; f(*x), which can easily be stored and updated during the optimization. However analogous discussion is presented in Appendix but considering only the final values f(x and the same conclusions are drew. More than simply looking the minimum values, would like verify whether L-BFGS-a

---

Under review as a conference paper at ICLR 2021
(2g Rgj
(1,91)
82) m 4000
fio 9.) (4,80)
20
20
20
201
10-
101
H
-10
-10
-10
-20
-20
-21
ADAM
RMiipepp
LAPPOS BTL5
ADAM
Ratippop
L.DEGS Ree
LAPGS BTLS
ADAM
RMlippop
LABnGG UTLS
Figure 5: Box plots for values of Ia, where a is the algorithm the x-axis, encountered for tasks in Trest by different algorithms and pairs (mi, ne,).
attains lower values f. when compared to other algorithms. To this end we present the index
(f) = in (f(0)/ (L)-Bo
(8)
where s!aر represents the minimum value reached by some algorithm for [xx). Hence, (J) implies that L-BFGS-N performs better than in the task associated to f(x) and its initial condition. Box plots of the obtained values for all tasks and each one of the other algorithms are presented in Fig. 5. In these plots competitors. Also, ADAM and RMSprop generalized very poorly to higher- higher-dimension problems, indicating that some re-tuning is required for these algorithms. Under this perspective, L-BFGS-T had a similar performance to L-BFGS-BTLS and L-BFGS-B, despite the presence of some outliers indicating cases where our policy reached bad local minima. This showed how the proposed policy was successful in learning to provide step sizes in a single shot that are as good as those generated by heuristically designed line search, which benefits from the possibility of re-evaluating the objective function as much as needed.
notice that L-BFGS-T reaches, in average, better values than all of its
Finally, as last experiment, we applied the learned policy and these competitors to a class of tasks comprising the training of Convolutional Neural Network (CNN) to classify images in CIFAR- 10, see (Krizhevsky al., 2009). The adopted architecture is described in Zhang (2016) but sigmoid activation functions were replaced by ReLU to make this problem even more distant from the one was trained on. A training and a test set of tasks, Teain and Teest were built similarly to Terain and
Trest but using images in CIFAR-10 instead. Evaluating these algorithms in Thest and computing the index I, (f) for each task allows us to present the first box plot in Fig. 6. This figure indicates that x do not perform as good as before in these problems. This could be expected as a different architecture directly affects the nature of the objective function. To investigate whether the learned policy и can be used as a warm-start for the training a new policy cC, we perform additional training steps corresponding 10 epochs in the training set Terain but eliminating 5/6 of its tasks. This done to show that even with very low effort placed in this retraining phase and considering fewer learning data, can benefit from previous learning to speed-up new training procedures. The corresponding results are presented the second box plot of Fig. 6, which shows that the new policy T performs comparably the competitors. Certainly, further investigation required but this suggests that some learning can be transferred across distinct problem domains.
7 CONCLUSIONS
In this work we demonstrate how to build and train neural network to work as step-size policy for the L-BFGS algorithm. The step sizes provided by this policy are the same quality as those of backtracking line searches, hence making the latter superfluous. Moreover, L-BFGS with our step-size policy outperforms, in wall-clock time and optimal/final value, ADAM and RMSprop with heuristically tuned parameters in training classifiers for the MNIST database. Also, showed how a learned policy can used to warm start the training of new policies to operate on different classes of problems. In future work, intend to extend this result for stochastic optimization,
8

---

Under review as a conference paper at ICLR 2021
CIFAR-10 (x)
on CIPAR-10
10
1+
-10
-10
-20
-20
ADAM
AMtSpup
L-OPOi
LUPGSETTS
ADAM
AMlSpup
UBPGS
Figure 6: Box plots for values of Ia, where a is the algorithm the x-axis, encountered for tasks in Test by different algorithms when compared against and TC on the training of CNN.
allowing us to learn policies to determine, for example, learning rates in other classic machine learning algorithms.
REFERENCES
A. Agrawal, B. Amos, S. Barratt, S. Boyd, S. Diamond, and Z. Kolter. Differentiable convex opti- mization layers. In Advances in Neural Information Processing Systems, pages 9562-9574, 2019.
M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingford, and N. de Freitas. Learning to learn by gradient descent by gradient descent. arXiv preprint arXiv: 1606.0444, 2016.
Y. Bengio. Gradient-based optimization of hyperparameters. Neural computation, 1899)1188,, 2000.
Bergstra and Y. Bengio. Random search for hyper- pppem-par optimization. The Journal of Ma- chine Learning Research, 13(1):281-305, 2012.
J. S. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl. Algorithms for hyper-parameter optimization. In Advances in neural information processing systems, pages 2546-2554, 2011.
R. Bollapragada, D. Mudigere, J. Nocedal, H.-J. M. Shi, and P. P. Tang. A progressive batching L-BEGS method for machine learning. arXiv preprint arXiv: arXiv:/802.055 1e02.0822..74 2018.
C. Daniel, Taylor, and S. Nowozin. Learning step size controllers for robust neural network training. In AAAI, pages 1519-1525, 2016.
X. Dong, J. Shen, W. Wang, Y. Liu, L. Shao, and F. Porikli. Hyperparameter optimization for tracking with continuous deep Q-learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 518-521 2018.
S. Hochreiter, A. S. Younger, and P. R. Conwell. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks, pages 87-94. Springer, 2001.
A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 22222224,,.. 1998.
K. Li and Malik. Learning optimize. arXiv preprint 1606:0606 2016.
D. C. Liu and Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical programming, 45(1-3):51) 45(1-3):503-528, 1989.
L. Metz, N. Maheswaranathan, J. Nixon, D. Freeman, and J. Sohl-Dickstein. Understanding and cor- recting pathologies in the training of learned optimizers. In International Conference on Machine Learning, pages 4556-4565, 2019.

---

Under review as a conference paper at ICLR 2021
P. Moritz, R. Nishihara, and M. Jordan. A linearly-convergeen stochastic L-BFGS algorithm. In Artificial Intelligence and Statistics, pages 249-258, 2016.
J. Nocedal and S. Wright. Numerical optimization. Springer Science & Business Media, 2006.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, et al. Pytorch: An imperative style, high- pesrooerrrmmece deep learning library. Advances Neural Information 2019.
Processing Systems 32, pages
J. Snoek, O. Rippel, K. Swersky, R. Kiros, N. Satish, N. Sundaram, M. Patwary, M. Prabhat, and R. Adams. Scalable bayesian optimization using deep neural networks. International Confer- on Machine Learning, pages 2171-2180, 2015.
S. Thrun and L.. Pratt. Learning learn. Springer Science & Business Media, 1998.
A. Wills and T. Schön. Stochastic quasi-newton with line-search regularization. arXiv preprint arXiv:190,.01238, 2019.
Z. Xu, A. M. Dai, J. Kemp, and L. Metz. Learning adaptive learning rate schedule. arXiv preprint 199.0129,, 2019.
M. D. Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:/212.5701, 2012.
Z. Zhang. Derivation of backpropagation in convolutional neural network (cnn). University of Tennessee, Knoxville, TN, 2016.
C. Zhou, W. Gao, and Goldfarb. Stochastic adaptive quasi- newtonn methods for minimizing expected values. In International Conference Machine Learning, pages 4150-4159, 2017.
10

---

Under review as a conference paper at ICLR 2021
A PARAMETERS FOR THE POLICY TO CALCULATE cos OR
In Section 4, was stated that the policy x(;0) is able to calculate
d
cos OR
v d degk gk
In the simplest case, this can be done by choosing Wo1. bus, Waz and buz adequate to make
u1 = in(-dd 9+) I.5 d+) 0.5 In(g gs) and u2 = 1. The optimization problem becomes
Tk arg min In cos (co ox)) TER
(10)
s.t. Tm
TM
(11)
Recalling that cos 6h > 0 as - di 9k > 0, letting TM = 0 and Tm small enough assures that l e² Φk. It is important say that this specific step size might not be good one, but this quantity can carry useful information when composing vectors and as it characterizes the deviation between the update direction dę and the steepest descend direction.
B SAMPLES OF TEST TASKS
In this appendix provide in Fig. the objective function (axx obtained in our tests for the 10 first tasks in Trest Differently form the results in Fig. 3 that were chosen by inspection, the plots in Figure should represent more uniform visualization of the policy behavior in this set.
C FINAL VALUE ANALYSIS
Here we present the results regarding the index (f) defined in (8) but in the case where one chooses to define f! based on the final value f(æx) obtained after applying the algorithm for K 800 iterations. The box plot in Fig. is reconstructed and presented in Fig. 8. The conclusion drawn from this analysis is the same as the one obtained in the former definition of f!al based on the minimum value f(xx) for all k. However, in the context of deterministic nonlinear optimization is a good idea to keep the best visited iterate so far and allow the algorithm to explore other areas of the decision space. This is the reasoning that motivates considering the minimum over the iterations in the main analysis.

---

Under review as a conference paper at ICLR 2021
Flab 1bok #40
#4
10
20-3
Tosk
10
10""
Flas).
10
10
0.0
0.0
0.5
06
10
01.0
01.2
0..3
10.6
0.0
0.22
0.4
0.6
time
time
time
10/
F(za), Task: 443
10-1
1017
F(20) Tisk
10
f(zx).
101
10/
100
10
10
0.0
0.2
0.4
0.G
10
36
0.0
0.2
0.G
31F
f(rs): Thok 866
((a3). TTak #47
11/
10
F(ar.) 1hak
10
10
10
10-5
6.0
0.9
0/4
0.0
0.0
600
21F
2.0
45
10
10-1
10
timee
timoe
time
30
(2ng)
30
-2
11.0
timo
Figure 7: Objective function f f(x)) at the current iterate with respect to wall-clock time for all optimizers and the 10 first tasks in Trest (top) and correspondent step sizes for T(.0) and BTLS (bottom).
12

---

Under review as a conference paper at ICLR 2021
Re) (1,20)
(io, n.) (2,400)
(R0, R4) (4,800)
30
70
10
101
H
-30
-10
-10
-20
-21
221
ADAM
L-OROS
ADAM
BMSpprog
L-FFOS
LBPGS-BT
ADAM
L-OFOS
L BUU
Figure 8: Box plots for values of Zo. where a is the algorithm in the x-axis, encountered for all tasks in Trast by different algorithms and pairs (ni, na) (for f(ol defined as the final value f(xx) obtained by algorithm a).
13


Reviews:
[{"review_id": "YV1hoR5RwuU", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. Paper contribution summary\n    This paper proposes a neural network architecture with local gradient/quasi gradient as input to learn the step-length policy from data for the specific L-BFGS algorithm. The step-length policy can be learned from the data with similar optimization problems. The designed neural network architecture for step-length policy is using inner product as input to allow the policy to be independent of the problem size, and the logarithm operation is used to enable products and division among powers of the inputs.\n    The numerical example shows that the learned policy is comparable to L-BFGS with backtracking line search and potentially better generalization to the same problem with higher dimension.\n\n2.  Strong and weak points of the paper\n    Strong part: 1) The network architecture for the step-length policy is very interesting, and may be useful for other problems with similar needs. 2) The numerical experiment shows that the learned step-length policy is comparable to the same algorithm with backtracking line searched step-length, which looks promising.\n    Week part: 1) The paper's goal is limited to design the step-length policy for one specific optimization algorithm L-BFGS. If the original L-BFGS algorithm is not effective for certain problems, then the learned step-length policy may be un-useful as well. 2) Based on Table 1's metric of gradient length smaller than \\epsilon, the learned step-length policy seems not better than Adam even in the higher dimension setup in which the Adam is using the pre-tuned learning rate. The result in Fig. 5 seems better for the learned step-length policy, but that uses the smallest value over training as compared value, which may not be 'fair'. Because Adam, RMSProp, and L-BFGS are different algorithms, which may result in different (local) minima in different problems. 3) The performance test setup is not realistic in neural network model training. The setup for training the model (inner optimization) along with step-length policy model (outer optimization) is with a fixed 1000 images (randomly selected from 60 batches of data) over T = 16 outer iteration and 10 epochs. When comparing different algorithms' performance, it also splits the test dataset into 10 batches of 1000 images. And record performance for each fixed batch dataset, which is not realistic in usual neural network model training. Usually, different batches are randomly selected at each time step as opposed to being fixed. This makes the claimed performance doubtful in realistic setting.\n\n3. Based on the strong and week points of this paper, I tend to reject it.\n\n4. Supporting arguments\n    Please refer to my above arguments.\n\n5. Questions\n    1) Can author/s please explain the first two plots in Figure 4? It seems to me that both Adam and RMSProp are faster reaching to the required gradient norm precision. The precision seems different between these two algorithms and the author/s claimed one. Why not use the same precision requirement so that we can compare these algorithms directly?\n    2) Can author/s provide any feedback on the third point of \"weak part\"?\n    3) Can this proposed architecture/idea be used as a step-size policy for not just L-BFGS?", "extra_scores": {}, "extra_text": {}}, {"review_id": "Be-ZHgVcbfw", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies a problem of learning step-size policy for L-BFGS algorithm. This paper falls into a general category of meta-learning algorithms that try to derive a data-driven approach to learn one of the parameters of the learning algorithm. In this case, it is the learning rate of L-BFGS. The paper is very similar in nature to the papers of Ravi & Larochelle, MAML and Andrychowicz.\n\nMy biggest issue with this paper is the evaluation. The paper itself cites in the introduction:  \"...for large-dimensional problems this procedure is likely to become the bottle-neck of the optimization task\". However, the paper doesn't not provide necessary evaluation to help resolving this issue. In fact, it is not very clear at all that the proposed method would work on a more general scenario when the evaluation dataset is wildly different from training the dataset.\n\nI'm also a bit surprised to see this paper tackling specifically L-BFGS algorithm, instead of more general case of learning rate parameter for any gradient basis algorithm. I would be curious to learn what is so special about L-BFGS that made the authors chose it. After all, the paper (and L-BFGS) deals with deterministic optimization problems on a full batch which limits the applicability of the paper.\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "AXMyWDcep89", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**:\n\nThe paper presents a novel steps-size adaptation for the L-BFGS algorithm inspired by the learning-to-learn idea. The step-size policy is determined by two linear layers which compare a higher dimensional mapping of curvature information which is trained to adapt the step-size.\n\n\n**Reasons for score**:\n\nWhile the idea of learning to predict a suitable step-size is intriguing and is definitely worth pursuing I am not convinced that the proposed algorithm results in an active policy that usefully adapts the step-size. There are too many concerns that I think needs to be addressed and it is not clear if the speed up improves over the reliability of a line search. I therefore vote to reject the paper in its current form.\n\n**Pros**:\n\n- It is clear that a lot of thought has gone into the project to come up with the policy. I think it might have merit but requires additional tests.\n\n- The figures were first difficult to understand but once the content had been explained in the text the benefit of the chosen presentation became clear.\n\n\n\n**Concerns**:\n\n- My main concern is best visualized in Figure 3. Both the learned policy and the BTLS seem to mostly favour a step of 1 which raises several questions. \n\t1) What would be the results of using no adaptation and rely on a step of 1 (or 0.9) constantly as a baseline? \n\t2) The $\\pi$-algorithm mostly uses a step-size of 1 which happens top be the upper boundary $\\tau_M$, which means it is not clear if the policy network has learned that 1 is a good step or if it has not learned at all and the results are just due to the clipping. What would happen if $\\tau_M>1$ for example? \n\t3) Given that both the BTLS and $\\pi$ mostly use $t_k=1$, is there any intuitive explanation as to why the results between the two algorithms differ by so much in figure 3? Are there additional figures where the BTLS similarly outperforms the competition (it did reach $10^{-5}$ first in ~60% of the tasks according to table 1, column 1)? This despite the fact that BTLS is at least 1 forward pass more expensive per iteration than the policy (for a fully connected network I think that is ~50% of the iteration cost).\n\n- The benefit of using the double network for the policy is not clear to me. What would be the result of using a single linear layer instead or a recurrent network that monitors temporal changes to the curvature information that is used as input?\n\n- Given that the input and output dimensionality of the policy network is of low dimension it would be interesting to see what the weights and biases look like for respective policy layers. By looking at the weights it would be possible to see what curvature information makes the network decide to adjust the step-length. Does the policy learn a cosine behaviour similar to the proposed possibility in the appendix? \n\n- Could the policy be used for another optimization algorithm for which $-g_t^\\intercal d_k>0$, such as RMSprop or GD? It might be easier to understand the influence of the policy in such a setting.\n\n\nComparably minor points:\n\n- Section 3 first paragraph ends with a statement regarding $\\rho_i$ to keep the Hessian approximation s.p.d by ignoring iterations. Is this used in the implementation? \n- Table 1: According to what metric is \"first\" defined (iteration, time, function evaluations)? It would be good to mention the average value of this metric for each optimizer at the end of $K=800$ inner steps.\n- In Section 6 it says that the learning rate of Adam and RMSprop was chosen to yield fast convergence. I understand this as quickly reaching a threshold, not achieving best performance for allocated time. Is this correct? That could help explain why so many settings in table 1 and figure 4 fail to converge. Personally I think the first-order algorithms should be allowed to change the learning rate between problems to prevent them from not converging (ex. RMSprop).\n- Eq.7 s.t. -> with, or is the outer problem actually constrained and I missed something?\n\n\n-------------------\n\n\n**Post Rebuttal**\n\nI have considered the revised article, additional reviews and rebuttal and decided to slightly raise my score but I am still in favor of rejecting the paper. Below is a summary of my reasoning.\n\n--------\n\nThe authors have provided a good rebuttal and I am overall pleased with the detailed response, additional experiments and figures, and overall exhibited transparency. \nUnfortunately my assumption about $t_k$ seemed correct when considering the additional L-BFGS-B results, which indicate that using standard $t_k=1$ is a really strong baseline that proved difficult to beat.\n\nI would suggest finding another set of problems where $t_k=1$ is not so good for L-BFGS or consider adapting another first-order algorithm for which it is clear that the step-length needs to change between tasks and architectures.", "extra_scores": {}, "extra_text": {}}, {"review_id": "AHmhtI6uF2Q", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "** Description\n\nThis paper makes two separate contributions to machine learning: (1) It proposes a neural network to learn the step-size policy of L-BFGS and (2) it uses the algorithm for training a deep neural network.\n\n** Pros\n\nIn practical terms it provides a solution to the problem of finding an appropriate step-size policy.  It also demonstrates that the policy out performs ADAM and RMSprop on a rather strangely chosen problem.\n\n** Cons\n\nLearning how to optimise by examining related problems is often a fruitless endeavour because small changes in a problem can drastically change the behaviour of optimisation algorithms.  It would have been nice to see more convincing evidence that learning a step-size polices generalises across at few more problem domains.\n\nFor me, the evaluation of their algorithm for training a neural network was slightly unconvincing.  Possibly this was just chosen as an example optimisation problem and the application shouldn't be taken too seriously (a comment to that effect would be useful).  Obviously the use of full-batch training is not that practical for most problems.  For neural network training, robustness to the noise produced by mini-batch training is important to understand.  Although ADAM and RMSProp are state-of-the-art optimisers for minibatch training when compared on full-batch it would be useful to compare with other standard approaches such as conjugate gradient, etc.  The choice of problem was puzzling.  Clearly an MLP on MNIST is not representative of modern machine learning.  It left the question of whether a small network was deliberately chosen because the algorithm does not scale to really large networks.  Again a comment about this would have strengthened the paper.\n\nI was left with the impression that the authors were being slightly selective in their choice of problems for showing the utility of their method.  I would have liked to see more conclusive evidence in this direction and a clearer discussion of the regime where this method is likely to perform well.\n\n** Typos\n\nThe paper is comprehensible, but would gain from being proof read by a native speaker.  This is not a consideration that affects the valuation.  Examples of rewordings that would make the paper flow slightly better are\n\np2 l4: good <-- well\np2 l8: exempts <-- frees\np2 l23: seek <-- search\np3 section 3 paragraph 2 line 2: it does ... <-- there does ...\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a novel procedure to set the steps-size for the L-BFGS algorithm using a neural network.
Overall, the reviewers found the paper interesting and the main idea well-thought. However, a baseline that was proposed by one of the reviewers seems to be basically on par with the performance of the proposed algorithm, at least in the experiments of the paper. For this reason, it is difficult to understand if the new procedure has merit or not. Also, the reviewers would have liked to see the same approach applied to different optimization algorithms.

For the reasons, the paper cannot be accepted in the current form. Yet, the idea might have potential, so I encourage the authors to take into account the reviewers' comments and resubmit the paper to another venue.
Decision: Reject