Try to understand how to use optim.LBFGS

PytorchLearning: <p>Hi all,</p>
<p>I want to use ‘optimiser = optim.LBFGS(model.parameters(), lr=1e-4)’ instead of ‘optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)’<br>
but I didn’t know how to introduce ‘def closure():’<br>
can someone please explain to me how to modify the following code to use optim.LBFGS</p>
<p>here is the code:</p>
<pre><code class="lang-auto">def train_model(model, trainDataset, valDataset, number_epochs):
  optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)

  criterion = nn.MSELoss().to(device)
  hist = dict(train=[], val=[])

  best_loss = 25.0
  best_m_wts = copy.deepcopy(model.state_dict())

  for epoch in range(1, number_epochs + 1):
    model = model.train()

    train_losses = []
    for sequence_true in trainDataset:
      optimizer.zero_grad()

      sequence_true = sequence_true.to(device)
      sequence_pred = model(sequence_true)

      loss = criterion(sequence_pred, sequence_true)

      loss.backward()
      optimizer.step()

      train_losses.append(loss.item())

    val_losses = []
    model = model.eval()
    with torch.no_grad():
      for sequence_true in valDataset:

        sequence_true = sequence_true.to(device)
        sequence_pred = model(sequence_true)

        loss = criterion(sequence_pred, sequence_true)
        val_losses.append(loss.item())

    train_loss = np.mean(train_losses)
    val_loss = np.mean(val_losses)

    hist['train'].append(train_loss)
    hist['val'].append(val_loss)

    if val_loss &lt; best_loss:
      best_loss = val_loss
      best_m_wts = copy.deepcopy(model.state_dict())

    print(f'Epoch {epoch}: train loss {train_loss} val loss {val_loss}')

  model.load_state_dict(best_m_wts)
  return model.eval(), hist
</code></pre>
<p>Thanks a lot for your help</p>

KFrank: <p>Hi Py!</p>
<aside class="quote no-group" data-username="PytorchLearning" data-post="1" data-topic="195602" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/d2c977/48.png" class="avatar"> PytorchLearning:</div>
<blockquote>
<p>can someone please explain to me how to modify the following code to use optim.LBFGS</p>
</blockquote>
</aside>
<p><code>LBFGS</code> is quite different in character than basically all of pytorch’s<br>
other optimizers (which are all more-or-less purely gradient-descent<br>
based).  So you can’t use <code>LBFGS</code> as a plug-and-play replacement for,<br>
say, <code>Adam</code>.</p>
<p>Rather than starting by rewriting some existing some code to use <code>LBFGS</code>,<br>
I would recommend getting <code>LBFGS</code> working with a toy computation.  The<br>
following links to an example that I haven’t tried running, but appears to<br>
contain the necessary ingredients: <a href="https://gist.github.com/tuelwer/0b52817e9b6251d940fd8e2921ec5e20" rel="noopener nofollow ugc">pytorch-L-BFGS-example.</a></p>
<p>If you still have questions, please post your attempt to use <code>LBFGS</code> on<br>
some toy problem (providing a fully-self-contained, runnable script)<br>
with specific questions about that attempt.</p>
<p>Good luck!</p>
<p>K. Frank</p>

PytorchLearning: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a>,</p>
<p>Thanks a lot for your help.<br>
I’ll try to understand the example to see how it works.</p>
<p>By the way, I’ve tried rewriting the code to use torch.optim.LBFGS by following the example. Could you please tell me if what I’ve written is correct? At least to know if I’ve written the code correctly.</p>
<pre><code class="lang-auto">def train_model(model, trainDataset, valDataset, number_epochs):
  optimizer = torch.optim.LBFGS(model.parameters(), lr=1e-4)
  # change the criterion
  criterion = nn.MSELoss().to(device)
  hist = dict(train=[], val=[])

  best_loss = 25.0
  best_m_wts = copy.deepcopy(model.state_dict())

  # Add closure function because of torch.optim.LBFGS
    def closure():
        optimizer.zero_grad()
        train_losses = []

        model.train()
        for sequence_true in train_dataset:
            sequence_true = sequence_true.to(device)
            sequence_pred = model(sequence_true)
            loss = criterion(sequence_pred, sequence_true)
            loss.backward()
            train_losses.append(loss.item())

        return np.mean(train_losses)
		
  for epoch in range(1, number_epochs + 1):
    model = model.train()
    train_loss = optimizer.step(closure)

    val_losses = []
    model = model.eval()
    with torch.no_grad():
      for sequence_true in valDataset:

        sequence_true = sequence_true.to(device)
        sequence_pred = model(sequence_true)

        loss = criterion(sequence_pred, sequence_true)
        val_losses.append(loss.item())

    val_loss = np.mean(val_losses)

    hist['train'].append(train_loss)
    hist['val'].append(val_loss)

    if val_loss &lt; best_loss:
      best_loss = val_loss
      best_m_wts = copy.deepcopy(model.state_dict())

    print(f'Epoch {epoch}: train loss {train_loss} val loss {val_loss}')

  model.load_state_dict(best_m_wts)
  return model.eval(), hist
</code></pre>
<p>On the other hand, I have a question about torch.optim.LBFGS. Do you have a link that explains the theory so that I can understand it a little more this optimiser? and understand what differentiates it from other optimisers?</p>
<p>Thanks a lot for your help</p>

KFrank: <p>Hi Py!</p>
<aside class="quote no-group" data-username="PytorchLearning" data-post="3" data-topic="195602" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/d2c977/48.png" class="avatar"> PytorchLearning:</div>
<blockquote>
<p>Could you please tell me if what I’ve written is correct?</p>
</blockquote>
</aside>
<p>Please post a super-simple, fully-self-contained, runnable script, together<br>
with its output, and ask a specific question if there is something you don’t<br>
understand or if something looks wrong.</p>
<aside class="quote no-group">
<blockquote>
<p>I have a question about torch.optim.LBFGS. Do you have a link that explains the theory so that I can understand it a little more this optimiser?</p>
</blockquote>
</aside>
<p>First, take a look at the <a href="https://www.cs.ubc.ca/~schmidtm/Software/minFunc.html" rel="noopener nofollow ugc">minFunc</a> link in pytorch’s <a href="https://pytorch.org/docs/2.1/generated/torch.optim.LBFGS.html" rel="noopener nofollow ugc">LBFGS documentation.</a><br>
You can also find a good overview in <a href="https://en.wikipedia.org/wiki/Limited-memory_BFGS" rel="noopener nofollow ugc">Wikipedia’s LBFGS entry.</a></p>
<aside class="quote no-group">
<blockquote>
<p>and understand what differentiates it from other optimisers?</p>
</blockquote>
</aside>
<p>Most pytorch optimizers use some form of gradient descent.  <code>LBFGS</code> also<br>
uses gradients to choose a so-called search direction, but then searches<br>
along that direction using the value of the loss function itself.</p>
<p>Best.</p>
<p>K. Frank</p>

PytorchLearning: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a>,</p>
<p>Thanks a lot for your answers and your help,<br>
I entered both codes because I wanted only to understand how worked:</p>
<pre><code class="lang-auto">def closure():
</code></pre>
<p>why and when to use this function ? And what do I need to put in this function? and why this optimizer use this function?<br>
I’ll now try to study a simpler example and try to understand this optimizer better and how to use closure().</p>
<p>Once again thank you for your help</p>

KFrank: <p>Hi Py!</p>
<aside class="quote no-group quote-modified" data-username="PytorchLearning" data-post="5" data-topic="195602" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/d2c977/48.png" class="avatar"> PytorchLearning:</div>
<blockquote>
<pre><code class="lang-auto">def closure():
</code></pre>
<p>…<br>
why this optimizer use this function?</p>
</blockquote>
</aside>
<p>The short story:  <code>closure()</code> basically packages the forward / backward<br>
pass of the training loop that you would use for a “conventional” pytorch<br>
optimizer such as <code>SGD</code>.  But why?</p>
<p>You are optimizing (specifically minimizing) your <code>loss</code> as a function of<br>
the parameters of your model.</p>
<p>With most of pytorch’s optimizers, it works like this:</p>
<p>Your code (by which I mean code that is not part of the optimizer) performs<br>
a forward and backward pass.  This populates the <code>.grad</code> properties of the<br>
model’s parameters with the gradient of the loss (for the current value of<br>
the model’s parameters).</p>
<p>You then call <code>optimizer.step()</code>.  <em>All</em> this does is to change the model’s<br>
parameters to some new values based on the gradient of the <code>loss</code>.<br>
(Not that it matters that much, but the <code>optimizer</code> doesn’t care about<br>
the value of <code>loss</code>, just its gradient.)</p>
<p>However, a <em>single step</em> of a <code>LBFGS</code> optimizer looks at the value of <code>loss</code><br>
(and its gradient) at multiple places in parameter space, and uses those<br>
values to decide where else to jump around in parameter space to look<br>
at more values of <code>loss</code>.</p>
<p><code>LBFGS</code> takes responsibility for causing <code>loss</code> and its gradient to be<br>
recomputed at each point in parameter space that it investigates, but<br>
it doesn’t actually know how to perform this recomputation.  So you<br>
tell it how to perform the recomputation by passing a function – named<br>
<code>closure()</code>, by convention – into the <code>optimizer.step (closure)</code><br>
call.  It is <code>closure()</code>'s responsibility to return the value of <code>loss</code> and<br>
populate the model’s <code>.grad</code> properties with the gradient (evaluated<br>
for whatever values the model’s parameters have when <code>closure()</code><br>
is called).  To be clear, <code>LBFGS</code> tells <code>closure()</code> what values of the model’s<br>
parameters to use when <code>closure()</code> recomputes <code>loss</code> and <code>.grad</code> by<br>
updating the model’s parameters before calling <code>closure()</code>.</p>
<p><code>closure()</code> normally does this by performing a single forward and backward<br>
pass based on the current state of the model, returning the <code>loss</code> that was<br>
computed by the forward pass and letting the backward pass populate<br>
the <code>.grad</code>s.  (<code>closure()</code> doesn’t <em>have</em> to perform a forward / backward<br>
pass – it can do whatever it wants as long as it returns some <code>loss</code> value<br>
and somehow populates the <code>.grad</code>s.)</p>
<p>Note, <code>closure()</code> will typically be called many times each time you call<br>
<code>optimizer.step()</code>.</p>
<p>So, unlike all the other pytorch optimizers (which are all basically<br>
single-step gradient-descent optimizers),  <code>LBFGS</code> takes much more<br>
responsibility about how to search through parameter space while<br>
optimizing <code>loss</code>.  (For what it’s worth, its a quasi-Newton optimization<br>
algorithm with line search.)</p>
<p>Best.</p>
<p>K. Frank</p>

PytorchLearning: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a>,</p>
<p>Thanks a lot for your very clear answer, now I have a clear understanding.</p>