Multiple return variables in LBFGS closure

robinho: <p>A typical LBFGS closure:</p>
<pre><code class="lang-auto">for input, target in dataset:
    def closure():
        optimizer.zero_grad()
        output = model(input)
        loss = loss_fn(output, target)
        loss.backward()
        return loss
    optimizer.step(closure)
</code></pre>
<p>But what if I need to return multiple variables in closure(), because I need to store and plot the loss and other variables etc out of the closure loop. However, I’m not allowed to write the following:</p>
<pre><code class="lang-auto">for input, target in dataset:
    def closure():
        optimizer.zero_grad()
        output = model(input)
        loss = loss_fn(output, target)
        loss.backward()
        u=loss+1
        return loss, u
    loss, u=closure
    optimizer.step(closure)
</code></pre>

KFrank: <p>Hi Robin!</p>
<aside class="quote no-group" data-username="robinho" data-post="1" data-topic="190381" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/c6cbf5/48.png" class="avatar"> robinho:</div>
<blockquote>
<p>I need to store and plot the loss and other variables etc out of the closure loop. However, I’m not allowed to write the following:</p>
<pre><code class="lang-auto">for input, target in dataset:
    def closure():
        optimizer.zero_grad()
        output = model(input)
        loss = loss_fn(output, target)
        loss.backward()
        u=loss+1
        return loss, u
    loss, u=closure
    optimizer.step(closure)
</code></pre>
</blockquote>
</aside>
<p>Define some empty lists:</p>
<pre data-code-wrap="python"><code class="lang-python">myLosses = []
myUs = []
</code></pre>
<p>and have your <code>closure</code> store the results you want to plot in them for future use:</p>
<pre data-code-wrap="python"><code class="lang-python">    def closure():
        optimizer.zero_grad()
        output = model(input)
        loss = loss_fn(output, target)
        loss.backward()
        u=loss+1
        myLosses.append (loss)
        myUs.append (u)
        return loss,
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

robinho: <p>Hi KFrank, many thanks. But I have to plot a lot of figures based on those variables, so it’ll be nicer to return them all and get back to them out of the loop. Is that impossible?</p>

KFrank: <p>Hi Robin!</p>
<aside class="quote no-group" data-username="robinho" data-post="3" data-topic="190381" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/c6cbf5/48.png" class="avatar"> robinho:</div>
<blockquote>
<p>I have to plot a lot of figures based on those variables, so it’ll be nicer to return them all and get back to them out of the loop.</p>
</blockquote>
</aside>
<p>I didn’t make it explicit, but <code>myLosses</code> and <code>myUs</code> are meant to be defined<br>
initially outside of the loop, so the values you store in them will persist<br>
after the loop exits.</p>
<p>As a refinement to what I posted above, you should probably use<br>
<code>myLosses.append (loss.item())</code> and <code>myUs.append (u.item())</code><br>
(or similarly use <code>.detach()</code>) so that you don’t keep around references<br>
to the computation graph.</p>
<p>Best.</p>
<p>K. Frank</p>

robinho: <p>Hi Frank,<br>
it seems that <code>myLosses</code> and <code>myUs</code> are meant to be defined<br>
initially outside of the loop, but loss needs to be re-defined in the loop as well. In other words, the loss needs to be defined/given a value twice (in and out of the loop)? Is there any way to circumvent this redundancy?</p>

KFrank: <p>Hi Robin!</p>
<aside class="quote no-group" data-username="robinho" data-post="5" data-topic="190381" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/c6cbf5/48.png" class="avatar"> robinho:</div>
<blockquote>
<p>In other words, the loss needs to be defined/given a value twice (in and out of the loop)?</p>
</blockquote>
</aside>
<p>I don’t understand what you are asking.</p>
<p>Here’s how you can do what I think you want to do:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print (torch.__version__)
2.1.0
&gt;&gt;&gt;
&gt;&gt;&gt; # outside of loop
&gt;&gt;&gt; myLosses = []
&gt;&gt;&gt; myUs = []
&gt;&gt;&gt;
&gt;&gt;&gt; for  i in range (1, 11):
...     loss = i * torch.tensor ([10.0])
...     u = loss + 1
...     # inside of loop
...     myLosses.append (loss.item())
...     myUs.append (u.item())
...
&gt;&gt;&gt; # outside of loop
&gt;&gt;&gt; myLosses
[10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]
&gt;&gt;&gt; myUs
[11.0, 21.0, 31.0, 41.0, 51.0, 61.0, 71.0, 81.0, 91.0, 101.0]
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

robinho: <p>Hi Frank,<br>
thanks so much. However you used a normal loop and got things right. But when this loop is a LBFGS closure, the loss (defined in the closure loop) isn’t recognized to be a defined variable unless it’s redefined out of the closure loop as far as I found.</p>

KFrank: <p>Hi Robin!</p>
<aside class="quote no-group" data-username="robinho" data-post="7" data-topic="190381" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/c6cbf5/48.png" class="avatar"> robinho:</div>
<blockquote>
<p>you used a normal loop and got things right. But when this loop is a LBFGS closure, the loss (defined in the closure loop) isn’t recognized to be a defined variable unless it’s redefined out of the closure loop as far as I found.</p>
</blockquote>
</aside>
<p>Please post a super-simplified, minimal, fully-self-contained, runnable script<br>
that reproduces your issue, together with the output you get when you run it.</p>
<p>Best.</p>
<p>K. Frank</p>

J_Johnson: <p>You might try using a class, as that contains a <code>self</code> reference in which you can call variables without needing to return them or pass them into the function. This seems to work:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn


class Trainer:
    def __init__(self):
        self.model = nn.Linear(10, 1)
        self.optimizer = torch.optim.SGD(self.model.parameters(), lr=0.1)
        self.loss_fn = nn.L1Loss()

        self.dataset = [(torch.rand(10, 10), torch.rand(10, 1)), (torch.rand(10, 10), torch.rand(10, 1))]

        self.loss_list = []
        self.u_list = []
        self.u = 0

    def train_loop(self):
        for input, target in self.dataset:
            self.input, self.target = input, target
            self.optimizer.step(self.closure)

    def closure(self):
        self.optimizer.zero_grad()
        output = self.model(self.input)
        loss = self.loss_fn(output, self.target)
        loss.backward()
        self.u+=1
        self.loss_list.append(loss.item())
        self.u_list.append(self.u)
        return loss

train = Trainer()
train.train_loop()

print(train.loss_list)
print(train.u_list)
</code></pre>

robinho: <p>Hi J_Johnson, many thanks for your help! But if</p>
<pre><code class="lang-auto">        self.u+=1
        self.loss_list.append(loss.item())
        self.u_list.append(self.u)
</code></pre>
<p>are put in train_loop like:</p>
<pre><code class="lang-auto">def train_loop(self):
        for input, target in self.dataset:
            self.input, self.target = input, target
            self.optimizer.step(self.closure)
            self.u+=1
            self.loss_list.append(loss.item())
            self.u_list.append(self.u)
</code></pre>
<p>loss isn’t defined again! Why I need to put these lines out of closure is because LBFGS calculates loss for quite a few times in each epoch but I only want to store the final loss in each epoch.</p>

J_Johnson: <p>In your original example, the definition “closure” is NOT being run until optimizer.step(closure). It is only being <em>defined</em> again and again. A definition does not run until it is actually called.</p>
<aside class="quote no-group" data-username="robinho" data-post="1" data-topic="190381">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/c6cbf5/48.png" class="avatar"> robinho:</div>
<blockquote>
<pre><code class="lang-auto">for input, target in dataset:
    def closure():
        optimizer.zero_grad()
        output = model(input)
        loss = loss_fn(output, target)
        loss.backward()
        u=loss+1
        return loss, u
    loss, u=closure
    optimizer.step(closure)
</code></pre>
</blockquote>
</aside>
<p>At this point, I’m unclear on what your objective is. If not the above, can you please update what you are trying to do.</p>

robinho: <p>oh sorry, then in my original example loss, u=closure is meant to follow optimizer.step(closure) rather than the other way round. I just need to get loss etc after closure is implemented.</p>
<pre><code class="lang-auto">for input, target in dataset:
    def closure():
        optimizer.zero_grad()
        output = model(input)
        loss = loss_fn(output, target)
        loss.backward()
        u=loss+1
        return loss, u
    optimizer.step(closure)
    loss, u=closure
</code></pre>

J_Johnson: <p>Did you try running the example I provided? Because it creates a list of the losses that occurred inside of the closure at each step. If you want to add those up over each epoch, you can add the code to do so between epochs. Or make a definition in the class to reset those values:</p>
<pre><code class="lang-auto">def reset_stats(self):
    self.ulist = []
    self.u = 0

</code></pre>
<p>And just call that where appropriate.</p>

robinho: <p>Yes I did. I suddenly got it now. The essence is that one has to define what we want to carry out of closure as self.*!</p>