Loss error when using a list for ensemble of network

weiguowilliam: <p>I just started to use PyTorch and I plan to build an ensemble of networks. I implemented it with a list. But I got the following error:<br>
I’m confused by the error. The network has just 2 fc layers. Could you please help me explain that? Thanks in advance.</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "test_list.py", line 107, in &lt;module&gt;
    list_of_loss[i].backward()
  File "/home/weiguo/anaconda3/envs/pomdp/lib/python3.6/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/weiguo/anaconda3/envs/pomdp/lib/python3.6/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

</code></pre>
<p>Here’s my code:</p>
<pre><code class="lang-auto">num_model = 3

list_of_model = []
list_of_cri = []
list_of_opt = []

for _ in range(num_model):
    tem_model = TestNet(input_size, hidden_size, num_classes).to(device)
    tem_cri = nn.CrossEntropyLoss()
    list_of_model.append(tem_model)
    list_of_cri.append(tem_cri)

for i in range(num_model):
    list_of_opt.append(torch.optim.Adam(list_of_model[i].parameters(), lr=learning_rate))

list_of_loss = []


total_step = len(train_loader)
for epoch in range(num_epochs):
    for j, (images, labels) in enumerate(train_loader):
        for i in range(num_model):
            # print(i)
            images = images.reshape(-1, 28*28).to(device)
            labels = labels.to(device)

            output = list_of_model[i](images)
            loss = list_of_cri[i](output, labels)
            list_of_loss.append(loss)

            list_of_opt[i].zero_grad()
            list_of_loss[i].backward()
            # loss.backward()
            list_of_opt[i].step()

            if (j+1) % 100 == 0:
                _, predicted = torch.max(output.data, 1)
                correct = (predicted == labels).sum().item()
                if i == 0:
                    print("########")
                print(correct/labels.size(0))
</code></pre>
<p>However, if I comment <code>list_of_loss.append(loss)</code> and <code>list_of_loss[i].backward()</code>, but use <code>loss.backward()</code> straightforwardly in each loop, there’s no error.</p>

KFrank: <p>Hi Wei!</p>
<aside class="quote no-group quote-modified" data-username="weiguowilliam" data-post="1" data-topic="120497" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/weiguowilliam/48/37987_2.png" class="avatar"> weiguowilliam:</div>
<blockquote>
<pre><code class="lang-auto">list_of_loss = []
...
for epoch in range(num_epochs):
    for j, (images, labels) in enumerate(train_loader):
        for i in range(num_model):
            ...
            list_of_loss.append(loss)
            ...
            list_of_loss[i].backward()
</code></pre>
<p>However, if I comment <code>list_of_loss.append(loss)</code> and <code>list_of_loss[i].backward()</code>, but use <code>loss.backward()</code> straightforwardly in each loop, there’s no error.</p>
</blockquote>
</aside>
<p>Your <code>list_of_loss</code> keeps growing with every iteration of the loops<br>
over <code>num_epochs</code> and <code>train_loader</code>.</p>
<p>Therefore you keep calling <code>list_of_loss[0].backward()</code> on the<br>
same loss over and over again that was created for the first sample<br>
in your first epoch.</p>
<p>Put another way, the newly appended <code>loss</code> could be accessed and<br>
backpropagated as:</p>
<pre data-code-wrap="python"><code class="lang-python">list_of_loss[-1].backward()
</code></pre>
<p>which is not the same as:</p>
<pre data-code-wrap="python"><code class="lang-python">list_of_loss[i].backward()
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

weiguowilliam: <p>Thank you, Frank!</p>
<p>I now use</p>
<pre><code class="lang-auto">
list_of_loss = []
...
for epoch in range(num_epochs):
    list_of_loss = [0 for _ in range(num_model)]

    for j, (images, labels) in enumerate(train_loader):
        for i in range(num_model):
            ...
            list_of_loss[i] = list_of_cri[i](output, labels)

            ...
            list_of_loss[i].backward()

</code></pre>
<p>To solve this.  Based on the numerical experiment, this seems to work. Is this correct?</p>
<p>My core question is: can we put all these terms into a list, use a list of optimizer/loss, update list_of_optimizer[i], backward the list_of_loss[i]? I wrote this demo code to numerically prove that.</p>

KFrank: <p>Hi Wei!</p>
<aside class="quote no-group" data-username="weiguowilliam" data-post="3" data-topic="120497" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/weiguowilliam/48/37987_2.png" class="avatar"> weiguowilliam:</div>
<blockquote>
<p>My core question is: can we put all these terms into a list, use a list of optimizer/loss, update list_of_optimizer[i], backward the list_of_loss[i]?</p>
</blockquote>
</aside>
<p>Yes, absolutely this will work.  It’s a perfectly reasonable approach<br>
to take.  Of course, as you saw with the issue in your first version,<br>
you have to do it correctly, but that’s true with any code.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">list_of_loss = []
...
for epoch in range(num_epochs):
    list_of_loss = [0 for _ in range(num_model)]

    for j, (images, labels) in enumerate(train_loader):
        for i in range(num_model):
            ...
            list_of_loss[i] = list_of_cri[i](output, labels)
            ...
            list_of_loss[i].backward()
</code></pre>
</blockquote>
</aside>
<p>A few comments, mostly stylistic:</p>
<p>Your first <code>list_of_loss = []</code> has no effect because you subsequently<br>
overwrite it with <code>list_of_loss = [0 for _ in range(num_model)]</code><br>
before you ever use it.</p>
<p>I would create the list of length <code>num_model</code> once, outside of all of<br>
the loops.  Stylistically, I would initialize the elements of the list with<br>
<code>None</code> rather than <code>0</code>.  (These initial values are never used, so which<br>
initialization you use has no practical effect.)  I would also not use<br>
a list comprehension merely to create a list of a given length.  Thus,<br>
I would probably do something like this:</p>
<pre data-code-wrap="python"><code class="lang-python">list_of_loss = [None] * num_model
only_need_one_cri = nn.CrossEntropyLoss()
...
for epoch in range(num_epochs):
    for j, (images, labels) in enumerate(train_loader):
        for i in range(num_model):
            ...
            list_of_loss[i] = only_need_one_cri (output, labels)
            ...
            list_of_loss[i].backward()
</code></pre>
<p>Also, I’ve illustrated that because an instance of <code>CrossEntropyLoss</code><br>
contains no state (that depends on <code>i</code>, <code>list_of_model[i]</code>, or whether<br>
its been called before), we don’t need a <code>list_of_cri</code> that contains<br>
multiple instances of <code>CrossEntropyLoss</code> – a single instance suffices.</p>
<p>Best.</p>
<p>K. Frank</p>

weiguowilliam: <p>Thank you so much, Frank! That really helps me a lot.</p>