Copy.deepcopy() does not seem to work

Alexey1: <p>In the following code, I am using a Neural Network (<code>net</code> ) to minimize the expectation of a complex stochastic function (<code>complex_function</code> ).</p>
<pre><code class="lang-auto">loss = torch.tensor(0., requires_grad=True)    
params = list(net.parameters())
optimizer = torch.optim.SGD(params, lr=learning_rate)
nb_train = 100
nb_mean = 1000

for i in range(nb_train):
    for j in range(nb_mean):
        value = complex_function(net)
        loss = loss + value
    loss = loss / nb_mean        
    loss.backward(retain_graph=True)        
    optimizer.step()
    optimizer.zero_grad()

    if i==0:
        best_loss = loss
        best_net = copy.deepcopy(net)
        
    else:
        if loss &lt; best_loss:
            best_net = copy.deepcopy(net)
            best_loss = loss
return best_net
</code></pre>
<p>Weirdly, this code does not return the best net that I have encountered. Could someone explain me why? I suppose it is related to <code>copy.deepcopy(net)</code> but I do not know how…</p>

ptrblck: <p>How did you verify that <code>best_net</code> is not the last copied <code>net</code>? Did you print any parameters from <code>net</code> inside the <code>if loss &lt; best_loss</code> condition and compared it to the final <code>best_net</code> parameters?</p>

Alexey1: <p>Hello, and thank you for your reply.</p>
<p>I have compared the parameters of the nets using <code>state_dict()</code> and the nets are indeed the same. But… I don’t understand why the loss I get when using this net after executing this code, is lower than the one I got previously in the print. Does it still converge to a better net even if I do not ask him to do so ?</p>
<pre><code class="lang-auto">loss = torch.tensor(0., requires_grad=True)    
params = list(net.parameters())
optimizer = torch.optim.SGD(params, lr=learning_rate)
nb_train = 100
nb_mean = 1000

for i in range(nb_train):
    for j in range(nb_mean):
        value = complex_function(net)
        loss = loss + value
    loss = loss / nb_mean        
    loss.backward(retain_graph=True)        
    optimizer.step()
    optimizer.zero_grad()

    if i==0:
        best_loss = loss
        best_net = copy.deepcopy(net)
        
    else:
        if loss &lt; best_loss:
            best_net = copy.deepcopy(net)
            best_loss = loss
            print(best_loss) #here !
return best_net
</code></pre>
<p>Thank you again for your time.</p>

ptrblck: <p>It depends what you are doing afterwards and how you are calculating the loss.<br>
So currently it seems the best model is indeed saved and <code>deepcopy</code> works as expected. However, the loss calculation seems to be “better” using the stored <code>best_net</code> compared to the original <code>net</code> which was used as the reference to create the <code>best_net</code>.<br>
Could you explain how you are comparing the losses? I.e. are you using the same data loader, are you calling <code>model.eval()</code> on both runs, are you disabling random data augmentation, etc.</p>

Alexey1: <p>To compute the loss, I have to randomly generate many time series and to do the mean. But the difference I described above is systematically positive, and so big, that it is practically impossible that this be due to noise only.<br>
Besides, I am not dealing with images, so I don’t think I am concerned by data augmentation.</p>
<p>Moreover, I was not using <code>model.eval()</code> since I don’t use BatchNorm or dropout. So I have just added it at the beginning of my code, and for the evaluation part I have added</p>
<pre><code class="lang-auto">with torch.no_grad():
    net.eval()
</code></pre>
<p>before evaluating the result.</p>
<p>But again there is a big difference. What do you think ?</p>

ptrblck: <p>You’ve already verified that the <code>state_dict</code>s are equal, so the input data would be the next part to check.<br>
I would check the data next and start with a simple pre-defined tensor, such as <code>torch.ones</code>, store the results, and compare them. If both models return the same result, your data generation might be wrong.</p>