Why .eval() is missing from this official Pytorch tutorial?

username: John_Deterious
cooked: <p>Here’s a DCGAN tutotial<br>
<a href="https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html</a></p>
<p>When evaluating the performance, .eval() and .train() is completely ignored, no mention of them in the entire code. Why is that? Thanks.</p>
<p>EDIT: here, I can point to the exact line where I think this is missing<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/tutorials/blob/master/beginner_source/dcgan_faces_tutorial.py#L646" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/tutorials/blob/master/beginner_source/dcgan_faces_tutorial.py#L646" target="_blank" rel="nofollow noopener">pytorch/tutorials/blob/master/beginner_source/dcgan_faces_tutorial.py#L646</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="636" style="counter-reset: li-counter 635 ;">
<li>        # Output training stats</li>
<li>        if i % 50 == 0:</li>
<li>            print('[%d/%d][%d/%d]\tLoss_D: %.4f\tLoss_G: %.4f\tD(x): %.4f\tD(G(z)): %.4f / %.4f'</li>
<li>                  % (epoch, num_epochs, i, len(dataloader),</li>
<li>                     errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))</li>
<li>        </li>
<li>        # Save Losses for plotting later</li>
<li>        G_losses.append(errG.item())</li>
<li>        D_losses.append(errD.item())</li>
<li>        </li>
<li class="selected">        # Check how the generator is doing by saving G's output on fixed_noise</li>
<li>        if (iters % 500 == 0) or ((epoch == num_epochs-1) and (i == len(dataloader)-1)):</li>
<li>            with torch.no_grad():</li>
<li>                fake = netG(fixed_noise).detach().cpu()</li>
<li>            img_list.append(vutils.make_grid(fake, padding=2, normalize=True))</li>
<li>            </li>
<li>        iters += 1</li>
<li>
</li>
<li>
</li>
<li>######################################################################</li>
<li># Results</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

username: Balamurali_M
cooked: <p>I have also noticed this. Even while I code, even if I miss  eval and train. It didn’t have any effect on the result.</p>
<p>But the thing is how we are going about it does matter. We use eval because we won’t be interested in updating the weight of the network. While train weight updation happens. If we can turn off the gradients using torch no grad like your example, be it. It’s another way of approaching I think.</p>

username: Swarchal
cooked: <aside class="quote no-group" data-username="Balamurali_M" data-post="2" data-topic="51539">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/balamurali_m/48/12574_2.png" class="avatar"> Balamurali_M:</div>
<blockquote>
<p>We use eval because we won’t be interested in updating the weight of the network.</p>
</blockquote>
</aside>
<p>My understanding is that <code>.eval()</code> is to tell the network to disable dropout and batchnorm layers, where as the <code>torch.no_grad()</code> context is to disable gradient calculations. They are different concepts which happen to be used together during inference.</p>
<p>One possibility why <code>.eval()</code> is missing is because there are no dropout or batchnorm layers?</p>

username: Balamurali_M
cooked: <p>Dropout, batchnorm also seems like a proper answer. Can you let us know about that ?</p>

username: John_Deterious
cooked: <p>There are. In both generator and discriminator.</p>

username: John_Deterious
cooked: <p>It does have an effect. If I take 1000 examples from the generator, the images get whiter and whiter. To fix the performance I go to eval() mode.</p>