Opinion: .eval() should be a context manager

username: hughperkins
cooked: <pre><code> with mymodel.eval():
     eval_res = mymodel(test_data)</code></pre>

username: ptrblck
cooked: <p>What is your opinion on this?<br>
Do you run into errors or annoying situations without the context manager?</p>
<p>I my opinion the less context managers the better. Especially, if you are using PEP8! <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=5" title=":wink:" class="emoji" alt=":wink:"></p>
<p>Whike the <code>torch.no_grad()</code> context manager makes totally sense, since the gradient calculation was previously a property of the data (<code>Variable(..., volatile=False)</code>) and is now a “property” of the work flow, the eval property belongs to the model in my opinion, since it changes the behavior of some internal <code>Modules</code>. Using this context manager, it might also be more complicated to set some layers into eval, while others stay in train.</p>
<p>Would love to hear other opinions on this!</p>

username: Christoph_Heindl
cooked: <p>Switching correctly train()/eval() during training/validation is tricky in the presence of exceptions or other early exits. Once in the wrong state your training might be totally off. Context managers help in such situations. I’m using the following context manager:</p>
<pre><code class="lang-auto"># Following snippet is licensed under MIT license

from contextlib import contextmanager

@contextmanager
def evaluating(net):
    '''Temporarily switch to evaluation mode.'''
    istrain = net.training
    try:
        net.eval()
        yield net
    finally:
        if istrain:
            net.train()
</code></pre>
<p>Intended usage: <code>with evaluating(net): ...</code>. Also in case you are concerned having too many nested blocks, you can collapse multiple context managers into a single line <code>with evaluating(net), torch.no_grad():</code> in py3.x and later. In 2.7 you need to use <code>nested</code> from contextlib.</p>

username: tom
cooked: <p>Clever, thank you for sharing!</p>

username: hughperkins
cooked: <p>Thank you <a class="mention" href="/u/christoph_heindl">@Christoph_Heindl</a> . Dont suppose… could you explicitly grant a license to use the code, eg MIT, (or something else that is relatively free, ie not GPLv3 or similar).</p>

username: Christoph_Heindl
cooked: <p><a class="mention" href="/u/hughperkins">@hughperkins</a> modified above snippet to contain license info.</p>
<p>One thing that should be clarified by someone who has internal insights is if <code>net.training</code> is always a primitive boolean and not something complex. Otherwise remembering the training state before switching to eval would require an explicit copy.</p>

username: hughperkins
cooked: <blockquote>
<p><a class="mention" href="/u/hughperkins">@hughperkins</a> modified above snippet to contain license info.</p>
</blockquote>
<p>Great! Thanks! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: jastern33
cooked: <p>With the current system, it’s also easy to make the error of calling model.eval() at the start of the validation loop, and model.train() at the end of the eval() loop, which can change model.training unexpectedly if the model was in eval mode when starting the validation loop. The correct way is to check the state of the model before eval mode and restore it after, and this context manager can help avoid this error.</p>
<p>Another thing: if eval() is a context manager, there should be a train context manager as well. The use case would be ensuring model state for a certain block of code, and restoring the model to its original state after finishing that block.</p>
<p>Something else to consider: the name of this context manager (and other, related context managers) should be carefully evaluated - for example, it would be easy to think that <code>with torch.inference_mode()</code> would put the model in eval() mode, when it doesn’t.</p>

username: Yann_POURCENOUX
cooked: <p>I agree with <a class="mention" href="/u/jastern33">@jastern33</a>, I landed here after being confused about <code>with torch.inference_mode()</code> not setting the model in eval mode.</p>