When should one call .eval() and .train() when doing MAML with the PyTorch higher library?

username: Brando_Miranda
cooked: <p>I was going through the omniglot maml example and saw that they have <code>net.train()</code> at the <a href="https://github.com/facebookresearch/higher/blob/1e20cf9696054277b2d760f64835d5d74a3115a2/examples/maml-omniglot.py#L192" rel="noopener nofollow ugc">top of their testing code</a>. This seems like a mistake since that means the stats from each task at meta-testing is shared:</p>
<pre><code class="lang-auto">def test(db, net, device, epoch, log):
    # Crucially in our testing procedure here, we do *not* fine-tune
    # the model during testing for simplicity.
    # Most research papers using MAML for this task do an extra
    # stage of fine-tuning here that should be added if you are
    # adapting this code for research.
    net.train()
    n_test_iter = db.x_test.shape[0] // db.batchsz

    qry_losses = []
    qry_accs = []

    for batch_idx in range(n_test_iter):
        x_spt, y_spt, x_qry, y_qry = db.next('test')


        task_num, setsz, c_, h, w = x_spt.size()
        querysz = x_qry.size(1)

        # TODO: Maybe pull this out into a separate module so it
        # doesn't have to be duplicated between `train` and `test`?
        n_inner_iter = 5
        inner_opt = torch.optim.SGD(net.parameters(), lr=1e-1)

        for i in range(task_num):
            with higher.innerloop_ctx(net, inner_opt, track_higher_grads=False) as (fnet, diffopt):
                # Optimize the likelihood of the support set by taking
                # gradient steps w.r.t. the model's parameters.
                # This adapts the model's meta-parameters to the task.
                for _ in range(n_inner_iter):
                    spt_logits = fnet(x_spt[i])
                    spt_loss = F.cross_entropy(spt_logits, y_spt[i])
                    diffopt.step(spt_loss)

                # The query loss and acc induced by these parameters.
                qry_logits = fnet(x_qry[i]).detach()
                qry_loss = F.cross_entropy(
                    qry_logits, y_qry[i], reduction='none')
                qry_losses.append(qry_loss.detach())
                qry_accs.append(
                    (qry_logits.argmax(dim=1) == y_qry[i]).detach())

    qry_losses = torch.cat(qry_losses).mean().item()
    qry_accs = 100. * torch.cat(qry_accs).float().mean().item()
    print(
        f'[Epoch {epoch+1:.2f}] Test Loss: {qry_losses:.2f} | Acc: {qry_accs:.2f}'
    )
    log.append({
        'epoch': epoch + 1,
        'loss': qry_losses,
        'acc': qry_accs,
        'mode': 'test',
        'time': time.time(),
    })
</code></pre>
<p>however whenever I do eval instead I get that my MAML model diverges:</p>
<pre><code class="lang-auto">&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5939, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5940, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5940, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5940, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5941, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5940, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5942, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5940, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5940, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_old (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;&gt;maml_old (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5939, grad_fn=&lt;NormBackward1&gt;)
eval_loss=0.9859228551387786, eval_acc=0.5907692521810531
args.meta_learner.lr_inner=0.01
==== in forward2
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(171440.6875, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(208426.0156, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(17067344., grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(40371.8125, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(1.0911e+11, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(21.3515, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(5.4257e+13, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(128.9109, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(3994.7734, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (before inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(9.5937, grad_fn=&lt;NormBackward1&gt;)
&gt;maml_new (after inner adapt): fmodel.model.features.conv1.weight.norm(2)=tensor(1682896., grad_fn=&lt;NormBackward1&gt;)
eval_loss_sanity=nan, eval_acc_santiy=0.20000000298023224
</code></pre>
<p>So what is one suppose to do to avoid this divergence?</p>
<hr>
<p>related:</p>
<ul>
<li>
<p><a href="https://github.com/facebookresearch/higher/issues/107" class="inline-onebox" rel="noopener nofollow ugc">Is there data leakage in the maml-omniglot example? · Issue #107 · facebookresearch/higher · GitHub</a></p>
</li>
<li>
<p><a href="https://discuss.pytorch.org/t/when-should-one-call-eval-and-train-when-doing-maml-with-the-pytorch-higher-library/136022" class="inline-onebox">When should one call .eval() and .train() when doing MAML with the PyTorch higher library?</a></p>
</li>
<li>
<p><a href="https://stackoverflow.com/questions/69845469/when-should-one-call-eval-and-train-when-doing-maml-with-the-pytorch-highe" class="inline-onebox" rel="noopener nofollow ugc">machine learning - When should one call .eval() and .train() when doing MAML with the PyTorch higher library? - Stack Overflow</a></p>
</li>
</ul>