[resolved] Implementing MAML in PyTorch

aitutakiv: <p>Hello,</p>
<p>I am re-implementing the supervised learning experiments from  <a href="https://arxiv.org/abs/1703.03400" rel="nofollow noopener">Model-Agnostic Meta Learning (MAML)</a> in PyTorch.</p>
<p>The goal is to learn features that are “most fine-tune-able.” This is achieved by taking gradient step(s) in the direction that maximizes performance on the validation set given a step(s) on the training set. This requires second derivatives with respect to the parameters. See Algorithm 1 in the paper.</p>
<p>Where I am stuck is that I need to do:<br>
(1) inner loop: a forward pass on the training example, take gradients with respect to the parameters (2) meta loop: do a forward pass with the <em>updated</em> parameters on a validation example, then take another gradient wrt the original parameters and backprop through the first gradient (thus the second derivative).</p>
<p>From the Improved WGAN implementation, I see that I can take the gradient and retain the graph, allowing me to then take another gradient. But I don’t see how I can do the second forward pass without updating the parameters via <code>opt.step()</code>. Do I need to have two graphs, one where I cache the old parameters for the meta-update, and one where I allow the parameters to update in the inner loop?</p>
<p>Thanks for your help!</p>
<p>EDIT: I understand now that I need to add variables to the graph for each gradient of each variable. Then in the meta-update, the gradient of these gradients can be taken with a backward pass over the augmented graph.</p>

rinuboney: <p>Were you able to successfully implement it? If yes, can you explain how you did it?</p>
<p>I have a related question <a href="https://discuss.pytorch.org/t/gradient-computation-in-meta-learning-algorithms/6039">here</a></p>

aitutakiv: <p>Yes I have an implementation, but I’ve been unable to replicate the Omniglot experiments from the paper. Let me clean up the code and I will make it public soon.</p>

aitutakiv: <p>Here is a link to the MAML code: <a href="https://github.com/katerakelly/pytorch-maml" rel="nofollow noopener">https://github.com/katerakelly/pytorch-maml</a><br>
I was able to replicate the Omniglot experiments with it.</p>

Sudarshan_VB: <p>is there a easy way to train networks within the maml framework without having to code up the functional form ? as this would make training of deeper networks easier</p>

Miao_txy_Aaron: <p>Can you post  your implemention? I am also curious about the “module” style implemention about maml.</p>

NotNANtoN: <p>This is super late, but there are now libraries that nicely implement this. See <a href="https://github.com/learnables/learn2learn" class="inline-onebox" rel="noopener nofollow ugc">GitHub - learnables/learn2learn: A PyTorch Library for Meta-learning Research</a> for a stateful solution and <a href="https://github.com/facebookresearch/higher" class="inline-onebox" rel="noopener nofollow ugc">GitHub - facebookresearch/higher: higher is a pytorch library allowing users to obtain higher order gradients over losses spanning training loops rather than individual training steps.</a> for a stateless solution <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>