How do I backpropagate through a module's parameters?

ihexx: <p>I’m trying to implement meta-gradient learning similarly to [<a href="https://arxiv.org/pdf/1805.09801.pdf" rel="nofollow noopener">https://arxiv.org/pdf/1805.09801.pdf</a>].<br>
I need to backpropagate through a parameter update.</p>
<p>The following is a minimum example that highlights the problem:</p>
<pre><code class="lang-auto">                meta_param = torch.tensor(5.0,requires_grad=True) # this is the variable i want to learn
                param = torch.nn.Parameter(torch.tensor(4.0,requires_grad=True)) # this is a module param
                param.data = meta_param * 2

                meta_objective = - (param * 2)
                meta_objective.mean().backward()
</code></pre>
<p>I’m trying to get the gradient of <code>meta_param</code> wrt meta_objective, but it has to pass through a nn.Parameter update for <code>param</code>. If <code>param</code> was a regular tensor as so:</p>
<pre><code class="lang-auto">                meta_param = torch.tensor(5.0,requires_grad=True) # this is the variable i want to learn
                param = meta_param * 2

                meta_objective = - (param * 2)
                meta_objective.mean().backward()
</code></pre>
<p><code>meta_param.grad</code> has a sensible value. But when <code>param</code> is an nn.Parameter, <code>meta_param.grad</code> is None.<br>
When assigning the new value to the nn.Parameter, i notice it has no <code>.grad_fn</code> method, and it is a <code>leaf_tensor=True</code> for it.<br>
This is true for all model weights i want to update. Is there a way to make an nn.Module’s weights differentiable? Perhaps making a temporary model and make its weights views of non-leaf tensors?<br>
How do I work around this?</p>

smth: <p>you could look at something like <a href="https://github.com/facebookresearch/higher" rel="nofollow noopener">https://github.com/facebookresearch/higher</a> for this purpose.<br>
It functionalizes the model, where it’s parameters can be detached and backproped through</p>

ihexx: <p>oh my god yes! this is exactly what i was looking for and SO MUCH MORE<br>
They even have differentiable optimizers that are 1000x more elegant than what I’d hacked together</p>
<p>For posterity, this is a minimal example of what I was trying to accomplish, and it works with higher:</p>
<pre><code class="lang-auto">import higher
import torch
from torch import nn

"""
Goal: use meta gradients to tune differentiable hyper params of an update step
by updating them to maximise a corss-validation step online against a second update
""" 
train_inputs = torch.tensor(10.0)
xval_inputs = torch.tensor(10.0)

meta_param = torch.tensor(10.0, requires_grad=True)
meta_optimizer = torch.optim.Adam([meta_param], lr=3e-4)

my_model = nn.Linear(1, 1)
opt_a = torch.optim.Adam(my_model.parameters(), lr=3e-4)

with higher.innerloop_ctx(my_model, opt_a) as (fmodel, diffopt):
    # training step with higher wrappers
    pred = fmodel(train_inputs.unsqueeze(0))
    pred = pred * meta_param
    loss = -pred
    diffopt.step(loss)
    meta_optimizer.zero_grad() # &lt;- does nothing, but I had to be sure 
    
    # meta param update ste[
    second_pred = fmodel(xval_inputs.unsqueeze(0))
    meta_objective = -second_pred
    meta_objective.backward() #  meta_param.grad now exists, is valid and doesn't sum grads from first loss 🎉🎉 
    meta_optimizer.step()
</code></pre>
<p>thanks</p>

Chow_Harry: <p>A solution for your minimum example without involving external lib.</p>
<p><a href="https://discuss.pytorch.org/t/non-leaf-variables-as-a-modules-parameters/65775">https://discuss.pytorch.org/t/non-leaf-variables-as-a-modules-parameters/65775</a></p>