Intermediate params update; set as a leaf variable

mk100000: <p>Hello,</p>
<p>I wish to create intermediate trainable parameters in my network. I am able to collect intermediate gradients with the help of the <code>retain_grad()</code> method, and perform updates manually, but I’d prefer to work with the <code>optim</code> module PyTorch provides so that I don’t have to write my own functions for the various gradient updates there are out there.</p>
<p>I’m relatively new to PyTorch, and my understanding is that the updates on the intermediate params are not working since they are non leaf nodes. Is there a way to detach previous computations to create these leaf nodes, so that <code>optimizer.step()</code> can perform updates successfully? Or is there a way to pass these intermediate gradients into the <code>optimizer</code> object itself?</p>
<p>The code below gives a vague illustration of what I am trying to accomplish. Any help is greatly appreciated!</p>
<pre><code class="lang-auto">network = MyNetwork()

fc_weights = fc_weights = torch.tensor([[1,0],[0,1]],requires_grad=False,dtype=torch.float)
input = torch.tensor([[2],[3]],requires_grad=False,dtype=torch.float) # sample output of several layers that are frozen

params = torch.tensor([[0.25],[0.125]],requires_grad=True,dtype=torch.float) # learnable parameters for first final layer
params2 = torch.rand(2,1,requires_grad=True) # learnable parameters for second final layer
params3 = torch.rand(2,1,requires_grad=True) # learnable parameters for third final (final) layer

learning_rate = 1000 # just some sample learning rate to probe changes
optimizer = optim.SGD([params], lr=learning_rate)
optimizer2 = optim.SGD([params2],lr=learning_rate)
optimizer3 = optim.SGD([params3],lr = learning_rate)
optimizer.zero_grad() # zero gradients initially to be safe
optimizer2.zero_grad()
optimizer3.zero_grad()

print(params) # check value before nudge
print(params2) # check value before nudge
print(params3) # check value before nudge
print()

x = torch.hstack((input,params))
params2 = network(x,fc_weights)
params2.retain_grad() # keep intermediate grads
print(params2.is_leaf) # prints False
params3 = network(params2,fc_weights)
params3.retain_grad() # keep intermdiate grads
print(params3.is_leaf) # prints False
print()

loss = MyLoss(params3)
loss.backward()

print(params.grad) # intermediate gradients successfully computed
print(params2.grad) # intermediate gradients successfully computed
print(params3.grad) # intermediate gradients successfully computed
print()

optimizer.step() # gradient update/ nudge
optimizer2.step() # gradient update/ nudge
optimizer3.step() # gradient update/ nudge

print(params) # this learnable parameter is updated successfully because it is a leaf node.
print(params2) # this learnable parameter is not updated, which is what I am trying to figure out
print(params3) # this learnable parameter is not updated, which is what I am trying to figure out
</code></pre>

InnovArul: <aside class="quote no-group" data-username="mk100000" data-post="1" data-topic="107360">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/91b2a8/48.png" class="avatar"> mk100000:</div>
<blockquote>
<pre><code class="lang-auto">params2 = network(x,fc_weights)
params2.retain_grad() # keep intermediate grads
print(params2.is_leaf) # prints False
params3 = network(params2,fc_weights)
params3.retain_grad() # keep intermdiate grads
print(params3.is_leaf) # prints False
print()
</code></pre>
</blockquote>
</aside>
<p>You are re-defining (or re-assigning) <code>params2</code> and <code>params3</code> again in these lines.<br>
They are not holding the variables that you defined initially.</p>

mk100000: <p><a class="mention" href="/u/innovarul">@InnovArul</a><br>
Appreciate the response. What’s wrong with reassigning them new values? Isn’t it stored as the same object internally? Reason I do it that way is because I want <code>param</code>, <code>param2</code>, and <code>param3</code> to undergo a nested set of computations; they should pick up on the computations they undergo relative to where they’re placed.</p>

InnovArul: <aside class="quote no-group" data-username="mk100000" data-post="3" data-topic="107360">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/91b2a8/48.png" class="avatar"> mk100000:</div>
<blockquote>
<p>What’s wrong with reassigning them new values? Isn’t it stored as the same object internally?</p>
</blockquote>
</aside>
<p>Initially <code>params2</code> holds the reference of  <code>torch.rand(2,1,requires_grad=True)</code>. When you assign again, it holds the reference of some intermediate value that’s computed by model. So, the answer is No. <code>params2</code> is not holding the same object.<br>
Also, your usage of <code>params2</code> tells me that it is just an intermediate variable and not something that you want to optimize.<br>
Maybe you have to elaborate a bit more on what is your end goal with context.</p>
<aside class="quote no-group" data-username="mk100000" data-post="3" data-topic="107360">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/91b2a8/48.png" class="avatar"> mk100000:</div>
<blockquote>
<p>I want <code>param</code>, <code>param2</code>, and <code>param3</code> to undergo a nested set of computations; they should pick up on the computations they undergo relative to where they’re placed.</p>
</blockquote>
</aside>
<p>Like you said, they “will” pick-up on the computations and pass on the gradients to previous computations. That is definitely handled by pytorch. But you cannot optimize <code>params2</code>, as it is an intermediate value calculated by model (i.e., it will change for different inputs).</p>
<p>With the current code example and definitions of <code>params*</code>, it is not understandable what is expected from pytorch (at least not understandable to me).</p>

mk100000: <p><a class="mention" href="/u/innovarul">@InnovArul</a></p>
<p>I actually do intend on having <code>params</code>, <code>params2</code> and <code>params3</code> be optimized, and not just stored as intermediate placeholder variables. They culminate in an unsupervised loss function <code>MyLoss</code>. The gradients obtained via <code>retain_grad()</code> generate the partial derivatives <code>MyLoss</code> wrt <code>params</code>, <code>MyLoss</code> wrt <code>params2</code> and  <code>MyLoss</code> wrt <code>params3</code>. Taking SGD as an example, the update for <code>params2</code> would be:<br>
<code>params2 -= lr*(MyLoss wrt params2)</code>.</p>
<p>Where <code>params2.data</code> is set to what was during instantiation (before being affected by the intermediate pass). Ditto for the other params.</p>
<p>But again, I wish to abstract this away by using the <code>optim</code> class instance. I think I see what you mean by reassigning values creates different objects, as the optimizer loses track of <code>params2</code> for example after the reassignment (please correct me if I’m wrong).</p>
<p>The reason why I reassign <code>params*</code> is because I wish to first instantiate them outside my training loop and pass them into an instance of a custom model I’ve built. Loosely speaking, what I’m trying to do is train a standard neural network, then freeze it (the trained fully connected weights), and train another model that takes these frozen weights and trains another set of parameters. These new parameters (the <code>params*</code>) act as activations similar to a fully connected net, but they are also trainable (they get corrected according to minimizing <code>myLoss</code>).</p>
<p>Please let me know if this is enough, and whether you can infer a solution to my problem. If not, I can provide additional details. I really appreciate your help!</p>

InnovArul: <p>Inside custom model <code>__init__</code>, you can define these <code>params*</code> as instances of <code>nn.Parameter</code> and use it in your computation (new model’s <code>forward</code>) along with freezed weights from your earlier model.<br>
Use optimizer to optimize <code>new_model.parameters()</code>. Will this work for you?</p>
<p>But remember that you can’t reassign <code>params*</code> in any case. You can use them in computation as <code>nn.Parameter</code>. Optimizer will take care of updating the `params* with its gradients automatically.</p>