Understanding how to use grad when getting grad wrt inputs

jcallahan4: <p>Hi all, I’m trying to use autograd to calculate the gradient of some outputs wrt some inputs on a<br>
pretrained neural network. Here is the code I use:</p>
<pre><code class="lang-auto">net = Custom_NN(input_size, hidden_layers, output_size)
net.load_state_dict(torch.load('network_weights.pt'))

x = torch.rand(input_size, requires_grad=True)
y = net(x)
gradient = torch.autograd.grad(y, x, grad_outputs = torch.ones_like(y))
</code></pre>
<p>This code gives me the following error:</p>
<blockquote>
<p>RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.</p>
</blockquote>
<p>I’ve consulted the following questions and it seems like I’m doing the same thing as them, but using their solutions doesn’t fix the error, so it seems like there’s something I’m fundamentally misunderstanding about how to use autograd. Can anyone help me understand what autograd is doing, why it cares about unused Tensors, and how to get the network to use my input Tensors in its graph? Thanks.</p>
<aside class="quote quote-modified" data-post="1" data-topic="58585">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tvdw/48/12334_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/gradient-of-output-wrt-specific-inputs/58585">Gradient of output wrt specific inputs</a> <a class="badge-category__wrapper " href="/c/autograd/7"><span data-category-id="7" data-drop-close="true" class="badge-category " title="A category of posts relating to the autograd engine itself."><span class="badge-category__name">autograd</span></span></a>
  </div>
  <blockquote>
    Hi there, 
I am trying to retrieve the gradients of the output variables wrt the input variables of a Neural Network model (loaded from a .pt file). Note that the neural network describes a dynamical system in which a history of states is taken into account. The network is too complicated to have an intuition of the range in which the gradients should lie. 
I am currently tackling it the following way: 
    x = torch.zeros(1,1500) # 1500 inputs to model
    x[0][:] = torch.from_numpy(array_of_in…
  </blockquote>
</aside>
<aside class="quote quote-modified" data-post="1" data-topic="24407">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/d78d45/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-calculate-gradients-wrt-one-of-inputs/24407">How to calculate gradients wrt one of inputs</a> <a class="badge-category__wrapper " href="/c/autograd/7"><span data-category-id="7" data-drop-close="true" class="badge-category " title="A category of posts relating to the autograd engine itself."><span class="badge-category__name">autograd</span></span></a>
  </div>
  <blockquote>
    Suppose that F(x) is a NN with 3 inputs and 1 output and I try to compute the gradient of F wrt the first input. But I got the error message: “One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.” What is the problem? 
My code is as follows: 
import torch 
import torch.nn as nn 
from torch.autograd import Variable,grad 
NN = nn.Sequential( 
torch.nn.Linear(3, 64), 
torch.nn.ReLU(), 
torch.nn.Linear(64, 64), 
torch.nn…
  </blockquote>
</aside>


AlphaBetaGamma96: <aside class="quote no-group" data-username="jcallahan4" data-post="1" data-topic="160233">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jcallahan4/48/52771_2.png" class="avatar"> jcallahan4:</div>
<blockquote>
<pre><code class="lang-auto">net.load_state_dict(torch.load('network_weights.pt'))
</code></pre>
</blockquote>
</aside>
<p>Could you try,</p>
<pre><code class="lang-auto">net = net.load_state_dict(torch.load('network_weights.pt'))
</code></pre>
<p>and try again? (and also share what <code>Custom_NN</code> is)</p>

jcallahan4: <p>Sure. Running it the way you have it throws</p>
<blockquote>
<p>TypeError: ‘_IncompatibleKeys’ object is not callable</p>
</blockquote>
<p>after the line</p>
<pre><code class="lang-auto">y = net(x)
</code></pre>
<p>(In this case <code>type(net)</code> is <code>torch.nn.modules.module._IncompatibleKeys</code>).</p>
<p>The class definition for my model is as follows:</p>
<pre><code class="lang-auto">class Custom_NN(torch.nn.Module):
    def __init__(self, input_size, hidden_size, output_size, scalers=None):
        super(VanillaNet, self).__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size
        if scalers is None:
            # Here I've hardcoded the mean_'s and scale_'s of input/output scalers.
            self.inp_mean = torch.Tensor([
                -4.59936018e+00,  1.31706962e+02,  3.80556213e+05,  1.05726545e+05,
                1.22070403e+01,  1.49188727e+02,  1.48334349e+01,  1.73222231e+04
            ])
            self.inp_std = torch.Tensor([
                5.35557343e-01, 2.00745698e-01, 1.22149250e+05, 2.86161126e+04,
                3.68246696e+00, 2.23304735e+01, 3.15135316e+00, 5.14069803e+03
            ])
            
            self.out_mean = torch.Tensor([ 
                1.70409856,  2.36874371, 14.85654644,  5.24019232,  1.51261909,
                1.0671637 , 39.82251636,  4.39834804,  0.99539551,  1.37684262,
                2.83503784
            ])
            self.out_std = torch.Tensor([
                0.54293611, 0.33864151, 3.57840622, 1.48489459, 0.31817987,
                0.20210712, 2.12708213, 0.6290501 , 0.15368718, 0.21272585,
                0.61394896
            ])
        else:
            inp_scaling, out_scaling = scalers
            self.inp_mean, self.inp_std = inp_scaling
            self.out_mean, self.out_std = out_scaling
        
        self.normalize_input = lambda x: (x - self.inp_mean)/self.inp_std
        self.unnormalize_output = lambda x: x*self.out_std + self.out_mean
        
        self.input_layer = torch.nn.Linear(self.input_size, self.hidden_size[0])
        self.input_act = torch.nn.ReLU()
        self.hidden_layers = torch.nn.Sequential(
            *chain.from_iterable(
                [torch.nn.Linear(a,b), torch.nn.ReLU()] for a,b in zip(hidden_size[:-1], hidden_size[1:])
            )
        )
        self.output_layer = torch.nn.Linear(self.hidden_size[-1], self.output_size)

        
    def forward(self, x):
        with torch.no_grad():
            normalized_x = self.normalize_input(x)
        y = self.input_act(self.input_layer(normalized_x))
        z = self.hidden_layers(y)
        return self.output_layer(z)
    
    
    def forward_unnormalized(self, x):
        with torch.no_grad():
            return self.unnormalize_output(self.forward(x))
</code></pre>

AlphaBetaGamma96: <aside class="quote no-group" data-username="jcallahan4" data-post="3" data-topic="160233">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jcallahan4/48/52771_2.png" class="avatar"> jcallahan4:</div>
<blockquote>
<pre><code class="lang-auto">        with torch.no_grad():
            normalized_x = self.normalize_input(x)
</code></pre>
</blockquote>
</aside>
<p>This is the issue. You’re breaking your comp graph, which is why you get,</p>
<pre><code class="lang-auto">RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.
</code></pre>
<p>Remove the <code>torch.no_grad</code> context manager and try again (and be careful of potential divide by zero errors on <code>self.inp_std</code>).</p>

jcallahan4: <p>That was it! Thank you so much!</p>

srishti-git1110: <p>Hi <a class="mention" href="/u/jcallahan4">@jcallahan4</a>,<br>
Good to see your error is already solved.</p>
<p>Since you wanted to understand what autograd is doing, and how to get the network to use the input tensors in the computation graph, I’m adding some details in that regard:</p>
<p><code>torch.autograd</code> is pytorch’s automatic differentiation engine that, as the name suggests, deals with automatically calculating gradients for any “computational graph”.</p>
<p>Computational graphs are what that get build by autograd as and when tensors are subjected to mathematical operations. While building these graphs, <code>autograd</code> also saves tensors that’ll be required to calculate the gradients wrt tensors having their <code>requires_grad</code> attribute set to <code>True</code>.</p>
<p>(So, when you use <code>torch.autograd.grad</code> or use the <code>.backward</code> call, these saved tensors are used).</p>
<p>Now, <code>torch.no_grad()</code> basically tells autograd to look away. It can be used as a context manager so that <em>for any piece of code occurring within this context</em>, autograd shall build no graph (or will not further populate any graph that’s already there).<br>
i.e. It’ll not track any operations.</p>
<p>Now, for your code, you are differentiating y (output) wrt x,<br>
where <code>y = net(x)</code> which essentially means <code>y = net.forward(x)</code>.</p>
<p>Inside forward, output_layer(z) which is returned (and hence is essentially what gets stored in <code>y = net(x)</code>) is a result of operations on <code>normalized_x</code>, but <code>normalized_x</code> is getting created as a result of operations on x under <code>torch.no_grad()</code>.</p>
<p>This means even if<br>
<code>y = self.input_act(self.input_layer(normalized_x))</code> and<br>
<code>z = self.hidden_layers(y)</code> are a part of the computation graph, <code>normalized_x</code> isn’t really.</p>
<p>And so when you tried to differentiate y (which is returned by forward) wrt x, it produced the error<br>
<code>One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.</code></p>
<p>So, here the error prompt is most probably talking about <code>normalized_x</code> as the tensor that appears to not have been used in the graph.</p>
<p>Note: Even if <code>normalized_x</code> is getting created as a result of operations on <code>x</code> <em>whose <code>requires_grad</code> is set to True</em>, it doesn’t matter. Under <code>torch.no_grad()</code>, nothing is tracked by autograd and so all resulting tensors have their <code>requires_grad=False</code>.</p>
<p>Hope this helps,<br>
Srishti</p>