Pytorch: torch.autograd.grad returns NoneType

Sankalp_Mittal: <p>Here is my code:</p>
<pre><code class="lang-auto">import torch
#Captum Attribution
from captum.attr import Saliency

model = torch.hub.load('pytorch/vision:v0.10.0', 'squeezenet1_1', pretrained=True)
model.eval()

sal = Saliency(model)

#X, y is an image and label
original_label = y
test_image = X.reshape([1,3,227,227]).float()

#I need gradient w.r.t. this test_image
test_image.requires_grad = True
test_image.retain_grad()

#Calculate saliency
attribution = sal.attribute(test_image, target=original_label)
attribution = torch.sum(torch.abs(attribution[0]), dim=0)
attribution = 227 * 227 * attribution / torch.sum(attribution)
attribution = attribution.view(-1)
elem1 = torch.argsort(attribution)[-1000:]
elements1 = torch.zeros(227 * 227)
elements1[elem1] = 1

#I need gradient of topK_loss w.r.t. test_image
topK_loss = torch.sum(attribution * elements1)
topK_loss.requires_grad = True
topK_loss.retain_grad()
gradients = -torch.autograd.grad(outputs=topK_loss, inputs=test_image, allow_unused=True)[0]
</code></pre>
<p>I get this error: <code>bad operand type for unary -: 'NoneType'</code></p>
<p>I was told and searched that it means that the code is not able to find the path to the gradient.</p>
<p>Can anyone please help me resolve this issue and guide me to it?</p>
<p>Thanks in advance!</p>

ptrblck: <p>What kind of error so you see if <code>allow_unused</code> is set the the default <code>False</code> value? If an error is raised it would mean <code>topK_loss</code> was not created by <code>test_image</code> in a differentiable way and you would need to check if e.g. you have accidentally detached variables from the computation graph.</p>

Sankalp_Mittal: <p>Hello <a class="mention" href="/u/ptrblck">@ptrblck</a> ,<br>
Thanks for replying.</p>
<blockquote>
<p>What kind of error so you see if <code>allow_unused</code> is set the the default <code>False</code> value?</p>
</blockquote>
<p>This gives,<br>
<code>RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.</code></p>
<blockquote>
<p>If an error is raised it would mean topK_loss was not created by test_image in a differentiable way and you would need to check if e.g. you have accidentally detached variables from the computation graph.</p>
</blockquote>
<p>I am also using an external library called captum that takes into input test_image and calculates attribution which is then used by topK_loss. Could that be an issue?</p>
<p>Otherwise the code is as is. I don’t see where I’m detaching a tensor from graph. Thanks.</p>

ptrblck: <p>I’m not familiar enough with Captum, but the original error explains why the gradient is <code>None</code> now.<br>
You could check if the code works fine without Captum (if that’s possible) or you could try to isolate the line of code detaching the tensor from the computation graph by printing the <code>.grad_fn</code> of intermediate tensors making sure they are set to a proper backward function.</p>

Sankalp_Mittal: <p>Thank you for your reply.</p>
<blockquote>
<p>You could check if the code works fine without Captum (if that’s possible)</p>
</blockquote>
<p>I just checked by commenting the captum’s use and gradients are getting calculated. It is not returning None anymore. Further, topK_loss is no more a leaf node.</p>
<pre data-code-pretrained="True)" data-code-wrap="model"><code class="lang-model">
model.eval()

#Captum Attribution

from captum.attr import Saliency

sal = Saliency(model)

# Calculate saliency

original_label = y[5]

test_image = X[5].reshape([1,3,227,227]).float()

print(test_image.shape)

test_image.requires_grad = True

test_image.retain_grad()

# attribution = sal.attribute(test_image, target=original_label)

attribution = torch.sum(torch.abs(test_image[0]), dim=0)

attribution = 227 * 227 * attribution / torch.sum(attribution)

attribution = attribution.view(-1)

print(attribution.shape)

elem1 = torch.argsort(attribution)[-1000:]

elements1 = torch.zeros(227 * 227)

elements1[elem1] = 1

topK_loss = torch.sum(attribution * elements1)

# topK_loss.requires_grad = True

# topK_loss.retain_grad()

# Calculate gradients

gradients = -torch.autograd.grad(outputs=topK_loss, inputs=test_image, allow_unused=False)[0]

print(gradients)
</code></pre>
<blockquote>
<p>or you could try to isolate the line of code detaching the tensor from the computation graph by printing the .grad_fn of intermediate tensors making sure they are set to a proper backward function.</p>
</blockquote>
<p>As soon as I use captum, the topK_loss is getting detached and is being treated as a leaf node. Further, <code>topK_loss.grad_fn</code> yields None. Indeed upon including captum’s <code>attribution = sal.attribute(test_image, target=original_label)</code>, every intermediate tensor is showing None for grad_fn.</p>
<p>Or, could the issue be not captum but actually loading the model instead of defining in the same file?</p>
<pre><code class="lang-auto">model = torch.hub.load('pytorch/vision:v0.10.0', 'squeezenet1_1', pretrained=True)
model.eval()
</code></pre>
<p>What could be a way to handle this? Thank you!</p>

ptrblck: <p>Loading the model from the hub should not make a difference unless the actual model implementation differs.<br>
I’m not familiar enough with the <code>Saliency</code> class, but maybe the gradient calculation is explicitly disabled since the returned <code>attributions</code> object would already contain the interesting gradients?</p>

Sankalp_Mittal: <p>Here is the link to the source code:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://captum.ai/api/_modules/captum/attr/_core/saliency.html#Saliency.attribute">
  <header class="source">
      <img src="https://captum.ai/img/captum.ico" class="site-icon" width="256" height="256">

      <a href="https://captum.ai/api/_modules/captum/attr/_core/saliency.html#Saliency.attribute" target="_blank" rel="noopener nofollow ugc">captum.ai</a>
  </header>

  <article class="onebox-body">
    <img src="https://captum.ai/img/captum-icon.png" class="thumbnail onebox-avatar" width="500" height="500">

<h3><a href="https://captum.ai/api/_modules/captum/attr/_core/saliency.html#Saliency.attribute" target="_blank" rel="noopener nofollow ugc">Captum · Model Interpretability for PyTorch</a></h3>

  <p>Model Interpretability for PyTorch</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>It seems like saliency returns gradients with respect to input anyway. The saliency is gradient of Target output (with a label) with respect to input. However, I want the gradient of this saliency output itself with respect to input image. That basically makes it a double derivative I guess?</p>
<p>Is there anyway to enable gradients in the code somehow?</p>
<p>Thanks.</p>

Sankalp_Mittal: <p>Any help would be appreciated!</p>

Sankalp_Mittal: <pre><code class="lang-auto">    def attribute(
        self,
        inputs: TensorOrTupleOfTensorsGeneric,
        baselines: Union[
            TensorOrTupleOfTensorsGeneric, Callable[..., TensorOrTupleOfTensorsGeneric]
        ],
        n_samples: int = 5,
        stdevs: Union[float, Tuple[float, ...]] = 0.0,
        target: TargetType = None,
        additional_forward_args: Any = None,
        return_convergence_delta: bool = False,
    ) -&gt; Union[
        TensorOrTupleOfTensorsGeneric, Tuple[TensorOrTupleOfTensorsGeneric, Tensor]
    ]:
        # since `baselines` is a distribution, we can generate it using a function
        # rather than passing it as an input argument
        print("HEEEERE")
        baselines = _format_callable_baseline(baselines, inputs)
        assert isinstance(baselines[0], torch.Tensor), (
            "Baselines distribution has to be provided in a form "
            "of a torch.Tensor {}.".format(baselines[0])
        )

        input_min_baseline_x_grad = InputBaselineXGradient(
            self.forward_func, self.multiplies_by_inputs
        )
        input_min_baseline_x_grad.gradient_func = self.gradient_func

        nt = NoiseTunnel(input_min_baseline_x_grad)

        # NOTE: using attribute.__wrapped__ to not log
        inputs.requires_grad_ = True
        attributions = nt.attribute.__wrapped__(
            nt,  # self
            inputs,
            nt_type="smoothgrad",
            nt_samples=n_samples,
            stdevs=stdevs,
            draw_baseline_from_distrib=True,
            baselines=baselines,
            target=target,
            additional_forward_args=additional_forward_args,
            return_convergence_delta=return_convergence_delta,
        ).requires_grad_()
        loss = torch.sum(attributions).requires_grad_()
        print(loss.grad_fn)
        print(torch.autograd.grad(loss, inputs))

        return attributions
</code></pre>
<p>I want to return the gradient of <code>loss</code> with respect to <code>inputs</code>. Right now it throws following error:<br>
<code>RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.</code><br>
If this is solved, then so will be my issue.</p>
<p>Thank you!</p>