Vision: RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

aswamy: <p>Getting follwoing error:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “gaussian_blur.py”, line 36, in <br>
sigma = estimate_sigma(original_image, blurred_image, args.init_sigma)<br>
File “gaussian_blur.py”, line 18, in estimate_sigma<br>
loss = F.mse_loss(blurred, blurred_image)<br>
File “/nfs/tools/humans/conda/envs/datacap/lib/python3.7/site-packages/torch/_tensor.py”, line 396, in backward<br>
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)<br>
File “/nfs/tools/humans/conda/envs/datacap/lib/python3.7/site-packages/torch/autograd/<strong>init</strong>.py”, line 175, in backward<br>
allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass<br>
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</p>
</blockquote>
<blockquote>
<p>import torch<br>
import torch.nn.functional as F<br>
from torchvision.transforms import GaussianBlur<br>
from PIL import Image<br>
import argparse<br>
import numpy as np<br>
from ipdb import set_trace as bb</p>
<p>def estimate_sigma(original_image, blurred_image, init_sigma=None):<br>
if init_sigma is None:<br>
init_sigma = 1.0<br>
bb()<br>
sigma = torch.tensor(init_sigma, requires_grad=True)<br>
optimizer = torch.optim.Adam([sigma], lr=0.1)</p>
<pre><code>for _ in range(100):
    blurred = GaussianBlur(kernel_size=(int(6 * sigma + 1), int(6 * sigma + 1)))(original_image)
    loss = F.mse_loss(blurred, blurred_image)
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
return sigma.detach()
</code></pre>
<p>if <strong>name</strong> == “<strong>main</strong>”:<br>
parser = argparse.ArgumentParser(description=“Estimate Gaussian blur sigma”)<br>
parser.add_argument(“original_image”, type=str, help=“Path to the original image”)<br>
parser.add_argument(“blurred_image”, type=str, help=“Path to the blurred image”)<br>
parser.add_argument(“–init_sigma”, type=float, default=None, help=“Initial guess for sigma”)<br>
args = parser.parse_args()</p>
<pre><code>original_image = torch.tensor(np.array(Image.open(args.original_image)), dtype=torch.float32).permute(2, 0, 1)[None, ...]
blurred_image = torch.tensor(np.array(Image.open(args.blurred_image)), dtype=torch.float32).permute(2, 0, 1)[None, ...]

sigma = estimate_sigma(original_image, blurred_image, args.init_sigma)
print(f"Estimated sigma: {sigma.item()}")
</code></pre>
</blockquote>

ptrblck: <p>None of your inputs requires gradients and also no trainable parameters were used in differentiable operations. <code>sigma</code> is used as an integer (integers are not usefully differentiable and the transformation will detach the tensor) only so the error is expected.</p>

aswamy: <p>Thanks for response!<br>
Sigma is float and I do not understand why inputs have to be differentiable. Below is the runnable code which gives error. Could you please suggest the modification?</p>
<pre><code class="lang-auto">import torch
import torch.nn.functional as F
from torchvision.transforms import GaussianBlur
from PIL import Image
import argparse
import numpy as np
from ipdb import set_trace as bb

def estimate_sigma(original_image, blurred_image, init_sigma=1.0):
    sigma = torch.tensor(init_sigma, requires_grad=True)
    optimizer = torch.optim.Adam([sigma], lr=0.01)
    for _ in range(100):
        blurred = GaussianBlur(kernel_size=(int(6 * sigma + 1), int(6 * sigma + 1)))(original_image)
        loss = torch.nn.MSELoss()(blurred, blurred_image)
        
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        
    return sigma.detach()

original_image = torch.rand(1, 3, 100, 100)
blurred_image = torch.rand(1, 3, 100, 100)
sigma = estimate_sigma(original_image, blurred_image)
</code></pre>

ptrblck: <aside class="quote no-group" data-username="aswamy" data-post="3" data-topic="202070">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/d2c977/48.png" class="avatar"> aswamy:</div>
<blockquote>
<p><code>int(6 * sigma + 1), int(6 * sigma + 1)</code></p>
</blockquote>
</aside>
<p><code>sigma</code> is transformed to an integer and thus the output is detached from the original trainable <code>sigma</code> as integer types are not usefully differentiable. Their gradient will be zero everywhere and either undefined or <code>Inf</code> at the points of rounding.</p>