How to calculate the gradient of images?

Michael: <p>In my network, I have a output variable A which is of size h<em>w</em>3, I want to get the gradient of A in the x dimension and y dimension, and calculate their norm as loss function. How should I do it?</p>
<p>Thanks.</p>

kmichaelkills: <p>I guess you could represent gradient by a convolution with sobel filters. Maybe implemented with Convolution 2d filter with require_grad=false (where you set the weights to sobel filters).</p>

yanp: <p><a class="mention" href="/u/michael">@Michael</a> have you been able to implement it? Is it possible to show the code snippet?</p>

Blade6570: <p>Here is the code. It works perfectly.</p>
<p>from PIL import Image<br>
import torch.nn as nn<br>
import torch<br>
import numpy as np<br>
from torchvision import transforms<br>
from torch.autograd import Variable<br>
<span class="hashtag">#img</span> = Image.open(’/home/soumya/Documents/cascaded_code_for_cluster/RGB256FullVal/frankfurt_000000_000294_leftImg8bit.png’).convert(‘LA’)<br>
img = Image.open(’/home/soumya/Downloads/PhotographicImageSynthesis_master/result_256p/final/frankfurt_000000_000294_gtFine_color.png.jpg’).convert(‘LA’)<br>
<span class="hashtag">#img</span>.save(‘greyscale.png’)<br>
T=transforms.Compose([transforms.ToTensor()])<br>
P=transforms.Compose([transforms.ToPILImage()])</p>
<p>ten=torch.unbind(T(img))<br>
x=ten[0].unsqueeze(0).unsqueeze(0)</p>
<p>a=np.array([[1, 0, -1],[2,0,-2],[1,0,-1]])<br>
conv1=nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1, bias=False)<br>
conv1.weight=nn.Parameter(torch.from_numpy(a).float().unsqueeze(0).unsqueeze(0))</p>
<p>G_x=conv1(Variable(x)).data.view(1,256,512)</p>
<p>b=np.array([[1, 2, 1],[0,0,0],[-1,-2,-1]])<br>
conv2=nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1, bias=False)<br>
conv2.weight=nn.Parameter(torch.from_numpy(b).float().unsqueeze(0).unsqueeze(0))<br>
G_y=conv2(Variable(x)).data.view(1,256,512)</p>
<p>G=torch.sqrt(torch.pow(G_x,2)+ torch.pow(G_y,2))<br>
X=P(G)<br>
X.save(‘fake_grad.png’)</p>

DhruvJawalkar: <p>Thanks ! this worked.<br>
We could simplify it a bit, since we don’t want to compute gradients, but the outputs look great <img src="https://discuss.pytorch.org/images/emoji/apple/+1/4.png?v=6" title=":+1:t4:" class="emoji" alt=":+1:t4:"></p>
<p><span class="hashtag">#Black</span> and white input image x, 1x1xHxW<br>
a = torch.Tensor([[1, 0, -1],<br>
[2, 0, -2],<br>
[1, 0, -1]])</p>
<p>a = a.view((1,1,3,3))<br>
G_x = F.conv2d(x, a)</p>
<p>b = torch.Tensor([[1, 2, 1],<br>
[0, 0, 0],<br>
[-1, -2, -1]])</p>
<p>b = b.view((1,1,3,3))<br>
G_y = F.conv2d(x, b)</p>
<p>G = torch.sqrt(torch.pow(G_x,2)+ torch.pow(G_y,2))<br>
res = P(G)</p>

NjuHaoZhang: <p>I have one of the simplest differentiable solutions. The idea comes from the implementation of tensorflow. The basic principle is:</p>
<p>dx = I(x+1) - I(x), dy = I(y+1) - I(y),</p>
<p>The code is as follows:</p>
<pre><code class="lang-auto">def gradient_loss(gen_frames, gt_frames, alpha=1):

    def gradient(x):
        # idea from tf.image.image_gradients(image)
        # https://github.com/tensorflow/tensorflow/blob/r2.1/tensorflow/python/ops/image_ops_impl.py#L3441-L3512
        # x: (b,c,h,w), float32 or float64
        # dx, dy: (b,c,h,w)

        h_x = x.size()[-2]
        w_x = x.size()[-1]
        # gradient step=1
        left = x
        right = F.pad(x, [0, 1, 0, 0])[:, :, :, 1:]
        top = x
        bottom = F.pad(x, [0, 0, 0, 1])[:, :, 1:, :]

        # dx, dy = torch.abs(right - left), torch.abs(bottom - top)
        dx, dy = right - left, bottom - top 
        # dx will always have zeros in the last column, right-left
        # dy will always have zeros in the last row,    bottom-top
        dx[:, :, :, -1] = 0
        dy[:, :, -1, :] = 0

        return dx, dy

    # gradient
    gen_dx, gen_dy = gradient(gen_frames)
    gt_dx, gt_dy = gradient(gt_frames)
    #
    grad_diff_x = torch.abs(gt_dx - gen_dx)
    grad_diff_y = torch.abs(gt_dy - gen_dy)

    # condense into one tensor and avg
    return torch.mean(grad_diff_x ** alpha + grad_diff_y ** alpha)
</code></pre>

edgarriba: <p>hi! you can also use <code>kornia.spatial_gradient</code> to compute gradients of an image.<br>
See: <a href="https://kornia.readthedocs.io/en/latest/filters.html#kornia.filters.SpatialGradient" rel="nofollow noopener">https://kornia.readthedocs.io/en/latest/filters.html#kornia.filters.SpatialGradient</a></p>

bumpybanana: <p>You defined h_x and w_x, however you do not use these in the defined function. May I ask what the purpose of h_x and w_x are?</p>