Unable to apply a basic filter using Functional Conv2D

Kushal_Kumar_Kasina: <p>Hi, I am trying to replicate the 2D convolutions using FFT. To test my understanding and my FFT convolution, I am applying a 5x5x3 sobel filter to a 256x256x3 image.<br>
Surprisingly, I do see some edges in the final convolved image with my FFT convolution and I believe its right. But I get an img full of noise when I try using torch’s Functional Conv2d.<br>
Code:<br>
The sobel filter:</p>
<pre><code class="lang-auto">kernel = np.array([[[1, 2, 0, -2, -1], [4, 8, 0, -8, -4], [6, 12, 0, -12, -6], [4, 8, 0, -8, -4], [1, 2, 0, -2, -1]],
                   [[1, 2, 0, -2, -1], [4, 8, 0, -8, -4], [6, 12, 0, -12, -6], [4, 8, 0, -8, -4], [1, 2, 0, -2, -1]],
                   [[1, 2, 0, -2, -1], [4, 8, 0, -8, -4], [6, 12, 0, -12, -6], [4, 8, 0, -8, -4], [1, 2, 0, -2, -1]]])
</code></pre>
<p>Functional Conv2D usage:</p>
<pre><code class="lang-auto">kernel = np.expand_dims(kernel, axis=0)
weight = nn.Parameter(torch.from_numpy(kernel.astype(np.int8)), requires_grad=False)
print(weight.shape)
# weight = torch.unsqueeze(weight, 0)

new_img = np.expand_dims(img, axis=0)
new_img = nn.Parameter(torch.from_numpy(new_img.astype(np.int8)), requires_grad=False)
print(new_img.shape)
# new_img = torch.unsqueeze(new_img, 0)

classic_2d_conv = F.conv2d(new_img, weight, bias=None, stride=1, padding=2, dilation=1, groups=1)
classic_2d_conv = classic_2d_conv.cpu().detach().numpy()
classic_2d_conv = classic_2d_conv.reshape((256, 256, 1))
</code></pre>
<ul>
<li>
<p>Edit: The kernel and the image have the dimensions in the right order. I just did not paste the snippet here. I have maintained (1, 3, 256, 256) for the image and (1, 3, 5, 5) for the kernel.</p>
</li>
<li>
<p>I intend to perform “same” convolution, hence a padding of 2.</p>
</li>
<li>
<p>1 output channel, 3 input channel is intentional.</p>
</li>
<li>
<p>In FFT convolution, I am convolving each image channel, with the corresponding filter channel. Then summing the 3 outputs into 1 thus arriving at the same behaviour as mentioned in the above point.</p>
</li>
</ul>
<p>The output images:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/b/9b84a241214e80ac50fefc449cc853854e49d214.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/9b84a241214e80ac50fefc449cc853854e49d214" title="sample_output"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/b/9b84a241214e80ac50fefc449cc853854e49d214_2_690x341.jpeg" alt="sample_output" data-base62-sha1="mbMaOVxGue6oFkldKDY8qmkXpiI" width="690" height="341" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/b/9b84a241214e80ac50fefc449cc853854e49d214_2_690x341.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/9/b/9b84a241214e80ac50fefc449cc853854e49d214_2_1035x511.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/b/9b84a241214e80ac50fefc449cc853854e49d214_2_1380x682.jpeg 2x" data-dominant-color="BDD5CB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">sample_output</span><span class="informations">1536×761 267 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I just cannot figure out why the Functional API is producing the gibberish image. Please point out if I am using the functional API in the wrong way or if I am missing some obvious thing. Thanks!<br>
(Please let me know if you need more code snippets)</p>

eqy: <p>This might not be the root cause of the problem but I think you might want to change the data dtypes to something like <code>int</code> or <code>long</code> before running your computation. The issue is that you will likely overflow <code>int8</code> if the original data is in 8-bit RGB, and this becomes even more likely during the accumulation in the convolution. After converting to a larger datatype, you can <code>torch.clamp</code> the values back to 0-255 for a viewable RGB image.</p>

Kushal_Kumar_Kasina: <p>Hi! Well it almost worked… and <strong>thank you very much</strong>. I say almost because I think the colors are inverted.<br>
With torch.clamp for output tensor and np.clip for my FFT convolution, the inversion (observe the back of the head of the bird) is much more prominent:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/8/f8109625ac2a6a3d287e3ab42456ff4a831c2f7d.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/f8109625ac2a6a3d287e3ab42456ff4a831c2f7d" title="sample_output_2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/8/f8109625ac2a6a3d287e3ab42456ff4a831c2f7d_2_690x341.jpeg" alt="sample_output_2" data-base62-sha1="zotWQnwyahbztshSZVibCy4KC1n" width="690" height="341" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/8/f8109625ac2a6a3d287e3ab42456ff4a831c2f7d_2_690x341.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/8/f8109625ac2a6a3d287e3ab42456ff4a831c2f7d_2_1035x511.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/8/f8109625ac2a6a3d287e3ab42456ff4a831c2f7d_2_1380x682.jpeg 2x" data-dominant-color="CCC8C0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">sample_output_2</span><span class="informations">1536×761 361 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The edges are very similar but its the color I cannot understand why, are different. I will keep digging but do let me know if you have any clues.</p>
<p>Edit: it seems like either me or torch flips the kernel and then applies convolution. I flipped the kernel before converting it into weights and now I have the perfect replica. A few details remain, yet to be discovered!</p>

eqy: <p>Ah, I think you have discovered that “convolution” from the perspective of deep learning frameworks is usually cross-correlation! <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Kushal_Kumar_Kasina: <p>Yes. I love the insight. I think the direction of sliding the kernel across the input image, in torch, is opposite (in the directional sense) to the effect of FFT dot-product convolution I have been trying.</p>