Understanding F.conv2d_weight and conv2d_input (Conv2D implementation in general)

username: PySimon
cooked: <p>Hi,</p>
<p>I am implementing a modified version of Conv2D and therefore want to understand what exactly <code>torch.nn.functional.conv2d</code> is doing.</p>
<p>I think it internally uses <code>torch.nn.grad.conv2d_weight</code> and <code>torch.nn.grad.conv2d_input</code>. However, I am having a hard time understanding what exactly these functions do. I am also wondering how the gradient of the bias is computed. <code>torch/nn/functional.py</code> only contains a docstring for this function, but no actual code. Where can I find it?</p>
<p>The problem is that I can’t really find the internal implementation of <code>torch.nn.functional.conv2d</code>, so I can’t see how the other functions I mentioned are used and how the bias grad is computed.</p>
<p>Does anybody have more insight into this function?</p>
<p>EDIT:  Pointers to the code:</p>
<ul>
<li><a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/grad.py#L164" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/torch/nn/grad.py#L164</a></li>
<li><a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/conv.py#L205" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/conv.py#L205</a></li>
</ul>

username: albanD
cooked: <p>Hi,</p>
<p>This function is implemented in cpp for better speed.<br>
You can check this question <a href="https://discuss.pytorch.org/t/source-code-for-conv2d/58398" class="inline-onebox">Source code for conv2d</a> to see where this implementation is.</p>

username: PySimon
cooked: <p>Thanks! But where are the functions conv2d_input and conv2d_weight used then?</p>

username: albanD
cooked: <p>Hi,</p>
<p>There were created for the user to have easy access to the backward function of convolutions. But this is not maintained anymore and will get deprecated in the future.</p>

username: aashaka
cooked: <p>What is the best way for users to reuse the backward function of convolutions then?</p>

username: albanD
cooked: <p>Hi,</p>
<p>“reuse” as call it directly?<br>
Unfortunately, we don’t really bind them at the moment as it can be tricky to be done in general.<br>
What is your use case for this?</p>

username: aashaka
cooked: <p>The use case is to compute the backward pass on slightly modified inputs than what the forward pass computed on. So, I am planning to save the changed input in the ctx variable of the forward pass, and would like to pass that input for gradient computation in the backward pass.</p>

username: albanD
cooked: <p>Why do you need to do a custom Function for this? Why not a regular function with your modification followed by a regular convolution?</p>

username: aashaka
cooked: <p>If I understand correctly, won’t this mean that both forward and backward pass use the modified input? I want the forward pass to be calculated on the original inputs and the backward pass gradients calculated with the help of modified inputs.</p>

username: albanD
cooked: <p>In that case, yes you can use that function.<br>
I would advise copying it into your code. Or the subset you need. As the one in F.grad is going to be removed in future version.</p>