Changing bias sign in Conv2d from positive to negative in the source code will affect convbackward?

aakash_saboo: <pre><code class="lang-auto">    def __init__(self, in_channels, out_channels, kernel_size, stride=1,
                 padding=0, dilation=1, groups=1,
                 bias=True, padding_mode='zeros'):
        kernel_size = _pair(kernel_size)
        stride = _pair(stride)
        padding = _pair(padding)
        dilation = _pair(dilation)
        super(Conv2d, self).__init__(
            in_channels, out_channels, kernel_size, stride, padding, dilation,
            False, _pair(0), groups, bias, padding_mode)

    def conv2d_forward(self, input, weight):
        if self.padding_mode == 'circular':
            expanded_padding = ((self.padding[1] + 1) // 2, self.padding[1] // 2,
                                (self.padding[0] + 1) // 2, self.padding[0] // 2)
            return F.conv2d(F.pad(input, expanded_padding, mode='circular'),
                            weight, self.bias, self.stride,
                            _pair(0), self.dilation, self.groups)
        return F.conv2d(input, weight, self.bias, self.stride,
                        self.padding, self.dilation, self.groups)

    def forward(self, input):
        return self.conv2d_forward(input, self.weight)
</code></pre>
<p>this is the official Conv2d function.</p>
<p>If we change the sign of “self.bias” in 4th to last line i.e<br>
from</p>
<pre><code class="lang-auto">return F.conv2d(input, weight, self.bias, self.stride,
                        self.padding, self.dilation, self.groups)
</code></pre>
<p>to</p>
<pre><code class="lang-auto">return F.conv2d(input, weight, (-1*self.bias), self.stride,
                        self.padding, self.dilation, self.groups)
</code></pre>
<p>will this affect the backward function?</p>

tom: <p>Given that the formula is  <code>conv_out = input *correl* weight + bias</code> (<code>*correl*</code> being the correlation operator), this will not change the ∂<code>conv_out</code> / ∂<code>weight</code> nor the ∂<code>conv_out</code> / ∂<code>input</code>. It will flip the sign in ∂<code>conv_out</code> / ∂<code>bias</code>. It will almost certainly change ∂<code>loss</code>  / ∂<code>conv_out</code> as that now is computed at a totally different location.</p>
<p>Best regards</p>
<p>Thomas</p>

aakash_saboo: <p>Hey Thomas<br>
Thanks for your quick response.<br>
I realised that the sign of ∂ <code>conv_out</code>  / ∂ <code>bias</code>  SHOULD change according to the equation <code>conv_out = input *correl* weight + bias</code></p>
<p>but what I want to know is- if i make a new function <code>Conv2d_modified</code> with the mentioned changes and then when I call <code>backward()</code> on the loss function ,then will the REQUIRED sign change of ∂ <code>conv_out</code>  / ∂ <code>bias</code> will take place?</p>

tom: <p>Yeah, so PyTorch will track your computation and then backpropagate through it. The idea of the functional interface is that you can do wild computation around the standard functions.<br>
For example, <a class="mention" href="/u/ptrblck">@ptrblck</a> and I did this when we implemented the <a href="https://github.com/lernapparat/lernapparat/blob/master/style_gan/pytorch_style_gan.ipynb" rel="nofollow noopener">style GAN’s weight-scaling</a> for convolutions (and linear layers, which have the commentary around it).</p>
<p>Best regards</p>
<p>Thomas</p>

aakash_saboo: <p>Thanks Thomas<br>
appreciate your quick responses. My doubt is cleared now.</p>