Updating an instance attribute within a custom Conv2D

Niamul_Quader: <p>Hello,</p>
<p>I am trying to modify a custom Conv2D’s custom attribute, but failing as of now.</p>
<p>For example, in the following custom Conv2D which inherits from PyTorch’s nn.Conv2d, I have a ‘count’ attribute that should get counted up each time the forward function in an instantiated Conv2d object is called. However, the value of count resets to 0 (i.e. the printed output is always 0) when the forward function is invoked every time. Can you please point out what I am doing wrong here?</p>
<p>Thanks in advance!</p>
<p>Code snippet:<br>
class Conv2d(nn.Conv2d):</p>
<pre><code>def __init__(self, in_channels, out_channels, kernel_size, stride=1,
             padding=0, dilation=1, groups=1, bias=True):
    super(Conv2d, self).__init__(in_channels, out_channels, kernel_size, stride,
             padding, dilation, groups, bias)

    self.count = 0

def forward(self, x):
    weight = self.weight
    print(self.count)  # always prints 0
    self.count += 1
    return F.conv2d(x, weight, self.bias, self.stride,
                    self.padding, self.dilation, self.groups)</code></pre>

Eta_C: <p>I run your code and it prints 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.</p>
<pre><code class="lang-python">import torch.nn as nn
import torch.nn.functional as F

class Conv2d(nn.Conv2d):
    def __init__(self, in_channels, out_channels, kernel_size, stride=1,
             padding=0, dilation=1, groups=1, bias=True):
        super(Conv2d, self).__init__(in_channels, out_channels, kernel_size, stride,
             padding, dilation, groups, bias)

        self.count = 0

    def forward(self, x):
        weight = self.weight
        print(self.count)  # always prints 0
        self.count += 1
        return F.conv2d(x, weight, self.bias, self.stride,
                    self.padding, self.dilation, self.groups)
        
net = Conv2d(3, 3, 3)
for _ in range(10):
    x = torch.rand(3, 3, 3, 3)
    net(x)
</code></pre>

Niamul_Quader: <p>Thank you for looking into this, Eta_C.</p>
<p>You are right - it counts up with CPU models, but this seems to break for the same model with DataParallel and CUDA. Please see the code snippet below.</p>
<p>import torch.nn<br>
import torch.nn as nn<br>
import torch.nn.functional as F</p>
<p>class Conv2d(nn.Conv2d):<br>
def <strong>init</strong>(self, in_channels, out_channels, kernel_size, stride=1,<br>
padding=0, dilation=1, groups=1, bias=True):<br>
super(Conv2d, self).<strong>init</strong>(in_channels, out_channels, kernel_size, stride,<br>
padding, dilation, groups, bias)</p>
<pre><code>    self.count = 0

def forward(self, x):
    weight = self.weight
    print(self.count)  # always prints 0
    self.count += 1
    return F.conv2d(x, weight, self.bias, self.stride,
                self.padding, self.dilation, self.groups)
</code></pre>
<p>net = Conv2d(3, 3, 3)<br>
net = torch.nn.DataParallel(net).cuda()<br>
for _ in range(10):<br>
x = torch.rand(3, 3, 3, 3)<br>
x = x.cuda()<br>
net(x)</p>

Eta_C: <p>You are right. I found that I have to use <code>multiprocessing</code>.</p>
<p>The following code will work but,</p>
<ul>
<li>it is not elegant and generic enough</li>
<li>If I have 2 GPUs, it will output 1 to 20</li>
<li>I don’t know if there are any other side effects</li>
</ul>
<pre><code class="lang-python">import torch
import torch.nn as nn
import torch.nn.functional as F
from torch import multiprocessing as mp

class Conv2d(nn.Conv2d):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.cnt = mp.Value("d", 0)
    
    def forward(self, x):
        self.cnt.value += 1
        return super().forward(x)

net = Conv2d(3, 3, 3)
net = nn.DataParallel(net).cuda()
for _ in range(10):
    x = torch.rand(3, 3, 3, 3)
    x = x.cuda()
    net(x)
</code></pre>

Niamul_Quader: <p>Thank you very much for looking into this. The solution you mentioned works for me. I do hope though that later PyTorch builds will have a more generic solution to this problem.</p>
<p>Thank you once again for helping out.</p>