Quantization with custom weights in nn.Conv2d()

louis: <p>I want to quantize a model that I have created that uses a custom <code>Parameter</code> to hold the weights of several <code>Conv2d()</code> layers. I pass this <code>Parameter</code> to the forward function, which then assigns the different parts of the <code>Parameter</code> to the weights of the <code>Conv2d()</code> layers, which requires a cast to <code>Parameter</code> on every forward function call. This works fine for normal use, if inefficient, but when I want to use the Quantization package, the assignment throws this error:</p>
<blockquote>
<p>KeyError: “attribute ‘weight’ already exists”</p>
</blockquote>
<p>When I don’t use quantization, I can use the functional <code>conv2d()</code>, but I don’t think that is supported yet with nn.quantize. What’s the difference between a quantized model and a regular model, and how can I fix this error?</p>
<p>Thanks</p>

louis: <p>Here is a minimal example:</p>
<pre><code class="lang-auto">
import torch
import torch.nn as nn
from torch.quantization import QuantStub, DeQuantStub
import copy

class Block(nn.Module):
    def __init__(self):
        super(Block, self).__init__()
        self.conv1 = nn.Conv2d(1,1,3, padding=1, bias=False)
    def forward(self, x, conv_parameter):
        self.conv1.weight = torch.nn.parameter.Parameter(conv_parameter) 
        return self.conv1(x)
    
class BigBlock(nn.Module):
    def __init__(self):
        super(BigBlock, self).__init__()
        self.conv_parameter = torch.nn.parameter.Parameter(torch.rand((1,1,3,3)))
        self.block = Block()
        self.quant = QuantStub()
        self.dequant = DeQuantStub()
    def forward(self, x):
        x = self.quant(x)
        x = self.block(x, self.conv_parameter)
        x = self.dequant(x)
        return x

x = torch.rand((1,1,5,5))
net = BigBlock()
print(net(x)) # works fine

qnet = copy.deepcopy(net)
qnet = qnet.eval()
qnet.qconfig = torch.quantization.default_qconfig
torch.quantization.prepare(qnet, inplace=True)
net(torch.rand((1,1,5,5)))
torch.quantization.convert(qnet, inplace=True)
print(qnet)
print(qnet(x)) # throws an error

</code></pre>
<p>I am using pytorch-1.6.0.dev202, Python 3.8.1.</p>

louis: <p>So the issue is that I was passing <code>floatTensor</code> to a now <code>torch.qint8</code>, and attempting to cast it to a <code>Parameter</code>. The solution is to put an if statement like so:</p>
<pre><code class="lang-auto">class Block(torch.nn.Module):
    def __init__(self):
        super(Block, self).__init__()
        self.conv1 = torch.nn.Conv2d(1,1,3, padding=1, bias=False)
    def forward(self, x, conv_parameter):
        if(isinstance(self.conv1, torch.nn.quantized.Conv2d)):
            return self.conv1(x)
        else:
            self.conv1.weight = torch.nn.parameter.Parameter(conv_parameter) 
        return self.conv1(x)
</code></pre>
<p>which works since my passed in <code>conv_parameter</code> should be equal to the weight stored in <code>self.conv1</code>, so we can just ignore the conversion to Parameter when using the quantized network.</p>

aryan_461: <p>Does this code reduce the model size? I implement your code on my model and it doesn’t change my model size.</p>