Static quantizing and batch norm error (could not run aten::native_batch_norm with args from QuantCPUTensorid backend')

lessw2020: <p>Hi all,</p>
<p>Working on static quantizing a few models and hitting this error on a basic Resnet18 - any insight into what is missing to complete the quantization?<br>
Did not ‘fuse’ the BN but unclear if that is the core issue?<br>
Any assistance would be appreciated - very hard to find much documentation.</p>
<pre><code class="lang-auto">/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in batch_norm(input, running_mean, running_var, weight, bias, training, momentum, eps)
   1921     return torch.batch_norm(
   1922         input, weight, bias, running_mean, running_var,
-&gt; 1923         training, momentum, eps, torch.backends.cudnn.enabled
   1924     )
   1925 
RuntimeError: Could not run 'aten::native_batch_norm' with arguments from the 'QuantizedCPUTensorId' backend. 'aten::native_batch_norm' is only available for these backends: [CPUTensorId, CUDATensorId, MkldnnCPUTensorId, VariableTensorId].


</code></pre>
<p>Thanks</p>

hx89: <p>I think you need to fuse BN since there’s no quantized BN layer.</p>

lessw2020: <p>Thanks very much - will try to fuse!  The docs implied it was more to boost accuracy vs a requirement but makes that it won’t otherwise quantize itself so to speak.</p>

lessw2020: <p>That was in fact the issue (lack of fusing).  Thanks very much <a class="mention" href="/u/hx89">@hx89</a>!</p>

blueskywwc: <p>There is no convolutional layer in front, I want to know how to merge the bn layer，thanks!</p>
<p>class MobileFaceNet(Module):<br>
def <strong>init</strong>(self, embedding_size):<br>
super(MobileFaceNet, self).<strong>init</strong>()<br>
self.conv1 = Conv_Block(3, 64, kernel=3, stride=2, padding=1)<br>
self.conv2 = Conv_Block(64, 64, kernel=3, stride=1, padding=1, groups=64)<br>
self.conv3 = InvertedResidual(64, 64, kernel=3, stride=2, padding=1, groups=128)<br>
self.conv4 = MakeBlocks(64, num_block=4, kernel=3, stride=1, padding=1, groups=128)<br>
self.conv5 = InvertedResidual(64, 128, kernel=3, stride=2, padding=1, groups=256)<br>
self.conv6 = MakeBlocks(128, num_block=6, kernel=3, stride=1, padding=1, groups=256)<br>
self.conv7 = InvertedResidual(128, 128, kernel=3, stride=2, padding=1, groups=512)<br>
self.conv8 = MakeBlocks(128, num_block=2, kernel=3, stride=1, padding=1, groups=256)<br>
self.conv9 = Conv_Block(128, 512, kernel=1, stride=1, padding=0)<br>
self.conv10 = Conv_Block(512, 512, kernel=7, stride=1, padding=0, groups=512, is_linear=True)<br>
self.ft = Flatten()<br>
self.ln = Linear(512, embedding_size, bias=False)<br>
self.bn = BatchNorm1d(embedding_size)</p>
<pre><code>    self.quant = QuantStub()
    self.dequant = DeQuantStub()

    for m in self.modules():
        if isinstance(m, Conv2d):
            n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
            m.weight.data.normal_(0, math.sqrt(2. / n))
        elif isinstance(m, BatchNorm2d):
            m.weight.data.fill_(1)
            m.bias.data.zero_()

def forward(self, x):
    x = self.quant(x)
    x = self.conv1(x)
    x = self.conv2(x)
    x = self.conv3(x)
    x = self.conv4(x)
    x = self.conv5(x)
    x = self.conv6(x)
    x = self.conv7(x)
    x = self.conv8(x)
    x = self.conv9(x)
    x = self.conv10(x)
    x = self.ft(x)
    x = self.ln(x)
    x = self.bn(x)
    x = self.dequant(x)
    return x

def fuse_model(self):
    for m in self.modules():
        if type(m) == Conv_Block:
            m.fuse_model()</code></pre>