How to avoid Quantization warning: "Must run observer before calling calculate_qparams."?

username: FabianSchuetze
cooked: <p>I worked with the pytorch tutorial for <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" rel="nofollow noopener">static quantization</a> and when running the line:</p>
<pre><code class="lang-auto">torch.quantization.convert(per_channel_quantized_model, inplace=True)
</code></pre>
<p>I receive the following warning:</p>
<pre><code class="lang-auto">.../torch/quantization/observer.py:845: 
UserWarning: must run observer before calling calculate_qparams. Returning default scale and zero point 
</code></pre>
<p>I call the <code>convert</code> function within the following lines of code:</p>
<pre><code class="lang-auto">per_channel_quantized_model = load_model(..)
per_channel_quantized_model.eval()
per_channel_quantized_model.fuse_model()
per_channel_quantized_model.qconfig = torch.quantization.get_default_qconfig('fbgemm')
print(per_channel_quantized_model.qconfig)
torch.quantization.prepare(per_channel_quantized_model, inplace=True)
evaluate(per_channel_quantized_model, ...)
torch.quantization.convert(per_channel_quantized_model, inplace=True)
</code></pre>
<p>Does somebody have an idea what the warning means and how I can avoid that? I appreciate any hints and suggestions!</p>

username: khizar-anjum
cooked: <p>Facing the same issue. <code>torch.quantization.convert</code> is supposed to run the observers, right. This warning does not make sense.</p>

username: supriyar
cooked: <p>The prepare script inserts the observers. After that when model forward is run it also runs the observers.<br>
If you call <code>convert</code> without calling prepare then can complain about not running observers.</p>
<p>Which model are you running this on? We can take a look if there is a repro.</p>

username: FabianSchuetze
cooked: <p>Thanks for your replies, <a class="mention" href="/u/khizar-anjum">@khizar-anjum</a> and <a class="mention" href="/u/supriyar">@supriyar</a>!</p>
<p>After <a class="mention" href="/u/khizar-anjum">@khizar-anjum</a> comments, I also filed a <a href="https://github.com/pytorch/pytorch/issues/40278" rel="nofollow noopener">issue</a>  on github. The warning is thrown when running the static quantization tutorial. I also received the warning in a SSD-type model I wrote. The quantization lead to a low accuracy and I began asking myself if it was caused by the improper quantization the observer warns against.</p>

username: blueskywwc
cooked: <p>Hello, I also encountered this problem, is there any latest solution, thank you!</p>

username: FabianSchuetze
cooked: <p>see <a href="https://github.com/pytorch/pytorch/issues/40278" rel="nofollow noopener">here</a> for a solution.</p>

username: blueskywwc
cooked: <p>Replace self.skip_add.add with torch.add</p>
<p>class InvertedResidual(nn.Module):<br>
def <strong>init</strong>(self,in_channel,out_channel,stride,expand_ratio):<br>
super(InvertedResidual, self).<strong>init</strong>()<br>
hidden_channel=int(round(in_channel*expand_ratio))<br>
self.shortcut=stride==1 and in_channel==out_channel</p>
<pre><code>    layers=[]
    if expand_ratio!=1:
        #1x1 pointwise conv
        layers.append(ConvBnRelu(in_channel,hidden_channel,kersize=1))
    layers.extend([
        # 3x3 depthwise conv
        ConvBnRelu(hidden_channel,hidden_channel,stride=stride,groups=hidden_channel),
        nn.Conv2d(hidden_channel,out_channel,kernel_size=1,bias=False),
        nn.BatchNorm2d(out_channel),
    ])

    self.conv=nn.Sequential(*layers)
    #self.skip_add = nn.quantized.FloatFunctional()

def forward(self,x):
    if self.shortcut:
        #return self.skip_add.add(x,self.conv(x))
        return torch.add(x,self.conv(x))
    else:
        return self.conv(x)
</code></pre>
<p>RuntimeError: Could not run ‘aten::add.Tensor’ with arguments from the ‘QuantizedCPUTensorId’ backend. ‘aten::add.Tensor’ is only available for these backends: [CPUTensorId, MkldnnCPUTensorId, SparseCPUTensorId, VariableTensorId].</p>
<p>Where did I write it wrong? thanks!</p>

username: Ashar_Ali
cooked: <p>I am still facing the same issue even after following the instructions present in the link you shared. Are there any further updates to it?</p>