Pytorch1.3 Quantization for resnet50, accuracy is zero after fused_modules

Tiru_B: <p>Hi</p>
<p>I am experimenting pytorch 1.3 Quantization for resent50. I had taken the pre-trained model from model zoo.</p>
<p>Please find below for accuracy (for 100 images) and size at different stages of my experiment</p>
<pre><code class="lang-auto">Size (MB): 102.491395
{"metric": "original_resnet50_val_accuracy", "value": 93.75}
Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
Size (MB): 102.145116
{"metric": "fused_resnet50_val_accuracy", "value": 0.0}
ConvReLU2d(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3))
  (1): ReLU()
)

{"metric": "quntize_per_tensor_resent50_val_accuracy", "value": 0.0}
{"metric": "quntize_per_channel_resent50_val_accuracy", "value": 0.0}
Size (MB): 25.65341
QuantizedConvReLU2d(3, 64, kernel_size=(7, 7), stride=(2, 2), scale=1.0, zero_point=0, padding=(3, 3))
Size (MB): 25.957137
QuantizedConvReLU2d(3, 64, kernel_size=(7, 7), stride=(2, 2), scale=1.0, zero_point=0, padding=(3, 3))
</code></pre>
<p>Not sure Where it went wrong</p>
<p>PFB code for fusing layers</p>
<pre><code class="lang-auto">def fuse_model(m):
    
    modules_to_fuse = [['conv1','bn1',"relu"]]
    torch.quantization.fuse_modules(m, modules_to_fuse,inplace=True)
   
    for mod in m.layer1:
        torch.quantization.fuse_modules(mod, [["conv1","bn1"]],inplace=True)
        torch.quantization.fuse_modules(mod, [["conv2","bn2"]],inplace=True)
        torch.quantization.fuse_modules(mod, [["conv3","bn3","relu"]],inplace=True)
        if mod.downsample:
            torch.quantization.fuse_modules(mod.downsample, [["0","1"]],inplace=True)

    for mod in m.layer2:
        torch.quantization.fuse_modules(mod, [["conv1","bn1"]],inplace=True)
        torch.quantization.fuse_modules(mod, [["conv2","bn2"]],inplace=True)
        torch.quantization.fuse_modules(mod, [["conv3","bn3","relu"]],inplace=True)
        if mod.downsample:
            torch.quantization.fuse_modules(mod.downsample, [["0","1"]],inplace=True)

    for mod in m.layer3:
        torch.quantization.fuse_modules(mod, [["conv1","bn1"]],inplace=True)
        torch.quantization.fuse_modules(mod, [["conv2","bn2"]],inplace=True)
        torch.quantization.fuse_modules(mod, [["conv3","bn3","relu"]],inplace=True)
        if mod.downsample:
            torch.quantization.fuse_modules(mod.downsample, [["0","1"]],inplace=True)

    for mod in m.layer4:
        torch.quantization.fuse_modules(mod, [["conv1","bn1"]],inplace=True)
        torch.quantization.fuse_modules(mod, [["conv2","bn2"]],inplace=True)
        torch.quantization.fuse_modules(mod, [["conv3","bn3","relu"]],inplace=True)
        if mod.downsample:
            torch.quantization.fuse_modules(mod.downsample, [["0","1"]],inplace=True)

    return m 
</code></pre>

Tiru_B: <p>modified My fused_model function now I am seeing same accuracy as non fused model but, Quantiation accuracy is zero.</p>
<pre><code class="lang-auto">Size (MB): 102.491395
{"metric": "original_resnet50_val_accuracy", "value": 93.75}
Size (MB): 102.143772
{"metric": "fused_resnet50_val_accuracy", "value": 93.75}
{"metric": "quntize_per_tensor_resent50_val_accuracy", "value": 0.0}
{"metric": "quntize_per_channel_resent50_val_accuracy", "value": 0.0}
Size (MB): 25.653416
Size (MB): 25.957149

</code></pre>
<p>Change in fused function is , I have added relu in layer 1 to 4</p>

raghuramank100: <p>Its great to see that the accuracy post fusion is high. What are you using for calibration before you quantize the model?</p>

Tiru_B: <p>issue with calibration data set, I have selected randomly 1024 training samples from imagenet dataset. Now I can I see some good results.</p>
<pre><code class="lang-auto">Size (MB): 102.491395
{"metric": "original_resnet50_val_accuracy", "value": 90.234375}
Size (MB): 102.143772
{"metric": "fused_resnet50_val_accuracy", "value": 90.234375}
calibration
{"metric": "quntize_per_tensor_resent50_val_accuracy", "value": 78.22265625}
after quantization
{"metric": "quntize_per_tensor_resent50_val_accuracy", "value": 88.28125}
calibration
{"metric": "quntize_per_tensor_resent50_val_accuracy", "value": 76.26953125}
after quantization
{"metric": "quntize_per_channel_resent50_val_accuracy", "value": 89.84375}
size of quantization per tensor model
Size (MB): 25.653446
size of quantization per channel model
Size (MB): 25.957137

</code></pre>

studML: <p>Hey Tiru, can you please share your full code because I am facing same issue.</p>

khizar-anjum: <p>Hi <a class="mention" href="/u/tiru_b">@Tiru_B</a>, can you share your code as I am having the same issue inspite of adding a relu layer.</p>

Tiru_B: <p>U can check in tiru1930  github  user Id</p>

Tiru_B: <aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">
      <a href="https://github.com/tiru1930/resnet_quantization" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars3.githubusercontent.com/u/12211287?s=400&amp;v=4" class="thumbnail onebox-avatar" width="400" height="400">

<h3><a href="https://github.com/tiru1930/resnet_quantization" target="_blank" rel="nofollow noopener">tiru1930/resnet_quantization</a></h3>

<p>Contribute to tiru1930/resnet_quantization development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>