Any method to optimize graph for QAT model?

username: icyhearts
cooked: <p>Hello!<br>
I want to deploy a pytorch model trained in quantization-aware training method and then convert the model to other inference deep learning frame works(such as MACE) to deploy on mobile devices.<br>
I am using a model based on <code>https://github.com/pytorch/vision/blob/master/torchvision/models/mobilenet.py</code><br>
my modification is:<br>
1: change change <code>bias=False</code> to <code>bias=True</code><br>
2: change <code>out_features</code> of classifier to 20 because I have only 20 classes.<br>
I use torch.jit to trace my model and save it.<br>
The traced model is available in:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/icyhearts/mobilenetv2_bias/blob/master/33.pt">
  <header class="source">

      <a href="https://github.com/icyhearts/mobilenetv2_bias/blob/master/33.pt" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/icyhearts/mobilenetv2_bias/blob/master/33.pt" target="_blank" rel="noopener nofollow ugc">icyhearts/mobilenetv2_bias/blob/master/33.pt</a></h4>


  This file is binary. <a href="https://github.com/icyhearts/mobilenetv2_bias/blob/master/33.pt" target="_blank" rel="noopener nofollow ugc">show original</a>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I find the graph created by jit is not optimized:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/c/cc188f75bb51914c95e6cc556d591403c777aceb.png" data-download-href="https://discuss.pytorch.org/uploads/default/cc188f75bb51914c95e6cc556d591403c777aceb" title="graph"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc188f75bb51914c95e6cc556d591403c777aceb_2_690x473.png" alt="graph" data-base62-sha1="t7w1vEQ9a0YDNVtrNMeu2ifLZdp" width="690" height="473" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc188f75bb51914c95e6cc556d591403c777aceb_2_690x473.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc188f75bb51914c95e6cc556d591403c777aceb_2_1035x709.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc188f75bb51914c95e6cc556d591403c777aceb_2_1380x946.png 2x" data-dominant-color="EEEEEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">graph</span><span class="informations">1411×968 122 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The consants are not folded so there are <code>add</code>, <code>sqrt</code>, <code>div</code> and <code>mul</code> before conv layer,  the bias of conv layer is added twice, <code>BN</code> parameters are not folded into <code>Conv</code> layer.</p>
<p>Is there any suggestion to optimize a jit model to do these folds?<br>
I want to deploy the model in mobile device so these folds can be useful.</p>
<p>Many thanks.</p>

username: dskhudia
cooked: <p>Did you quantize the model before transferring? See here <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html#quantization-aware-training" rel="noopener nofollow ugc">https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html#quantization-aware-training</a> for the steps.</p>