How to save the quantized model?

username: RichardMrLu
cooked: <p>I used linear quantization, but the quantized model’s size unchanged,It seems that ‘torch.save()’ still save weights in float format…<br>
How to save the quantized weights? I am really appreciate your help.</p>

username: Hyer_Chen
cooked: <p>Have solve the problem? Or any idea to do quantization with pytorch?</p>

username: RichardMrLu
cooked: <p>no… I quantized the model to 2 bit but it is still save in 32bit</p>

username: dlmacedo
cooked: <p>Can tou provide the github link to the code to allow us to help?</p>

username: indrajitsg
cooked: <p>I have attempted this and am facing the same issues. I used the approach from the following repo:</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/aaron-xichen/pytorch-playground/blob/master/quantize.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/aaron-xichen/pytorch-playground/blob/master/quantize.py" target="_blank" rel="nofollow noopener">aaron-xichen/pytorch-playground/blob/master/quantize.py</a></h4>
<pre><code class="lang-py">import argparse
from utee import misc, quant, selector
import torch
import torch.backends.cudnn as cudnn
cudnn.benchmark =True
from collections import OrderedDict

parser = argparse.ArgumentParser(description='PyTorch SVHN Example')
parser.add_argument('--type', default='cifar10', help='|'.join(selector.known_models))
parser.add_argument('--quant_method', default='linear', help='linear|minmax|log|tanh')
parser.add_argument('--batch_size', type=int, default=100, help='input batch size for training (default: 64)')
parser.add_argument('--gpu', default=None, help='index of gpus to use')
parser.add_argument('--ngpu', type=int, default=8, help='number of gpus to use')
parser.add_argument('--seed', type=int, default=117, help='random seed (default: 1)')
parser.add_argument('--model_root', default='~/.torch/models/', help='folder to save the model')
parser.add_argument('--data_root', default='/tmp/public_dataset/pytorch/', help='folder to save the model')
parser.add_argument('--logdir', default='log/default', help='folder to save to the log')

parser.add_argument('--input_size', type=int, default=224, help='input size of image')
parser.add_argument('--n_sample', type=int, default=20, help='number of samples to infer the scaling factor')
</code></pre>

  This file has been truncated. <a href="https://github.com/aaron-xichen/pytorch-playground/blob/master/quantize.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>When I try to save the model with torch.save the file size does not show any decrease.</p>

username: indrajitsg
cooked: <p>Hi Richard - were you able to quantize your PyTorch models successfully?</p>

username: raghuramank100
cooked: <p>With quantization support in pytorch 1.3, this should work if you follow the flow in the pytorch tutorials for quantization: <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html</a></p>