How to set quantization aware training scaling factors?

sunkr1995: <p>when i use quantization aware training , The weight tensor scaling factors is a standard floating point number.<br>
I want to convert my model as 8bit at FPGA, so the weight tensor scaling factor must be an integer power-of-two value exponent. Is there such an option? what should I do</p>

robotcator123: <p>It seems that the quantization scheme is a little bit different. You can see from this <a href="https://github.com/pytorch/pytorch/wiki/torch_quantization_design_proposal" rel="nofollow noopener">https://github.com/pytorch/pytorch/wiki/torch_quantization_design_proposal</a></p>

dskhudia: <p>Depending on the fixed-point arithmetic you use, you can convert float multiplier to quantized_multiplier (integer) and right shift (integer). Please checkout <a href="https://github.com/pytorch/FBGEMM/blob/master/src/QuantUtils.cc#L107-L157" rel="nofollow noopener">https://github.com/pytorch/FBGEMM/blob/master/src/QuantUtils.cc#L107-L157</a></p>

bigtree: <p><a class="mention" href="/u/sunkr1995">@sunkr1995</a><br>
I am facing the same issue. Did you find a way to do that?<br>
Thanks</p>

aimen123: <p>do you find a way to do that?</p>

aimen123: <p>do you have a good way to set scale —&gt; 2^n</p>

jerryzh168: <p>you’ll need to implement your own fake quantize module: <a href="https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fake_quantize.py" class="inline-onebox" rel="noopener nofollow ugc">pytorch/fake_quantize.py at master · pytorch/pytorch · GitHub</a> to restrict the scaling factor to power of two, we had an intern recently implemented additive power of two actually: <a href="https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/experimental/fake_quantize.py#L7" class="inline-onebox" rel="noopener nofollow ugc">pytorch/fake_quantize.py at master · pytorch/pytorch · GitHub</a>,<br>
the code for using it in the flow can be found in <a href="https://github.com/pytorch/pytorch/blob/master/test/quantization/core/experimental/apot_fx_graph_mode_qat.py" class="inline-onebox" rel="noopener nofollow ugc">pytorch/apot_fx_graph_mode_qat.py at master · pytorch/pytorch · GitHub</a><br>
paper: <a href="https://arxiv.org/abs/1909.13144" class="inline-onebox" rel="noopener nofollow ugc">[1909.13144] Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks</a></p>
<p>for converting the model to 8bit FPGA, I think you might need to follow the reference flow, which is only available in fx graph mode quantization right now, please take a look at <a href="https://github.com/pytorch/rfcs/blob/master/RFC-0019-Extending-PyTorch-Quantization-to-Custom-Backends.md" class="inline-onebox" rel="noopener nofollow ugc">rfcs/RFC-0019-Extending-PyTorch-Quantization-to-Custom-Backends.md at master · pytorch/rfcs · GitHub</a>, you will get a model with q/dq/fp32 ops which represents a quantized model, and you can lower the model to FPGA (I guess you need to expose ops implemented in FPGA in pytorch?) lowering code for native pytorch backend (fbgemm/qnnpack) can be found in <a href="https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/_lower_to_native_backend.py" class="inline-onebox" rel="noopener nofollow ugc">pytorch/_lower_to_native_backend.py at master · pytorch/pytorch · GitHub</a></p>