Reproduce qconv kernel for x86

Gurkirt: <p>Hi am using <a href="https://pytorch.org/tutorials/prototype/pt2e_quant_x86_inductor.html?highlight=x86" rel="noopener nofollow ugc">x86_inductor_qunatizer</a> following the given tutorial. I want to implement this on FPGA. However, I am having trouble find x86 kernel c++/python implementation of these quantized qconv which are suppose to be defined in torch.ops.qconv but going <a href="https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/quantized/library.cpp" rel="noopener nofollow ugc">/ATen/native/quantized/library.cpp</a> wasn’t much helpful.</p>
<p>Here is my understanding of the whole quantized conv execution for x86_inductor.<br>
Data types:<br>
weights: qint8 [-128 to 127]<br>
input output: qunit8 [0-255]<br>
Bias: float32 converted to bias_q: vector<br>
Multiplication: int32<br>
Accumulation: int32<br>
Bias addition: int32<br>
input scale: float32<br>
input zero point: int32<br>
per channel scales: vector<br>
output scale: float32<br>
output zero point: int32</p>
<p>I used symmetric quantization so all zeros points are actually zero.</p>
<p>Here is python version in <a href="https://gist.github.com/gurkirt/98b4c12c9618687e3d9262cea27fef70" rel="noopener nofollow ugc">this gist</a>. It include my understanding of bias_q from  <a href="https://discuss.pytorch.org/t/is-bias-quantized-while-doing-pytorch-static-quantization/146416/6">this thread</a></p>
<p>Can someone please help me sort this out, either by helping me verify the gist, i.e. ideas how can I make sure it mimics x86 kernel implementation so I can write FPGA code for it OR point me to kernel implementation of it.</p>
<p>I want to use x86 unlike Xnnpack because it produces [0-255] outputs so that make my life easier as my activation function is ReLU.</p>
<p>Many thanks in advance.<br>
Gurkirt</p>

jerryzh168: <p>x86 conv ops can be found here I think: <a href="https://github.com/pytorch/pytorch/blob/dcfa415e6e10b250e56a1793d45e886fd910358e/aten/src/ATen/native/quantized/library.cpp#L255-L277" class="inline-onebox" rel="noopener nofollow ugc">pytorch/aten/src/ATen/native/quantized/library.cpp at dcfa415e6e10b250e56a1793d45e886fd910358e · pytorch/pytorch · GitHub</a>, tests can be found <a href="https://github.com/pytorch/pytorch/blob/dcfa415e6e10b250e56a1793d45e886fd910358e/test/quantization/core/test_quantized_op.py#L6858" class="inline-onebox" rel="noopener nofollow ugc">pytorch/test/quantization/core/test_quantized_op.py at dcfa415e6e10b250e56a1793d45e886fd910358e · pytorch/pytorch · GitHub</a></p>

Gurkirt: <p>thank you <a class="mention" href="/u/jerryzh168">@jerryzh168</a>  for your answer. I see the test.py and librarry.cpp points to wrapper for backend kernel, I want to see the how backend implements qconv?<br>
Would be right to say that x86 backend uses onednn backend and compute call goes here <a href="https://github.com/oneapi-src/oneDNN/blob/main/src/cpu/gemm_x8s8s32x_convolution.cpp#L437" class="inline-onebox" rel="noopener nofollow ugc">oneDNN/src/cpu/gemm_x8s8s32x_convolution.cpp at main · oneapi-src/oneDNN · GitHub</a> ?</p>

Gurkirt: <p>Hi <a class="mention" href="/u/jerryzh168">@jerryzh168</a> I made some progress. I am getting almost the same result as one would get after executing graph module.<br>
However, manual quantization execution outputs are on sum:0.1534857451915741 mean: 0.00213174638338387 max: 0.006238460540771484 min: 3.343820571899414e-05 different per output pixel with bias_quantized = True.</p>
<p>Can you please have a look at <a href="https://gist.github.com/gurkirt/98b4c12c9618687e3d9262cea27fef70" rel="noopener nofollow ugc">this gist</a> and see if there is mistake or there some something in gemm that might be causing this difference. It works with pytorch2.4.0+cpu.</p>
<p>I set the quantized bias to be True as it this is what you mentioned earlier in another <a href="https://discuss.pytorch.org/t/is-bias-quantized-while-doing-pytorch-static-quantization/146416/6">thread.</a></p>

Gurkirt: <p>If anyone is stubling here, I think I figured it out, you can have look at <a href="https://github.com/gurkirt/pt2e_quantize_bias" class="inline-onebox" rel="noopener nofollow ugc">GitHub - gurkirt/pt2e_quantize_bias</a> for answer/tests.</p>