TypeError: quantized_add() missing 2 required positional arguments: 'op_scale' and 'op_zero_point'

username: ramson_k
cooked: <p>Hi I’m trying to export the quantized model to onnx. But im facing the following error while doing so:<br>
<strong>TypeError: quantized_add() missing 2 required positional arguments: ‘op_scale’ and ‘op_zero_point’</strong></p>
<p>Here is my skeleton code:</p>
<pre><code class="lang-auto">model = # my_model
model.eval()
from torch.quantization import quantize_fx
qconfig_dict = {"": torch.quantization.get_default_qconfig("qnnpack")}

model_prepared = quantize_fx.prepare_fx(model, qconfig_dict,(image_tensor[None],))

model_prepared.eval()
with torch.no_grad():
    for i in range(0,len(eval_ds)):
        image_tensor = eval_ds[i]
        model_prepared(image_tensor[None])

model_int8 = torch.quantization.quantize_fx.convert_fx(model_prepared)
torch.onnx.export(model_int8,image_tensor[None],"onnx_model.onnx",verbose=True,opset_version=15)

</code></pre>
<p>Can anyone please help me out with this?<br>
Thanks</p>

username: supriyar
cooked: <p>at what line do you see the error? If you share a stack trace that can help narrow it down.</p>
<p>Does convert_fx produce a valid quantized model in this case?</p>

username: ramson_k
cooked: <p>The convert_fx produces a GraphModule, which i presume is a valid model. please let me know if there is any other method to verify it.</p>
<p>The error occurs on the onnx export. Here is the stack trace:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/home/econsy/projects/yolov7-exploration/dev/v7/evaluate_quantized_model_US21_char.py", line 279, in &lt;module&gt;
    main()
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/func_to_script/core.py", line 108, in scripted_function
    return func(**args)
  File "/home/econsy/projects/yolov7-exploration/dev/v7/evaluate_quantized_model_US21_char.py", line 238, in main
    torch.onnx.export(model_int8,image_tensor[None],"onnx_model.onnx",verbose=True,opset_version=15)
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 506, in export
    _export(
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 1548, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 1117, in _model_to_graph
    graph = _optimize_graph(
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 665, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 1891, in _run_symbolic_function
    return symbolic_fn(graph_context, *inputs, **attrs)
TypeError: quantized_add() missing 2 required positional arguments: 'op_scale' and 'op_zero_point' 
</code></pre>

username: ramson_k
cooked: <p>Hi <a class="mention" href="/u/supriyar">@supriyar</a> ,<br>
Got any leads from the above stack trace ?</p>
<p>Thanks</p>

username: supriyar
cooked: <p>you can try running the model produced by convert_fx to make sure it is a valid model.</p>
<p>If the issue is specific to onnx lowering you can reach out to the onnx support team for more help, see <a href="https://pytorch.org/docs/stable/quantization.html#frequently-asked-questions" class="inline-onebox" rel="noopener nofollow ugc">Quantization — PyTorch 2.1 documentation</a></p>

username: ramson_k
cooked: <p>Hi <a class="mention" href="/u/supriyar">@supriyar</a> ,<br>
Yes i could run the model produces by convert_fx and able to get the output from the model.</p>
<p>I have created a github issue in onnx repo. Is there any other way to reach out to the support team?</p>

username: parakh08
cooked: <p>it could be possible that there are some float multiplications in the network, make them tensor multiplications, i.e. wrap the float value in torch.tensor()</p>

username: Thanh_vu
cooked: <p>hi, did you solve that problem, i got the same error with you.</p>

username: HDCharles
cooked: <p>i would look in the onnx repo as it sounds like <a class="mention" href="/u/ramson_k">@ramson_k</a> created a github issue there</p>