The ONNX exported by my QAT quantization training does not have a fake operator, the code is as follows?

username: scuizhibin
cooked: <p>import torch<br>
from thop import profile<br>
import torchvision.models as models<br>
import copy<br>
from torch.quantization.quantize_fx import prepare_qat_fx,convert_fx,fuse_fx<br>
import torch.nn as nn</p>
<p>resnet18_model = models.resnet34(pretrained=True)<br>
tensorrt_qconfig = torch.ao.quantization.QConfig(<br>
activation=  torch.ao.quantization.FakeQuantize.with_args(observer= torch.ao.quantization.MinMaxObserver,<br>
dtype=torch.qint8, qscheme=torch.per_tensor_symmetric, reduce_range=False),<br>
weight= torch.ao.quantization.FakeQuantize.with_args(observer= torch.ao.quantization.PerChannelMinMaxObserver,<br>
dtype=torch.qint8, qscheme=torch.per_channel_symmetric, reduce_range=False),<br>
)<br>
def qat_version_model(model):</p>
<pre><code># print(tensorrt_qconfig )

qconfig_dict = {
    # Global Config
    "":  tensorrt_qconfig ,

    # # Disable by layer-name
    # "module_name": [(m, None) for m in disable_layers],

    # Or disable by layer-type
}
example_inputs = (torch.randn(1, 3, 32, 32),)#通过一个例子获取graph 的实例
model_to_quantize = copy.deepcopy(model)

model_fp32_prepared = prepare_qat_fx(
    model_to_quantize, qconfig_dict,    example_inputs )
return model_fp32_prepared
</code></pre>
<p>qat_model=qat_version_model(resnet18_model)<br>
model=convert_fx(qat_model)</p>
<p>input1=torch.ones(1, 3, 32, 32)<br>
torch.onnx.export(model, (input1), “quantized_model.onnx”,<br>
input_names=[‘input1’], output_names=[‘result’],<br>
opset_version=13, export_params=True)</p>
<p>There are no QDQ operators in the exported onnx file，how to solve?</p>

username: jerryzh168
cooked: <p>you don’t need to call convert_fx if you want to get a QAT model for training, but I don’t think you can export the QAT model to onnx though</p>

username: scuizhibin
cooked: <p>Why can’t ONNX export QAT models? Except for the problem there?thank</p>

username: scuizhibin
cooked: <p>I improved the code based on the official example, the code is as follows：</p>
<p>import torch<br>
from thop import profile<br>
import torchvision.models as models<br>
import copy</p>
<p>from torch.ao.quantization.quantize_fx import (<br>
convert_to_reference_fx,<br>
prepare_fx,<br>
prepare_qat_fx,<br>
)<br>
import torch.nn as nn<br>
from torch.quantization.quantize_fx import prepare_qat_fx,convert_fx, prepare_fx</p>
<p>from torch.ao.quantization.backend_config import (<br>
get_tensorrt_backend_config_dict,<br>
ObservationType,<br>
)</p>
<p>resnet18_model = models.resnet34(pretrained=False)<br>
from torch.testing._internal.common_quantization import (<br>
NodeSpec as ns,<br>
QuantizationTestCase,<br>
)</p>
<p>prepare = prepare_qat_fx<br>
trt_qconfig = torch.ao.quantization.QConfig(<br>
activation=torch.ao.quantization.observer.HistogramObserver.with_args(<br>
qscheme=torch.per_tensor_symmetric, dtype=torch.qint8<br>
),<br>
weight=torch.ao.quantization.default_weight_observer,<br>
)<br>
example_inputs = (torch.randn(1, 3, 32, 32),)#<br>
trt_backend_config_dict = get_tensorrt_backend_config_dict()<br>
prepared = prepare(<br>
resnet18_model ,<br>
{“”: trt_qconfig},<br>
example_inputs,<br>
backend_config=trt_backend_config_dict,<br>
)</p>
<p>quantized = convert_to_reference_fx(<br>
prepared,<br>
backend_config=trt_backend_config_dict,<br>
)</p>
<h1><a name="qq_modelconvert_fxquantized-1" class="anchor" href="#qq_modelconvert_fxquantized-1"></a>qq_model=convert_fx(quantized)</h1>
<p>a_test=QuantizationTestCase()</p>
<p>input1=torch.ones(1, 3, 32, 32)<br>
a_test.checkGraphModuleNodes(quantized)</p>
<h1><a name="printquantized-2" class="anchor" href="#printquantized-2"></a>print(quantized)</h1>
<p>torch.onnx.export(quantized , (input1), “quantized_model.onnx”,<br>
input_names=[‘input1’], output_names=[‘result’],<br>
opset_version=11, operator_export_type=torch.onnx.OperatorExportTypes.ONNX_ATEN_FALLBACK)</p>
<p>There was a problem exporting ONNX files：AttributeError: ‘torch.qscheme’ object has no attribute ‘detach’</p>

username: jerryzh168
cooked: <p>ONNX path is not officially supported by us, so you might need to open an issue and tag onnx people, see: <a href="https://pytorch.org/docs/main/quantization.html#frequently-asked-questions" class="inline-onebox" rel="noopener nofollow ugc">Quantization — PyTorch main documentation</a></p>