[ONNX] Tried to trace <submodule> but it is not part of the active trace

zetyquickly: <p><a class="hashtag" href="/c/jit/13">#<span>jit</span></a> <a class="hashtag" href="/c/quantization/17">#<span>quantization</span></a></p>
<p>Hello!</p>
<p>I am trying to convert quantized model to Caffe2. I know that it is needed to use TorchScript tracing before ONNX exporting. I am trying to convert RCNN model the following way: 1) perform quantization, 2) trace quantized backbone to torchscript, 3) swap original backbone with quantized one (other parts of the network as they were), 4) patch and export network with ONNX</p>
<p>It is possible to convert original (non-quantized) network to Caffe2 without any errors, but when I do swapping of backbone and export ONNX I see the following:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "./tools/torchscript_converter.py", line 158, in &lt;module&gt;
    caffe2_model = export_caffe2_model(cfg, orig_model, first_batch)
  File "/root/some_detectron2/detectron2/export/api.py", line 157, in export_caffe2_model
    return Caffe2Tracer(cfg, model, inputs).export_caffe2()
  File "/root/some_detectron2/detectron2/export/api.py", line 95, in export_caffe2
    predict_net, init_net = export_caffe2_detection_model(model, inputs)
  File "/root/some_detectron2/detectron2/export/caffe2_export.py", line 148, in export_caffe2_detection_model
    onnx_model = export_onnx_model(model, (tensor_inputs,))
  File "/root/some_detectron2/detectron2/export/caffe2_export.py", line 67, in export_onnx_model
    export_params=True,
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/__init__.py", line 172, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/utils.py", line 92, in export
    use_external_data_format=use_external_data_format)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/utils.py", line 525, in _export
    fixed_batch_size=fixed_batch_size)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/utils.py", line 364, in _model_to_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/utils.py", line 317, in _trace_and_get_graph_from_model
    torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/jit/__init__.py", line 277, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 562, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/jit/__init__.py", line 359, in forward
    self._force_outplace,
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/jit/__init__.py", line 345, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 560, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 546, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/root/some_detectron2/detectron2/export/caffe2_modeling.py", line 326, in forward
    features = self._wrapped_model.backbone(images.tensor)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 560, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 536, in _slow_forward
    return self.forward(*input, **kwargs)
RuntimeError: Tried to trace &lt;__torch__.densepose.modeling.layers.bifpn.BiFPN object at 0x5570e7633550&gt; but it is not part of the active trace. Modules that are called during a trace must be registered as
 submodules of the thing being traced.
</code></pre>
<p>Is this because I’ve mixed TracedModule with eager mode modules?</p>

jerryzh168: <p>yeah according to the error message, the model being tracing is calling something outside of the traced model.</p>

wanchaol: <p>I think the step 2 here is probably wrong, you should not trace a quantized model before swapping it in? You can swap a quantized backbone model in first, then do onnx export, there’s no need to do explicit tracing as ONNX export will do it underlying.</p>

zetyquickly: <p>Thank you <a class="mention" href="/u/wanchaol">@wanchaol</a>, <a class="mention" href="/u/jerryzh168">@jerryzh168</a>. I’ll try again</p>
<p>But I’ve already opened a topic with such question and <a class="mention" href="/u/supriyar">@supriyar</a> has said that it is needed to <code>jit.trace</code> before <code>onnx.export</code> if we work with quantized model.<br>
Here’s the refference <a href="https://discuss.pytorch.org/t/onnx-export-of-quantized-model/76884/8" class="inline-onebox">ONNX export of quantized model</a></p>

zetyquickly: <p>Yes. I’ve tried and finally it’s given other errors <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>For now it is model with quantized backbone and untouched other parts. It is possible to <code>onnx.export</code> original model, but quantized behave the following way:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "./tools/torchscript_converter.py", line 145, in &lt;module&gt;
    caffe2_model = export_caffe2_model(cfg, torch_model, first_batch)
  File "/root/some_detectron2/detectron2/export/api.py", line 157, in export_caffe2_model
    return Caffe2Tracer(cfg, model, inputs).export_caffe2()
  File "/root/some_detectron2/detectron2/export/api.py", line 95, in export_caffe2
    predict_net, init_net = export_caffe2_detection_model(model, inputs)
  File "/root/some_detectron2/detectron2/export/caffe2_export.py", line 145, in export_caffe2_detection_model
    onnx_model = export_onnx_model(model, (tensor_inputs,))
  File "/root/some_detectron2/detectron2/export/caffe2_export.py", line 62, in export_onnx_model
    operator_export_type=OperatorExportTypes.ONNX_ATEN_FALLBACK,
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/__init__.py", line 172, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/utils.py", line 92, in export
    use_external_data_format=use_external_data_format)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/utils.py", line 530, in _export
    fixed_batch_size=fixed_batch_size)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/utils.py", line 366, in _model_to_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/onnx/utils.py", line 319, in _trace_and_get_graph_from_model
    torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/jit/__init__.py", line 283, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 574, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/jit/__init__.py", line 371, in forward
    self._force_outplace,
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/jit/__init__.py", line 357, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 572, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 558, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/root/some_detectron2/detectron2/export/caffe2_modeling.py", line 326, in forward
    features = self._wrapped_model.backbone(images.tensor)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 572, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 558, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/root/DensePose_ADASE/densepose/modeling/quantize_caffe2.py", line 166, in new_forward
    p5, p4, p3, p2 = self.bottom_up(x)  # top-&gt;down
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 572, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 558, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/timm/models/efficientnet.py", line 350, in forward
    x = self.conv_stem(x)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 572, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/modules/module.py", line 558, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/root/anaconda2/envs/pytorch-gpu/lib/python3.7/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py", line 37, in forward
    input, self._packed_params, self.scale, self.zero_point)
RuntimeError: Tried to trace &lt;__torch__.torch.classes.quantized.Conv2dPackedParamsBase object at 0x5632a7b49180&gt; but it is not part of the active trace. Modules that are called during a trace must be registered as submodules of the thing being traced.
</code></pre>
<p>Why this traceable on original but not on quantized? Any thoughts?</p>

Janani_Sundara_gandh: <p>I’m facing the same error when I try to export pytorch’s quantized MobilenetV2 model to ONNX. Is there any updates on this?</p>

zetyquickly: <p>Have a look at this thread maybe you’ll find something relevant</p>
<aside class="quote quote-modified" data-post="10" data-topic="82244">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/db5fbb/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/onnx-quantized-fused-conv2d-wont-trace/82244/10">[ONNX] Quantized fused Conv2d won't trace</a> <a class="badge-category__wrapper " href="/c/quantization/17"><span data-category-id="17" data-drop-close="true" class="badge-category " title="This category is for questions, discussion and issues related to PyTorch’s quantization feature."><span class="badge-category__name">quantization</span></span></a>
  </div>
  <blockquote>
    Please re-try with pytorch nightly build, we recently fixed this so you shouldn’t be seeing this error anymore. 

Seems like the conv layer is not quantized so it produces onnx::Conv as opposed to the _caffe2::Int8Conv operator. Currently the onnx export path to caffe2 does not support partially quantized model, so it expects the entire pytorch model to be able to get quantized.
  </blockquote>
</aside>