[FX][ONNX][exporter] Failed to export traced fx graph to onnx model
Labels: module: onnx, triaged, module: export

### 🐛 Describe the bug

I traced and quantized a ResNet50 model (only backbone and fused some skip-adds) and tried to export it to a onnx model format, but using `torch.onnx.dynamo_export`, here're informations:

Exporting codes:
```python
qmodel = xxxx # trace and quantize
tensor_x = torch.randn(1, 3, 224, 224)
onnx_model = dynamo_export(qmodel, tensor_x)  # failed!
```

Model graph:
```txt
GraphModule(
  (conv1): QuantizedConvReLU2d(3, 64, kernel_size=(3, 3), stride=(1, 1), scale=0.013380219228565693, zero_point=128, padding=(1, 1))
  (layer1): Module(
    (0): Module(
      (conv1): QuantizedConvReLU2d(64, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.005257370416074991, zero_point=128)
      (conv2): QuantizedConvReLU2d(64, 64, kernel_size=(3, 3), stride=(1, 1), scale=0.008329497650265694, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(64, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.010099029168486595, zero_point=128)
      (downsample): Module(
        (0): QuantizedConv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.005250006914138794, zero_point=128)
      )
    )
    (1): Module(
      (conv1): QuantizedConvReLU2d(256, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.005096939858049154, zero_point=128)
      (conv2): QuantizedConvReLU2d(64, 64, kernel_size=(3, 3), stride=(1, 1), scale=0.004832049366086721, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(64, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.00993015430867672, zero_point=128)
    )
    (2): Module(
      (conv1): QuantizedConvReLU2d(256, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.0038794134743511677, zero_point=128)
      (conv2): QuantizedConvReLU2d(64, 64, kernel_size=(3, 3), stride=(1, 1), scale=0.005039558280259371, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(64, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.010828116908669472, zero_point=128)
    )
  )
  (layer2): Module(
    (0): Module(
    (conv1): QuantizedConvReLU2d(256, 128, kernel_size=(1, 1), stride=(1, 1), scale=0.004017043858766556, zero_point=128)
      (conv2): QuantizedConvReLU2d(128, 128, kernel_size=(3, 3), stride=(2, 2), scale=0.004348098766058683, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(128, 512, kernel_size=(1, 1), stride=(1, 1), scale=0.005679312162101269, zero_point=128)
      (downsample): Module(
        (0): QuantizedConv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), scale=0.004687752574682236, zero_point=128)
      )
    )
    (1): Module(
      (conv1): QuantizedConvReLU2d(512, 128, kernel_size=(1, 1), stride=(1, 1), scale=0.0029946595896035433, zero_point=128)
      (conv2): QuantizedConvReLU2d(128, 128, kernel_size=(3, 3), stride=(1, 1), scale=0.002591489814221859, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(128, 512, kernel_size=(1, 1), stride=(1, 1), scale=0.0066427248530089855, zero_point=128)
    )
    (2): Module(
      (conv1): QuantizedConvReLU2d(512, 128, kernel_size=(1, 1), stride=(1, 1), scale=0.003675809595733881, zero_point=128)
      (conv2): QuantizedConvReLU2d(128, 128, kernel_size=(3, 3), stride=(1, 1), scale=0.004037801176309586, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(128, 512, kernel_size=(1, 1), stride=(1, 1), scale=0.0076212105341255665, zero_point=128)
    )
    (3): Module(
      (conv1): QuantizedConvReLU2d(512, 128, kernel_size=(1, 1), stride=(1, 1), scale=0.0036621815524995327, zero_point=128)
      (conv2): QuantizedConvReLU2d(128, 128, kernel_size=(3, 3), stride=(1, 1), scale=0.003685015020892024, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(128, 512, kernel_size=(1, 1), stride=(1, 1), scale=0.006834834348410368, zero_point=128)
    )
  )
  (layer3): Module(
    (0): Module(
      (conv1): QuantizedConvReLU2d(512, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.00595456175506115, zero_point=128)
      (conv2): QuantizedConvReLU2d(256, 256, kernel_size=(3, 3), stride=(2, 2), scale=0.006026321556419134, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), scale=0.007073363289237022, zero_point=128)
      (downsample): Module(
        (0): QuantizedConv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), scale=0.004362074658274651, zero_point=128)
      )
    )
    (1): Module(
      (conv1): QuantizedConvReLU2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.003083687275648117, zero_point=128)
      (conv2): QuantizedConvReLU2d(256, 256, kernel_size=(3, 3), stride=(1, 1), scale=0.0031905779615044594, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), scale=0.007137070409953594, zero_point=128)
    )
    (2): Module(
      (conv1): QuantizedConvReLU2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.004007844254374504, zero_point=128)
      (conv2): QuantizedConvReLU2d(256, 256, kernel_size=(3, 3), stride=(1, 1), scale=0.0028449269011616707, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), scale=0.00658777728676796, zero_point=128)
    )
    (3): Module(
      (conv1): QuantizedConvReLU2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.00378667120821774, zero_point=128)
      (conv2): QuantizedConvReLU2d(256, 256, kernel_size=(3, 3), stride=(1, 1), scale=0.00343075068667531, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), scale=0.006993501912802458, zero_point=128)
    )
    (4): Module(
      (conv1): QuantizedConvReLU2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.0034908351954072714, zero_point=128)
      (conv2): QuantizedConvReLU2d(256, 256, kernel_size=(3, 3), stride=(1, 1), scale=0.004998135380446911, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), scale=0.008047958835959435, zero_point=128)
    )
    (5): Module(
      (conv1): QuantizedConvReLU2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.00446851784363389, zero_point=128)
      (conv2): QuantizedConvReLU2d(256, 256, kernel_size=(3, 3), stride=(1, 1), scale=0.004318995401263237, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), scale=0.008282049559056759, zero_point=128)
    )
  )
  (layer4): Module(
    (0): Module(
      (conv1): QuantizedConvReLU2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), scale=0.006188110448420048, zero_point=128)
      (conv2): QuantizedConvReLU2d(512, 512, kernel_size=(3, 3), stride=(2, 2), scale=0.006452975329011679, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), scale=0.014389239251613617, zero_point=128)
      (downsample): Module(
        (0): QuantizedConv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), scale=0.005995641928166151, zero_point=128)
      )
    )
    (1): Module(
      (conv1): QuantizedConvReLU2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), scale=0.004341470077633858, zero_point=128)
      (conv2): QuantizedConvReLU2d(512, 512, kernel_size=(3, 3), stride=(1, 1), scale=0.0054984972812235355, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), scale=0.02057485468685627, zero_point=128)
    )
    (2): Module(
      (conv1): QuantizedConvReLU2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), scale=0.003666398348286748, zero_point=128)
      (conv2): QuantizedConvReLU2d(512, 512, kernel_size=(3, 3), stride=(1, 1), scale=0.006707869004458189, zero_point=128, padding=(1, 1))
      (conv3): QuantizedConvAddReLU2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), scale=0.0357082337141037, zero_point=128)
    )
  )
)
```

Error message:
```
Traceback (most recent call last):
  File "/workspace/torchint/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 595, in dynamo_export
    ).export()
  File "/workspace/torchint/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 446, in export
    graph_module = self.options.fx_tracer.generate_fx(
  File "/workspace/torchint/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 196, in generate_fx
    graph_module, graph_guard = torch._dynamo.export(
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 948, in export
    result_traced = opt_f(*args, **kwargs)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 295, in _fn
    return fn(*args, **kwargs)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 153, in wrapped
    return output_adapter.apply(model_func(*args, **kwargs))                                                                                     File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 448, in catch_errors
    return callback(frame, cache_size, hooks, frame_state)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 127, in _fn
    return fn(*args, **kwargs)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 360, in _convert_frame_assert
    return _compile(
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 180, in time_wrapper
    r = func(*args, **kwargs)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 430, in _compile
    out_code = transform_code_object(code, transform)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1000, in transform_code_object
    transformations(instructions, code_options)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 415, in transform
    tracer.run()
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2026, in run
    super().run()
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 708, in run
    and self.step()
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 668, in step
    getattr(self, inst.opname)(inst)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 390, in wrapper
    return inner_fn(self, inst)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1100, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 559, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/variables/torch.py", line 610, in call_function
    tensor_variable = wrap_fx_proxy(
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1115, in wrap_fx_proxy
    return wrap_fx_proxy_cls(
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1151, in wrap_fx_proxy_cls
    example_value = get_fake_value(proxy.node, tx)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1313, in get_fake_value
    raise TorchRuntimeError(str(e)).with_traceback(e.__traceback__) from None
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1281, in get_fake_value
    return wrap_fake_exception(
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 871, in wrap_fake_exception
    return fn()
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1282, in <lambda>
    lambda: run_node(tx.output, node, args, kwargs, nnmodule)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1347, in run_node
    raise RuntimeError(fn_str + str(e)).with_traceback(e.__traceback__) from e
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1334, in run_node
    return node.target(*args, **kwargs)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
    return fn(*args, **kwargs)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1161, in __torch_dispatch__
    return self.dispatch(func, types, args, kwargs)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1410, in dispatch
    return run_fallback_kernel(self, func, args, kwargs, not_implemented_error)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1657, in run_fallback_kernel
    return tree_map(map_out, r)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/utils/_pytree.py", line 323, in tree_map
    return tree_unflatten([fn(i) for i in flat_args], spec)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/utils/_pytree.py", line 323, in <listcomp>
    return tree_unflatten([fn(i) for i in flat_args], spec)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1653, in map_out
    return fake_mode.fake_tensor_converter(fake_mode, e)
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 341, in __call__
    return self.from_real_tensor(
  File "/workspace/torchint/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 274, in from_real_tensor
    raise UnsupportedFakeTensorException("quantized nyi in meta tensors")
  raise UnsupportedFakeTensorException("quantized nyi in meta tensors")
torch._dynamo.exc.TorchRuntimeError: Failed running call_function <built-in method quantize_per_tensor of type object at 0x7f3ab3e18160>(*(Fake
Tensor(..., size=(24, 3, 32, 32)), FakeTensor(..., size=()), FakeTensor(..., size=(), dtype=torch.int64), torch.quint8), **{}):
quantized nyi in meta tensors
```

### Versions

```txt
PyTorch version: 2.1.0.dev20230621+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.2 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.4.0-42-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA A100 80GB PCIe
GPU 1: NVIDIA A100 80GB PCIe
GPU 2: NVIDIA A100 80GB PCIe
GPU 3: NVIDIA A100 80GB PCIe
GPU 4: NVIDIA A100 80GB PCIe
GPU 5: NVIDIA A100 80GB PCIe
GPU 6: NVIDIA A100 80GB PCIe
GPU 7: NVIDIA A100 80GB PCIe
Nvidia driver version: 515.48.07                                                                                                      [28/4606]
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   46 bits physical, 57 bits virtual
Byte Order:                      Little Endian
CPU(s):                          64
On-line CPU(s) list:             0-63
Vendor ID:                       GenuineIntel
Model name:                      Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz
CPU family:                      6
Model:                           106
Thread(s) per core:              2
Core(s) per socket:              16
Socket(s):                       2
Stepping:                        6
Frequency boost:                 enabled
CPU max MHz:                     3600.0000
CPU min MHz:                     800.0000
BogoMIPS:                        6200.00
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss
ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqd
q dtes64 ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c
rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_single ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept
vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt
 avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local wbnoinvd dtherm ida arat pl
n pts avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpid md_clear pconfig flush_l
1d arch_capabilities
Virtualization:                  VT-x
L1d cache:                       1.5 MiB (32 instances)
L1i cache:                       1 MiB (32 instances)
L2 cache:                        40 MiB (32 instances)
L3 cache:                        72 MiB (2 instances)
NUMA node(s):                    2
NUMA node0 CPU(s):               0-15,32-47
NUMA node1 CPU(s):               16-31,48-63
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.25.1
[pip3] pytorch-triton==2.1.0+440fd1bf20
[pip3] torch==2.1.0.dev20230621+cu117
[pip3] torch-int==0.0.0
[pip3] torchvision==0.15.2
[conda] Could not collect
```