Repository: pytorch
Issue: Torch.export.export fails with SAM and returns a non-informative Error message 
Labels: triaged,actionable,oncall: pt2,oncall: export

### 🐛 Describe the bug

## Problem:
- SAM cannot be exported with `torch.export.export`
- The export fails and produces a non-informative error message. The entire stack comes from `_dynamo` or `_export` without any source location from SAM.
- I have tried the same with the `predict_torch` function but experienced the same problems.

What can I do to export the SAM? Did I do something wrong in the export process? I appreciate any suggestions. 

## Reproduce:
From the SAM root dir (with vit-b) and installed SAM, please run the following:

```python
import torch
import cv2

from segment_anything import sam_model_registry, SamPredictor
sam = sam_model_registry["vit_b"](checkpoint="./sam_vit_b_01ec64.pth")
image = cv2.imread('./notebooks/images/truck.jpg')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)


predictor = SamPredictor(sam)


torch_img = predictor.transform.apply_image(image)
torch_img = torch.as_tensor(torch_img)
torch_img = torch_img.permute(2, 0, 1).contiguous()[None, :, :, :]

img_shape = (torch_img.shape[0], torch_img.shape[1])
predictor.set_torch_image(torch_img, img_shape)
torch.export.export(predictor.set_torch_image, (torch_img, img_shape),
        kwargs=None, constraints=None)
```
This will yield the following error message:
```
[2023-12-11 10:46:57,194] torch._dynamo.eval_frame: [DEBUG] skipping __init__ /usr/lib/python3.10/contextlib.py
[2023-12-11 10:46:57,194] torch._dynamo.eval_frame: [DEBUG] skipping __enter__ /usr/lib/python3.10/contextlib.py
[2023-12-11 10:46:57,194] torch._dynamo.eval_frame: [DEBUG] skipping helper /usr/lib/python3.10/contextlib.py
[2023-12-11 10:46:57,194] torch._dynamo.eval_frame: [DEBUG] skipping __init__ /usr/lib/python3.10/contextlib.py
[2023-12-11 10:46:57,194] torch._dynamo.eval_frame: [DEBUG] skipping __enter__ /usr/lib/python3.10/contextlib.py
[2023-12-11 10:46:57,194] torch._dynamo.eval_frame: [DEBUG] skipping enable_dynamic /home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py
[2023-12-11 10:46:57,195] [0/0] torch._dynamo.symbolic_convert: [INFO] Step 1: torchdynamo start tracing inner /home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/external_utils.py:15
[2023-12-11 10:46:57,196] [0/0] torch._dynamo.symbolic_convert.__trace_source: [DEBUG] TRACE starts_line inner /home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/external_utils.py:15
[2023-12-11 10:46:57,196] [0/0] torch._dynamo.symbolic_convert.__trace_source: [DEBUG]         @functools.wraps(fn)
[2023-12-11 10:46:57,197] [0/0] torch._dynamo.variables.builder: [DEBUG] wrap_to_fake L['args'][0] (1, 3, 683, 1024) [<DimDynamic.STATIC: 2>, <DimDynamic.STATIC: 2>, <DimDynamic.STATIC: 2>, <DimDynamic.STATIC: 2>] [None, None, None, None]
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert.__trace_source: [DEBUG] TRACE starts_line inner /home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/external_utils.py:17
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert.__trace_source: [DEBUG]             return fn(*args, **kwargs)
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_DEREF fn []
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST args [UserDefinedObjectVariable(set_torch_image)]
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_MAP 0 [UserDefinedObjectVariable(set_torch_image), TupleVariable()]
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST kwargs [UserDefinedObjectVariable(set_torch_image), TupleVariable(), ConstDictVariable()]
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert: [DEBUG] TRACE DICT_MERGE 1 [UserDefinedObjectVariable(set_torch_image), TupleVariable(), ConstDictVariable(), ConstDictVariable()]
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_EX 1 [UserDefinedObjectVariable(set_torch_image), TupleVariable(), ConstDictVariable()]
[2023-12-11 10:46:57,207] [0/0] torch._dynamo.symbolic_convert: [DEBUG] empty checkpoint
Traceback (most recent call last):
  File "/home/fabian/Documents/work/github/segment-anything/export.py", line 22, in <module>
    torch.export.export(predictor.set_torch_image, (torch_img, img_shape),
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/export/__init__.py", line 1018, in export
    return export(f, args, kwargs, constraints)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_export/__init__.py", line 270, in export
    gm_torch_level, _ = torch._dynamo.export(
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1140, in inner
    result_traced = opt_f(*args, **kwargs)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 328, in _fn
    return fn(*args, **kwargs)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 490, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 133, in _fn
    return fn(*args, **kwargs)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 389, in _convert_frame_assert
    return _compile(
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 569, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 189, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 491, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1028, in transform_code_object
    transformations(instructions, code_options)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 458, in transform
    tracer.run()
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2069, in run
    super().run()
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 719, in run
    and self.step()
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 683, in step
    getattr(self, inst.opname)(inst)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 392, in wrapper
    return inner_fn(self, inst)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1150, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars.items)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 557, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 385, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 306, in call_function
    unimplemented(f"call_function {self} {args} {kwargs}")
  File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 172, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: call_function UserDefinedObjectVariable(set_torch_image) [TensorVariable(), TupleVariable()] OrderedDict()

from user code:
   File "/home/fabian/.local/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 17, in inner
    return fn(*args, **kwargs)

[2023-12-11 10:46:57,212] torch._dynamo.utils: [INFO] TorchDynamo compilation metrics:
[2023-12-11 10:46:57,212] torch._dynamo.utils: [INFO] Function                           Runtimes (s)
[2023-12-11 10:46:57,212] torch._dynamo.utils: [INFO] -------------------------------  --------------
[2023-12-11 10:46:57,212] torch._dynamo.utils: [INFO] _compile.<locals>.compile_inner 
```



### Versions

Collecting environment information...
PyTorch version: 2.1.1+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: 14.0.0-1ubuntu1.1
CMake version: version 3.27.6
Libc version: glibc-2.35

Python version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-91-generic-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      46 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             24
On-line CPU(s) list:                0-23
Vendor ID:                          GenuineIntel
Model name:                         12th Gen Intel(R) Core(TM) i7-12850HX
CPU family:                         6
Model:                              151
Thread(s) per core:                 2
Core(s) per socket:                 16
Socket(s):                          1
Stepping:                           2
CPU max MHz:                        4800,0000
CPU min MHz:                        800,0000
BogoMIPS:                           4838.40
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb intel_pt sha_ni xsaveopt xsavec xgetbv1 xsaves split_lock_detect avx_vnni dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req umip pku ospke waitpkg gfni vaes vpclmulqdq tme rdpid movdiri movdir64b fsrm md_clear serialize pconfig arch_lbr flush_l1d arch_capabilities
Virtualization:                     VT-x
L1d cache:                          640 KiB (16 instances)
L1i cache:                          768 KiB (16 instances)
L2 cache:                           14 MiB (10 instances)
L3 cache:                           25 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-23
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.3
[pip3] onnx==1.14.1
[pip3] onnx-tf==1.10.0
[pip3] onnxruntime==1.16.0
[pip3] onnxsim==0.4.33
[pip3] pytorch-lightning==2.1.2
[pip3] tf2onnx==1.15.1
[pip3] torch==2.1.1+cpu
[pip3] torchaudio==2.1.1+cpu
[pip3] torchmetrics==1.2.0
[pip3] torchpack==0.3.1
[pip3] torchprofile==0.0.4
[pip3] torchvision==0.16.1+cpu
[pip3] triton==2.0.0
[conda] Could not collect


cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan @angelayi @suo @ydwu4