torch.compile() BackendCompilerFailed: _compile_fn raised RuntimeError
Labels: triaged, bug, oncall: pt2, module: inductor

### 🐛 Describe the bug

Could not run the code with torch.compile(). 

BackendCompilerFailed: _compile_fn raised RuntimeError: It appears that you're trying to get value out of a tracing tensor with aten._local_scalar_dense.default - erroring out! It's likely that this is caused by data-dependent control flow or similar.

I am not sure what this error means. 
Note: my code uses a custom LSTM and consists of nested models with multiple input parameters (>2). 

**Full trace** 

>  RuntimeError                              Traceback (most recent call last)
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/output_graph.py in call_user_compiler(self, gm)
>     585             else:
> --> 586                 compiled_fn = compiler_fn(gm, self.fake_example_inputs())
>     587             _step_logger()(logging.INFO, f"done compiler function {name}")
> 
>   ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/debug_utils.py in debug_wrapper(gm, 
>  example_inputs, **kwargs)
>     914         else:
>  --> 915             compiled_gm = compiler_fn(gm, example_inputs, **kwargs)
>     916 
> 
>   ~/anaconda3/lib/python3.7/site-packages/torch/__init__.py in _compile_fn(model_, inputs_)
>    1202             with cm:
>   -> 1203                 return compile_fn(model_, inputs_)
>    1204 
> 
>   ~/anaconda3/lib/python3.7/site-packages/torch/_inductor/compile_fx.py in compile_fx(model_, 
>   example_inputs_, inner_compile)
>     407             force_compile_tiny_graphs=True,
>     --> 408         )(model_, example_inputs_)
>     409 
> 
>   ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/optimizations/training.py in 
>   compiler_fn(gm, example_inputs)
>      79             # NB: NOT cloned!
>    ---> 80             cg = aot_module_simplified(gm, example_inputs, **kwargs)
>      81             counters["aot_autograd"]["ok"] += 1
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/_functorch/aot_autograd.py in 
>   aot_module_simplified(mod, args, fw_compiler, bw_compiler, partition_fn, decompositions, 
>    hasher_type, static_argnums)
>    2095         full_args,
>    -> 2096         aot_config,
>    2097     )
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/utils.py in time_wrapper(*args, **kwargs)
>      89         t0 = time.time()
>     ---> 90         r = func(*args, **kwargs)
>      91         latency = time.time() - t0
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/_functorch/aot_autograd.py in 
>     create_aot_dispatcher_function(flat_fn, flat_args, aot_config)
>    1791 
>    -> 1792         compiled_fn = compiler_fn(flat_fn, fake_flat_tensor_args, aot_config)
>    1793 
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/_functorch/aot_autograd.py in 
>      aot_wrapper_dedupe(flat_fn, flat_args, aot_config, compiler_fn)
>    1196         if ok:
>     -> 1197             return compiler_fn(flat_fn, leaf_flat_args, aot_config)
>    1198 
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/_functorch/aot_autograd.py in 
>     aot_dispatch_autograd(flat_fn, flat_args, aot_config)
>    1378                 joint_forward_backward, aot_config.decompositions
>     -> 1379             )(*joint_inputs)
>    1380 
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/fx/experimental/proxy_tensor.py in wrapped(*args)
>     682              sym_mode, proxy_mode, disable_autocast_cache():  # type: ignore[attr-defined]
>     --> 683             t = dispatch_trace(wrap_key(func, args, fx_tracer), tracer=fx_tracer, 
>     concrete_args=tuple(phs))
>     684 
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/eval_frame.py in _fn(*args, **kwargs)
>     210             try:
>     --> 211                 return fn(*args, **kwargs)
>     212             finally:
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/fx/experimental/proxy_tensor.py in 
>    dispatch_trace(root, tracer, concrete_args)
>     440 ) -> GraphModule:
>     --> 441     graph = tracer.trace(root, concrete_args)
>     442     name = root.__class__.__name__ if isinstance(root, torch.nn.Module) else root.__name__
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/eval_frame.py in _fn(*args, **kwargs)
>     210             try:
>    --> 211                 return fn(*args, **kwargs)
>     212             finally:
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/fx/_symbolic_trace.py in trace(self, root, 
>     concrete_args)
>     738                     "output",
>      --> 739                     (self.create_arg(fn(*args)),),
>     740                     {},
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/fx/_symbolic_trace.py in flatten_fn(*args)
>     613                 tree_args = pytree.tree_unflatten(list(args), in_spec)
>     --> 614                 tree_out = root_fn(*tree_args)
>     615                 out_args, out_spec = pytree.tree_flatten(tree_out)
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/fx/experimental/proxy_tensor.py in 
>    wrapped(*proxies)
>     456 
>       --> 457         out = f(*tensors)
>     458         out = pytree.tree_map_only(
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/_functorch/aot_autograd.py in 
>    functionalized_joint(primals, tangents)
>     745             # Run the joint
>     --> 746             outs = joint_forward_backward(f_primals, f_tangents)
>     747         finally:
> 
>   ~/anaconda3/lib/python3.7/site-packages/torch/_functorch/aot_autograd.py in 
>    joint_forward_backward(primals, tangents)
>     716                     grad_outputs=needed_tangents,
>     --> 717                     allow_unused=True,
>     718                 )
> 
>   ~/anaconda3/lib/python3.7/site-packages/torch/autograd/__init__.py in grad(outputs, inputs, 
>   grad_outputs, retain_graph, create_graph, only_inputs, allow_unused, is_grads_batched)
>     275             allow_unused=allow_unused,
>    --> 276             is_grads_batched=is_grads_batched,
>     277         )
> 
>   ~/anaconda3/lib/python3.7/site-packages/torch/overrides.py in handle_torch_function(public_api, 
>    relevant_args, *args, **kwargs)
>    1519         with _pop_mode_temporarily() as mode:
>     -> 1520             result = mode.__torch_function__(public_api, types, args, kwargs)
>    1521         if result is not NotImplemented:
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/_inductor/overrides.py in __torch_function__(self, 
>    func, types, args, kwargs)
>      36             return replacements[func](*args, **kwargs)
>     ---> 37         return func(*args, **kwargs)
>      38 
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/autograd/__init__.py in grad(outputs, inputs, 
>    grad_outputs, retain_graph, create_graph, only_inputs, allow_unused, is_grads_batched)
>     301             t_outputs, grad_outputs_, retain_graph, create_graph, t_inputs,
>    --> 302             allow_unused, accumulate_grad=False)  # Calls into the C++ engine to run the 
>    backward pass
>     303 
> 
>     ~/anaconda3/lib/python3.7/site-packages/torch/fx/experimental/proxy_tensor.py in 
>    __torch_dispatch__(self, func, types, args, kwargs)
>     482         with self.sym_mode.enable(False):
>     --> 483             return self.inner_torch_dispatch(func, types, args, kwargs)
>     484 
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/fx/experimental/proxy_tensor.py in 
>    inner_torch_dispatch(self, func, types, args, kwargs)
>     507 
>    --> 508         out = proxy_call(self, func, args, kwargs)
>     509         return out
> 
>   ~/anaconda3/lib/python3.7/site-packages/torch/fx/experimental/proxy_tensor.py in 
>   proxy_call(proxy_mode, func, args, kwargs)
>     284         raise RuntimeError(
>   --> 285             f"It appears that you're trying to get value out of a tracing tensor with {func} - 
>     erroring out! "
>     286             "It's likely that this is caused by data-dependent control flow or similar."
> 
>    
> 
>  RuntimeError:` It appears that you're trying to get value out of a tracing tensor with 
> 
>  aten._local_scalar_dense.default - erroring out! It's likely that this is caused by data-dependent 
>  control flow or similar.
> 
> The above exception was the direct cause of the following exception:
> 
>   BackendCompilerFailed                     Traceback (most recent call last)
>   <ipython-input-16-d2e9339dd69c> in <module>
>      76            
>      77             t1 = time.time()
>    ---> 78             output_N = model_N(inputs)
>      79             t2 = time.time()
>      80             print ("time noncustom", t2-t1)
> 
>    ~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *args, 
>    **kwargs)
>    1478                 or _global_backward_pre_hooks or _global_backward_hooks
>    1479                 or _global_forward_hooks or _global_forward_pre_hooks):
> -> 1480             return forward_call(*args, **kwargs)
>    1481         # Do not call functions when jit is used
>    1482         full_backward_hooks, non_full_backward_hooks = [], []
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/eval_frame.py in forward(self, *args, **kwargs)
>      80 
>      81     def forward(self, *args, **kwargs):
> ---> 82         return self.dynamo_ctx(self._orig_mod.forward)(*args, **kwargs)
>      83 
>      84 
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/eval_frame.py in _fn(*args, **kwargs)
>     209             dynamic_ctx.__enter__()
>     210             try:
> --> 211                 return fn(*args, **kwargs)
>     212             finally:
>     213                 set_eval_frame(prior)
> 
> <ipython-input-10-c07d89dd9fcd> in forward(self, x)
>     437         out = self.relu(out)
>     438         out = self.fcLayer(out)
> --> 439         out = softmax(out)
>     440         return out
>     441 
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/eval_frame.py in catch_errors(frame, cache_size)
>     330 
>     331         with compile_lock:
> --> 332             return callback(frame, cache_size, hooks)
>     333 
>     334     catch_errors._torchdynamo_orig_callable = callback  # type: ignore[attr-defined]
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/convert_frame.py in _convert_frame(frame, cache_size, hooks)
>     474         counters["frames"]["total"] += 1
>     475         try:
> --> 476             result = inner_convert(frame, cache_size, hooks)
>     477             counters["frames"]["ok"] += 1
>     478             return result
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/convert_frame.py in _fn(*args, **kwargs)
>     101         torch.fx.graph_module._forward_from_src = fx_forward_from_src_skip_result
>     102         try:
> --> 103             return fn(*args, **kwargs)
>     104         finally:
>     105             torch._C._set_grad_enabled(prior_grad_mode)
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/utils.py in time_wrapper(*args, **kwargs)
>      88             compilation_metrics[key] = []
>      89         t0 = time.time()
> ---> 90         r = func(*args, **kwargs)
>      91         latency = time.time() - t0
>      92         # print(f"Dynamo timer: key={key}, latency={latency:.2f} sec")
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/convert_frame.py in _convert_frame_assert(frame, cache_size, hooks)
>     346             export,
>     347             hooks,
> --> 348             frame,
>     349         )
>     350 
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/convert_frame.py in _compile(code, globals, locals, builtins, compiler_fn, one_graph, export, hooks, frame)
>     393         for attempt in itertools.count():
>     394             try:
> --> 395                 out_code = transform_code_object(code, transform)
>     396                 orig_code_map[out_code] = code
>     397                 break
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/bytecode_transformation.py in transform_code_object(code, transformations, safe)
>     339     propagate_line_nums(instructions)
>     340 
> --> 341     transformations(instructions, code_options)
>     342 
>     343     fix_vars(instructions, code_options)
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/convert_frame.py in transform(instructions, code_options)
>     380             export,
>     381         )
> --> 382         tracer.run()
>     383         output = tracer.output
>     384         assert output is not None
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/symbolic_convert.py in run(self)
>    1619     def run(self):
>    1620         _step_logger()(logging.INFO, f"torchdynamo start tracing {self.f_code.co_name}")
> -> 1621         super().run()
>    1622 
>    1623     def match_nested_cell(self, name, cell):
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/symbolic_convert.py in run(self)
>     483                 self.instruction_pointer is not None
>     484                 and not self.output.should_exit
> --> 485                 and self.step()
>     486             ):
>     487                 pass
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/symbolic_convert.py in step(self)
>     452             if not hasattr(self, inst.opname):
>     453                 unimplemented(f"missing: {inst.opname}")
> --> 454             getattr(self, inst.opname)(inst)
>     455 
>     456             return inst.opname != "RETURN_VALUE"
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/symbolic_convert.py in inner(self, inst)
>     233                 self,
>     234                 reason=GraphCompileReason(
> --> 235                     f"generic_jump {typestr(value)}", [self.frame_summary()]
>     236                 ),
>     237             )
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/output_graph.py in compile_subgraph(self, tx, partial_convert, reason)
>     437             # optimization to generate better code in a common case
>     438             self.add_output_instructions(
> --> 439                 self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
>     440                 + [create_instruction("UNPACK_SEQUENCE", len(stack_values))]
>     441             )
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/output_graph.py in compile_and_call_fx_graph(self, tx, rv, root)
>     508 
>     509         assert_no_fake_params_or_buffers(gm)
> --> 510         compiled_fn = self.call_user_compiler(gm)
>     511         compiled_fn = disable(compiled_fn)
>     512 
> 
> ~/anaconda3/lib/python3.7/site-packages/torch/_dynamo/output_graph.py in call_user_compiler(self, gm)
>     589         except Exception as e:
>     590             compiled_fn = gm.forward
> --> 591             raise BackendCompilerFailed(self.compiler_fn, e) from e
>     592         return compiled_fn
>     593 
> `

Thank you.

### Versions

Collecting environment information...
PyTorch version: 1.14.0.dev20221208
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6.1 (x86_64)
GCC version: Could not collect
Clang version: 3.4 (tags/RELEASE_34/final)
CMake version: version 3.18.0
Libc version: N/A

Python version: 3.7.3 (default, Mar 27 2019, 16:54:48)  [Clang 4.0.1 (tags/RELEASE_401/final)] (64-bit runtime)
Python platform: Darwin-21.6.0-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.6
[pip3] numpydoc==0.8.0
[pip3] torch==1.14.0.dev20221208
[pip3] torch-ac==1.1.0
[pip3] torchaudio==0.14.0.dev20221208
[pip3] torchmetrics==0.11.0
[pip3] torchrec-nightly-cpu==2022.5.12
[pip3] torchvision==0.15.0.dev20221208
[pip3] torchx-nightly==2022.11.30
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.3                      199  
[conda] mkl-service               1.1.2            py37hfbe908c_5  
[conda] mkl_fft                   1.0.10           py37h5e564d8_0  
[conda] mkl_random                1.0.2            py37h27c97d8_0  
[conda] numpy                     1.21.6                   pypi_0    pypi
[conda] numpydoc                  0.8.0                    py37_0  
[conda] torch                     1.14.0.dev20221208          pypi_0    pypi
[conda] torch-ac                  1.1.0                    pypi_0    pypi
[conda] torchaudio                0.14.0.dev20221208          pypi_0    pypi
[conda] torchmetrics              0.11.0                   pypi_0    pypi
[conda] torchrec-nightly-cpu      2022.5.12                pypi_0    pypi
[conda] torchvision               0.15.0.dev20221208          pypi_0    pypi
[conda] torchx-nightly            2022.11.30               pypi_0    pypi


cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @mlazos @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire