AssertionError: slice.Tensor is not supported with cpp wrapper (llama)
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

```
2023-05-08T10:46:35.4955096Z cuda eval  llama                               ERROR:common:backend='inductor' raised:
2023-05-08T10:46:35.4956007Z AssertionError: slice.Tensor is not supported with cpp wrapper
2023-05-08T10:46:35.4956583Z 
2023-05-08T10:46:35.4956590Z 
2023-05-08T10:46:35.4956888Z You can suppress this exception and fall back to eager by setting:
2023-05-08T10:46:35.4957450Z     import torch._dynamo
2023-05-08T10:46:35.4957901Z     torch._dynamo.config.suppress_errors = True
2023-05-08T10:46:35.4958374Z Traceback (most recent call last):
2023-05-08T10:46:35.4959003Z   File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1448, in check_accuracy
2023-05-08T10:46:35.4959721Z     new_result = optimized_model_iter_fn(model_copy, example_inputs)
2023-05-08T10:46:35.4960737Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 286, in _fn
2023-05-08T10:46:35.4961257Z     return fn(*args, **kwargs)
2023-05-08T10:46:35.4961779Z   File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1291, in run_n_iterations
2023-05-08T10:46:35.4962443Z     self.model_iter_fn(mod, inputs, collect_outputs=False)
2023-05-08T10:46:35.4963963Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 439, in catch_errors
2023-05-08T10:46:35.4964599Z     return callback(frame, cache_size, hooks, frame_state)
2023-05-08T10:46:35.4965812Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 519, in _convert_frame
2023-05-08T10:46:35.4966506Z     result = inner_convert(frame, cache_size, hooks, frame_state)
2023-05-08T10:46:35.4967425Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 122, in _fn
2023-05-08T10:46:35.4967995Z     return fn(*args, **kwargs)
2023-05-08T10:46:35.4969267Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 355, in _convert_frame_assert
2023-05-08T10:46:35.4969878Z     return _compile(
2023-05-08T10:46:35.4970691Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 177, in time_wrapper
2023-05-08T10:46:35.4971306Z     r = func(*args, **kwargs)
2023-05-08T10:46:35.4972150Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 425, in _compile
2023-05-08T10:46:35.4972781Z     out_code = transform_code_object(code, transform)
2023-05-08T10:46:35.4973787Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1000, in transform_code_object
2023-05-08T10:46:35.4974489Z     transformations(instructions, code_options)
2023-05-08T10:46:35.4975399Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 410, in transform
2023-05-08T10:46:35.4975970Z     tracer.run()
2023-05-08T10:46:35.4976818Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2010, in run
2023-05-08T10:46:35.4977355Z     super().run()
2023-05-08T10:46:35.4978121Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 703, in run
2023-05-08T10:46:35.4978685Z     and self.step()
2023-05-08T10:46:35.4979578Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 663, in step
2023-05-08T10:46:35.4980213Z     getattr(self, inst.opname)(inst)
2023-05-08T10:46:35.4980931Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2098, in RETURN_VALUE
2023-05-08T10:46:35.4981543Z     self.output.compile_subgraph(
2023-05-08T10:46:35.4982068Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 736, in compile_subgraph
2023-05-08T10:46:35.4982780Z     self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
2023-05-08T10:46:35.4983236Z   File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner
2023-05-08T10:46:35.4983542Z     return func(*args, **kwds)
2023-05-08T10:46:35.4984078Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 813, in compile_and_call_fx_graph
2023-05-08T10:46:35.4984491Z     compiled_fn = self.call_user_compiler(gm)
2023-05-08T10:46:35.4985117Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 177, in time_wrapper
2023-05-08T10:46:35.4985460Z     r = func(*args, **kwargs)
2023-05-08T10:46:35.4985954Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 872, in call_user_compiler
2023-05-08T10:46:35.4986401Z     raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
2023-05-08T10:46:35.4986981Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 868, in call_user_compiler
2023-05-08T10:46:35.4987396Z     compiled_fn = compiler_fn(gm, self.example_inputs())
2023-05-08T10:46:35.4987936Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/repro/after_dynamo.py", line 108, in debug_wrapper
2023-05-08T10:46:35.4988344Z     compiled_gm = compiler_fn(gm, example_inputs)
2023-05-08T10:46:35.4989228Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/backends/inductor.py", line 9, in inductor
2023-05-08T10:46:35.4989584Z     return compile_fx(*args, **kwargs)
2023-05-08T10:46:35.4990091Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 628, in compile_fx
2023-05-08T10:46:35.4990458Z     return compile_fx_with_cpp_wrapper(
2023-05-08T10:46:35.4991007Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 575, in compile_fx_with_cpp_wrapper
2023-05-08T10:46:35.4991350Z     return compile_fx(
2023-05-08T10:46:35.4991948Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 728, in compile_fx
2023-05-08T10:46:35.4992309Z     return aot_autograd(
2023-05-08T10:46:35.4992802Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/backends/common.py", line 56, in compiler_fn
2023-05-08T10:46:35.4993214Z     cg = aot_module_simplified(gm, example_inputs, **kwargs)
2023-05-08T10:46:35.4993786Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 3334, in aot_module_simplified
2023-05-08T10:46:35.4994189Z     compiled_fn = create_aot_dispatcher_function(
2023-05-08T10:46:35.4994686Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 177, in time_wrapper
2023-05-08T10:46:35.4995026Z     r = func(*args, **kwargs)
2023-05-08T10:46:35.4995671Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 2975, in create_aot_dispatcher_function
2023-05-08T10:46:35.4996199Z     compiled_fn = compiler_fn(flat_fn, fake_flat_args, aot_config, fw_metadata=fw_metadata)
2023-05-08T10:46:35.4996794Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1911, in aot_wrapper_dedupe
2023-05-08T10:46:35.4997237Z     return compiler_fn(flat_fn, leaf_flat_args, aot_config, fw_metadata=fw_metadata)
2023-05-08T10:46:35.4997851Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 2082, in aot_wrapper_synthetic_base
2023-05-08T10:46:35.4998281Z     return compiler_fn(flat_fn, flat_args, aot_config, fw_metadata=fw_metadata)
2023-05-08T10:46:35.4998904Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1348, in aot_dispatch_base
2023-05-08T10:46:35.4999318Z     compiled_fw = compiler(fw_module, adjusted_flat_args)
2023-05-08T10:46:35.4999841Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 177, in time_wrapper
2023-05-08T10:46:35.5000170Z     r = func(*args, **kwargs)
2023-05-08T10:46:35.5000673Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 684, in fw_compiler_base
2023-05-08T10:46:35.5001026Z     return inner_compile(
2023-05-08T10:46:35.5001505Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/repro/after_aot.py", line 83, in debug_wrapper
2023-05-08T10:46:35.5001906Z     inner_compiled_fn = compiler_fn(gm, example_inputs)
2023-05-08T10:46:35.5002432Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/debug.py", line 220, in inner
2023-05-08T10:46:35.5002768Z     return fn(*args, **kwargs)
2023-05-08T10:46:35.5003067Z   File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner
2023-05-08T10:46:35.5003371Z     return func(*args, **kwds)
2023-05-08T10:46:35.5003878Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 210, in compile_fx_inner
2023-05-08T10:46:35.5004228Z     graph.run(*example_inputs)
2023-05-08T10:46:35.5004711Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 177, in time_wrapper
2023-05-08T10:46:35.5005376Z     r = func(*args, **kwargs)
2023-05-08T10:46:35.5005872Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/graph.py", line 249, in run
2023-05-08T10:46:35.5006375Z     return super().run(*args)
2023-05-08T10:46:35.5006855Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/interpreter.py", line 138, in run
2023-05-08T10:46:35.5007208Z     self.env[node] = self.run_node(node)
2023-05-08T10:46:35.5007678Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/graph.py", line 476, in run_node
2023-05-08T10:46:35.5008076Z     result = fallback_handler(n.target, add_to_fallback_set=False)(
2023-05-08T10:46:35.5008612Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/lowering.py", line 1043, in handler
2023-05-08T10:46:35.5009313Z     TensorBox.create, ir.FallbackKernel.create(kernel, *args, **kwargs)
2023-05-08T10:46:35.5009905Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 3173, in create
2023-05-08T10:46:35.5010251Z     packed = FallbackKernel(
2023-05-08T10:46:35.5010743Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 3122, in __init__
2023-05-08T10:46:35.5011048Z     assert (
2023-05-08T10:46:35.5011444Z torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
2023-05-08T10:46:35.5011834Z AssertionError: slice.Tensor is not supported with cpp wrapper
2023-05-08T10:46:35.5012039Z 
2023-05-08T10:46:35.5012045Z 
2023-05-08T10:46:35.5012212Z You can suppress this exception and fall back to eager by setting:
2023-05-08T10:46:35.5012597Z     import torch._dynamo
2023-05-08T10:46:35.5012910Z     torch._dynamo.config.suppress_errors = True
2023-05-08T10:46:35.5013095Z 
2023-05-08T10:46:35.5013274Z TorchDynamo optimized model failed to run because of following error
2023-05-08T10:46:35.5014490Z fail_to_run
```

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh @anijain2305