Repository: pytorch
Issue: [AudioLM] Inductor `LoweringException: NotImplementedError: View`
Labels: triaged,oncall: pt2,module: inductor

### 🐛 Describe the bug

This is a carryover from https://github.com/pytorch/pytorch/issues/121345#issuecomment-2492686541. The model no longer fails in Dynamo on latest main, but fails in Inductor.

Repro code: https://gist.github.com/ezyang/64c24c9fc5529f3afed4ee4266f6adc5

### Error logs

```
2024-11-25 11:49:55 | INFO | fairseq.tasks.text_to_speech | Please install tensorboardX: pip install tensorboardX
training with dataset of 2 samples and validating with randomly splitted 1 samples
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0] Graph break from `Tensor.item()`, consider setting:
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]     torch._dynamo.config.capture_scalar_outputs = True
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0] or:
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]     env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0] to include these operations in the captured graph.
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0] Graph break: from user code at:
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]   File "/home/ryanguo99/.conda/envs/audiolm/lib/python3.10/site-packages/audiolm_pytorch/soundstream.py", line 840, in forward
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]     x, indices, commit_loss = self.rq(x)
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]   File "/home/ryanguo99/.conda/envs/audiolm/lib/python3.10/site-packages/vector_quantize_pytorch/residual_vq.py", line 498, in forward
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]     rand_quantize_dropout_fixed_seed = get_maybe_sync_seed(device) if self.training else None
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]   File "/home/ryanguo99/.conda/envs/audiolm/lib/python3.10/site-packages/vector_quantize_pytorch/residual_vq.py", line 50, in get_maybe_sync_seed
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]     return rand_int.item()
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]
W1125 11:50:00.561000 1134905 torch/_dynamo/variables/tensor.py:869] [0/0]
W1125 11:50:30.329000 1134905 torch/_dynamo/convert_frame.py:915] [23/8] torch._dynamo hit config.cache_size_limit (8)
W1125 11:50:30.329000 1134905 torch/_dynamo/convert_frame.py:915] [23/8]    function: 'forward' (/home/ryanguo99/pt/torchaudio/src/torchaudio/transforms/_transforms.py:100)
W1125 11:50:30.329000 1134905 torch/_dynamo/convert_frame.py:915] [23/8]    last reason: 23/0: tensor 'L['self']._buffers['window']' size mismatch at index 0. expected 64, actual 1024
W1125 11:50:30.329000 1134905 torch/_dynamo/convert_frame.py:915] [23/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W1125 11:50:30.329000 1134905 torch/_dynamo/convert_frame.py:915] [23/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html.
Traceback (most recent call last):
  File "/home/ryanguo99/pt/scratch/compile.py", line 139, in <module>
    trainer.train()
  File "/home/ryanguo99/.conda/envs/audiolm/lib/python3.10/site-packages/audiolm_pytorch/trainer.py", line 710, in train
    logs = self.train_step()
  File "/home/ryanguo99/.conda/envs/audiolm/lib/python3.10/site-packages/audiolm_pytorch/trainer.py", line 578, in train_step
    loss, (recon_loss, multi_spectral_recon_loss, adversarial_loss, feature_loss, all_commitment_loss) = self.soundstream(wave, return_loss_breakdown = True)
  File "/home/ryanguo99/pt/pytorch-310/torch/nn/modules/module.py", line 1738, in _wrapped_call_impl
    return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/eval_frame.py", line 569, in _fn
    return fn(*args, **kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/nn/modules/module.py", line 1751, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ryanguo99/.conda/envs/audiolm/lib/python3.10/site-packages/audiolm_pytorch/soundstream.py", line 840, in forward
    x, indices, commit_loss = self.rq(x)
  File "/home/ryanguo99/.conda/envs/audiolm/lib/python3.10/site-packages/audiolm_pytorch/soundstream.py", line 956, in torch_dynamo_resume_in_forward_at_840
    (stft_real_logits, stft_real_intermediates), (stft_fake_logits, stft_fake_intermediates) = map(partial(self.stft_discriminator, return_intermediates=True), (real, fake))
  File "/home/ryanguo99/pt/pytorch-310/torch/nn/modules/module.py", line 1740, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/nn/modules/module.py", line 1751, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/convert_frame.py", line 1414, in __call__
    return self._torchdynamo_orig_callable(
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/convert_frame.py", line 1198, in __call__
    result = self._inner_convert(
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/convert_frame.py", line 550, in __call__
    return _compile(
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/convert_frame.py", line 995, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/convert_frame.py", line 717, in compile_inner
    return _compile_inner(code, one_graph, hooks, transform)
  File "/home/ryanguo99/pt/pytorch-310/torch/_utils_internal.py", line 95, in wrapper_function
    return function(*args, **kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/convert_frame.py", line 752, in _compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/bytecode_transformation.py", line 1349, in transform_code_object
    transformations(instructions, code_options)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/convert_frame.py", line 234, in _fn
    return fn(*args, **kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/convert_frame.py", line 665, in transform
    tracer.run()
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/symbolic_convert.py", line 2865, in run
    super().run()
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/symbolic_convert.py", line 1054, in run
    while self.step():
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/symbolic_convert.py", line 964, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/symbolic_convert.py", line 3045, in RETURN_VALUE
    self._return(inst)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/symbolic_convert.py", line 3030, in _return
    self.output.compile_subgraph(
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/output_graph.py", line 1117, in compile_subgraph
    self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/output_graph.py", line 1358, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/output_graph.py", line 1408, in call_user_compiler
    return self._call_user_compiler(gm)
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/output_graph.py", line 1459, in _call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/output_graph.py", line 1438, in _call_user_compiler
    compiled_fn = compiler_fn(gm, self.example_inputs())
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/repro/after_dynamo.py", line 130, in __call__
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/home/ryanguo99/pt/pytorch-310/torch/__init__.py", line 2301, in __call__
    return compile_fx(model_, inputs_, config_patches=self.config)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/compile_fx.py", line 1735, in compile_fx
    return aot_autograd(
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/backends/common.py", line 73, in __call__
    cg = aot_module_simplified(gm, example_inputs, **self.kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_functorch/aot_autograd.py", line 1103, in aot_module_simplified
    compiled_fn = dispatch_and_compile()
  File "/home/ryanguo99/pt/pytorch-310/torch/_functorch/aot_autograd.py", line 1079, in dispatch_and_compile
    compiled_fn, _ = create_aot_dispatcher_function(
  File "/home/ryanguo99/pt/pytorch-310/torch/_functorch/aot_autograd.py", line 527, in create_aot_dispatcher_function
    return _create_aot_dispatcher_function(
  File "/home/ryanguo99/pt/pytorch-310/torch/_functorch/aot_autograd.py", line 778, in _create_aot_dispatcher_function
    compiled_fn, fw_metadata = compiler_fn(
  File "/home/ryanguo99/pt/pytorch-310/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py", line 655, in aot_dispatch_autograd
    compiled_fw_func = aot_config.fw_compiler(fw_module, adjusted_flat_args)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/compile_fx.py", line 1548, in fw_compiler_base
    return _fw_compiler_base(model, example_inputs, is_inference)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/compile_fx.py", line 1617, in _fw_compiler_base
    return inner_compile(
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/compile_fx.py", line 600, in compile_fx_inner
    return wrap_compiler_debug(_compile_fx_inner, compiler_name="inductor")(
  File "/home/ryanguo99/pt/pytorch-310/torch/_dynamo/repro/after_aot.py", line 102, in debug_wrapper
    inner_compiled_fn = compiler_fn(gm, example_inputs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/compile_fx.py", line 757, in _compile_fx_inner
    compiled_graph = FxGraphCache.load(
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/codecache.py", line 1588, in load
    compiled_graph = compile_fx_fn(
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/compile_fx.py", line 664, in codegen_and_compile
    compiled_graph = fx_codegen_and_compile(gm, example_inputs, **fx_kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/compile_fx.py", line 956, in fx_codegen_and_compile
    graph.run(*example_inputs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/graph.py", line 828, in run
    return super().run(*args)
  File "/home/ryanguo99/pt/pytorch-310/torch/fx/interpreter.py", line 167, in run
    self.env[node] = self.run_node(node)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/graph.py", line 1450, in run_node
    result = super().run_node(n)
  File "/home/ryanguo99/pt/pytorch-310/torch/fx/interpreter.py", line 228, in run_node
    return getattr(self, n.op)(n.target, args, kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/graph.py", line 1097, in call_function
    raise LoweringException(e, target, args, kwargs).with_traceback(
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/graph.py", line 1087, in call_function
    out = lowerings[target](*args, **kwargs)  # type: ignore[index]
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/lowering.py", line 401, in wrapped
    out = decomp_fn(*args, **kwargs)
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/kernel/conv.py", line 581, in convolution
    bias.freeze_layout()
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/ir.py", line 6808, in freeze_layout
    return self.data.freeze_layout()
  File "/home/ryanguo99/pt/pytorch-310/torch/_inductor/ir.py", line 539, in freeze_layout
    raise NotImplementedError(type(self).__name__)
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
LoweringException: NotImplementedError: View
  target: aten.convolution.default
  args[0]: TensorBox(
    ReinterpretView(
      StorageBox(
        MultiOutput(
          python_kernel_name=None,
          name=buf14,
          layout=FixedLayout('cpu', torch.float32, size=[1, 1, 513, 41, 2], stride=[42066, 1026, 2, 1026, 1]),
          inputs=[FallbackKernel(
            python_kernel_name='torch.ops.aten.view_as_real.default',
            name=buf13,
            layout=MultiOutputLayout(device=device(type='cpu')),
            inputs=[MultiOutput(
              python_kernel_name=None,
              name=buf8,
              layout=FixedLayout('cpu', torch.complex64, size=[1, 1, 513, 41], stride=[21033, 513, 1, 513]),
              inputs=[FallbackKernel(
                python_kernel_name='torch.ops.aten.reshape.default',
                name=buf7,
                layout=MultiOutputLayout(device=device(type='cpu')),
                inputs=[MultiOutput(
                  python_kernel_name=None,
                  name=buf6,
                  layout=FixedLayout('cpu', torch.complex64, size=[1, 513, 41], stride=[21033, 1, 513]),
                  inputs=[FallbackKernel(
                    python_kernel_name='torch.ops.aten.permute.default',
                    name=buf5,
                    layout=MultiOutputLayout(device=device(type='cpu')),
                    inputs=[MultiOutput(
                      python_kernel_name=None,
                      name=buf4,
                      layout=FixedLayout('cpu', torch.complex64, size=[1, 41, 513], stride=[21033, 513, 1]),
                      inputs=[FallbackKernel(
                        python_kernel_name='torch.ops.aten._fft_r2c.default',
                        name=buf3,
                        layout=MultiOutputLayout(device=device(type='cpu')),
                        inputs=[ComputedBuffer(name='buf2', layout=FixedLayout('cpu', torch.float32, size=[1, 41, 1024], stride=[41984, 1024, 1]), data=Pointwise(device=device(type='cpu'), dtype=torch.float32, inner_fn=<function make_pointwise.<locals>.inner.<locals>.inner_fn at 0x7f927f995870>, ranges=[1, 41, 1024]))],
                        constant_args=(2, 0, True),
                        kwargs={},
                        output_view=None,
                        python_kernel_name=torch.ops.aten._fft_r2c.default,
                        cpp_kernel_name=None,
                        ordered_kwargs_for_cpp_kernel=[],

                    kwarg_properties=None,
                    unbacked_bindings=None,
                    mutation_outputs=[],
                    origin_node=permute,
                    origins=OrderedSet([permute])
                  )],
                  constant_args=(),
                  kwargs={},
                  output_view=None,
                  python_kernel_name=None,
                  cpp_kernel_name=None,
                  ordered_kwargs_for_cpp_kernel=(),
                  op_overload=None,
                  arg_properties=[{}],
                  kwarg_properties=None,
                  unbacked_bindings={},
                  mutation_outputs=[],
                  origin_node=permute,
                  origins=OrderedSet([permute])
                )],
                constant_args=(1, 1, 513, 41),
                kwargs={},
                output_view=None,
                python_kernel_name=torch.ops.aten.reshape.default,
                cpp_kernel_name=None,
                ordered_kwargs_for_cpp_kernel=[],
                op_overload=aten.reshape.default,
                arg_properties=[{'name': 'self', 'type': Tensor, 'default_value': None}, {'name': 'shape', 'type': List[int], 'default_value': None}],
                kwarg_properties=None,
                unbacked_bindings=None,
                mutation_outputs=[],
                origin_node=view_3,
                origins=OrderedSet([view_3])
              )],
              constant_args=(),
              kwargs={},
              output_view=None,
              python_kernel_name=None,
              cpp_kernel_name=None,
              ordered_kwargs_for_cpp_kernel=(),
              op_overload=None,
              arg_properties=[{}],
              kwarg_properties=None,
              unbacked_bindings={},
              mutation_outputs=[],
              origin_node=view_3,
              origins=OrderedSet([view_3])
            )],
            constant_args=(),
            kwargs={},
            output_view=None,
            python_kernel_name=torch.ops.aten.view_as_real.default,
            cpp_kernel_name=None,
            ordered_kwargs_for_cpp_kernel=[],
            op_overload=aten.view_as_real.default,
            arg_properties=[{'name': 'self', 'type': Tensor, 'default_value': None}],

                kwargs={},
                output_view=None,
                python_kernel_name=torch.ops.aten.view_as_complex.default,
                cpp_kernel_name=None,
                ordered_kwargs_for_cpp_kernel=[],
                op_overload=aten.view_as_complex.default,
                arg_properties=[{'name': 'self', 'type': Tensor, 'default_value': None}],
                kwarg_properties=None,
                unbacked_bindings=None,
                mutation_outputs=[],
                origin_node=view_as_complex,
                origins=OrderedSet([view_as_complex])
              )],
              constant_args=(),
              kwargs={},
              output_view=None,
              python_kernel_name=None,
              cpp_kernel_name=None,
              ordered_kwargs_for_cpp_kernel=(),
              op_overload=None,
              arg_properties=[{}],
              kwarg_properties=None,
              unbacked_bindings={},
              mutation_outputs=[],
              origin_node=view_as_complex,
              origins=OrderedSet([view_as_complex])
            )],
            constant_args=(),
            kwargs={},
            output_view=None,
            python_kernel_name=torch.ops.aten.view_as_real.default,
            cpp_kernel_name=None,
            ordered_kwargs_for_cpp_kernel=[],
            op_overload=aten.view_as_real.default,
            arg_properties=[{'name': 'self', 'type': Tensor, 'default_value': None}],
            kwarg_properties=None,
            unbacked_bindings=None,
            mutation_outputs=[],
            origin_node=view_as_real_1,
            origins=OrderedSet([view_as_real_1])
          )],
          constant_args=(),
          kwargs={},
          output_view=None,
          python_kernel_name=None,
          cpp_kernel_name=None,
          ordered_kwargs_for_cpp_kernel=(),
          op_overload=None,
          arg_properties=[{}],
          kwarg_properties=None,
          unbacked_bindings={},
          mutation_outputs=[],
          origin_node=view_as_real_1,
          origins=OrderedSet([view_as_real_1])
        )
      ),
      FixedLayout('cpu', torch.float32, size=[32, 1, 7, 7], stride=
                unbacked_bindings={},
                mutation_outputs=[],
                origin_node=view_as_complex_1,
                origins=OrderedSet([view_as_complex_1])
              )],
              constant_args=(),
              kwargs={},
              output_view=None,
              python_kernel_name=torch.ops.aten.view_as_real.default,
              cpp_kernel_name=None,
              ordered_kwargs_for_cpp_kernel=[],
              op_overload=aten.view_as_real.default,
              arg_properties=[{'name': 'self', 'type': Tensor, 'default_value': None}],
              kwarg_properties=None,
              unbacked_bindings=None,
              mutation_outputs=[],
              origin_node=view_as_real_2,
              origins=OrderedSet([view_as_real_2])
            )],
            constant_args=(),
            kwargs={},
            output_view=None,
            python_kernel_name=None,
            cpp_kernel_name=None,
            ordered_kwargs_for_cpp_kernel=(),
            op_overload=None,
            arg_properties=[{}],
            kwarg_properties=None,
            unbacked_bindings={},
            mutation_outputs=[],
            origin_node=view_as_real_2,
            origins=OrderedSet([view_as_real_2])
          )
        ),
        FixedLayout('cpu', torch.float32, size=[32, 1], stride=[2, 1]),
        origins=OrderedSet([select_4])
      ),
      size=[32],
      reindex=lambda i0: [i0, 0],
      origins=OrderedSet([select_4])
    )
  )
  args[3]: [1, 1]
  args[4]: [3, 3]
  args[5]: [1, 1]
  args[6]: False
  args[7]: [0, 0]
  args[8]: 1

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True
```

### Versions

Python 3.10, da94ab0, CUDA 12.4

cc @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @aakhundov