Repository: pytorch
Issue: Non actionable perf hint: reduction over non-contiguous dims
Labels: feature,triaged,enhancement,oncall: pt2,module: inductor

### 🐛 Describe the bug

This is the exact same repro as https://github.com/pytorch/pytorch/issues/103715 but a different issue

When using SAM with SDPA and torch.compile I get this perf hint coming from `torch/_inductor/codegen/triton.py`

```
reduction over non-contiguous dims
reduction over non-contiguous dims
```

But it doesn't really tell me what I can change in my model code to fix this or which line of code in my program is causing this


## Setup

```
pip install git+https://github.com/cpuhrsch/saf.git@msaroufim/sdpa
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121
pip install nvtx
wget https://gist.githubusercontent.com/msaroufim/1002c17469fc670fc5af27dda12fa7e4/raw/c22a381f359da08c7efb4c87959efbd00f688bac/benchmark_segment_anything.py
```

## Command

```
python benchmark_segment_anything.py --model_type "vit_h" --checkpoint_path "/data/home/vasiliy/cluster/sam_dataset/sam_vit_h_4b8939.pth" --compiled 1  --profile 0 --max_batch_size 30 --block_only 0 --quantized 0
```

## Logs

OOM issue was reported seperately

```
(base) marksaroufim@a100-st-p4d24xlarge-42:~$ python bench2.py --model_type "vit_h" --checkpoint_path "/data/home/vasiliy/cluster/sam_dataset/sam_vit_h_4b8939.pth" --compiled 1  --profile 0 --max_batch_size 30 --block_only 0 --quantized 0
/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/utils/benchmark/utils/timer.py:16: UserWarning: 'has_cuda' is deprecated, please use 'torch.backends.cuda.is_built()'
  if torch.has_cuda and torch.cuda.is_available():
Namespace(model_type='vit_h', checkpoint_path='/data/home/vasiliy/cluster/sam_dataset/sam_vit_h_4b8939.pth', max_batch_size=30, quantized=0, compiled=1, block_only=0, profile=0)
AUTOTUNE convolution(30x3x1024x1024, 1280x3x16x16)
  convolution 14.1885 ms 100.0%
  triton_convolution_0 35.4304 ms 40.0%
  triton_convolution_3 51.1130 ms 27.8%
  triton_convolution_1 53.7979 ms 26.4%
  triton_convolution_6 57.7577 ms 24.6%
  triton_convolution_5 62.8029 ms 22.6%
  triton_convolution_4 93.8199 ms 15.1%
  triton_convolution_2 104.7224 ms 13.5%
SingleProcess AUTOTUNE takes 13.5646 seconds
AUTOTUNE mm(147000x1280, 1280x3840)
  mm 6.2659 ms 100.0%
  triton_mm_8 7.3764 ms 84.9%
  triton_mm_9 7.5295 ms 83.2%
  triton_mm_10 8.7255 ms 71.8%
  triton_mm_11 8.7378 ms 71.7%
  triton_mm_7 9.6256 ms 65.1%
  triton_mm_15 10.3578 ms 60.5%
  triton_mm_14 11.8845 ms 52.7%
  triton_mm_17 14.8060 ms 42.3%
  triton_mm_16 19.0935 ms 32.8%
SingleProcess AUTOTUNE takes 9.0483 seconds
AUTOTUNE bmm(12000x196x80, 12000x80x196)
  triton_bmm_21 1.6814 ms 100.0%
  triton_bmm_20 1.6988 ms 99.0%
  triton_bmm_26 1.7674 ms 95.1%
  triton_bmm_19 1.8811 ms 89.4%
  triton_bmm_23 1.8862 ms 89.1%
  triton_bmm_22 1.9661 ms 85.5%
  triton_bmm_29 2.0142 ms 83.5%
  bmm 2.3460 ms 71.7%
  triton_bmm_30 2.4054 ms 69.9%
  triton_bmm_27 3.0884 ms 54.4%
SingleProcess AUTOTUNE takes 6.8304 seconds
AUTOTUNE bmm(12000x196x196, 12000x196x80)
  triton_bmm_32 1.4879 ms 100.0%
  triton_bmm_34 1.6292 ms 91.3%
  triton_bmm_33 1.6312 ms 91.2%
  triton_bmm_35 1.7828 ms 83.5%
  bmm 1.7992 ms 82.7%
  triton_bmm_41 2.0500 ms 72.6%
  triton_bmm_31 2.0634 ms 72.1%
  triton_bmm_39 2.4177 ms 61.5%
  triton_bmm_38 2.4668 ms 60.3%
  triton_bmm_42 2.4893 ms 59.8%
SingleProcess AUTOTUNE takes 6.5458 seconds
AUTOTUNE mm(122880x1280, 1280x5120)
  mm 7.3851 ms 100.0%
  triton_mm_44 8.4147 ms 87.8%
  triton_mm_45 8.5765 ms 86.1%
  triton_mm_46 9.8478 ms 75.0%
  triton_mm_47 9.9625 ms 74.1%
  triton_mm_43 11.2773 ms 65.5%
  triton_mm_51 11.6705 ms 63.3%
  triton_mm_50 13.6346 ms 54.2%
  triton_mm_53 16.8397 ms 43.9%
  triton_mm_52 21.2070 ms 34.8%
SingleProcess AUTOTUNE takes 7.3337 seconds
AUTOTUNE mm(122880x5120, 5120x1280)
  triton_mm_57 8.2811 ms 100.0%
  triton_mm_56 8.2949 ms 99.8%
  triton_mm_58 9.3793 ms 88.3%
  triton_mm_59 9.6061 ms 86.2%
  mm 10.5482 ms 78.5%
  triton_mm_63 11.1032 ms 74.6%
  triton_mm_55 11.1974 ms 74.0%
  triton_mm_62 13.4717 ms 61.5%
  triton_mm_65 17.1244 ms 48.4%
  triton_mm_60 20.5896 ms 40.2%
SingleProcess AUTOTUNE takes 7.2971 seconds
AUTOTUNE mm(122880x1280, 1280x3840)
  mm 5.2255 ms 100.0%
  triton_mm_428 6.1379 ms 85.1%
  triton_mm_429 6.2868 ms 83.1%
  triton_mm_430 7.3503 ms 71.1%
  triton_mm_431 7.4537 ms 70.1%
  triton_mm_427 8.1193 ms 64.4%
  triton_mm_435 8.6641 ms 60.3%
  triton_mm_434 9.9072 ms 52.7%
  triton_mm_437 12.3535 ms 42.3%
  triton_mm_436 15.9247 ms 32.8%
SingleProcess AUTOTUNE takes 7.0755 seconds
AUTOTUNE convolution(30x1280x64x64, 256x1280x1x1)
  convolution 0.8335 ms 100.0%
  triton_convolution_1831 2.1627 ms 38.5%
  triton_convolution_1832 2.2426 ms 37.2%
  triton_convolution_1834 2.2477 ms 37.1%
  triton_convolution_1836 2.2958 ms 36.3%
  triton_convolution_1835 2.4504 ms 34.0%
  triton_convolution_1837 2.6307 ms 31.7%
  conv1x1_via_mm 11.3367 ms 7.4%
  triton_convolution_1833 17.4961 ms 4.8%
SingleProcess AUTOTUNE takes 7.2176 seconds
AUTOTUNE convolution(30x256x64x64, 256x256x3x3)
  convolution 1.3814 ms 100.0%
  triton_convolution_1839 3.9660 ms 34.8%
  triton_convolution_1844 4.0305 ms 34.3%
  triton_convolution_1841 4.9162 ms 28.1%
  triton_convolution_1842 7.7573 ms 17.8%
  triton_convolution_1843 7.8566 ms 17.6%
  triton_convolution_1838 8.1787 ms 16.9%
  triton_convolution_1840 33.1822 ms 4.2%
SingleProcess AUTOTUNE takes 6.9797 seconds
reduction over non-contiguous dims
reduction over non-contiguous dims
/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/cudagraph_trees.py:85: UserWarning: 'has_cuda' is deprecated, please use 'torch.backends.cuda.is_built()'
  if torch.has_cuda:
Traceback (most recent call last):
  File "/data/home/marksaroufim/bench2.py", line 144, in <module>
    main()
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/data/home/marksaroufim/bench2.py", line 123, in main
    model(image)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 295, in _fn
    return fn(*args, **kwargs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/segment_anything/modeling/image_encoder.py", line 106, in forward
    def forward(self, x: torch.Tensor) -> torch.Tensor:
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 295, in _fn
    return fn(*args, **kwargs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 17, in inner
    return fn(*args, **kwargs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 3761, in forward
    return compiled_fn(full_args)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1449, in g
    return f(*args)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 2407, in runtime_wrapper
    all_outs = call_func_with_args(
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1473, in call_func_with_args
    out = normalize_as_list(f(args))
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1561, in rng_functionalization_wrapper
    return compiled_fw(args)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 457, in run
    return model(new_inputs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 499, in run
    return compiled_fn(new_inputs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/cudagraph_trees.py", line 360, in deferred_cudagraphify
    fn, out = cudagraphify(model, inputs, static_input_idxs, *args, **kwargs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/cudagraph_trees.py", line 384, in cudagraphify
    return manager.add_function(
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/cudagraph_trees.py", line 1856, in add_function
    return fn, fn(inputs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/cudagraph_trees.py", line 1676, in run
    out = self._run(new_inputs, function_id)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/cudagraph_trees.py", line 1717, in _run
    return self.run_eager(new_inputs, function_id)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/cudagraph_trees.py", line 1832, in run_eager
    return node.run(new_inputs)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/_inductor/cudagraph_trees.py", line 572, in run
    out = self.wrapped_function.model(new_inputs)
  File "/tmp/torchinductor_marksaroufim/va/cva4viinufgivaehrtjftdor756fg7lfvjjbvhjw525cii43a76w.py", line 3329, in call
    buf156 = empty_strided((480, 4096, 4096), (16777216, 4096, 1), device='cuda', dtype=torch.float16)
  File "/data/home/marksaroufim/miniconda/lib/python3.10/site-packages/torch/utils/_device.py", line 76, in __torch_function__
    return func(*args, **kwargs)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 15.00 GiB. GPU 0 has a total capacty of 39.56 GiB of which 11.15 GiB is free. Including non-PyTorch memory, this process has 28.41 GiB memory in use. Of the allocated memory 25.28 GiB is allocated by PyTorch, and 1009.77 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

```

### Error logs

_No response_

### Minified repro

_No response_

### Versions

nightlies on aws cluster

cc @ezyang @wconstab @bdhirsh @anijain2305 @zou3519 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @Xia-Weiwen @ngimel