StableDiffusion with dynamic=True still recompiles
Labels: triaged, oncall: pt2, module: dynamic shapes

              @ezyang It completed to run, but still took a long time if the input shape changed. Already upgraded pytorch to `2.1.0.dev20230706`.

```python
from diffusers import StableDiffusionPipeline
import torch
import torch._dynamo
import datetime


torch._dynamo.config.dynamic_shapes = True
torch._dynamo.config.automatic_dynamic_shapes = True
torch._dynamo.config.assume_static_by_default = True


pipe = StableDiffusionPipeline.from_pretrained(
    "SG161222/Realistic_Vision_V2.0", torch_dtype=torch.float16
)
pipe.to("cuda:0")
pipe.unet = torch.compile(pipe.unet)

start = datetime.datetime.now()
pipe(prompt="prompt", height=512, width=512)
first_done = datetime.datetime.now()
pipe(prompt="prompt", height=768, width=768)
second_done = datetime.datetime.now()

print("first elapsed:", first_done - start)
print("second elapsed:", second_done - first_done)
```

Log:

```
  0%|          | 0/50 [00:00<?, ?it/s]Using FallbackKernel: aten._scaled_dot_product_efficient_attention
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
100%|██████████| 50/50 [01:14<00:00,  1.49s/it]
  0%|          | 0/50 [00:00<?, ?it/s]Using FallbackKernel: aten._scaled_dot_product_efficient_attention
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
reduction over non-contiguous dims
100%|██████████| 50/50 [00:51<00:00,  1.04s/it]
first elapsed: 0:01:15.543955
second elapsed: 0:00:52.173735
```

Not sure if this `reduction over non-contiguous dims` warning indicates anything ?

_Originally posted by @sunhs in https://github.com/pytorch/pytorch/issues/103587#issuecomment-1625324835_
            

cc @msaroufim @wconstab @bdhirsh @anijain2305