Repository: pytorch
Issue: NVFuser bad "reshape" performance
Labels: triaged,module: nvfuser

### 🐛 Describe the bug

Repro script:
```python
import torch
from torch.utils.benchmark import Timer
from torch.utils.jit.log_extract import no_fuser

"""
graph(%0 : Float(1, 12, 1, 64, 64, strides=[768, 64, 49152, 768, 1], requires_grad=0, device=cuda:0)):
  %1 : int[] = prim::Constant[value=[1, 12, 64, 64, 1, 1]]()
  %2 : int[] = prim::Constant[value=[1, 12, 64, 64, 1]]()
  %3 : int[] = prim::Constant[value=[1, 12, 64, 64]]()
  %4 : Float(1, 12, 64, 64, strides=[768, 64, 768, 1], requires_grad=0, device=cuda:0) = aten::reshape(%0, %3)
  %5 : Float(1, 12, 64, 64, 1, strides=[768, 64, 768, 1, 1], requires_grad=0, device=cuda:0) = aten::reshape(%4, %2)
  %6 : Float(1, 12, 64, 64, 1, 1, strides=[768, 64, 768, 1, 1, 1], requires_grad=0, device=cuda:0) = aten::reshape(%5, %1)
"""

source = torch.rand(49152).cuda()
inp = source.as_strided([1, 12, 1, 64, 64], [768, 64, 49152, 768, 1])

def fn(a):
    b = a.reshape((1, 12, 64, 64))
    c = b.reshape((1, 12, 64, 64, 1))
    d = c.reshape((1, 12, 64, 64, 1, 1))
    return d

fn.__disable_jit_function_caching__ = True

def time_fn(fn, inputs):
    for _ in range(5):
        fn(*inputs)
    t = Timer(stmt="fn(*inputs)", globals={"fn": fn, "inputs": inputs})
    times = t.blocked_autorange()
    return times.median * 1000  # in ms

def time_no_fuser(fn, inputs):
    with no_fuser():
        fn_s = torch.jit.script(fn)
        return time_fn(fn_s, inputs)

def time_nvfuser(fn, inputs):
    with torch.jit.fuser("fuser2"):
        fn_s = torch.jit.script(fn)
        return time_fn(fn_s, inputs)

print("No fuser:", time_no_fuser(fn, [inp]))
print("NVfuser :", time_nvfuser(fn, [inp]))
```

More context: this is based on nvfuser microbenchmarks.

Below is a CSV showing benchmark results (on A100) from microbenchmarks https://github.com/pytorch/benchmark/pull/801. The CSV is sorted by increasing "nvfuser vs none", which is the performance improvement (positive is good). Note that a number of microbenchmarks are omitted because they fail fallbacks.

```csv
model,no fuser - ms,nnc - ms,nvfuser - ms,nvfuser vs none,nvfuser vs nnc,
autogen-47,0.207,0.206,0.206,39.415,-99.47%,-99.48%,-99.48%
autogen-15,0.217,0.219,0.217,27.95,-99.22%,-99.22%,-99.22%
autogen-38,0.253,0.249,0.247,28.641,-99.12%,-99.13%,-99.14%
autogen-24,0.497,0.191,0.222,34.181,-98.55%,-99.44%,-99.35%
autogen-33,0.374,0.254,0.281,24.615,-98.48%,-98.97%,-98.86%
autogen-42,0.569,0.294,0.327,36.55,-98.44%,-99.20%,-99.11%
autogen-35,0.521,0.379,0.4,32.585,-98.40%,-98.84%,-98.77%
autogen-69,0.439,0.3,0.333,25.057,-98.25%,-98.80%,-98.67%
autogen-46,0.927,0.293,0.326,38.361,-97.58%,-99.24%,-99.15%
autogen-09,0.132,0.13,0.13,0.242,-45.45%,-46.28%,-46.28%
autogen-31,0.096,0.094,0.095,0.167,-42.51%,-43.71%,-43.11%
autogen-50,0.177,0.174,0.172,0.247,-28.34%,-29.55%,-30.36%
autogen-57,0.167,0.163,0.159,0.229,-27.07%,-28.82%,-30.57%
autogen-01,0.165,0.162,0.165,0.21,-21.43%,-22.86%,-21.43%
autogen-41,0.146,0.145,0.144,0.173,-15.61%,-16.18%,-16.76%
autogen-10,0.156,0.156,0.154,0.182,-14.29%,-14.29%,-15.38%
autogen-23,0.303,0.301,0.3,0.351,-13.68%,-14.25%,-14.53%
autogen-07,0.178,0.178,0.174,0.204,-12.75%,-12.75%,-14.71%
autogen-18,0.274,0.273,0.271,0.303,-9.57%,-9.90%,-10.56%
autogen-16,0.224,0.219,0.231,0.245,-8.57%,-10.61%,-5.71%
autogen-68,0.351,0.332,0.356,0.38,-7.63%,-12.63%,-6.32%
autogen-32,0.198,0.194,0.189,0.212,-6.60%,-8.49%,-10.85%
autogen-70,0.178,0.176,0.174,0.19,-6.32%,-7.37%,-8.42%
autogen-65,0.246,0.246,0.246,0.262,-6.11%,-6.11%,-6.11%
autogen-12,0.178,0.133,0.166,0.185,-3.78%,-28.11%,-10.27%
autogen-26,1.181,1.181,1.18,1.225,-3.59%,-3.59%,-3.67%
autogen-74,0.218,0.156,0.192,0.217,0.46%,-28.11%,-11.52%
autogen-21,0.313,0.291,0.319,0.292,7.19%,-0.34%,9.25%
autogen-34,0.37,0.364,0.365,0.345,7.25%,5.51%,5.80%
autogen-11,0.207,0.201,0.201,0.187,10.70%,7.49%,7.49%
autogen-56,0.373,0.37,0.371,0.335,11.34%,10.45%,10.75%
autogen-72,0.259,0.237,0.274,0.219,18.26%,8.22%,25.11%
autogen-59,0.286,0.245,0.271,0.237,20.68%,3.38%,14.35%
autogen-54,0.212,0.209,0.212,0.172,23.26%,21.51%,23.26%
autogen-61,0.277,0.272,0.272,0.224,23.66%,21.43%,21.43%
autogen-13,0.36,0.36,0.36,0.289,24.57%,24.57%,24.57%
autogen-27,0.37,0.364,0.365,0.297,24.58%,22.56%,22.90%
autogen-36,0.469,0.475,0.473,0.342,37.13%,38.89%,38.30%
autogen-49,0.42,0.344,0.374,0.293,43.34%,17.41%,27.65%
autogen-20,0.425,0.403,0.436,0.292,45.55%,38.01%,49.32%
autogen-17,0.454,0.332,0.398,0.311,45.98%,6.75%,27.97%
autogen-08,0.362,0.181,0.198,0.246,47.15%,-26.42%,-19.51%
autogen-29,0.39,0.385,0.386,0.264,47.73%,45.83%,46.21%
autogen-67,0.441,0.332,0.341,0.297,48.48%,11.78%,14.81%
autogen-25,0.411,0.268,0.283,0.273,50.55%,-1.83%,3.66%
autogen-37,0.332,0.189,0.221,0.214,55.14%,-11.68%,3.27%
autogen-55,0.544,0.415,0.463,0.349,55.87%,18.91%,32.66%
autogen-60,0.45,0.336,0.36,0.276,63.04%,21.74%,30.43%
autogen-62,0.594,0.557,0.592,0.345,72.17%,61.45%,71.59%
autogen-51,0.413,0.287,0.322,0.238,73.53%,20.59%,35.29%
autogen-44,0.361,0.358,0.356,0.208,73.56%,72.12%,71.15%
autogen-14,1.485,1.447,1.374,0.85,74.71%,70.24%,61.65%
autogen-64,0.424,0.172,0.182,0.241,75.93%,-28.63%,-24.48%
autogen-40,0.58,0.562,0.589,0.325,78.46%,72.92%,81.23%
autogen-02,0.473,0.433,0.47,0.26,81.92%,66.54%,80.77%
autogen-05,0.571,0.569,0.589,0.302,89.07%,88.41%,95.03%
autogen-52,0.746,0.705,0.726,0.359,107.80%,96.38%,102.23%
autogen-39,0.422,0.297,0.355,0.195,116.41%,52.31%,82.05%
autogen-43,0.76,0.698,0.712,0.349,117.77%,100.00%,104.01%
autogen-73,1.088,0.391,1.296,0.475,129.05%,-17.68%,172.84%
autogen-22,1.275,0.442,0.432,0.55,131.82%,-19.64%,-21.45%
autogen-06,4.883,1.148,1.105,1.327,267.97%,-13.49%,-16.73%
autogen-03,3.909,0.821,0.867,0.947,312.78%,-13.31%,-8.45%
```

### Versions

A100, master