Repository: pytorch
Issue: [bug] FALLBACK path has been taken inside: runCudaFusionGroup
Labels: oncall: jit

### 🐛 Describe the bug

We are facing a similar issue as https://github.com/pytorch/pytorch/issues/88050.
```sh
WARNING/MainProcess] /opt/venv/lib/python3.8/site-packages/torch/nn/modules/module.py:1130: UserWarning: FALLBACK path has been taken inside: runCudaFusionGroup. This is an indication that codegen Failed for some reason.
```
After setting PYTORCH_NVFUSER_DISABLE=fallback, I get the following:
```sh
[ERROR] RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
RuntimeError: stride == cur_contig_stride || size == 1 || (still_rightmost && stride == 1) || (!still_rightmost && stride % word_size == 0) INTERNAL ASSERT FAILED at "../third_party/nvfuser/csrc/executor_utils.cpp":621, please report a bug to PyTorch. Vectorization of T0_g[ iS63{( ceilDiv(( ceilDiv(( ceilDiv(( 1 * ( T0.size[1] * ( T0.size[2] * 1 ) ) ), 2) ), 1) ), 128) )}, iS62{1}, iS60{2}, iS64{128} ] with word size 2 not possible due to invalid stride. Domain: iS60{2}, stride: 42

Traceback (most recent call last):
  File "/back/src/jobs/timeout_runner.py", line 33, in run_with_timeout
    behaviour(shared_dict)
  File "/back/src/jobs/quick_api_pipeline_job.py", line 32, in <lambda>
    behaviour=lambda shared_dict: __do_work(photo_redis_key, options, shared_dict),
  File "/back/src/jobs/quick_api_pipeline_job.py", line 87, in __do_work
    output_data, medias = pipeline_service.process_face(
  File "/back/src/serve/pipeline_service.py", line 164, in process_face
    runner.run_beautifying(client=beautify_client)
  File "/back/src/serve/pipeline_run.py", line 132, in run_beautifying
    beautified_raw = client.beautify(self.__previous_segmap())
  File "/back/src/serve/clients/beautify_client.py", line 29, in beautify
    beautified_tensors = self.service.beautify(segmap.tensor_one_hot)
  File "/back/src/serve/models_unified/src/service/beautify_service.py", line 60, in beautify
    out_seg = self.codec_model.decoder(out_seg, seg_mask)
  File "/opt/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
RuntimeError: stride == cur_contig_stride || size == 1 || (still_rightmost && stride == 1) || (!still_rightmost && stride % word_size == 0) INTERNAL ASSERT FAILED at "../third_party/nvfuser/csrc/executor_utils.cpp":621, please report a bug to PyTorch. Vectorization of T0_g[ iS63{( ceilDiv(( ceilDiv(( ceilDiv(( 1 * ( T0.size[1] * ( T0.size[2] * 1 ) ) ), 2) ), 1) ), 128) )}, iS62{1}, iS60{2}, iS64{128} ] with word size 2 not possible due to invalid stride. Domain: iS60{2}, stride: 42
```


### Versions

```sh
$ python collect_env.py
Collecting environment information...
PyTorch version: 2.0.1+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: Could not collect
CMake version: version 3.27.5
Libc version: glibc-2.31

Python version: 3.8.10 (default, May 26 2023, 14:05:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.107+-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 525.105.17
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          6
On-line CPU(s) list:             0-5
Thread(s) per core:              2
Core(s) per socket:              3
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           63
Model name:                      Intel(R) Xeon(R) CPU @ 2.30GHz
Stepping:                        0
CPU MHz:                         2299.998
BogoMIPS:                        4599.99
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       96 KiB
L1i cache:                       96 KiB
L2 cache:                        768 KiB
L3 cache:                        45 MiB
NUMA node0 CPU(s):               0-5
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Mmio stale data:   Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Retbleed:          Mitigation; IBRS
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat md_clear arch_capabilities

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.23.5
[pip3] torch==2.0.1+cu117
[pip3] torchvision==0.15.2+cu117
[pip3] triton==2.0.0
[conda] Could not collect

```
```[tasklist]
### Tasks
```


cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel