Repository: pytorch
Issue: TypeError: slice indices must be integers or None or have an __index__ method
Labels: oncall: quantization

### 🐛 Describe the bug

 @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo I am utilizing pytorch 2.3.0 and facing this particular issue. I have tried using `int()` on `start_pos` and `seqlen`, but that did not help

```
Traceback (most recent call last):
  File "/home/sagemaker-user/sshai/research/code-llama/CodeLLaMa_Classifier/train_fn_binary_quantized.py", line 250, in quantized_aware_train
    model.module.transformer = quantize_fx.prepare_qat_fx(
  File "/opt/conda/lib/python3.10/site-packages/torch/ao/quantization/quantize_fx.py", line 489, in prepare_qat_fx
    return _prepare_fx(
  File "/opt/conda/lib/python3.10/site-packages/torch/ao/quantization/quantize_fx.py", line 135, in _prepare_fx
    graph_module = GraphModule(model, tracer.trace(model))
  File "/opt/conda/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 793, in trace
    (self.create_arg(fn(*args)),),
  File "/home/sagemaker-user/sshai/research/code-llama/CodeLLaMa_Classifier/cl_snn_peft_binary_quantized.py", line 343, in forward
    freqs_cis = self.freqs_cis[start_pos: start_pos + seqlen]
TypeError: slice indices must be integers or None or have an __index__ method
```

### Versions

PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0] (64-bit runtime)
Python platform: Linux-4.14.343-260.564.amzn2.x86_64-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: Tesla T4
GPU 1: Tesla T4
GPU 2: Tesla T4
GPU 3: Tesla T4

Nvidia driver version: 535.129.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      46 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             48
On-line CPU(s) list:                0-47
Vendor ID:                          GenuineIntel
Model name:                         Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
CPU family:                         6
Model:                              85
Thread(s) per core:                 2
Core(s) per socket:                 24
Socket(s):                          1
Stepping:                           7
BogoMIPS:                           4999.99
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke avx512_vnni
Hypervisor vendor:                  KVM
Virtualization type:                full
L1d cache:                          768 KiB (24 instances)
L1i cache:                          768 KiB (24 instances)
L2 cache:                           24 MiB (24 instances)
L3 cache:                           35.8 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-47
Vulnerability Gather data sampling: Unknown: Dependent on hypervisor status
Vulnerability Itlb multihit:        KVM: Vulnerable
Vulnerability L1tf:                 Mitigation; PTE Inversion
Vulnerability Mds:                  Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Meltdown:             Mitigation; PTI
Vulnerability Mmio stale data:      Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Retbleed:             Vulnerable
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] flake8==6.1.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.3
[pip3] pytorch-lightning==2.0.9
[pip3] pytorch-metric-learning==1.7.3
[pip3] torch==2.3.0
[pip3] torchmetrics==1.0.3
[pip3] torchvision==0.15.2a0+072ec57
[pip3] triton==2.3.0
[conda] cudatoolkit               11.8.0              h4ba93d1_12    conda-forge
[conda] libblas                   3.9.0            16_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            16_linux64_mkl    conda-forge
[conda] liblapack                 3.9.0            16_linux64_mkl    conda-forge
[conda] libmagma                  2.7.1                hc72dce7_6    conda-forge
[conda] libmagma_sparse           2.7.1                h8354cda_6    conda-forge
[conda] magma                     2.7.1                ha770c72_6    conda-forge
[conda] mkl                       2022.2.1         h84fe81f_16997    conda-forge
[conda] numexpr                   2.8.7           mkl_py310h5e07f1c_4    conda-forge
[conda] numpy                     1.26.3          py310hb13e2d6_0    conda-forge
[conda] pytorch-gpu               2.0.0           cuda112py310h9871d0b_200    conda-forge
[conda] pytorch-lightning         2.0.9              pyhd8ed1ab_0    conda-forge
[conda] pytorch-metric-learning   1.7.3              pyhd8ed1ab_0    conda-forge
[conda] torch                     2.3.0                    pypi_0    pypi
[conda] torchmetrics              1.0.3              pyhd8ed1ab_0    conda-forge
[conda] torchvision               0.15.2          cuda112py310h0801bf5_1    conda-forge
[conda] triton                    2.3.0                    pypi_0    pypi

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel