mps device bug - a weird inconsistency on tensor indexing operations
Labels: triaged, module: mps

### 🐛 Describe the bug

With a mps device, there is a weird inconsistency on tensor indexing operations, maybe depending on different python/pytorch combinations. See the example below where I regenerated the issue.

```
>> Testing environment

Python 3.8.16
Pytorch 2.0.1

>> Machine info

Apple M1 Max
OSX 13.4 (22F66)

```

```
>>> a = torch.tensor([31843, 16644, 31844,   629,  1382,   322,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0], device='mps:0',
       dtype=torch.int32)
>>> b = torch.tensor([1568], device='mps:0')
>>> a[10] = b # See the weird behavior below, it put 1568 into 0'th element instead of the 10th element.
>>> a = torch.tensor([1568, 16644, 31844,   629,  1382,   322,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0], device='mps:0',
       dtype=torch.int32)
>>> a[10] = b.item() # However, b.item() behaves normally as expected
>>> a = torch.tensor([1568, 16644, 31844,   629,  1382,   322,     0,     0,     0,     0,
         1568,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0], device='mps:0',
       dtype=torch.int32)
```

### Versions

PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.4 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: version 3.14.3
Libc version: N/A

Python version: 3.8.16 (default, Mar  1 2023, 21:18:45)  [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-13.4-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Max

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] pytorch-lightning==2.0.2
[pip3] torch==2.0.1
[pip3] torchmetrics==0.11.4
[conda] numpy                     1.24.3                   pypi_0    pypi
[conda] pytorch-lightning         2.0.2                    pypi_0    pypi
[conda] torch                     2.0.1                    pypi_0    pypi
[conda] torchmetrics              0.11.4                   pypi_0    pypi

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev