Repository: pytorch
Issue: Inconsistent list indexing behavior
Labels: triaged,module: advanced indexing

## 🐛 Bug

Sometimes `t[indices] == t[indices, :]`, other times it throws an `IndexError`.
* `t` is a 2D Tensor 
* `indices : list[list[int]]` is a list of list of indices

The error appears depending on the length of `indices`: if it is too short, there's the `IndexError`, if it is "long enough", then `t[indices] == t[indices, :]`.

The above might be true for Tensors other than 2D ones, but i found the problem with a 2D one.

## To Reproduce

Steps to reproduce the behavior:

Paste this example into a python interpreter:
```python
import torch
N, D = 1000, 42 # number of rows, number of features
t = torch.randn(N, D) # the dataset (time series data)
W = 10 # window length
# these indices create windows to feed to e.g. an LSTM
indices =  [ range(i, i + W) for i in range(0, N - W) ]

windowed_data = t[indices]           # this works
windowed_data = t[indices[:100] ] # this works too
# first index where the problem appears is 31:
windowed_data = t[indices[:32] ]   # this works
windowed_data = t[indices[:31] ]   # IndexError: too many indices for tensor of dimension 2
```

Substituting `range(...)` with `list(range(...))` does not affect the problem.

Casting `indices` to Tensor via `indices = torch.tensor(indices)` gives the same result without ever throwing the `IndexError`.

Also, using `t[indices, :]` instead of `t[indices]` gives the same result without ever throwing the `IndexError`.

I have written a function that finds the lowest index for which the `IndexError` appears:

```python
def test_it(N=1000, D=42, W=10):
    t = torch.randn(N, D)
    indices = [range(i, i + W) for i in range(0, N - W)]

    for i in reversed(range(0, len(indices))):
        try:
            t[indices[:i]]
        except IndexError:
            return i
    return None
```

I ran it with many values for `W`, `D` and `N`, and found out that for the majority of cases no `IndexError` is ever thrown, but with some combinations of N,D,W it presents itself. 
In my specific case, `N `didn't matter and the "critical length" for the `indices` list was always 31 regardless of `N`.


## Expected behavior

Either `t[indices]` always throws an error, or `t[indices]` is always equal to `t[indices, :]` (which it is when it doesn't throw the `IndexError`).

## Environment

```
Collecting environment information...
PyTorch version: 1.10.0
Is debug build: False
CUDA used to build PyTorch: 11.5
ROCM used to build PyTorch: N/A

OS: Arch Linux (x86_64)
GCC version: (GCC) 11.1.0
Clang version: 13.0.0
CMake version: version 3.21.4
Libc version: glibc-2.33

Python version: 3.9.7 (default, Oct 10 2021, 15:13:22)  [GCC 11.1.0] (64-bit runtime)
Python platform: Linux-5.10.79-1-lts-x86_64-with-glibc2.33
Is CUDA available: True
CUDA runtime version: 11.5.50
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 2080 Ti
Nvidia driver version: 495.44
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.8.2.4
/usr/lib/libcudnn_adv_infer.so.8.2.4
/usr/lib/libcudnn_adv_train.so.8.2.4
/usr/lib/libcudnn_cnn_infer.so.8.2.4
/usr/lib/libcudnn_cnn_train.so.8.2.4
/usr/lib/libcudnn_ops_infer.so.8.2.4
/usr/lib/libcudnn_ops_train.so.8.2.4
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] byol-pytorch==0.5.1
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.2
[pip3] performer-pytorch==0.14.0
[pip3] pytorch-fast-transformers==0.3.0
[pip3] pytorch-ignite==0.4.2
[pip3] pytorch-lightning==1.1.1
[pip3] torch==1.10.0
[pip3] torchvision==0.8.2
[pip3] vit-pytorch==0.6.2
[conda] Could not collect
```

I originally posted the problem [here](https://discuss.pytorch.org/t/inconsistent-behavior-for-tensor-list-indexing/137225), where someone suggested this is unexpected and worth raising an issue for.