hf_BigBird failing on torchdynamo
Author: shingjan
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.12.1
- Platform: Linux-5.13.0-44-generic-x86_64-with-glibc2.17
- Python version: 3.8.13
- PyTorch version (GPU?): 1.13.0.dev20220609 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no
```


### Who can help?

@ydshieh 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

./torchbench.py --only hf_BigBird --speed-ts

### Expected behavior

```shell
This model should run just fine. But right now I am seeing:

  File "/home/xx/anaconda3/envs/torchdynamo/lib/python3.8/site-packages/transformers/models/big_bird/modeling_big_bird.py", line 607, in bigbird_block_sparse_attention
    gathered_key = self.torch_gather_b2(blocked_key_matrix, rand_attn)
  File "/home/xx/anaconda3/envs/torchdynamo/lib/python3.8/site-packages/transformers/models/big_bird/modeling_big_bird.py", line 969, in torch_gather_b2
    raise ValueError(
  File "/home/xx/anaconda3/envs/torchdynamo/lib/python3.8/site-packages/transformers/models/big_bird/modeling_big_bird.py", line 964, in torch_gather_b2
    @staticmethod
ValueError: Make sure that the first two dimensions of params and indices are identical,                 but they are params: (1, 12) vs. indices: (1, 12)
```
```


Comments:
Hi @shingjan 

The error message is very strange as it says `they are params: (1, 12) vs. indices: (1, 12)`, so it should be identical.
So far I am not able to reproduce as the installation of `benchmark` and `torchdynamo` gives several errors.

Could you try to identity the input that is passed to the model in `modeling_big_bird.py` when `torchbench.py` is running that causes the issue?

Also, you might try to check with the latest stable `transformers` + `PyTorch` version. Thanks!


Hi @ydshieh Thanks for your prompt reply! I am using pytorch nightly, version `1.13.0.dev20220609`, as it is a requirement for `torchdynamo` and `torchbench` so I can't really fall back to pytorch stable like 1.11.0. This error seems odd to me as well since those two indices looks identical. 
This is the setup I have for repro: https://github.com/pytorch/torchdynamo#minimal-developer-setup
I will dig in and see if there is more I can provide you with about a repro.

The two values seems identical because there was a bug in the error message, see #17840. You can re-run your code to see what is the actual values.

If you can only use pytorch nightly, could you also check what's your `torchvision`, `torchaudio` and `torchtext` version?
Also, for these (torch), did you install the version with CUDA, or the version with CPU only?


```
torchtext                 0.14.0.dev20220609            py38    pytorch-nightly
torchvision               0.14.0.dev20220609      py38_cu113    pytorch-nightly
torchaudio                0.13.0.dev20220609      py38_cu113    pytorch-nightly
```
The above are the specs. My installation does include cuda 11.3. This model is supposed to be compiled on cpu/llvm only if that helps.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.