Error using SpecAugment feature masking in Wav2Vec 2.0
Author: Nithin-Holla
Pull request: no

When fine-tuning Wav2Vec 2.0, turning on SpecAugment and setting a non-zero value for `mask_feature_prob` results in a size mismatch error at the line `spec_aug_mask = torch.where(attention_mask.bool(), spec_aug_mask, False)`. There are no issues when `mask_feature_prob` is set to zero. 

## Environment info

- `transformers` version: 4.9.2
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0 (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten 

## Information

Model I am using (Bert, XLNet ...): Wav2Vec 2.0

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Load the Wav2Vec 2.0 model, e.g., `facebook/wav2vec2-large-960h-lv60-self` with non-zero value for `mask_feature_prob`. 
2. Train the model on a batch of data.

Sample code to replicate the error:

```python
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
import numpy as np

model_name = "facebook/wav2vec2-large-960h-lv60-self"

processor = Wav2Vec2Processor.from_pretrained(model_name)

model = Wav2Vec2ForCTC.from_pretrained(model_name,
                                       mask_feature_prob=0.2)
model.train()

batch_duration_in_seconds = [1, 3, 2, 6]
input_features = [np.random.random(16_000 * s) for s in batch_duration_in_seconds]


batch = processor(input_features,
                  padding=True,
                  sampling_rate=16_000,
                  return_tensors="pt")

model(**batch)
```

The stacktrace is as follows:

```bash
Traceback (most recent call last):
  File "spec.py", line 21, in <module>
    model(**batch)
  File "/Users/nithinholla/opt/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/Users/nithinholla/opt/anaconda3/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1478, in forward
    outputs = self.wav2vec2(
  File "/Users/nithinholla/opt/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/Users/nithinholla/opt/anaconda3/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1064, in forward
    hidden_states = self._mask_hidden_states(
  File "/Users/nithinholla/opt/anaconda3/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1004, in _mask_hidden_states
    mask_feature_indices = _compute_mask_indices(
  File "/Users/nithinholla/opt/anaconda3/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 186, in _compute_mask_indices
    spec_aug_mask = torch.where(attention_mask.bool(), spec_aug_mask, False)
RuntimeError: The size of tensor a (299) must match the size of tensor b (1024) at non-singleton dimension 1
```

## Expected behavior

Successful forward and backward pass without errors.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

@patrickvonplaten Any idea what's causing this?

Hey @Nithin-Holla, very sorry to reply so late! 

It's a bug on our side - I'm attaching a PR to fix it, should be merged to master today/tomorrow :-)