Issues with custom dataset in Wav2Vec2
Author: peregilk
Pull request: no

@patrickvonplaten @anton-l 

We are training Wav2Vec using the run_speech_recognition_ctc_bnb.py-script.

This is working fine with Common Voice datasets, however using our custom dataset and data loader at [NbAiLab/NPSC]( https://huggingface.co/datasets/NbAiLab/NPSC) it crashes after roughly 1 epoch with the following stack trace:

![image](https://user-images.githubusercontent.com/9079808/151355893-6d5887cc-ca19-4b12-948a-124eb6dac372.png)


We are able to work around the issue, for instance by adding this check in line#222 in transformers/models/wav2vec2/modeling_wav2vec2.py:
```python
if input_length - (mask_length - 1) < num_masked_span:
           num_masked_span = input_length - (mask_length - 1)
```
Interestingly, these are the variable values before the adjustment:
```
input_length=10
mask_length=10
num_masked_span=2
````
After adjusting num_masked_spin to 1, the training script runs. The issue is also fixed by setting “replace=True” in the same function.

Do you have any  idea what is causing this, and how to fix this error permanently? If you do not think this is an Datasets issue, feel free to move the issue.


Comments:
Hey @peregilk, 

I'm happy to look into it! Could you give me a minimal reproducible code example (if the training data is not too huge a simple bash command is fine as well)

Hi @patrickvonplaten.
We are not exactly sure where the error is, but running any the default training scripts with these parameters, reproduces the error. Note that the error happens after roughly one epoch:

```bash
python run_speech_recognition_ctc_bnb.py \
      --dataset_name="NbAiLab/NPSC" \
      --dataset_config_name="16K_mp3" \
      --text_column_name="text" \ 
      --model_name_or_path="facebook/wav2vec2-xls-r-300m" \
     ...
    # I would assume this will give you a smaller train set (but have not tested it)
    --train_split_name="test" \
```





Hey @peregilk,

I'm sadly getting a couple of errors when running your command, *e.g.*:

```bash
Traceback (most recent call last):
  File "run_speech_recognition_ctc_bnb.py", line 760, in <module>
    main()
  File "run_speech_recognition_ctc_bnb.py", line 490, in main
    vocab_dict = create_vocabulary_from_data(
  File "run_speech_recognition_ctc_bnb.py", line 322, in create_vocabulary_from_data
    remove_columns=datasets["train"].column_names,
  File "/home/patrick/python_bin/datasets/dataset_dict.py", line 41, in __getitem__
    return super().__getitem__(k)
KeyError: 'train'
```

Could you try to make the command runable? 


See next comment

Actually now having read through your comment more carefully, I think we don't even need to reproduce it - the problem is (as you probably found) that there is an example of very short input length. An input_length of 10 is very short it corresponds to less than 0.1 seconds. 

The fix that you proposed actually looks like a very good one! Would be interested in opening a PR? I'd be happy to help with adding a nice test and making the function a bit more robust. Otherwise happy to open a PR myself :-)

Thanks @patrickvonplaten. I can submit a PR on this but it has to wait until after the weekend.

I verified that there are quite a few small sentences in the corpus. Most are valid though, like "OK", "Thanks", "Yes" etc. Many between 0.1 and 0.2 second. We should probably keep this in the corpus, and fix the error here.

A slightly related question. During initialisation of the model, I keep getting this warning:
```"The following columns in the training set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length."```

I see this also refers to the same ```ìnput_length```-variable. Is this in any way related, and can it affect this?

Hey @peregilk, 

The warning is totally normal and you can discard this!

I created a pull request for the num_masked_span issue: "Update modeling_wav2vec2.py #15423"

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

This can be closed I think no?