text: 
  
  TITLE
  Can't reproduce training of wav2vec2-large from documentation
  
  COMMENTS
  2
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ### System Info
  
  ```shell
  - `transformers` version: 4.19.0.dev0
  - Platform: Linux-5.4.0-109-generic-x86_64-with-glibc2.29
  - Python version: 3.8.10
  - Huggingface_hub version: 0.5.1
  - PyTorch version (GPU?): 1.11.0+cu102 (True)
  - Tensorflow version (GPU?): not installed (NA)
  - Flax version (CPU?/GPU?/TPU?): not installed (NA)
  - Jax version: not installed
  - JaxLib version: not installed
  - Using GPU in script?: Yes
  - Using distributed or parallel set-up in script?: No
  ```
  
  
  ### Who can help?
  
  @patrickvonplaten 
  
  ### Information
  
  - [X] The official example scripts
  - [ ] My own modified scripts
  
  ### Tasks
  
  - [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
  - [ ] My own task or dataset (give details below)
  
  ### Reproduction
  
  Pretraining a wav2vec-large model using the documentation under [examples/speech-pretraining](https://github.com/huggingface/transformers/tree/main/examples/pytorch/speech-pretraining) does not work. 
  
  Running the following code (copy-pasted from the README) gives an error due to `model_path_or_dir` not found:
  ```
  accelerate launch run_wav2vec2_pretraining_no_trainer.py \ 
  	--dataset_name=librispeech_asr \
  	--dataset_config_names clean clean other \
  	--dataset_split_names train.100 train.360 train.500 \
  	--output_dir=./test \
  	--max_train_steps=200000 \
  	--num_warmup_steps=32000 \
  	--gradient_accumulation_steps=8 \
  	--learning_rate=0.001 \
  	--weight_decay=0.01 \
  	--max_duration_in_seconds=20.0 \
  	--min_duration_in_seconds=2.0 \
  	--model_name_or_path=./ 
  	--logging_steps=1 \
  	--saving_steps=10000 \
  	--per_device_train_batch_size=2 \
  	--per_device_eval_batch_size=4 \
  	--adam_beta1=0.9 \
  	--adam_beta2=0.98 \
  	--adam_epsilon=1e-06 \
  	--gradient_checkpointing \
  ```
  
  I tried using ´facebook/wav2vec-large-lv60' in `model_name_or_path` but receive the following error:
  ```
  Traceback (most recent call last):
    File "run_wav2vec2_pretraining_no_trainer.py", line 730, in <module>
      main()
    File "run_wav2vec2_pretraining_no_trainer.py", line 572, in main
      for step, batch in enumerate(train_dataloader):
    File "/home/ucloud/.local/lib/python3.8/site-packages/accelerate/data_loader.py", line 303, in __iter__    for batch in super().__iter__():  File "/home/ucloud/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 530, in __next__    data = self._next_data()
    File "/home/ucloud/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 570, in _next_data
      data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    File "/home/ucloud/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
      return self.collate_fn(data)
    File "run_wav2vec2_pretraining_no_trainer.py", line 326, in __call__
      sampled_negative_indices = _sample_negative_indices(
    File "/home/ucloud/.local/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 336, in _sample_negative_indices
      sampled_indices = np.random.randint(0, high, size=(high + 1, num_negatives))
    File "mtrand.pyx", line 748, in numpy.random.mtrand.RandomState.randint
    File "_bounded_integers.pyx", line 1247, in numpy.random._bounded_integers._rand_int64
  ValueError: high <= 0
  ```
  
  The demo script trains without issue. Using the parameters from the demo script and changing `model_name_or_path` from 'patrickvonplaten/wav2vec2-base-v2` to ´facebook/wav2vec-large-lv60´ gives the above error. 
  
  Training on a single T4 GPU (benchmarking purposes)
  
  ### Expected behavior
  
  ```shell
  Wav2vec-large pretraining to run.
  ```
  