BERT for Regression predicts constant

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.2
- Platform: Ubuntu?
- Python version: 3.8
- Tensorflow version (GPU?): 2.4.1 (Yes)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help

- albert, bert, xlm: @LysandreJik
- tensorflow: @jplu

## Information

Model I am using (Bert, XLNet ...): BERT.

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts:

I am doing BERT for a regression `]0-1]` in a deep (sequential) genomic data. Similar to [DNABERT](https://github.com/jerryji1993/DNABERT) and [this medium post](https://towardsdatascience.com/bringing-bert-to-the-field-how-to-predict-gene-expression-from-corn-dna-9287af91fcf8)

My code is quite simple, basically I am using BERT from the lib, averaging the outputs and passing it to a head model.

```python
class Predictor(Model):

    def __init__(
        self, batch_size: int, sequence_size: int, hidden_layers: List[int], bert_params: dict
    ):
        super().__init__()
        self._embedder = Embedder(sequence_size, bert_params)
        self._head_model = _create_head_model(batch_size, hidden_layers, bert_params["hidden_size"])

    def call(self, inputs):
        embedding = self._embedder(inputs)
        return self._head_model(embedding)

class Embedder(Model):
    def __init__(self, sequence_size: int, bert_params: dict):
        super().__init__()
        self._bert = _create_bert_model(sequence_size, bert_params)
        self.avg_pooling = GlobalAveragePooling1D()

    def call(self, sequence):
        x = self._bert(sequence)
        return self.avg_pooling(x.last_hidden_state)

def _create_bert_model(sequence_size: int, bert_params: dict) -> TFBertModel:
    tokenizer = KmerTokenizer.load()
    sequence_length = sequence_size - tokenizer.k
    config = BertConfig(
        vocab_size=tokenizer.vocab_size,
        max_position_embeddings=sequence_length,
        **bert_params,
    )
    return TFBertModel(config)

def _create_head_model(batch_size: int, n_mixtures: int, hidden_layers: List[int], input_size: int):
    embedding = Input(input_size, batch_size)
    x = embedding
    for n_neurons in hidden_layers:
        x = Dense(n_neurons, activation=nn.gelu)(x)
    output = Dense(1, x)
    return Model(embedding, output)
```

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

```
X -> Tokenized sequence of integers ([1,5,10010, 2,200, 304,1001,535,341])
y -> Float, ]0,1]
```

My Y variable is distributed like this:
![image](https://user-images.githubusercontent.com/11489228/111401808-c30d7980-86a8-11eb-86f4-21ce677ecf7d.png)

## Problem

The problem I am having is that my model is predicting constant.

Scatter (y_true x y_pred)

![image](https://user-images.githubusercontent.com/11489228/111402202-8a21d480-86a9-11eb-96fc-4679c7e81217.png)

Predictions histogram:

![image](https://user-images.githubusercontent.com/11489228/111402564-2e0b8000-86aa-11eb-9b77-31239674e377.png)

Parameters:

```yaml
    batch_size: 16
    training_steps: 20
    sequence_size: 600
    bert:
        hidden_size: 32
        num_attention_heads: 8
        num_hidden_layers: 2
    hidden_layers: [32]
    early_stopping:
        patience: 15
    optimizer:
        type: "RectifiedAdam"
        lr: 0.0004
        epsilon: 0.000001
        beta_2: 0.98
        total_steps: 100
        weight_decay: 0.01
    loss: "mean_squared_error"
```

## What I have tried

* Smaller/Larger learning rate
* Smaller/Larger batch size
* Shallower/Deeper network
* Changing Y distribution (Std Scaling)
* Mixture Density Networks

I simply cannot get through this constant prediction


Google Colab TypeError: expected str, bytes or os.PathLike object, not NoneType

## Environment info

- `transformers` version: 4.4.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.0+cu101 (True)
- Tensorflow version (GPU?): 2.4.1 (True)

The problem arises when using:

I started getting this error this morning without any changes on my side just loading my old Colab notebook (that worked few hours ago without any problem!)

The code that breaks is:
```
tokenizer = AutoTokenizer.from_pretrained('xlm-clm-ende-1024')

/usr/local/lib/python3.7/dist-packages/transformers/models/auto/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    417             else:
    418                 if tokenizer_class_py is not None:
--> 419                     return tokenizer_class_py.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
    420                 else:
    421                     raise ValueError(

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
   1703
   1704         return cls._from_pretrained(
-> 1705             resolved_vocab_files, pretrained_model_name_or_path, init_configuration, *init_inputs, **kwargs
   1706         )
   1707

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in _from_pretrained(cls, resolved_vocab_files, pretrained_model_name_or_path, init_configuration, *init_inputs, **kwargs)
   1774         # Instantiate tokenizer.
   1775         try:
-> 1776             tokenizer = cls(*init_inputs, **init_kwargs)
   1777         except OSError:
   1778             raise OSError(

/usr/local/lib/python3.7/dist-packages/transformers/models/xlm/tokenization_xlm.py in __init__(self, vocab_file, merges_file, unk_token, bos_token, sep_token, pad_token, cls_token, mask_token, additional_special_tokens, lang2id, id2lang, do_lowercase_and_remove_accent, **kwargs)
    645             self.encoder = json.load(vocab_handle)
    646         self.decoder = {v: k for k, v in self.encoder.items()}
--> 647         with open(merges_file, encoding="utf-8") as merges_handle:
    648             merges = merges_handle.read().split("\n")[:-1]
    649         merges = [tuple(merge.split()[:2]) for merge in merges]

TypeError: expected str, bytes or os.PathLike object, not NoneType

```

## To reproduce

Steps to reproduce the behavior:

'''
!pip install transformers
!pip install pytorch-transformers
!pip install tensorboardX

'''

## Expected behavior

It all worked this morning without problem and for many month before that I have not touched the code.


Online decoding for ASR

# 🚀 Feature request

Are there plans to implement online decoding for the speech recognition models such as wav2vec2 and XLSR? More specifically, to be able to receive audio in short chunks, and output partial transcripts as they become available.

## Motivation

Many use cases are covered by the current wav2vec2 model in the library, involving batch recognition of pre-recorded text. However for an online application that wanted to continuously recognize speech on a live input stream, this may not be sufficient.


run_clm.py gpt-2 training example in documentation runs out of memory on a 32gb v100, should be verified and/or modified

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.3
- Platform: Linux-4.15.0-135-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: aye!
- Using distributed or parallel set-up in script?:  no

Models:

- gpt2: @patrickvonplaten, @LysandreJik

Library:

- benchmarks: @patrickvonplaten

Documentation: @sgugger

## Information

Model I am using (Bert, XLNet ...): running the run_clm.py fine-tuning script on gpt-2

The problem arises when using:
* [x] the official example scripts: tranformers/example/language-modeling/run_clm.py
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)  Using the official huggingface wikitext-2-raw-v1 dataset
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Using the example here:
https://github.com/huggingface/transformers/tree/master/examples/language-modeling

When fine-tuning gpt-2 with run_clm.py, this should run on a k80 (24gb of RAM) in about an hour according to the example.  However, I'm running out of memory with default settings... and I'm using a v100 inside a DXG-9, with 32gb of memory:

```
nvidia-smi
Tue Mar 16 15:19:23 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67       Driver Version: 418.67       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:06:00.0 Off |                    0 |
| N/A   34C    P0    44W / 300W |      0MiB / 32480MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
```

Pasting the command here for clarity.

```
python run_clm.py \
    --model_name_or_path gpt2 \
    --dataset_name wikitext \
    --dataset_config_name wikitext-2-raw-v1 \
    --do_train \
    --do_eval \
    --output_dir /tmp/test-clm
```
and the eponymous error:

`RuntimeError: CUDA out of memory. Tried to allocate 512.00 MiB (GPU 0; 31.72 GiB total capacity; 30.32 GiB already allocated; 187.88 MiB free; 30.38 GiB reserved in total by PyTorch)`

That's a ton of memory... is this right?  Or is there some type of memory leak?

Now, this can be fixed by setting `--per_device_train_batch_size 4` but I highly doubt the current format will work out of the box on a k80 w/out changing anything (which I can't test because I don't have access to one) using the default batch size of `8`, and this should be reflected in the example and/or verified with the current `run_clm.py`.

Now, on a v100, it did finish in a little under 14 minutes, which is incredibly fast-- so I'm not complaining-- but I know batch size should be as high as possible on these things to get the best results, and I was really hoping it would work with 8 (in fact I was hoping I could jack it up to 16 by wrapping it in nn.torchDataParallel, but that's for another day.)

This leads me to another question-- I know you can do torch.distributed.launch with these scripts, is there one that wraps the model in `nn.parallel.DistrubtedDataParallel` so that you can chunk a larger batch size across multiple GPU's and utilize the extra memory, or should this be done by hand?  If so, maybe I will create a PR and add an option for this inside the three  example scripts as it would be quite beneficial.  Example:

```
model = torch.nn.parallel.DistributedDataParallel(model,
                                                      device_ids=[args.local_rank],
                                                      output_device=args.local_rank)
```

Results:

```
Training completed. Do not forget to share your model on huggingface.co/models =)

{'train_runtime': 820.2706, 'train_samples_per_second': 2.121, 'epoch': 3.0}
100%|##########################################################################################################################| 1740/1740 [13:40<00:00,  2.12it/s]
[INFO|trainer.py:1408] 2021-03-16 15:39:31,084 >> Saving model checkpoint to /tmp/test-clm
[INFO|configuration_utils.py:304] 2021-03-16 15:39:31,085 >> Configuration saved in /tmp/test-clm/config.json
[INFO|modeling_utils.py:817] 2021-03-16 15:39:32,049 >> Model weights saved in /tmp/test-clm/pytorch_model.bin
03/16/2021 15:39:32 - INFO - __main__ -   ***** Train results *****
03/16/2021 15:39:32 - INFO - __main__ -     epoch = 3.0
03/16/2021 15:39:32 - INFO - __main__ -     train_runtime = 820.2706
03/16/2021 15:39:32 - INFO - __main__ -     train_samples_per_second = 2.121
03/16/2021 15:39:32 - INFO - __main__ -   *** Evaluate ***
[INFO|trainer.py:1600] 2021-03-16 15:39:32,118 >> ***** Running Evaluation *****
[INFO|trainer.py:1601] 2021-03-16 15:39:32,118 >>   Num examples = 240
[INFO|trainer.py:1602] 2021-03-16 15:39:32,119 >>   Batch size = 8
100%|##############################################################################################################################| 30/30 [00:08<00:00,  3.52it/s]
03/16/2021 15:39:40 - INFO - __main__ -   ***** Eval results *****
03/16/2021 15:39:40 - INFO - __main__ -     perplexity = 20.967772820757663
```


[DeepSpeed] ZeRO Stage 3

This PR implements DeepSpeed ZeR0 3 integration:

* [x] removes the "wind-down" of the deepspeed setup at the end of train, since zero3 can't do inference w/o this setup - we will have some other ways to reclaim memory for the no longer needed optimizer in the future.
* [x] adds initial support for eval w/o train - more work will be done in the future
* [x] to support `predict_with_generate` extends `generate` and its 5 beam search ways to support a new `synced_gpus` flag which is needed by ZeRO stage3 - under ZeRO3 parallelization if this gpu finished before max_length was reached, it must continue running forward so that the other gpus who may have not finished their generate yet, can complete as they rely on this gpu to received its slice of params. Currently deployed for DeepSpeed - but may need to do the same for fairscale elsewhere.
* [x] because now we are forced to run all gpus in sync, the `generate` logic is also now equipped with a stopping early mechanism that is synchronized across all participating gpus
* [x] reworks how pretrained model is loaded - `from_pretrained` is now zero3 aware and does a whole lot to efficiently preload massive models
* [x] since `state_dict` is fake under `zero3` it can't be saved and used - so care is taken to either not save the bogus model, or the weights get reconsolidated before saving if `stage3_gather_fp16_weights_on_model_save` is enabled
* [x] adds new DeepSpeed configuration docs and basic tuning recommendation
* [x] adds lots of new tests, now testing zero2 and zero3 separately
* [x] fixes a disappearing std stream problem in an older test using a workaround
* [x] a new DS feature: `deepspeed.zero.register_external_parameter(self, self.layer1.weight)` - haven't needed it so far - need to find which models may need this feature. this is needed for when a layer accesses weights of another layer. but most our models don't do that. so just documented this for now.

DeepSpeed PRs that need to be merged and a new release is made 0.3.14:
* [ ] https://github.com/microsoft/DeepSpeed/pull/881 (memory issue)
* [ ] https://github.com/microsoft/DeepSpeed/pull/884 support lists of params in `GatheredParameters` (needed for pretrained model load)
* [x] https://github.com/microsoft/DeepSpeed/pull/892 script to extract consolidated fp32 weights for zero2 and zero3
* [x] https://github.com/microsoft/DeepSpeed/pull/893 save the consolidated fp16 weights under zero3
* [ ]  https://github.com/microsoft/DeepSpeed/pull/896 leak memory fix needed for tests
* [ ] `deepspeed==0.3.14` released

Future PR TODO:

* [ ] make loading and resuming more efficient - gotta find a way to not preload the model from weights when we are resuming from a checkpoint. and of course not init the weights. Currently we do it 3 times! A huge overhead for big models.


Patches full import failure when sentencepiece is not installed

The `M2M100Tokenizer` and `DebertaV2Tokenizer` should be under the `if is_sentencepiece_available()`

cc @patil-suraj


Tensorflow Keras model.loads_weights() breaks on TFElectraModel trained with v4.3.0

## Environment info

- `transformers` version: 4.4.0
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.0+cu101 (True)
- Tensorflow version (GPU?): 2.4.1 (True)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
@jplu

## Information

Model I am using TFElectraModel:

The problem arises when using:
* my own modified scripts: see attachment

The tasks I am working on is:
* Information Retrieval on SQuAD dataset

## To reproduce

Steps to reproduce the behavior:

1. train a keras model whith a TFElectraModel as a layer on transformers 4.3.0.
2. switch to transformers 4.4.0
3. call model.load_weights on the keras model

OR:
1. "run all" on attached notebook, wait for weights save after (very short) training
2. change pip tranasformers version to 4.4.0 in the first cell
3. comment model.fit to avoid override of weights
4. restart an run all, wait for model.load_weights to fail

[IR_TFElectraModel.ipynb.zip](https://github.com/huggingface/transformers/files/6151831/IR_TFElectraModel.ipynb.zip)

ValueError: Cannot assign to variable tf_electra_model_1/electra/embeddings/token_type_embeddings/embeddings:0 due to variable shape (2, 128) and value shape (512, 128) are incompatible

## Expected behavior

I would expect the weights to load as it correctly does with both train and load performed on transformers 4.3.0

Anyway keep up with the great job! 🤗


Patches the full import failure and adds a test

The full import currently fails because some layers are imported when they do not exist.

This adds a test in `test_file_utils.py` by trying to import the entire transformers. This failed before the proposed fix.

Fixes https://github.com/huggingface/transformers/issues/10749


bug in new version 4.4.0 sentencepiece is not available

Hi, I am using Colab for a sentiment analysis model.
The code suddenly stopped working after a fresh run from yesterday.
I noticed that a new version of transformers was released which caused this issue to appear.

when trying to import I get this error message:
`ModuleNotFoundError: No module named 'sentencepiece'`

and after installing sentencepiece using pip I get this error message:
`AttributeError: module transformers.models.ibert has no attribute IBertLayer`


Fix URLs from #10744

# What does this PR do?

Forgot the `resole/main` in the URLs in #10744 (cc @julien-c)


Issues with MODEL_FOR_MASKED_LM_MAPPING.keys(), and transformer.utils.check_min_version()

Hey,

I just recently wanted to pre-train on top of a BERT model, and ran into some issues. When I run `python run_mlm.py`,
I get the following error:
```
Traceback (most recent call last):
  File "run_mlm.py", line 46, in <module>
    from transformers.utils import check_min_version
ImportError: cannot import name 'check_min_version' from 'transformers.utils' (/PATH/TO/site-packages/transformers/utils/__init__.py)
```
https://github.com/huggingface/transformers/blob/d3d388b934ef515e96246ba643c924d675f6515d/examples/language-modeling/run_mlm.py#L46

After commenting out that line, and it's import (I know, shame on me), I get the following error:
```
Traceback (most recent call last):
  File "run_mlm.py", line 53, in <module>
    MODEL_CONFIG_CLASSES = list(MODEL_FOR_MASKED_LM_MAPPING.keys())
AttributeError: 'NoneType' object has no attribute 'keys'
```
https://github.com/huggingface/transformers/blob/d3d388b934ef515e96246ba643c924d675f6515d/examples/language-modeling/run_mlm.py#L54

Tried with python 3.7.10 and 3.8.3

Thanks


Add DistributedSamplerWithLoop

# What does this PR do?

This PR adds a new distributed sampler that will provide a round multiple of the batch size samples on all processes by looping back at the beginning of the (shuffled) dataset. This is useful:
- for TPUs to avoid triggering a new XLA compilation for the last training batch
- for model parallelism to have batches of the same size on all processes

This PR also refactors some logic regarding the wold_size and process_rank in the `TrainingArguments`, as well as adds a test of the new `DistributedSamplerWithLoop`.

Tested on:
- single-GPU
- multi-GPU
- TPU
- SageMaker MP


fix M2M100 example


Remove old links to CDN

# What does this PR do?

This PR removes a few links left pointing to `https://cdn.huggingface.co` instead of `https://huggingface.co` (purely cosmetic since they are not actually used anymore, normally).


Fix DeBERTa + Conversational pipeline slow tests


DialoGPT- cannot increase number of conversation turns

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.5.1
- Platform: Linux-5.4.0-58-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.6.13
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten @LysandreJik

## Information

Model I am using (Bert, XLNet ...): DialoGPT

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

I fine-tuned DialoGPT on some sitcom subtitles, and now I am trying to chat with it using the commands listed [here](https://gist.github.com/albusdemens/9cd5602f088720e403f84038e088d696) (in the example, I use the Microsoft fine-tuned model).

1. When I increase the number of conversation turns from 5 to 10, everything goes OK with the Microsoft model.
2. If I increase the number of conversation turns to 10 using my fine-tuned model, reply number six is something like

```
>> User:hello
OurBot: Hello.
>> User:how are things?
OurBot: They're fine.
>> User:cool. Did you have lunch already?
OurBot: I did, actually.
>> User:what did you have?
OurBot: Oh, um, i just wanted to toast.
>> User:where?
**OurBot: !!!hello!**
```

## Expected behavior

Using the DialoGPT model I fine-tuned, I would like to be able to have the same number of turns that I have using the Microsoft model.


Fix S2T example

cc @patil-suraj


BigBird

## Environment info
- `transformers` version: 4.4.0.dev0
- Platform: Linux-5.3.0-64-generic-x86_64-with-glibc2.30
- Python version: 3.9.2
- PyTorch version (GPU?): 1.8.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Y
- Using distributed or parallel set-up in script?: N

### Who can help
@patrickvonplaten, @patil-suraj (guessing)
## Information

Model I am using **BigBird**:

The problem arises when using: the official example scripts: **seq2seq**

The tasks I am working on is: the official **BigPatent** dataset

## To reproduce
I'm trying to use BigBird for a summarization task (on the BigPatent dataset). I'm using the official seq2seq script, which I run as (all lengths/batches are small for testing)

```
python run_seq2seq.py \
    --model_name_or_path google/bigbird-roberta-base \
    --dataset_name big_patent \
    --max_source_length 3 \
    --max_target_length 3 \
    --val_max_target_length 3 \
    --do_eval --do_predict \
    --per_gpu_train_batch_size 1 \
    --per_gpu_eval_batch_size 1 \
    --num_train_epochs 1 \
    --output_dir tmp
```

However, I get the following error message:
```
Traceback (most recent call last):
  File "/home/scasola/factuality/factuality/transformers/examples/seq2seq/run_seq2seq.py", line 657, in <module>
    main()
  File "/home/scasola/factuality/factuality/transformers/examples/seq2seq/run_seq2seq.py", line 344, in main
    config = AutoConfig.from_pretrained(
  File "/home/scasola/anaconda3/envs/factuality/lib/python3.9/site-packages/transformers/models/auto/configuration_auto.py", line 382, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
KeyError: 'big_bird'
```


Tokenizer becomes very slow after adding new tokens

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1
- Platform: Linux-5.4.0-58-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik

## Information

Hi,

When I am trying to add a large number (50k) new tokens to BERT's tokenizer, the tokenizer becomes very slow, taking 29 seconds to tokenize a single short sentence.

## To reproduce

```
from transformers import BertTokenizer
import time

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")

new_tokens = ["aaa"+str(i) for i in range(50000)]
tokenizer.add_tokens(new_tokens) # takes some time

sentence = "a short sentence."
start = time.time()
tokenizer.tokenize(sentence)
print(time.time() - start)

> 29.049
```


load wav2vec model from local path

I'm trying to run the wav2vec based ASR on my machine.  This is my code:
```
import soundfile as sf
import torch
from transformers import Wav2Vec2ForMaskedLM, Wav2Vec2Tokenizer

# load pretrained model
cp = "./my_model_directory/wav2vec_small.pt"
tokenizer = Wav2Vec2Tokenizer.from_pretrained(cp)
model = Wav2Vec2ForMaskedLM.from_pretrained(cp)

# load audio
audio_input, _ = sf.read("/home/robot/Music/dinesh.flac")

# transcribe
input_values = tokenizer(audio_input, return_tensors="pt").input_values
logits = model(input_values).logits
predicted_ids = torch.argmax(logits, dim=-1)
transcription = tokenizer.batch_decode(predicted_ids)[0]
print(transcription)

```
Here cp is the path to the wav2ved local model file. But when i try to run this i'm getting error;
`- or './my_model_directory' is the correct path to a directory containing relevant tokenizer files`
Here when i use the model present in the cloud eg. cp = "facebook/wav2vec2-base-960h". This works perfectly.
Isn't this possible to run the transformers wav2vec without cloud?


`group_texts` duplicates special tokens

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.2
- Platform: Linux-5.4.0-66-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 --> @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run [run_mlm.py](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py#L381) with pdb break point at line 409.
2. see value for `tokenized_datasets['train'][0]`
```
ipdb> tokenized_datasets['train'][0]
{'attention_mask': ..., 'input_ids': [2, 11699, 6139, 23923, 6354, 6216, 3, 2, 124, 6149, 6228, 6164, 5125, 27479, 6228, 11699, 6139, 23923, 6354, 6216, 11961, 9121, 9804, 10602, 10293, 5328, 6721, 121, 6997, 15520, 16117, 10602, 11302, 5328, 6721, 121, 6997, 13014, 6177, 22111, 25147,
6189, 6106, 6315, 6110, 5084, 5158, 6291, 12836, 6108, 15512, 6726, 18139, 25596, 12701, 6291, 6106, 6315, 6616, 112, 5171, 113, 4363, 6380, 14946, 13769, 13928, 17518, 10216, 12299, 12571, 12850, 26355, 5315, 6457, 6117, 6303, 6213, 19358, 117, 122, 5201, 6361, 6211, 6377, 6312, 22259, 6631, 9268, 112, 10538, 113, 10728, 22278, 117, 14870, 13905, 142, 15214, 112, 10538, 113, 10728, 22278, 117, 14870, 14934, 7575, 10524, 186, 14921, 30912, 10758, 118, 10022, 680, 6275, 117, 181, 9860, 186, 14921, 30912, 10758, 136, 20107, 18973, 6358, 118, 3, 2, 10022, 7539, 25147, 6189, 116, 24690, 6915, 128, 116, 11134, 14216, 15650, 15373, 117, 13531, 20100, 117, 10028, 6132, 117, 127, 112, 124, 3866, 113, 15798, 15650, 15373, 117,
13531, 20100, 117, 9840, 6403, 117, 23999, 25006, 6131, 112, 14604, 113, 5084, 15466, 112, 5171, 113, 4363, 6380, 14946, 6213, 13579, 10393, 11023, 6187, 9218, 13014, 6236, 23534, 4587, 12827, 11069, 9422, 25686, 9112, 9220, 12112, 13538, 10112, 9427, 9215, 9260, 19036, 10393, 13514, 6187, 10112, 14882, 6130, 20150, 9279, 118, 3, 2, 14233, 15466, 18609, 16080, 118, 3, 2, 25147, 6189, 24864, 28007, 13581, 6149, 6228, 6164, 5125, 27479, 6228, 124, 5134, 16109, 28372, 3814, 6224, 20116, 6158, 12221, 6595, 105, 3, 2, 5466, 11794, 10393, 4700, 6224, 12819, 10694, 6187, 4671, 6628, 6119, 5502, 24468, 5743, 6125, 7111, 18452, 105, 3, 2, 14368, 6164, 5125, 27479, 6228, 6309, 6221, 6139, 4174, 6428, 6243, 167, 11699, 6139, 13295, 16589, 18619, 15924, 6131, 22573, 19515, 11914, 23850, 11914, 11512, 11346, 25763, 5134, 16109, 28372, 5134, 16109, 28372, 7031, 6114, 6114, 6626, 7020, 118, 3, 2, 5476, 6214, 116, 4121, 7788, 6107, 7788, 118, 4822, 6503, 6236, 15053, 4606, 9117, 118, 3, 2, 5676, 26156, 4973, 7088, 6114, 23122, 6114, 25444, 6422, 4218, 14246, 11920, 6147, 12097, 4011, 9117, 118, 3, 2, 4417, 25703, 9205, 9271, 9165, 19235, 4202, 6115, 14810, 6187, 19915, 6164, 4839, 6361, 11721, 4378, 7063, 15482, 9156, 11976, 30627, 9291, 3788, 19018, 20146, 4202, 9172, 118, 9868, 16712, 29634, 6115, 5206, 6203, 4469, 5294, 11019, 10250, 4973, 6284, 6203, 9691, 118, 3, 2, 9310, 10574, 5330, 9799, 11042, 13237, 6149, 9237, 118, 3, 2, 4378, 7063, 9126, 9271, 9242, 9822, 6236, 15472, 23041, 16135, 18119, 15314, 118, 3, 2, 5134, 6341, 6187, 9159, 14990, 5656, 4241, 14059, 6139, 4913, 12802, 9822, 9181, 9841, 4788, 18037, 116, 14059, 10825, 5087, 5178, 11699, 6213, 15171, 6333, 9242, 4645, 10212, 9691, 118, 3, 2, 9620, 10021, 11699, 6139, 17626, 6236, 12258, 4378, 7063, 6185, 16269, 26623, 30683, 12901, 118, 3, 2, 18276, 6130, 4378, 7063, 9126, 11699, 6187, 9159, 9242, 9319, 13793, 17451, 6260, 4184, 9242, 17561, 10724, 20756, 12126, 4789, 9172, 118, 9567, 4144, 12062, 10780, 5466, 6803, 4202, 4378, 7063, 9126, 4422, 12303, 6164, 9165, 10350, 571, 9467, 20853, 7177, 11947, 10441, 9270, 18480, 3795, 9207, 12098, 11725, 118], 'special_tokens_mask':  ... , 'token_type_ids': ...}
```
When `group_texts` is `map`ed to `tokenized_datasets`, whose examples already contain special tokens (e.g. [CLS] and [SEP]), the mapped results have the following format: `[CLS] ... [SEP][CLS] ... [SEP][CLS] ... [SEP]`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The output of `group_texts` applied `tokenized_datasets` has the following format `[CLS] ... [SEP]` or `[CLS] ... [SEP] ... [SEP]`.
<!-- A clear and concise description of what you would expect to happen. -->

The current input format is different from the original implementation, [ELECTRA](https://github.com/google-research/electra/blob/f93f3f81cdc13435dd3e85766852d00ff3e00ab5/build_pretraining_dataset.py#L100) for example. Is this a trivial issue? I think the downstream task performance of the model pretrained with the current script could tell if this is a serious bug or not. Could someone share the results?


Position ids in RoBERTa

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Ubuntu 20.04
- Python version: Python 3.7.10
- PyTorch version (GPU?): 1.6.0_py3.7_cuda10.1.243_cudnn7.6.3_0
- Tensorflow version (GPU?): No
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik

## Information
Position ids in RoBERTa is not implemented properly.
The problem arises when using:
create_position_ids_from_input_ids in transformers.models.roberta.modeling_roberta.py
Based on this function, position id 0 is never used. This may cause problem when the sequence is long, for example, 512. Token whose id >= 511 will not get its corresponding token ids.


Release utils

# What does this PR do?

This PR adds utilities to help with the release process and four make commands to use them easily:

1. `make pre-release` will do all the necessary steps prior to the commit with the release tag (put the right version in all the good places and clean the README from the references to the master documentation).
2. `make post-release` will do all the necessary steps after the release has been made (put the right dev version in all the good places and add the latest version in the deploy doc/doc navbar)
3. `make pre-patch` will do all the necessary steps prior to the commit with the release patch tag (put the right version in all the good places).
4. `make post-past` will do all the necessary steps after the patch release has been made and we are back on master (put the right dev version in all the good places and add the latest version in the deploy doc/doc navbar)


[examples/seq2seq/README.md] fix t5 examples

This PR:
* switches the summarization example to use CNN/DailyMail as with t5-small it provides high scores out of the box.
* fixes T5 examples to include `--source_prefix` - it's **not** optional. If you give it a try you will see that you get 10x worse bleu scores w/o it. w/ `27.6849`, w/o `2.374`
* adds a normal translation example w/o the peculiarities of MBart and T5
* reduces the default max samples to 50 so it's much faster to test quickly
* fixes the reference to the last custom dataset that I incorrectly added in the first place (was missing a username, but worked locally when I created it w/o it)
* removes the 3 `--max*samples` from this README and puts a section on how to use these in the top-level `examples/README.md`

@sgugger, @patrickvonplaten


[examples run_summarization.py] t5 worse score w/ --source_prefix "summarize: " than w/o

I don't think the latest incarnation of summarization examples works for t5.

I'm lost with all the proposed let's-not-do-anything special for t5, except as you will see from numbers something isn't right:

With the latest master:
```
python examples/seq2seq/run_summarization.py --model_name_or_path t5-small --do_train --do_eval \
--dataset_name xsum --output_dir /tmp/tst-summarization --per_device_train_batch_size=4 \
--per_device_eval_batch_size=4 --overwrite_output_dir --predict_with_generate --max_train_samples 50 \
--max_val_samples 50

***** eval metrics *****
  epoch                     =     3.0
  eval_gen_len              =   60.14
  eval_loss                 =  3.3003
  eval_rouge1               = 19.3055
  eval_rouge2               =  2.4192
  eval_rougeL               =  13.931
  eval_rougeLsum            = 16.3446
  eval_runtime              =  6.2317
  eval_samples              =      50
  eval_samples_per_second   =   8.023
```

Then let's add the required `--source_prefix "summarize: "`

```
python examples/seq2seq/run_summarization.py --model_name_or_path t5-small --do_train --do_eval \
--dataset_name xsum --output_dir /tmp/tst-summarization --per_device_train_batch_size=4 \
--per_device_eval_batch_size=4 --overwrite_output_dir --predict_with_generate --max_train_samples 50 \
--max_val_samples 50 --source_prefix "summarize: "

***** eval metrics *****
  epoch                     =     3.0
  eval_gen_len              =   52.94
  eval_loss                 =  3.3734
  eval_rouge1               = 18.7997
  eval_rouge2               =  2.2857
  eval_rougeL               = 13.4997
  eval_rougeLsum            = 14.7778
  eval_runtime              =  5.2697
  eval_samples              =      50
  eval_samples_per_second   =   9.488
```

As you can see the scores are worse than w/o  `--source_prefix "summarize: "` and it should be in reverse.

Where are we adding `task_specific_params`:
```
    "summarization": { "early_stopping": true, "length_penalty": 2.0, "max_length": 200,
    "min_length": 30, "no_repeat_ngram_size": 3, "num_beams": 4, "prefix": "summarize: " },
```
so that the model knows to do the right thing?

**edit**: found where this was last discussed: https://github.com/huggingface/transformers/pull/10133#issuecomment-778071812 So should `README.md` just say that currently `run_summarization.py` cannot be used for T5 models and then find another summarization model instead.

Of course, a lot of these repetitive breakages would have been avoided if we had quality-measuring tests for examples - perhaps when the dust settles around the examples we could have some of those added.

@sgugger


run_clm.py does not work with any other block_size other than 1024

**Note:** This issue can be fixed with one character change as described in last section.

## Environment info
- `transformers` version: 4.4.0.dev0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
- maintained examples (not research project or legacy): @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): GPT2

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: Causal language modelling
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Use GPT based model which has block_size different than 1024
2. Try to train it or fine tune with run_clm.py setting setting block_size in data_args.

In GPU mode, you will get following error:
```
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)`
```

In CPU mode, you will get following error:

```
index out of range in self
```

## Expected behavior
Above error should not occur.

## Cause and Proposed Fix

The issue is because block_size on [line 337](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_clm.py#L337) always get set to 1024 because of wrong indentation:

```
    if data_args.block_size is None:
        block_size = tokenizer.model_max_length
        if block_size > 1024:
            logger.warn(
                f"The tokenizer picked seems to have a very large `model_max_length` ({tokenizer.model_max_length}). "
                "Picking 1024 instead. You can change that default value by passing --block_size xxx."
            )
        block_size = 1024 # <<< THIS LINE NEEDS TO BE INTENDED!!!
```
 So just tabbing that line should fix the issue.


Fix log message for training from checkpoint with global step

# 🚀 Feature request

I think the log message here is wrong:

https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L978

I think it should read something along these lines:

```
logger.info(
                    f"  Will skip the first {epochs_trained} epochs then the first {steps_trained_in_current_epoch} "
                    "batches in the current epoch {epochs_trained + 1}."
                )
```

The batches that are skipped are not skipped in the first epoch since the training was already done for `epochs_trained`. As the variable name indicates, the training is going to skip the steps already trained in the current epoch. The current epoch is `epochs_trained + 1`.

## Motivation

The log message was confusing. I went and traced the code to ensure that it does the right skipping (which I think it does).


Stacked Roberta run_mlm.py

# 🖥 Benchmarking `transformers`

## Benchmark

I try to run [transformers/experiments/language_modeling/run_mlm.py](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py) in order to train Roberta from scratch on Wikipedia dataset.

## Set-up
Transformers version: 4.4.0.dev0
4 GPUs: NVIDIA Tesla V100 16 GB; 4096 Bit, PCI Express 3.0 x16.
Used bash run script:
```bash
#!/bin/bash

export CUDA_LAUNCH_BLOCKING=1

source /data/env/bin/activate

nohup python3 transformers/examples/language-modeling/run_mlm.py \
--dataset_name wikipedia \
--tokenizer_name roberta-base \
--model_type roberta \
--dataset_config_name 20200501.en \
--do_train \
--do_eval \
--learning_rate 1e-5 \
--num_train_epochs 5 \
--save_steps 5000 \
--warmup_steps=10000 \
--output_dir /data/models/wikipedia_roberta \
&
```
Tested with the script and also directly with python command (without nohup)

## Results

Code seems stacked in `trainer.py`, at the first compute_loss step, when performing inference:
```python
    def compute_loss(self, model, inputs, return_outputs=False):
        """
        How the loss is computed by Trainer. By default, all models return the loss in the first element.

        Subclass and override for custom behavior.
        """
        if self.label_smoother is not None and "labels" in inputs:
            labels = inputs.pop("labels")
        else:
            labels = None
        print("STACKED HERE")
        outputs = model(**inputs)
...
```
I can't understand which running parameters are wrong. Could inference for a single batch take more than 30 mins?
Thanks in advance!

UPDATE: Without --warmup_steps is working.


Multi-node training with the latest transformers/examples code

Hi,

I am trying to follow the instructions on how to use the examples from [https://huggingface.co/transformers/examples.html] and I notice there is a difference between version 4.3 and 1.2 in the distributed training session. In the older version it seems that it supports multi-node training with
"    --node_rank=$THIS_MACHINE_INDEX \
    --master_addr="192.168.1.1" \
    --master_port=1234 run_bert_classifier.py \"
  But these options no longer exist in the latest tutorial. Does the latest version still support multi-node training? Thanks.


[Issue template] need to update/extend who to tag

This PR

* [x] adds an entry for what to do what to do when someone has model hub issues - thank you, @julien-c!

TODO/Input needed:

* [ ] need to update who to tag for `tensorflow` Issues

@LysandreJik


Rename zero-shot pipeline multi_class argument

Renames the `multi_class` argument to `multi_label` in the `ZeroShotClassificationPipeline` and adds a deprecation warning to the former. Typically, "multi-label classification" is used to refer to this type of classification (where each class is evaluated independently).

The name is changed in the zero-shot distillation script as well. Resolves #6668.


broken models on the hub

Go to https://huggingface.co/sshleifer/distill-mbart-en-ro-12-6 click on "use in transformers", copy-n-paste and nope can't use this in `transformers`:

```
python -c 'from transformers import AutoTokenizer; AutoTokenizer.from_pretrained("sshleifer/distill-mbart-en-ro-12-6")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/models/auto/tokenization_auto.py", line 410, in from_pretrained
    return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 1704, in from_pretrained
    return cls._from_pretrained(
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 1717, in _from_pretrained
    slow_tokenizer = (cls.slow_tokenizer_class)._from_pretrained(
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 1776, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/models/roberta/tokenization_roberta.py", line 159, in __init__
    super().__init__(
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/models/gpt2/tokenization_gpt2.py", line 179, in __init__
    with open(vocab_file, encoding="utf-8") as vocab_handle:
TypeError: expected str, bytes or os.PathLike object, not NoneType
```

this is with the latest master.

These for example I tested to work fine:

- `sshleifer/distill-mbart-en-ro-12-4`
- `sshleifer/distill-mbart-en-ro-12-9`

Perhaps we need a sort of CI that goes over the public models, validates that `run in transformers` code succeeds and sends an alert if it doesn't? We have no idea how many other models are broken on the hub right now.


Flax testing should not run the full torch test suite

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR adds a new `run_tests_torch_and_flax` circle ci job so that the flax test don't have to run the full pytorch test suite anymore.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Add minimum version check in examples

# What does this PR do?

This PR adds a minimum version check to all examples, in order to avoid the waves of issues created each time they use a functionality that was just released into `Trainer`. The script will immediately error if the version of Transformers does not match the required minimum version. At each release, a script will set that to the version released automatically (work in progress for a second PR with other release utils) so that the examples associated with one tag will require the minimum version of that tag.

The user can still remove that line to avoid the error (at their own risks). The error points out to:
- the instruction for a source install
- the examples README that now lists all examples folders with the various version tags.


Train tokenizer for Deberta

Hi, I would like to know how can I train a DeBERTa tokenizer. From the paper I saw it uses BPETokenizer, but the BPETokenizer from huggingface/tokenizers doesn't work for this. Could you recommend me another implementation or library or a correct configuration for huggingface/tokenizers implementation to be able to train a DeBERTa model from scratch?


iterative evaluation in Trainer to save memory

# 🚀 Feature request

In Trainer.prediction_loop, compute metrics batch per batch (or allow to tweak this number) instead of gathering the whole predictions in a single array (nb: I'm aware of `eval_accumulation_steps` but it only allows to save GPU memory)

## Motivation

Running a ses2seq evaluation on a dataset of 200K examples with a vocabulary of 50K and context of 77 words, gathering all of the output amounts to an array of 2.77 TiB, which I'm not sure everyone can afford:
```
                                        Traceback (most recent call last):
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/bin/clip-train", line 33, in <module>
    sys.exit(load_entry_point('clip', 'console_scripts', 'clip-train')())
  File "/mnt/beegfs/home/lerner/CLIP/clip/train.py", line 196, in main
    trainer.train(**config.get("checkpoint", {}))
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 983, in train
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 1058, in _maybe_log_save_evaluate
    metrics = self.evaluate()
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/trainer_seq2seq.py", line 74, in evaluate
    return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 1513, in evaluate
    metric_key_prefix=metric_key_prefix,
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 1644, in prediction_loop
    preds_gatherer.add_arrays(self._gather_and_numpify(preds_host, "eval_preds"))
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/trainer_pt_utils.py", line 330, in add_arrays
    self._storage = nested_new_like(arrays, self.total_samples, padding_index=self.padding_index)
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/lib/python3.7/site-packages/transformers/trainer_pt_utils.py", line 238, in nested_new_like
    return np.full_like(arrays, padding_index, shape=(num_samples, *arrays.shape[1:]))
  File "<__array_function__ internals>", line 6, in full_like
  File "/mnt/beegfs/home/lerner/anaconda3/envs/transformers/lib/python3.7/site-packages/numpy/core/numeric.py", line 382, in full_like
    res = empty_like(a, dtype=dtype, order=order, subok=subok, shape=shape)
  File "<__array_function__ internals>", line 6, in empty_like
MemoryError: Unable to allocate 2.77 TiB for an array with shape (200206, 77, 49408) and data type float32
 60%|██████    | 3000/5000 [17:53<11:55,  2.80it/s]

```

### Who can help
Library:
 * trainer: @sgugger


Run Time Error: RuntimeError: Expected hidden[0] size (2, 1, 512), got [2, 128, 512] - Seq2Seq Model with PreTrained BERT Model

Hi,

I am facing this run time error while training a seq2seq model with pretrained bert model.

```
RuntimeError                              Traceback (most recent call last)

<ipython-input-63-472071541d41> in <module>()
      8     start_time = time.time()
      9
---> 10     train_loss = train(model, train_iterator, optimizer, criterion, CLIP)
     11     valid_loss = evaluate(model, valid_iterator, criterion)
     12

8 frames

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/rnn.py in check_hidden_size(self, hx, expected_hidden_size, msg)
    221                           msg: str = 'Expected hidden size {}, got {}') -> None:
    222         if hx.size() != expected_hidden_size:
--> 223             raise RuntimeError(msg.format(expected_hidden_size, list(hx.size())))
    224
    225     def check_forward_args(self, input: Tensor, hidden: Tensor, batch_sizes: Optional[Tensor]):

RuntimeError: Expected hidden[0] size (2, 1, 512), got [2, 128, 512]
```

Related code snippets:

```
from torchtext.legacy.data import BucketIterator,TabularDataset

BATCH_SIZE = 128

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

train_iterator, valid_iterator = data.BucketIterator.splits(
    (train_data, valid_data),
    batch_size = BATCH_SIZE,
    device = device)

```
# Encoder

class Encoder(nn.Module):
    def __init__(self, bert, hid_dim, n_layers, dropout):
        super().__init__()

        self.hid_dim = hid_dim
        self.n_layers = n_layers

        self.bert = bert

        emb_dim = bert.config.to_dict()['hidden_size']

        self.rnn = nn.LSTM(emb_dim, hid_dim, n_layers, batch_first = True,dropout = dropout)

        self.dropout = nn.Dropout(dropout)

    def forward(self, sent1):

        #sent1 = [sent1 len, batch size]

        with torch.no_grad():
            embedded = self.bert(sent1)[0]

        #embedded = [sent1 len, batch size, emb dim]

        outputs, (hidden, cell) = self.rnn(embedded)

        #outputs = [sent1 len, batch size, hid dim * n directions]
        #hidden = [n layers * n directions, batch size, hid dim]
        #cell = [n layers * n directions, batch size, hid dim]

        #outputs are always from the top hidden layer

        return hidden, cell

The detailed code with error description is available here for your reference:
https://github.com/Ninja16180/BERT/blob/main/Training_Seq2Seq_Model_using_Pre-Trained_BERT_Model.ipynb

Kindly help me in resolving the issue

Thanks in advance!
```
```


Cannot use custom roberta tokenizer with run_mlm_wwm.py

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.dev0
- Platform: Ubuntu 18
- Python version: 3.7
- PyTorch version (GPU?): 1.7.1 (YES)
- Tensorflow version (GPU?):
- Using GPU in script?: YES
- Using distributed or parallel set-up in script?:

### Who can help

@patrickvonplaten @LysandreJik @
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information
When I try to use the BPE Tokenizer trained with huggingface/tokenizers with Roberta directly, it works:

```{python}

tok = RobertaTokenizer.from_pretrained("bpe_tokenizer_0903", use_fast=True)

```

However, when I try to use this same tokenizer for training a language model, it fails:

```{bash}
python -u  transformers/examples/language-modeling/run_mlm_wwm.py \
    --model_type deberta \
    --config_name ./bpe_tokenizer_0903/config.json \
    --tokenizer_name ./bpe_tokenizer_0903 \
    --train_file ./prueba_tr.txt \
    --validation_file ./final_valid.txt  \
    --output_dir ./roberta_1102 \
    --overwrite_output_dir \
    --do_train \
    --do_eval \
    --evaluation_strategy steps \
    --per_device_train_batch_size 1  \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 2  \
    --learning_rate 6e-4  \
    --save_steps 10  \
    --logging_steps 10 \
    --overwrite_cache \
    --max_seq_length 128 \
    --eval_accumulation_steps 10 \
    --load_best_model_at_end \
    --run_name deberta_0902 \
    --save_total_limit 10 --warmup_steps 1750  \
    --adam_beta2 0.98 --adam_epsilon 1e-6 --weight_decay 0.01 --num_train_epochs 1

```

The error message is the following:

```
Traceback (most recent call last):
  File "transformers/examples/language-modeling/run_mlm_wwm.py", line 399, in <module>
    main()
  File "transformers/examples/language-modeling/run_mlm_wwm.py", line 286, in main
    use_fast=model_args.use_fast_tokenizer,
  File "/home/alejandro.vaca/data_rigoberta/transformers/src/transformers/models/auto/tokenization_auto.py", line 401, in from_pretrained
    return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/home/alejandro.vaca/data_rigoberta/transformers/src/transformers/tokenization_utils_base.py", line 1719, in from_pretrained
    resolved_vocab_files, pretrained_model_name_or_path, init_configuration, *init_inputs, **kwargs
  File "/home/alejandro.vaca/data_rigoberta/transformers/src/transformers/tokenization_utils_base.py", line 1790, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/alejandro.vaca/data_rigoberta/transformers/src/transformers/models/roberta/tokenization_roberta_fast.py", line 173, in __init__
    **kwargs,
  File "/home/alejandro.vaca/data_rigoberta/transformers/src/transformers/models/gpt2/tokenization_gpt2_fast.py", line 145, in __init__
    **kwargs,
  File "/home/alejandro.vaca/data_rigoberta/transformers/src/transformers/tokenization_utils_fast.py", line 87, in __init__
    fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
Exception: data did not match any variant of untagged enum ModelWrapper at line 1 column 1138661
```

Why doesn't it fail when I try to load the tokenizer with RobertaTokenizer.from_pretrained() but it does fail when I try to run run_mlm_wwm.py ? @sgugger @patrickvonplaten @LysandreJik


[WIP] Extend LayoutLMTokenizer to handle bounding boxes

# What does this PR do?

LayoutLMTokenizer does not take care properly of additional model input `bbox` (Bounding Boxes for words/tokens on a document), see https://github.com/huggingface/transformers/issues/10349.

With this PR, LayoutLMTokenizer will take care of bounding boxes when doing tokenization, that is repeating a bounding box for a split text as is done [in the official LayoutLM code](https://github.com/microsoft/unilm/blob/23a7ea35b55279a171a118ac767e863aa92e692c/layoutlm/layoutlm/data/funsd.py#L252).

Additionally, bounding box coordinates may be normalized to a target width and height.

`LayoutLMTokenizerFast` is removed as it is currently only a copy of `BertTokenizerFast` and does not have the added functionality yet.

Marked as WIP as I'm not sure this is the best way to tackle this problem, please see the discussion in the linked issue.

Fixes #10349


Fix backward compatibility with EvaluationStrategy

# What does this PR do?

As mentioned in #10666, the switch from `EvaluationStrategy` to `IntervalStrategy` is not fully backward compatible. This PR fixes that.


How can I get the exact position von answers?

I want to load a local model for Question and Answers task. I have copied the almost Code from Pipeline like follow:

model = MyBert()
checkpoint = torch.load('./training_mode_aim/checkpoint.pth.tar')
model.load_state_dict(checkpoint['model'])
model.eval()
#read the context in local file
texts = utils.readTxt()
question = Question
tokenizer = BertTokenizerFast.from_pretrained('')

all_answers = []
for text in texts[:1]:
    encoding = tokenizer(question,text,truncation = True,
                            max_length = 512,padding = True,stride = 256,
                            return_tensors = 'np',
                            return_overflowing_tokens=True,
                            return_token_type_ids=True,
                            return_offsets_mapping=True,
                            return_special_tokens_mask=True,)
    num_span = len(encoding['input_ids'])
    answers = []
    for span_idx in range(num_span):
        _,start_logits,end_logits = model(torch.tensor([encoding['input_ids'][span_idx]]),
                                        torch.tensor([encoding['attention_mask'][span_idx]]),
                                        torch.tensor([encoding['token_type_ids'][span_idx]]))
        with torch.no_grad():
            start_logits,end_logits = start_logits.cpu().numpy(),end_logits.cpu().numpy()
        undesired_tokens = np.abs(np.array(encoding['token_type_ids'][span_idx]) - 1) & np.array(encoding['attention_mask'][span_idx])
        undesired_tokens_mask = undesired_tokens == 0.0
        start = np.where(undesired_tokens_mask,start_logits, -10000.0)
        end = np.where(undesired_tokens_mask, end_logits, -10000.0)
        start = np.exp(start - np.log(np.sum(np.exp(start), axis=-1, keepdims=True)))
        end = np.exp(end- np.log(np.sum(np.exp(end), axis=-1, keepdims=True)))
        starts,ends,scores = decode(start,end,5,128)
        print('starts: {}, end: {}'.format(starts,ends))
        answers += [{
                'score':score.item(),
                'start':encoding.token_to_word(s),
                'end':encoding.token_to_word(e)}
                        for s,e,score in zip(starts,ends,scores)]
    answers = sorted(answers, key=lambda x: x["score"], reverse=True)[: 5]
    print(answers)

Then I got the relative positions for each subencodings. But finally I want to take the absolute positions of the answers. So someone know how to solve this problem?


Language model for wav2vec2.0 decoding

Hello, I implemented [wav2vec2.0 code](https://huggingface.co/blog/fine-tune-xlsr-wav2vec2) and a language model is not used for decoding. How can I add a language model (let's say a language model which is trained with KenLM) for decoding?

thanks in advance.


Pegasus-Large Question

Is [Pegasus Large model checkpoint](https://huggingface.co/google/pegasus-large) trained on any downstream task? Or is it only trained on pre-training task of gap sentence?


[Wav2Vec2] Make wav2vec2 test deterministic

# What does this PR do?

The Wav2Vec2 tests are not deterministic on some machines. This PR should force all tests to use the expected samples.


Is any possible with pipeline for using local model?

Hallo
I have fine tuned a model in my local PC. I have read the Doc for Pipeline to search a way to use local model in Pipeline. But didn't find. Someone who knows how should I do?


Update modeling_tf_pytorch_utils.py

# What does this PR do?
Fix a bug in convert_tf_weight_name_to_pt_weight_name().
Similar to the kernel parameters, the recurrent_kernel parameters in the LSTM networks need to be transposed, too.

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

## Memo
If the recurrent_kernel parameters are not transposed, it will cause the model parameters not to be loaded correctly, resulting in model migration failure.


## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:4.3.3
- Platform:Pytorch
- Python version:3.7.0
- PyTorch version (GPU?):GPU
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:NO

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.python run_mlm.py     --model_name_or_path release_model/     --dataset_name wikitext     --dataset_config_name wikitext-2-raw-v1     --do_train     --do_eval     --output_dir test-mlm --max_seq_length 128
2.Got an error:
```
[INFO|trainer.py:1408] 2021-03-15 11:10:32,884 >> Saving model checkpoint to test-mlm
[INFO|configuration_utils.py:304] 2021-03-15 11:10:32,886 >> Configuration saved in test-mlm/config.json
[INFO|modeling_utils.py:817] 2021-03-15 11:10:33,863 >> Model weights saved in test-mlm/pytorch_model.bin
Traceback (most recent call last):
  File "run_mlm.py", line 475, in <module>
    main()
  File "run_mlm.py", line 450, in main
    trainer.log_metrics("train", metrics)
AttributeError: 'Trainer' object has no attribute 'log_metrics'
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
The script finish without error.


independent training / eval with local files

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Allows running evaluation on local files without specifying a train file.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
- maintained examples (not research project or legacy): @sgugger


Wrong link to super class

# What does this PR do?

Documentation was referring to slow tokenizer class while it should be the fast tokenizer.

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

Documentation: @sgugger


I am trying to prediction on a data-point, but keep getting an error,

```
from transformers import TFDistilBertForSequenceClassification, DistilBertTokenizerFast

# initialize longformer tokenizer and model
tokenizer = DistilBertTokenizerFast.from_pretrained("distilbert-base-uncased", do_lower_case=True)
model = TFDistilBertForSequenceClassification.from_pretrained("MODEL")

data = tokenizer.encode_plus(
        sentence,
        padding="max_length",
        add_special_tokens=True,
        max_length=512,
        truncation=True,
    )
data['input_ids'] = tf.convert_to_tensor(np.reshape(data['input_ids'], (1, -1)))
data['attention_mask'] = tf.convert_to_tensor(np.reshape(data['attention_mask'], (1, -1)))

model.predict(data)
```

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-95-c8644be457c1> in <module>
----> 1 model.predict(data)

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py in predict(self, x, batch_size, verbose, steps, callbacks, max_queue_size, workers, use_multiprocessing)
   1692           for step in data_handler.steps():
   1693             callbacks.on_predict_batch_begin(step)
-> 1694             tmp_batch_outputs = self.predict_function(iterator)
   1695             if data_handler.should_sync:
   1696               context.async_wait()

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    865     tracing_count = self.experimental_get_tracing_count()
    866     with trace.Trace(self._name) as tm:
--> 867       result = self._call(*args, **kwds)
    868       compiler = "xla" if self._jit_compile else "nonXla"
    869       new_tracing_count = self.experimental_get_tracing_count()

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    900       # In this case we have not created variables on the first call. So we can
    901       # run the first trace but we should fail if variables are created.
--> 902       results = self._stateful_fn(*args, **kwds)
    903       if self._created_variables:
    904         raise ValueError("Creating variables on a non-first call to a function"

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
   3015     with self._lock:
   3016       (graph_function,
-> 3017        filtered_flat_args) = self._maybe_define_function(args, kwargs)
   3018     return graph_function._call_flat(
   3019         filtered_flat_args, captured_inputs=graph_function.captured_inputs)  # pylint: disable=protected-access

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _maybe_define_function(self, args, kwargs)
   3395
   3396     cache_key_context = self._cache_key_context()
-> 3397     cache_key = self._cache_key(args, kwargs, cache_key_context)
   3398
   3399     try:

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _cache_key(self, args, kwargs, cache_key_context, include_tensor_ranks_only)
   3176       input_signature = pywrap_tfe.TFE_Py_EncodeArg(inputs,
   3177                                                     include_tensor_ranks_only)
-> 3178       hashable_input_signature = _make_input_signature_hashable(input_signature)
   3179     else:
   3180       del args, kwargs

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _make_input_signature_hashable(elem)
    112   """
    113   try:
--> 114     hash(elem)
    115   except TypeError:
    116     # TODO(slebedev): consider using nest.

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/framework/type_spec.py in __hash__(self)
    311
    312   def __hash__(self):
--> 313     return hash(self.__get_cmp_key())
    314
    315   def __reduce__(self):

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/framework/type_spec.py in __get_cmp_key(self)
    349     """Returns a hashable eq-comparable key for `self`."""
    350     # TODO(b/133606651): Decide whether to cache this value.
--> 351     return (type(self), self.__make_cmp_key(self._serialize()))
    352
    353   def __make_cmp_key(self, value):

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/framework/type_spec.py in __make_cmp_key(self, value)
    367       ])
    368     if isinstance(value, tuple):
--> 369       return tuple([self.__make_cmp_key(v) for v in value])
    370     if isinstance(value, list):
    371       return (list, tuple([self.__make_cmp_key(v) for v in value]))

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/framework/type_spec.py in <listcomp>(.0)
    367       ])
    368     if isinstance(value, tuple):
--> 369       return tuple([self.__make_cmp_key(v) for v in value])
    370     if isinstance(value, list):
    371       return (list, tuple([self.__make_cmp_key(v) for v in value]))

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/framework/type_spec.py in __make_cmp_key(self, value)
    367       ])
    368     if isinstance(value, tuple):
--> 369       return tuple([self.__make_cmp_key(v) for v in value])
    370     if isinstance(value, list):
    371       return (list, tuple([self.__make_cmp_key(v) for v in value]))

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/framework/type_spec.py in <listcomp>(.0)
    367       ])
    368     if isinstance(value, tuple):
--> 369       return tuple([self.__make_cmp_key(v) for v in value])
    370     if isinstance(value, list):
    371       return (list, tuple([self.__make_cmp_key(v) for v in value]))

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/framework/type_spec.py in __make_cmp_key(self, value)
    379       return (np.ndarray, value.shape,
    380               TypeSpec.__nested_list_to_tuple(value.tolist()))
--> 381     raise ValueError("Unsupported value type %s returned by "
    382                      "%s._serialize" %
    383                      (type(value).__name__, type(self).__name__))

ValueError: Unsupported value type BatchEncoding returned by IteratorSpec._serialize
```

This is the input, I don't see any mistake in the input as well,

`Tensorflow :- 2.5.0-dev20210311`

```
{'input_ids': <tf.Tensor: shape=(1, 512), dtype=int64, numpy=
array([[  101, 10373, 25933, 20974, 13006,  2980, 21397,  4012,  4813,
         2086,  4341,  6823,  2086, 20519,  4852, 10198,  3325,  6605,
         3375,  3934,  2312,  2235,  4094,  2565,  2622,  2968,  2203,
         2203,  3001,  2458,  2166, 23490,  2968,  2974,  7300,  6959,
         2974,  3136,  6032,  3325,  2551, 10317,  5338,  4044,  2832,
         8048,  3003,  2844,  4813,  6581,  4105, 11647,  2583,  6464,
         2958,  6578, 22859,  8754, 21466,  3893, 10266,  3798,  2136,
         2372,  5198,  3237,  3247, 11153,  3237,  2968,  2195,  3454,
         2764,  3266,  4800, 10521,  6895, 28296,  5649,  8185,  2098,
         5500,  2780,  9605,  2136,  2372,  2164,  6327, 17088,  6605,
         3674, 18402,  3934,  2780,  7453,  3144,  2650,  2501, 12771,
         3934,  9531,  2051,  5166,  3737,  4118, 20792,  6605,  6502,
         9871,  3325,  2086,  2551,  2976,  2231,  6401,  2780, 12746,
        16134,  6310,  2544, 16134,  6605,  3934,  5994, 28585,  2974,
         4526,  9084,  2449,  6194,  7142,  7620,  2306,  6605,  7781,
         2279,  4245,  4684,  5097, 11924, 16380,  5281, 19905, 25870,
         2500,  6605,  4736,  4114,  6581, 17826,  6970, 28823,  4807,
         4813,  3001,  2933,  3266, 11103,  2892,  8360,  3934,  2434,
         4145,  2345,  7375,  4208,  2968,  4073, 12725,  2536,  3450,
        14206,  4187,  4130,  3450,  4719,  3934,  5147,  2949,  2051,
         2306,  5166, 25276,  2152,  3737,  4781,  3113,  8013, 14206,
         3450,  8518,  2426,  2622,  2136,  2372,  4722,  7640,  6327,
         4411,  2734,  3113,  2622,  3289,  5676,  2622,  6503,  6134,
         6413,  4807,  4722,  6327, 22859,  4953, 14670,  9531,  3570,
         5166,  2622,  8503,  3450,  2164,  5337,  3550, 24162,  2622,
         9920,  5935,  8220,  4342,  2949, 12653,  2192,  2622,  8116,
         3085,  2015, 23946, 20271,  3530,  2588,  3314, 13248,  2967,
         2949,  2695,  7375,  3319,  6709,  2752,  7620, 10779,  2136,
         2372,  8013,  8048,  3921, 10908,  2836,  4806,  7411,  4781,
         2836,  9312,  2873,  5704,  2443,  7748,  2780, 12706,  5300,
         6481,  5326,  7142,  7620,  6078, 10471,  5906,  2109,  2622,
         2968,  3120,  3642,  2544,  2491, 16473, 10813,  3451,  2689,
         8678,  3488,  5461, 29494, 10787,  3029,  8651,  2366,  2195,
         3934,  5704,  2443, 25505,  2724,  2164,  3772,  3040, 10912,
         6605,  2051, 27554, 11376, 19795,  3145,  4391,  6567, 12725,
         3463,  4041,  2680,  7396, 10908,  2635,  2599,  3788,  4722,
        22859,  7846,  2622, 20283,  4187,  5246,  9990,  4415,  2396,
         2594, 10924,  4751, 11157, 10035,  8635,  2147,  7528, 29445,
         6194,  3266,  4684, 16380,  7375,  2081,  3154,  3266,  2048,
         2590,  2951, 25095,  3934,  2951,  2697,  4600,  2109,  3934,
        11506,  2367,  4127,  2951,  2066,  2865,  4684,  2951,  2449,
         3563,  2592,  2136,  2565,  2622, 10489,  3375,  3795,  3454,
         5884, 10843, 16316,  3454, 12139, 11100,  5225,  3144, 23259,
         6503,  3454,  6162,  2449,  3289,  2164,  8720,  5813, 10831,
         3314,  4254,  2565,  6959,  2195,  3454,  3934,  6401,  2976,
         4034,  2306,  1044,  7898,  2565,  3208,  2877,  3947,  7396,
         3674,  4411,  3266,  2832,  7620,  2967,  3271, 10938,  3151,
        29003, 16134,  3024, 11433,  6143,  3257,  8182,  2449,  5918,
         2864,  6578,  4106,  2203,  2203,  6653,  2346,  2863,  2361,
        24977,  4208,  2968,  4073, 12725,  2536,  3450, 14206,  4187,
         4130,  3450,  4719,  3934,  5147,  2949,  2051,  2306,  5166,
        25276,  2152,  3737,  4781,  3113,  8013, 14206,  3450,  8518,
         2426,  2622,  2136,  2372,  4722,  7640,  6327,  4411,  2734,
         3113,  2622,  3289,  5676,  2622,  6503,  6134,  2780,  3454,
         2458,  2190,  6078,  2109,  2622,  2968,  3120,  3642,  2544,
         2491, 16473, 10813,  3266,  9123,  3433,  3934,  4083, 19875,
         6364, 17953,  2361,  2109, 22969,  2376,  8013,   102]])>, 'attention_mask': <tf.Tensor: shape=(1, 512), dtype=int64, numpy=
array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1]])>}
```


Inheriting from BartForConditionalGeneration into a new class - weight  not initializing


Trainer crashes when saving checkpoint

## Environment info

- `transformers` version: 4.3.3
- Platform: Linux-4.4.0-161-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.8.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes, distributed
- Using distributed or parallel set-up in script?: Yes

```
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.87.00    Driver Version: 418.87.00    CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  Off  | 00000000:02:00.0 Off |                    0 |
| N/A   47C    P0    33W / 250W |  16181MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla P100-PCIE...  Off  | 00000000:81:00.0 Off |                    0 |
| N/A   42C    P0    34W / 250W |  16173MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
```

### Who can help

- trainer: @sgugger

## Information

Model I am using (Bert, XLNet ...): "bert-base-multilingual-cased"

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name, do_lower_case=False)

def preprocess_function(examples):
    tokenized = tokenizer(examples["text"], truncation=True)
    tokenized["labels"] = [1 if elem == "sexist" else 0 for elem in examples["task1"]]
    return tokenized

encoded_dataset = dataset.map(preprocess_function, batched=True)

from transformers import AutoConfig, AutoModelForSequenceClassification, TrainingArguments, Trainer
import datasets

model_name = "bert-base-multilingual-cased"
config = AutoConfig.from_pretrained(
    model_name,
    num_labels=len(np.unique(encoded_dataset["train"]["labels"])),
)

model = AutoModelForSequenceClassification.from_pretrained(model_name, config=config)
training_arguments = TrainingArguments(
    output_dir=f"{model_name}",
    do_train=True,
    do_eval=True,
    evaluation_strategy="steps",
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-5,
    num_train_epochs=5,
    label_names=["labels"],
    load_best_model_at_end=True,
    metric_for_best_model=datasets.load_metric("accuracy"),
    eval_steps=50,
)
def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    predictions = np.argmax(predictions, axis=1)
    return metric.compute(predictions=predictions, references=labels)

trainer = Trainer(
    model,
    training_arguments,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["validation"],
    tokenizer=tokenizer,
    compute_metrics=compute_metrics,
)
trainer.train()
```

The error:

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-44-974384763956> in <module>
     10     #callbacks=[EarlyStoppingCallback(early_stopping_patience=4)], # Por alguna razón, casca
     11 )
---> 12 trainer.train()

~/.local/lib/python3.8/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, **kwargs)
    981                     self.control = self.callback_handler.on_step_end(self.args, self.state, self.control)
    982
--> 983                     self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
    984
    985                 if self.control.should_epoch_stop or self.control.should_training_stop:

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _maybe_log_save_evaluate(self, tr_loss, model, trial, epoch)
   1060
   1061         if self.control.should_save:
-> 1062             self._save_checkpoint(model, trial, metrics=metrics)
   1063             self.control = self.callback_handler.on_save(self.args, self.state, self.control)
   1064

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _save_checkpoint(self, model, trial, metrics)
   1085             self.store_flos()
   1086
-> 1087         self.save_model(output_dir)
   1088         if self.deepspeed:
   1089             self.deepspeed.save_checkpoint(output_dir)

~/.local/lib/python3.8/site-packages/transformers/trainer.py in save_model(self, output_dir)
   1376             self._save_tpu(output_dir)
   1377         elif self.is_world_process_zero():
-> 1378             self._save(output_dir)
   1379
   1380         # If on sagemaker and we are saving the main model (not a checkpoint so output_dir=None), save a copy to

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _save(self, output_dir)
   1419
   1420         # Good practice: save your training arguments together with the trained model
-> 1421         torch.save(self.args, os.path.join(output_dir, "training_args.bin"))
   1422
   1423     def store_flos(self):

~/miniconda3/envs/transformers4/lib/python3.8/site-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
    370         if _use_new_zipfile_serialization:
    371             with _open_zipfile_writer(opened_file) as opened_zipfile:
--> 372                 _save(obj, opened_zipfile, pickle_module, pickle_protocol)
    373                 return
    374         _legacy_save(obj, opened_file, pickle_module, pickle_protocol)

~/miniconda3/envs/transformers4/lib/python3.8/site-packages/torch/serialization.py in _save(obj, zip_file, pickle_module, pickle_protocol)
    474     pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
    475     pickler.persistent_id = persistent_id
--> 476     pickler.dump(obj)
    477     data_value = data_buf.getvalue()
    478     zip_file.write_record('data.pkl', data_value, len(data_value))

TypeError: cannot pickle '_thread.lock' object
```
It doesn't matter the save steps, etc. When it tries to save the model, I get that error. I don't think that I can post the dataset here but it doesn't look like a dataset problem. I'm following the sequence classification notebook but I changed some things to use binary classification and load my own dataset.

I also get a different error when using the early stopping callback:

```python
from transformers import EarlyStoppingCallback

trainer = Trainer(
    model,
    training_arguments,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["validation"],
    tokenizer=tokenizer,
    compute_metrics=compute_metrics,
    callbacks=[EarlyStoppingCallback(early_stopping_patience=4)],
)
trainer.train()
```

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-46-17322e623d42> in <module>
     10     callbacks=[EarlyStoppingCallback(early_stopping_patience=4)], # Por alguna razón, casca
     11 )
---> 12 trainer.train()

~/.local/lib/python3.8/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, **kwargs)
    981                     self.control = self.callback_handler.on_step_end(self.args, self.state, self.control)
    982
--> 983                     self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
    984
    985                 if self.control.should_epoch_stop or self.control.should_training_stop:

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _maybe_log_save_evaluate(self, tr_loss, model, trial, epoch)
   1056         metrics = None
   1057         if self.control.should_evaluate:
-> 1058             metrics = self.evaluate()
   1059             self._report_to_hp_search(trial, epoch, metrics)
   1060

~/.local/lib/python3.8/site-packages/transformers/trainer.py in evaluate(self, eval_dataset, ignore_keys, metric_key_prefix)
   1522             xm.master_print(met.metrics_report())
   1523
-> 1524         self.control = self.callback_handler.on_evaluate(self.args, self.state, self.control, output.metrics)
   1525         return output.metrics
   1526

~/.local/lib/python3.8/site-packages/transformers/trainer_callback.py in on_evaluate(self, args, state, control, metrics)
    360     def on_evaluate(self, args: TrainingArguments, state: TrainerState, control: TrainerControl, metrics):
    361         control.should_evaluate = False
--> 362         return self.call_event("on_evaluate", args, state, control, metrics=metrics)
    363
    364     def on_save(self, args: TrainingArguments, state: TrainerState, control: TrainerControl):

~/.local/lib/python3.8/site-packages/transformers/trainer_callback.py in call_event(self, event, args, state, control, **kwargs)
    375     def call_event(self, event, args, state, control, **kwargs):
    376         for callback in self.callbacks:
--> 377             result = getattr(callback, event)(
    378                 args,
    379                 state,

~/.local/lib/python3.8/site-packages/transformers/trainer_callback.py in on_evaluate(self, args, state, control, metrics, **kwargs)
    527     def on_evaluate(self, args, state, control, metrics, **kwargs):
    528         metric_to_check = args.metric_for_best_model
--> 529         if not metric_to_check.startswith("eval_"):
    530             metric_to_check = f"eval_{metric_to_check}"
    531         metric_value = metrics.get(metric_to_check)

AttributeError: 'Accuracy' object has no attribute 'startswith'

```


Please provide format of the dataset to finetuning wav2vec using run_asr.py script

Hi @patrickvonplaten, thanks for the great work, could you please provide some examples for the dataset format to train the wav2vec model using run_asr.py script.


How to generate texts in huggingface in a batch way?

I am new to huggingface. My task is quite simple, where I want to generate contents based on the given titles. The below codes is of low efficiency, that the GPU Util is only about 15%. It seems that it makes generation one by one. How can I improve the code to process and generate the contents in a batch way?

```
df_test = pd.read_csv("./ag_news/test.csv").sample(frac=1)
from transformers import pipeline
text_generator = pipeline("text-generation")
rows = df_test.sample(1000)
titles = rows['title'].tolist()
contents = rows['content'].tolist()
generate_texts = text_generator(titles, max_length=40, do_sample=False)
```