create model cards for qg models

Related to pretrained model cards

cc @julien-c , @danielduckworth


Duplicate grouped entities when using 'ner' pipeline

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): 'ner' pipeline

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Have transformers 3.0.2 installed
2. Run the below code

```python
from transformers import pipeline
nlp = pipeline('ner', grouped_entities=True)
nlp('Welcome to New York')
```

## Expected behavior

We should receive `[{'entity_group': 'I-LOC', 'score': 0.9984402656555176, 'word': 'New York'}`, but instead the output has duplicated 'New York': `[{'entity_group': 'I-LOC', 'score': 0.9984402656555176, 'word': 'New York'}, {'entity_group': 'I-LOC', 'score': 0.9984402656555176, 'word': 'New York'}]`.

### The Cause of the Issue According to Me

After reading 3.0.2, I noticed that lines 1047-1049 were added.  I think this was done to fix a prior issue that caused the last named entity in the sequence to be occasionally omitted when `grouped_entities=True`.  Long story short, I think this snippet was a patch that only shifted the problem from being an occasional named entity omission to an occasional named entity duplicate.

The for-loop that precedes this snippet is inconsistent in that sometimes the last named entity gets successfully added anyway (e.g. if the `if` clause on 1025 (first iteration) or 1032 is entered on the last iteration).  In this case, there is a duplicate entry upon the calling of the new code at 1047.  On the converse, the last named entity won’t be added if the `else` clause in line 1041 is entered on the last iteration.  In this case, the final named entity correctly gets added after the new code snippet is run.

In short, there is a duplicate (I think) if (i) there is only one recognized named entity or (ii) the last named entity is one such that the tokenizer cut it up into multiple tokens.  Otherwise, there is no duplicate.

nlp(‘Welcome to Dallas’) -> duplicate 'Dallas' because 'Dallas' is the only named entity
nlp(‘HuggingFace is not located in Dallas’) -> no duplicate because there are multiple entities and the final one 'Dallas' is not tokenized into multiple tokens
nlp(‘HuggingFace is located in New York City’) -> duplicate ‘New York City’ because the final named entity 'New York City' is tokenized into multiple tokens

## Environment info

- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-1031-azure-x86_64-with-glibc2.10
- Python version: 3.8.1
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


Is there an implementation of BERT architecture in PyTorch that I can modify here?

Hello Team,

  Firstly, thanks for this amazing repo.

I am doing my own research and I want access to a native implementation of BERT in PyTorch so I can modify the architecture and play with it by including a few of my own modules.

Is that possible with the codebase in HuggingFace repo here?

Thanks


docs(wandb): explain how to use W&B integration

Documentation on how to use W&B integration has been added to clear up confusion on how to customize logging.

fix #5262


OSError using FlauBERT

Hello everyone,

I'm trying to run Flaubert model on my data using ktrain.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):  flaubert/flaubert_base_cased

Language I am using the model on (English, Chinese ...): French

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

I'm using ktrain to load my model.
After preprocess my data when i want to get the classifier with function get_classifier() i get this error :

---------------------------------------------------------------------------
```
OSError                                   Traceback (most recent call last)
~\Anaconda3\lib\site-packages\transformers\modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    461                 if resolved_archive_file is None:
--> 462                     raise EnvironmentError
    463             except EnvironmentError:

OSError:

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
~\Anaconda3\lib\site-packages\ktrain\text\preprocessor.py in _load_pretrained(self, mname, num_labels)
    958             try:
--> 959                 model = self.model_type.from_pretrained(mname, config=self.config)
    960             except:

~\Anaconda3\lib\site-packages\transformers\modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1046             if isinstance(config, config_class):
-> 1047                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
   1048         raise ValueError(

~\Anaconda3\lib\site-packages\transformers\modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    468                 )
--> 469                 raise EnvironmentError(msg)
    470             if resolved_archive_file == archive_file:

OSError: Can't load weights for 'flaubert/flaubert_base_cased'. Make sure that:

- 'flaubert/flaubert_base_cased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'flaubert/flaubert_base_cased' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
~\Anaconda3\lib\site-packages\ktrain\text\preprocessor.py in _load_pretrained(self, mname, num_labels)
    961                 try:
--> 962                     model = self.model_type.from_pretrained(mname, config=self.config, from_pt=True)
    963                 except:

~\Anaconda3\lib\site-packages\transformers\modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1046             if isinstance(config, config_class):
-> 1047                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
   1048         raise ValueError(

~\Anaconda3\lib\site-packages\transformers\modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    481             # Load from a PyTorch checkpoint
--> 482             return load_pytorch_checkpoint_in_tf2_model(model, resolved_archive_file, allow_missing_keys=True)
    483

~\Anaconda3\lib\site-packages\transformers\modeling_tf_pytorch_utils.py in load_pytorch_checkpoint_in_tf2_model(tf_model, pytorch_checkpoint_path, tf_inputs, allow_missing_keys)
     92     return load_pytorch_weights_in_tf2_model(
---> 93         tf_model, pt_state_dict, tf_inputs=tf_inputs, allow_missing_keys=allow_missing_keys
     94     )

~\Anaconda3\lib\site-packages\transformers\modeling_tf_pytorch_utils.py in load_pytorch_weights_in_tf2_model(tf_model, pt_state_dict, tf_inputs, allow_missing_keys)
    124     if tf_inputs is not None:
--> 125         tf_model(tf_inputs, training=False)  # Make sure model is built
    126

~\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    821               self._compute_dtype):
--> 822             outputs = self.call(cast_inputs, *args, **kwargs)
    823           self._handle_activity_regularization(inputs, outputs)

~\Anaconda3\lib\site-packages\transformers\modeling_tf_xlm.py in call(self, inputs, attention_mask, langs, token_type_ids, position_ids, lengths, cache, head_mask, inputs_embeds, output_attentions, output_hidden_states, labels, training)
    803             output_hidden_states=output_hidden_states,
--> 804             training=training,
    805         )

~\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    821               self._compute_dtype):
--> 822             outputs = self.call(cast_inputs, *args, **kwargs)
    823           self._handle_activity_regularization(inputs, outputs)

~\Anaconda3\lib\site-packages\transformers\modeling_tf_flaubert.py in call(self, inputs, attention_mask, langs, token_type_ids, position_ids, lengths, cache, head_mask, inputs_embeds, training, output_attentions, output_hidden_states)
    259             if not self.pre_norm:
--> 260                 attn_outputs = self.attentions[i]([tensor, attn_mask, None, cache, head_mask[i]], training=training)
    261                 attn = attn_outputs[0]

~\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    821               self._compute_dtype):
--> 822             outputs = self.call(cast_inputs, *args, **kwargs)
    823           self._handle_activity_regularization(inputs, outputs)

~\Anaconda3\lib\site-packages\transformers\modeling_tf_xlm.py in call(self, inputs, training)
    140         """
--> 141         input, mask, kv, cache, head_mask, output_attentions = inputs
    142         # Input is (bs, qlen, dim)

ValueError: not enough values to unpack (expected 6, got 5)

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-38-dc13d8280fd1> in <module>
----> 1 model = t.get_classifier()

~\Anaconda3\lib\site-packages\ktrain\text\preprocessor.py in get_classifier(self, fpath, multilabel, metrics)
    997         num_labels = len(self.get_classes())
    998         mname = fpath if fpath is not None else self.model_name
--> 999         model = self._load_pretrained(mname, num_labels)
   1000         if multilabel:
   1001             loss_fn =  keras.losses.BinaryCrossentropy(from_logits=True)

~\Anaconda3\lib\site-packages\ktrain\text\preprocessor.py in _load_pretrained(self, mname, num_labels)
    962                     model = self.model_type.from_pretrained(mname, config=self.config, from_pt=True)
    963                 except:
--> 964                     raise ValueError('could not load pretrained model %s using both from_pt=False and from_pt=True' % (mname))
    965         else:
    966             model = self.model_type.from_pretrained(mname, num_labels=num_labels)

ValueError: could not load pretrained model flaubert/flaubert_base_cased using both from_pt=False and from_pt=True
```

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: windows 10
- Python version: 3.8
- PyTorch version (No GPU): 1.0.0
- Tensorflow version (No GPU): 2.1.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?:

Thank you a lot for your help


Here maybe a bug, when we load staged checkpoint

https://github.com/huggingface/transformers/blob/40d98ebf50c4662bcd6dce6395bbed0b2142ea52/src/transformers/trainer.py#L458

I met this bug when I used the setting below:

global_steps = 2748
len(train_dataloader) = 27484
gradient_accumulation_steps = 4

In the original code, "steps_trained_in_current_epoch" will be 2748 !  BUT this variable should be  2748*4 = 10,992

the code I suggested is below:

```
epochs_trained = (self.global_step * self.args.gradient_accumulation_steps) // len(train_dataloader)
steps_trained_in_current_epoch = (self.global_step * self.args.gradient_accumulation_steps) % len(train_dataloader)
```


[Benchmark] TFGPT2LMHeadModel is five times slower than GPT2LMHeadModel

Here are two scripts I ran.
```python
from time import time
from transformers import TFGPT2LMHeadModel, GPT2Tokenizer
import tensorflow as tf

tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = TFGPT2LMHeadModel.from_pretrained('gpt2')

text = "What are you doing after you have finished working?"
generated = tokenizer.encode(text)
context = tf.constant([generated])
past = None
start = time()
for i in range(100):
    output, past = model(context, past = past)
    logits = output[0, -1, :]
    tok = tf.argmax(logits)
    generated.append(tok.numpy())
    context = tf.expand_dims(tf.expand_dims(tok, 0), 0)
sequence = tokenizer.decode(generated)
print(time() - start, sequence)
```
and
```python
from time import time
from transformers import GPT2LMHeadModel, GPT2Tokenizer
import torch

tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained('gpt2')

text = "What are you doing after you have finished working?"
generated = tokenizer.encode(text)
context = torch.tensor([generated])
past = None
start = time()
for i in range(100):
    output, past = model(context, past=past)
    token = torch.argmax(output[..., -1, :])
    generated += [token.tolist()]
    context = token.unsqueeze(0)
sequence = tokenizer.decode(generated)
print(time() - start, sequence)
```
On my computer with the models running on the cpu, the PyTorch version finishes in about six seconds while the TensorFlow version takes 30 seconds. So something must be wrong with the TF implementation because it shouldn't be that much slower.


Update benchmark notebook

Small update


MarianMT: "CUDA out of memory" when translating many times with the MarianMT Model

# 🐛 Bug
RuntimeError('CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 4.00 GiB total capacity; 3.03 GiB already allocated; 4.72 MiB free; 3.06 GiB reserved in total by PyTorch)')

## Information
I wrote a python notebook to translate datasets using  MarianMT. Therefore I wrote a function, that gets called a couple of thousand times in this translation process. The function looks like this:

```
def translate(data):
    batch = tok.prepare_translation_batch(data).to('cuda')
    gen = model.generate(**batch).to('cuda')
    data: List[str] = tok.batch_decode(gen, skip_special_tokens=True)
    return data
```

After about 1000 function calls with a size of data of about 300 words, the error occurs. I tried freeing the GPU memory with torch.cuda.empty_cache() and calling the garbage collector with gc.collect(), but nothing changes. To my understanding I need to use 'torch.no_grad()' but thats already implemented in the generate function of the model.

Model I am using (Bert, XLNet ...): MarianMT

Language I am using the model on (English to German): Helsinki-NLP/opus-mt-en-de

## To reproduce

Steps to reproduce the behavior:

1. Translate an english text using the translate function provided below a couple of thousand times on a cuda enabled device
2. The error occurs corresponding to your gpu after some time

```
src = 'en'  # source language
trg = 'de'  # target language
mname = f'Helsinki-NLP/opus-mt-{src}-{trg}'
model = MarianMTModel.from_pretrained(mname)
tok = MarianTokenizer.from_pretrained(mname)
model.to('cuda')

def translate(data):
    batch = tok.prepare_translation_batch(data).to('cuda')
    gen = model.generate(**batch).to('cuda')
    data: List[str] = tok.batch_decode(gen, skip_special_tokens=True)
    return data
```

## Expected behavior
No Cuda out of memory error. The Cuda memory gets cleared after each translation process is done.

## Environment info

- `transformers` version: 3.0.0
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.7.6
- PyTorch version (GPU?): 1.5.1 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

Thank you in advance for your help, I´m struggling with this error for a while!


Create README.md

model card

Related to pretrained model cards


[MarianMT{


Add newly trained `calbert-tiny-uncased`

Related to pretrained model cards

Calbert is an open-source ALBERT trained on the Catalan OSCAR dataset. This is the `tiny` version, newly trained from a completely rewrite [in this repo](https://github.com/codegram/calbert), now using SentencePiece which makes everything work much better with pipelines and the default tooling, as demonstrated in the model card.


returned value from parse method of MeCab >= 1.0.0 was changed

As shown below, returned value from MeCab's parse method was changed,
we need to fix MecabTokenizer in BertJapaneseTokenizer.

# mecab-python3==1.0.0 (latest version)

```
root@713173e4bace:/# pip freeze | grep mecab
mecab-python3==1.0.0
root@713173e4bace:/# python
Python 3.8.3 (default, Jul  7 2020, 11:33:46)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import MeCab
>>> m = MeCab.Tagger()
>>> m.parse('こんにちは')
'こんにちは\tコンニチワ\tコンニチハ\t今日は\t感動詞-一般\t\t\t5\nEOS\n'
```

# mecab-python3==0.996.5 (previous version)

```
root@713173e4bace:/# pip install mecab-python3==0.996.5 --upgrade
Collecting mecab-python3==0.996.5
  Downloading mecab_python3-0.996.5-cp38-cp38-manylinux2010_x86_64.whl (17.1 MB)
     |████████████████████████████████| 17.1 MB 1.2 MB/s
Installing collected packages: mecab-python3
  Attempting uninstall: mecab-python3
    Found existing installation: mecab-python3 1.0.0
    Uninstalling mecab-python3-1.0.0:
      Successfully uninstalled mecab-python3-1.0.0
Successfully installed mecab-python3-0.996.5
root@713173e4bace:/# pip freeze | grep mecab
mecab-python3==0.996.5
root@713173e4bace:/# python
Python 3.8.3 (default, Jul  7 2020, 11:33:46)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import MeCab
>>> m = MeCab.Tagger()
>>> m.parse('こんにちは')
'こんにちは\t感動詞,*,*,*,*,*,こんにちは,コンニチハ,コンニチワ\nEOS\n'
```

I found other issues related to MeCab>=1.0.0 ( like #5392 ) and understood the latest version of MeCab will not be supported soon. So if this PR is too early to merge, do not hesitate to close.
Thanks.


DPR model examples / notebook / pipeline

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
An notebook or own script in the examples folders would be pretty nice for the new DPR model, implemented by @lhoestq
Maybe an implementation to the pipeline would be cool too

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

Such an notebook or script would make it easier to train an custom dpr model

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

I did not read the paper and had no closer look to the code, but with some advice I could try to start an PR


Add data_collator with attention_mask feature

For models like BERT, we should mask the padding ids as invalidate to avoid attention on them. Therefore, in addition to `input_ids`, the feature `attention_mask` should also be fed to the model.
Add a data collator `DataCollatorForMaskedLanguageModeling` which will return all of above data, and add a dataset class `LineByLineTextMaskDataset` which is related with the former.


transformer dataset and masked LM

# ❓ Questions & Help
I was wondering about the masked LM models and the Datasets for the transformer library.

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:
https://stackoverflow.com/questions/62757772/hugging-face-tokenizer-for-masked-lm-question


[Benchmark] Add benchmarks for TF Training

This PR adds train functions to TF benchmarks including tests.
The notebook is updated accordingly as well.


# ❓ Questions & Help

Hello,

when I run the run_generation.py file I got this error:

Traceback (most recent call last):
  File "C:\Users\imen.benamor\AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\tokenization_utils_base.py", line 507, in convert_to_tensors
    if tensor.ndim > 2:
AttributeError: 'Tensor' object has no attribute 'ndim'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_generation.py", line 274, in <module>
    main()
  File "run_generation.py", line 227, in main
    encoded_prompt = tokenizer.encode(prompt_text, add_special_tokens=False, return_tensors="pt")
  File "C:\Users\imen.benamor\AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\tokenization_utils_base.py", line 1425, in encode
    **kwargs,
  File "C:\Users\imen.benamor\AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\tokenization_utils_base.py", line 1737, in encode_plus
    **kwargs,
  File "C:\Users\imen.benamor\AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\tokenization_utils.py", line 473, in _encode_plus
    verbose=verbose,
  File "C:\Users\imen.benamor\AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\tokenization_utils_base.py", line 2098, in prepare_for_model
    encoded_inputs, tensor_type=return_tensors, prepend_batch_axis=prepend_batch_axis
  File "C:\Users\imen.benamor\AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\tokenization_utils_base.py", line 159, in __init__
    self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)
  File "C:\Users\imen.benamor\AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\tokenization_utils_base.py", line 515, in convert_to_tensors
    "Unable to create tensor, you should probably activate truncation and/or padding "
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.

Any suggestions?
Thank you


Allow to set Adam beta1, beta2 in TrainingArgs

In some models, `beta1` and `beta2` in Adam optimizer are set to be different from the default values `(0.9, 0.999)`. For example, RoBERTa set `beta2 = 0.98`. It is thereby necessary to add `beta1` and `beta2` in `TrainingArgs` if the user wants to fine-tune RoBERTa and other similar models. Also, another hyperparameter of Adam, `adam_epsilon`, has already been added to `TrainingArgs`. For the purpose of consistency, it would be better of `adam_beta1` and `adam_beta2` are also added.


KeyError Issue in Question answering

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Distillbert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* my own modified scripts: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Just run this code.

```
from transformers import pipeline

MODEL_LOC = "distilbert-base-uncased-distilled-squad"
TOKENIZER_LOC = "distilbert-base-uncased-distilled-squad"

qa = pipeline(
    "question-answering",
    model=MODEL_LOC,
    tokenizer=TOKENIZER_LOC
)
context = " ".join(["The incubation period is around 5 days (range: 4-7 days) with a maximum of 12-13 day"]*10)
qa({"question": "incubation period?", "context": context})
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
It should just provide a json output. If I change *10 to *5, it works.
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: OS X
- Python version: 3.6
- PyTorch version (GPU?): 1.5
- Tensorflow version (GPU?):
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: no


HF Trainer Segmentation Fault

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT2-medium & large

Language I am using the model on (English, Chinese ...): Korean (with custom trained tokenizer)

The problem arises when using:
* [ O ] the official example scripts: (give details below)
https://huggingface.co/blog/how-to-train
* [ O ] my own modified scripts: (give details below)
```
from transformers import Trainer, TrainingArguments, DataCollatorForLanguageModeling, LineByLineTextDataset
from transformers import GPT2Config, GPT2LMHeadModel, GPT2TokenizerFast

tokenizer = GPT2TokenizerFast.from_pretrained("./data/TOKEN")

config = GPT2Config.from_pretrained('gpt2-medium')
model = GPT2LMHeadModel(config=config)
tokenizer = GPT2TokenizerFast.from_pretrained("./data/TOKEN", model_max_length=1024)

print('loading dataset...')
dataset = LineByLineTextDataset(
    tokenizer=tokenizer,
    file_path="./data/kowiki.txt",
    block_size=512,
)

training_args = TrainingArguments(
    output_dir='./m',          # output directory
    num_train_epochs=1,              # total # of training epochs
    per_device_train_batch_size=1,  # batch size per device during training - the higher the better, but may OOM
    per_device_eval_batch_size=1,   # batch size for evaluation
    logging_dir='./logs',            # directory for storing logs
    save_steps=10000,
    do_train=True
)

trainer = Trainer(
    model=model,                         # the instantiated Transformers model to be trained
    args=training_args,                  # training arguments, defined above
    train_dataset=dataset,         # training dataset
)
faulthandler.enable()
trainer.train()
```

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ O ] my own task or dataset: (give details below)
Text generation with prompt (trained on wiki & novel)

## To reproduce

Steps to reproduce the behavior:

1. Modify path to data file
2. Use any file(tested with Korean - UTF8)
3. Use any tokenizer(tested with self & GPT2 tokenizers)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

### Error message
```
loading dataset...
Epoch:   0%|                                              | 0/1 [00:00<?, ?it/s]
Fatal Python error: Segmentation fault                | 0/99996 [00:00<?, ?it/s]

Thread 0x00007f872dfff700 (most recent call first):
  File "/opt/conda/lib/python3.6/threading.py", line 299 in wait
  File "/opt/conda/lib/python3.6/threading.py", line 551 in wait
  File "/opt/conda/lib/python3.6/site-packages/tqdm/_monitor.py", line 69 in run
  File "/opt/conda/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/opt/conda/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x00007f8736bb5700 (most recent call first):
  File "/opt/conda/lib/python3.6/threading.py", line 299 in wait
  File "/opt/conda/lib/python3.6/queue.py", line 173 in get
  File "/opt/conda/lib/python3.6/site-packages/tensorboard/summary/writer/event_file_writer.py", line 205 in run
  File "/opt/conda/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/opt/conda/lib/python3.6/threading.py", line 884 in _bootstrap

Current thread 0x00007f88273e7740 (most recent call first):
  File "/opt/conda/lib/python3.6/site-packages/torch/cuda/comm.py", line 39 in broadcast_coalesced
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/_functions.py", line 21 in forward
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/replicate.py", line 71 in _broadcast_coalesced_reshape
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/replicate.py", line 88 in replicate
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 159 in replicate
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 154 in forward
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 577 in __call__
  File "/opt/conda/lib/python3.6/site-packages/transformers/trainer.py", line 622 in _training_step
  File "/opt/conda/lib/python3.6/site-packages/transformers/trainer.py", line 499 in train
  File "trainer.py", line 34 in <module>
Segmentation fault (core dumped)

```

## Expected behavior
Process through training(as normal)

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-4.4.0-178-generic-x86_64-with-debian-buster-sid
- Python version: 3.6.10
- PyTorch version (GPU?): 1.6.0a0+9907a3e (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Planning to (don't see any flags!)


Datasets & collators for NER

# 🚀 Feature request

Is there a plan to add a dataset subclass and loader/collator for token classification tasks in the CONLL format? Or has that been deliberately avoided for some reason.

## Motivation

We have TextDatasets and DataCollatorForLanguageModeling. Can we have something similar for token classification tasks with some CONLL format input text file? That way we can just point to a file & tokenizer and generate a dataset like the language modeling tasks.

## Your contribution

I have a working version of this that I use in my projects. I can package and share it if it would be useful to add directly to transformers library.


[Some weights or buffers of the PyTorch model TFGPT2LMHeadModel were not initialized] convert GPT2 pytorch to tensorflow model

# ❓ Questions & Help

How can we convert gpt2 fine tuned model from run_language_model.py from pytorch to tensorflow model? I came into the following warning when importing pytorch model GPT2LMHeadModel into TFGPT2LMHeadModel.

```
WARNING:transformers.modeling_tf_pytorch_utils:Some weights or buffers of the PyTorch model TFGPT2LMHeadModel were not initialized from the TF 2.0 model and are newly initialized: ['transformer.h.2.attn.bias', 'transformer.h.4.attn.masked_bias', 'transformer.h.4.attn.bias', 'transformer.h.3.attn.bias', 'transformer.h.3.attn.masked_bias', 'transformer.h.0.attn.masked_bias', 'transformer.h.5.attn.masked_bias', 'transformer.h.1.attn.bias', 'transformer.h.0.attn.bias', 'transformer.h.5.attn.bias', 'transformer.h.1.attn.masked_bias', 'lm_head.weight', 'transformer.h.2.attn.masked_bias']
```

Or do we have any tutorial to train a GPT2 language model by TFTrainer?

## Details
<!-- Description of your issue -->

Reproduce

```
from transformers import *
model = GPT2LMHeadModel.from_pretrained("distilgpt2")
model.save_pretrained("./gpt2_pt")
model = TFGPT2LMHeadModel.from_pretrained("./gpt2_pt", from_pt=True)
```

However we won't see the above warning if we use
```
model = TFGPT2LMHeadModel.from_pretrained("distilgpt2")
```

If we already use `run_language_model.py` to train a GPT2LMHeadModel model and import it into TFGPT2LMHeadModel, can we safely use the converted model even if we see the warning?


Difference between AutoTokenizer.from_pretrained and BertTokenizer.from_pretrained

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

While loading pretrained BERT model, what's the difference between AutoTokenizer.from_pretrained and BertTokenizer.from_pretrained? I'm very new to transformers and still confused about some basic things.
Thanks.
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


GPT2 past usage

Hello everyone,

I tried to answer this [stackoverflow question](https://stackoverflow.com/questions/62703391/estimate-token-probability-logits-given-a-sentence-without-computing-the-entire) and stumbled about a strange beheaviour I can't explain.

The following code will calculate the loss for a sentence with different single words injected:
```
from  transformers import GPT2Tokenizer, GPT2LMHeadModel
import torch

model = GPT2LMHeadModel.from_pretrained("gpt2")
model.eval()
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")

def score(sentence):
    tokenize_input = tokenizer.tokenize(sentence)
    tensor_input = torch.tensor([tokenizer.convert_tokens_to_ids(tokenize_input)])
    loss = model(tensor_input, labels=tensor_input)
    return -loss[0].item()

candidates = ["watch", "run", "think", "apple", "light"]
sent_template = "I like sitting in my new chair and {} about life"
print({candidate: score(sent_template.format(candidate)) for candidate in candidates})
```
Output:
```
{'watch': -5.406847953796387, 'run': -5.533411502838135, 'think': -4.525279521942139, 'apple': -6.158637046813965, 'light': -5.835141658782959}
```
Now I wanted to use the past parameter according to [documentation](https://huggingface.co/transformers/quickstart.html#using-the-past) and expected the same result:
```
from  transformers import GPT2Tokenizer, GPT2LMHeadModel
import torch

model = GPT2LMHeadModel.from_pretrained("gpt2")
model.eval()
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")

past = "I like sitting in my new chair and"
past_tokenize_input = tokenizer.tokenize(past)
past_tensor_input = torch.tensor([tokenizer.convert_tokens_to_ids(past_tokenize_input)])

_, _, past = model(past_tensor_input, labels=past_tensor_input)

def score(sentence, past):
    tokenize_input = tokenizer.tokenize(sentence, )
    tensor_input = torch.tensor([tokenizer.convert_tokens_to_ids(tokenize_input)])

    loss = model(tensor_input, labels=tensor_input, past=past)
    return -loss[0].item()

candidates = ["watch", "run", "think", "apple", "light"]
sent_template = " {} about life"
print({candidate: score(sent_template.format(candidate), past) for candidate in candidates})
```
but the loss is different:
```
{'watch': -7.811002731323242, 'run': -6.370519638061523, 'think': -3.460831642150879, 'apple': -9.08120346069336, 'light': -8.28120231628418}
```
Is this the intended behaviour or am I doing something wrong?


Update question template

Point people to the forum.


On running finetune.py for seq2seq, the following error comes up: optimizer_step() got an unexpected keyword argument 'using_native_amp'

File "finetune.py", line 344, in <module>
    main(args)
  File "finetune.py", line 322, in main
    logger=logger,
  File "/content/drive/My Drive/Colab Notebooks/transformers/examples/lightning_base.py", line 336, in generic_train
    trainer.fit(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 979, in fit
    self.single_gpu_train(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/distrib_parts.py", line 185, in single_gpu_train
    self.run_pretrain_routine(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 1156, in run_pretrain_routine
    self.train()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 370, in train
    self.run_training_epoch()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 452, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 657, in run_training_batch
    grad_norm_dic = self.run_batch_backward_pass(split_batch, batch_idx, opt_idx, optimizer)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 709, in run_batch_backward_pass
    self.call_optimizer_step(optimizer, opt_idx, batch_idx, split_batch)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 747, in call_optimizer_step
    using_native_amp=native_amp)
TypeError: optimizer_step() got an unexpected keyword argument 'using_native_amp'

I am using pytorch-lightning==0.8.1.
Can I get some help.
Thank you


Test XLA examples

Add a script to test examples on XLA.


Rename files

As discussed in #4829


[mbart] prepare_translation_batch passes **kwargs to allow DeprecationWarning

Since we deleted the old `pad_to_max_length` kwarg, we want to give the user a deprecation warning if it is passed. By passing all `**kwargs` to tokenizer.__call__, all improper parameter usage will cause appropriate warnings/errors.


TypeError: 'BertTokenizer' object is not callable

```
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
sequence_a = "HuggingFace is based in NYC"
sequence_b = "Where is HuggingFace based?"
encoded_dict = tokenizer(sequence_a, sequence_b)
```
This will produce error 'BertTokenizer' object is not callable.  Maybe I should call `tokenizer.tokenize()`? But I think the [doc](https://huggingface.co/transformers/glossary.html#token-type-ids) should also be updated!


# 🐛 Bug

## Information

Model I am using (Bart):**facebook/bart-large-cnn**

Language I am using the model on English

The problem arises when using:
* [x] the official example scripts: summarization

## To reproduce

Steps to reproduce the behavior:

run the code below
```python
from transformers import BartTokenizer, BartForConditionalGeneration, BartConfig
tokenizer = BartTokenizer.from_pretrained('facebook/bart-large-cnn')
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
load the model
<!-- A clear and concise description of what you would expect to happen. -->
## Error infomation

```
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-5-3a96f15c5285> in <module>
----> 1 tokenizer = BartTokenizer.from_pretrained("facebook/bart-large-cnn")

~/.conda/envs/hug/lib/python3.7/site-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, *inputs, **kwargs)
   1138
   1139         """
-> 1140         return cls._from_pretrained(*inputs, **kwargs)
   1141
   1142     @classmethod

~/.conda/envs/hug/lib/python3.7/site-packages/transformers/tokenization_utils_base.py in _from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
   1244                     ", ".join(s3_models),
   1245                     pretrained_model_name_or_path,
-> 1246                     list(cls.vocab_files_names.values()),
   1247                 )
   1248             )

OSError: Model name 'facebook/bart-large-cnn' was not found in tokenizers model name list (facebook/bart-base, facebook/bart-large, facebook/bart-large-mnli, facebook/bart-large-cnn, facebook/bart-large-xsum, yjernite/bart_eli5). We assumed 'facebook/bart-large-cnn' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'] but couldn't find such vocabulary files at this path or url.
```

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: centos
- Python version: python 3.7.0
- PyTorch version (GPU?): pytorch   1.5.0           cpu_py37hd91cbb3_0
- Tensorflow version (GPU?): tensorflow-gpu            2.0.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: no


[Reformer] - Cache hidden states and buckets to speed up inference

As discussed with the authors in Reformer we cache the hidden states and buckets to speed up inference for language generation. Caching only the hidden states can save at least twice the memory versus caching the key and value output vectors (often more since the `hidden_size` can be smaller than key and query projections).

The idea is to only recompute key and value projections within the same chunk so that the output will be equal.

- [x] Implement caching mechanism for Local Attention
- [x] Implement caching mechanism for LSH Attention
- [x] Add test
- [x] Add logic to generation
- [x] Refactoring and better naming

=> This results in a 10x speed up when generating up to 1000 tokens.

**Some fun trials trying to make Reformer generate 16000 tokens for its own Wikipedia article [here](https://colab.research.google.com/drive/1Oao8vBtDkz6v1E1efUhTlug5uuxC_BnE?usp=sharing).**

*Review*:
Not urgent. Merging this PR can wait a bit. Added a couple of tests to make sure caching gives same output as non-caching.


Fix tokenizers pretrained saving/loading

Fix #5571

I managed to fix the saving part, but I don't really know what to do for the loading part (in `_from_pretrained`).


Fix tests imports dpr

There were changes in the locations of some functions used for tests in #5350.
When #5279 was merged some tests couldn't run.

I fixed the imports of those functions.
When I re-ran the tests I noticed that some didn't pass for the DPRReaderTokenizer so I fixed them.


Seperating premise and hypothesis in MNLI

# ❓ Questions & Help

I'm adding it here since I didn't receive any reply on SO. I think this query might be relevant for people who are working in few-shot, contrastive/metric learning space.

I'm trying to implement Siamese like transformer architecture. Similar work has been done in SentenceBERT paper. I'm facing an issue. To seperate hypothesis and premise, I modify [this line](https://github.com/huggingface/transformers/blob/3dcb748e31be8c7c9e4f62926c5c144c62d07218/src/transformers/data/processors/glue.py#L131) from `_glue_convert_examples_to_features`. Instead I do
```
batch_encoding_a = tokenizer(
        [example.text_a for example in examples],
        max_length=max_length,
        padding="max_length",
        truncation=True,
    )
```
Did the same thing for `examples.text_b` to obtain `batch_encoding_b`. Then I modify the GlueDataset by modifying [this line mainly](https://github.com/huggingface/transformers/blob/3dcb748e31be8c7c9e4f62926c5c144c62d07218/src/transformers/data/datasets/glue.py#L125), since this will return two items now (segregated "hypothesis" and premise").
Then `__getitem__` is modified accordingly to return `self.features_a[i], self.features_b[i]`.

That's the gist of how I'm obtaining segregated "hypothesis" and "premise". These are then passed to two BERTs (or one BERT if its weights are kept frozen).

This is how I've defined the `collate_fn`
```
def siamese_data_collator(batch):
    features_a, features_b = [], []
    for item in batch:
        for k, v in item.items():
            if k == "a":
                features_a.append(v)
            else:
                features_b.append(v)
    return {
        "a": default_data_collator(features_a),
        "b": default_data_collator(features_b),
    }
```
Then the `dataloader` is created in the usual way. So when we iterate like this:
```
def _training_step(...):
        model.train()
        for k, v in inputs["a"].items():
            if isinstance(v, torch.Tensor):
                inputs["a"][k] = v.to(self.args.device)

    # we get inputs['a'] and inputs['b'] which is passed to the model
```
I had to modify `_training_step` and `evaluate` accordingly in the `Trainer` class.

Now the problem is, the model doesn't learn at all (`bert-base-uncased`). I tried with using my `model` and modified `Trainer` with standard `GlueDataset`, and it works. This leads to the conclusion that something is off with the data. The model should learn something (even if is not being fed concatenated "hypothesis" and "premise").

The model basically has one BERT and one linear layer. The logits come from linear layer which is then used to compute loss function (typical siamese like architecture).

Can you suggest if there's an issue in how the `Dataset` is being created here, or propose something of your own to segregate "hypothesis" and "premise" so that they can be fed separately to BERT.

Link to [Stack Overflow question](https://stackoverflow.com/questions/62771502/seperating-premise-and-hypothesis-in-mnli)


Create README.md for electra-base-squad2

Related to pretrained model cards

Readme file for deepset/electra-base-squad2


MBARTTokenizer set_lang logic will only work for src_lang=en_XX

translation

machine translation utilities and models


Create README.md

model card

Related to pretrained model cards


Tokenizers save_pretrained doesn't work with custom vocabs (v3.0.2)

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

I'm using an instance of `RobertaTokenizerFast` with my custom vocab (pretrained using `tokenizers` library). When I try to save the tokenizer using `tokenizer.save_pretrained(<path>)` method a type error occurs.

I did a bit of investigation why this happens and got a workaround https://github.com/huggingface/transformers/issues/5393#issuecomment-654342933

## To reproduce

Steps to reproduce the behavior:
1. Choose any tokenizer that adds `AddedToken` to kwargs on init (e.g. `RobertaTokenizerFast` which adds a `mask_token` in the following manner)
2. Train a custom vocab using `tokenizers` library or download a pretrained vocab
3. Create an instance of chosen tokenizer by passing vocab and merges files to its constructor
4. Call `save_pretrained(<path>)` method

```python
tokenizer_kwargs = RobertaTokenizerFast.from_pretrained("roberta-base").init_kwargs
vocab_file = tokenizer_kwargs.get("vocab_file")
merges_file = tokenizer_kwargs.get("merges_file")

tokenizer = RobertaTokenizerFast(vocab_file, merges_file)
tokenizer.save_pretrained(".")
```

This will result in the following error:
```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
 in
----> 1 tokenizer.save_pretrained(".")

~/app/.venv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py in save_pretrained(self, save_directory)
   1360
   1361         with open(tokenizer_config_file, "w", encoding="utf-8") as f:
-> 1362             f.write(json.dumps(tokenizer_config, ensure_ascii=False))
   1363
   1364         with open(special_tokens_map_file, "w", encoding="utf-8") as f:

~/.pyenv/versions/3.7.6/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    236         check_circular=check_circular, allow_nan=allow_nan, indent=indent,
    237         separators=separators, default=default, sort_keys=sort_keys,
--> 238         **kw).encode(obj)
    239
    240

~/.pyenv/versions/3.7.6/lib/python3.7/json/encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

~/.pyenv/versions/3.7.6/lib/python3.7/json/encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

~/.pyenv/versions/3.7.6/lib/python3.7/json/encoder.py in default(self, o)
    177
    178         """
--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '
    180                         f'is not JSON serializable')
    181

TypeError: Object of type AddedToken is not JSON serializable
```

## Expected behavior

I expect the same behavior as if you would save a pretrained tokenizer from the hub:

```python
tokenizer = RobertaTokenizerFast.from_pretrained("roberta-base")
tokenizer.save_pretrained(".")
```

Which in its turn results in producing 4 files:

```
('/app/vocab.json',
 '/app/merges.txt',
 '/app/special_tokens_map.json',
 '/app/added_tokens.json')
```

## Possible solution

A possible approach would be to pass hooks to json methods for (de)serializing objects of complex type like `AddedToken`. Generally, a solution must preserve the type information but since types are a subject of change it's unclear whether such generalization needs to be followed at all.

This is a sketch for the solution described above.
```python
def deserialize_json_object(json_obj: Dict[Text, Any]) -> Any:
    classname = json_obj.get("__class__", "dict")
    try:
        obj = eval(classname)(**json_obj)
    except NameError:
        obj = json_obj
    return obj

def serialize_object_to_json(obj: Any) -> Dict[Text, Any]:
    get_state = getattr(obj, "__getstate__", None)
    if callable(get_state):
        json_obj = get_state()
        json_obj["__class__"] = type(obj).__name__
    else:
        json_obj = obj.__dict__
    return json_obj

json.load(<obj>, object_hook=deserialize_json_object)

json.dumps(<obj>, default=serialize_object_to_json)
```

Another solution would be to map `AddedToken` instances to dicts in `tokenizer_config` similar to constructing a `write_dict` object here:
https://github.com/huggingface/transformers/blob/d6b0b9d451e7ffe0da72a5f532cc9bec1563e801/src/transformers/tokenization_utils_base.py#L1368

This will require to deserialize dicts into `AddedToken` instances on loading vocabs and symbols.

## Environment info

- `transformers` version: 3.0.2
- Platform: macOS Catalina 10.15.5
- Python version: 3.7.6
- PyTorch version (GPU?): 1.5.1 (No)
- Tensorflow version (GPU?): N/A
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Freeze the token embeddings for finetuning

Hi ,

I have finetuned the T5 model using the community notebooks given . But when i looked into the code under examples/seq2seq finetuning code token embedding weights are frozen.

Can some one throw some light on how does this effect the finetuning process?

Thanks


BertEmbeddings code for position_embeddings and word_embeddings

   I am trying to figure out how exactly positions and words are embedded in Bert.  I checked the code, and it seems following code is used for the purpose.   This implies that both the embedding function are calling same `__init__' function from the class `nn.Embedding`.  Where exactly is the code doing the actual position embedding (sine and cosine based). Where can I see that code ?.

    self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size, padding_idx=config.pad_token_id)
    self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)
    inputs_embeds = self.word_embeddings(input_ids)
    position_embeddings = self.position_embeddings(position_ids)`


Use pretrained bert withou embedding layers.

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->

https://arxiv.org/pdf/2005.07421.pdf

I was inspirited by the above paper. But the author set the input of bert is there soft-masked embeddings which is not the original format of bert.

So are there any way for me to get Bert without its embedding layers.


How to get gradient wrt to a word embedding layer pytorch?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->'
The idea behind this is I want to try some old school gradient ascent style visualization with Bert Model.

I want to know the effect of changes of embedding layer on a specific layer's specific dimension. Thus, I took the gradient of the output of a specific layer's specific dimension wrt the first word embedding layer's output.

The best thing I can do here is the following:

```
from transformers import BertTokenizer, BertModel
model = BertModel.from_pretrained('bert-base-uncased', output_attentions=True,output_hidden_states=True)
tokenizer = BertTokenizer.from_pretrained(model_version, do_lower_case=True)
s = 'I want to sleep'
inputs = tokenizer.encode_plus(s,return_tensors='pt', add_special_tokens=False,is_pretokenized=True)
input_ids = inputs['input_ids']
output = model(input_ids)
hidden_states = output[-2]

X = hidden_states[0] #embedding space, shape: [1,4,768] (batch_size,sentence_length,embedding dimension)
y = hidden_states[3][0][0][0] ##the 0th position and 0th dimension of output of 3rd hidden layer. Dimension should just be [1], a scalar.

torch.autograd.grad(y,X,retain_graph=True, create_graph=True) #I take the gradient of y wrt. Since y is scalar. The dimension of the gradient is just the dimension of X.
```

This is, however, not good enough. I want the gradient wrt the actual word embedding layer. However, Transformer's embedding contains "position_embedding" and "token_type_embedding". Here's the code for the first layer embedding:

```
class BertEmbeddings(nn.Module):
    """Construct the embeddings from word, position and token_type embeddings.
    """
    def __init__(self, config):
        super(BertEmbeddings, self).__init__()
        self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size, padding_idx=0)
        self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)
        self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.hidden_size)

        # self.LayerNorm is not snake-cased to stick with TensorFlow model variable name and be able to load
        # any TensorFlow checkpoint file
        self.LayerNorm = BertLayerNorm(config.hidden_size, eps=config.layer_norm_eps)
        self.dropout = nn.Dropout(config.hidden_dropout_prob)

    def forward(self, input_ids, token_type_ids=None, position_ids=None):
        seq_length = input_ids.size(1)
        if position_ids is None:
            position_ids = torch.arange(seq_length, dtype=torch.long, device=input_ids.device)
            position_ids = position_ids.unsqueeze(0).expand_as(input_ids)
        if token_type_ids is None:
            token_type_ids = torch.zeros_like(input_ids)

        words_embeddings = self.word_embeddings(input_ids)
        position_embeddings = self.position_embeddings(position_ids)
        token_type_embeddings = self.token_type_embeddings(token_type_ids)

        embeddings = words_embeddings + position_embeddings + token_type_embeddings
        embeddings = self.LayerNorm(embeddings)
        embeddings = self.dropout(embeddings)
        return embeddings
```

Ideally, I want the gradient wrt JUST “words_embeddings" Rather than, wrt "words_embeddings + position_embeddings + token_type_embeddings" and follows by layerNorm and dropout.

I think I can do this by modifying changing the model. Is there a way to it without changing the model?

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:  https://stackoverflow.com/questions/62750798/how-to-get-gradient-wrt-to-a-specific-layers-output-pytorch


[docs] fix model_doc links in model summary

Possible fix for #5561

@sgugger


❓ Why multiplying the output of T5 by some scalar before LM head ?

# ❓ Questions & Help

I'm wondering why multiplying the outputs of T5 by some scalar before inputting in the LM head :

https://github.com/huggingface/transformers/blob/1d2332861f5225aef17bd7e75abc670a72239081/src/transformers/modeling_tf_t5.py#L1147


Where is the documentation on migrating to the 3.0 tokenizer API?

I see that you folks have completely changed the API to do tokenizing, e.g. for BertTokenizer. I have a lot of code using the two methods `encode_plus()` and `batch_encode_plus()`, and when I went to the [documentation](https://huggingface.co/transformers/main_classes/tokenizer.html) to look up an argument, I found that these methods are completely gone. All that remains is a little blurb saying:

> `BatchEncoding` holds the output of the tokenizer’s encoding methods (`__call__`, `encode_plus` and `batch_encode_plus`) and is derived from a Python dictionary.

Are these two methods deprecated now? Did you post a migration guide for users?

On the main [Huggingface Transformers page](https://github.com/huggingface/transformers), you have sections for `Migrating from pytorch-transformers to transformers` and `Migrating from pytorch-pretrained-bert to transformers`, so it's not like there's no precedent for you to provide some information to users on major API changes.


Bug in Question Answering pipeline when question is weird (unanswerable)

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): TFDistilBertForQuestionAnswering

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

 - Code
```python
from transformers import pipeline
qanlp = pipeline("question-answering", framework="tf")  # even PyTorch gives the error
qanlp(context="I am a company", question="When is the bill due?", handle_impossible_answer=True) # happens even without handle_impossible_answer
```
 - Error
```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-4-4da7a3b5ca0e> in <module>()
----> 1 qanlp(context="I am a company", question="When is the bill due?", handle_impossible_answer=True)

/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py in __call__(self, *args, **kwargs)
   1314                         ),
   1315                     }
-> 1316                     for s, e, score in zip(starts, ends, scores)
   1317                 ]
   1318

/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py in <listcomp>(.0)
   1314                         ),
   1315                     }
-> 1316                     for s, e, score in zip(starts, ends, scores)
   1317                 ]
   1318

KeyError: 0
```
 - Happens even on this hosted inference API https://huggingface.co/distilbert-base-cased-distilled-squad?text=When+is+the+bill+due%3F&context=I+am+a+company%0A

## Expected behavior
Either give a wrong answer or a blank answer without any errors

## Environment info
- `transformers` version: 3.0.2
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: Tried with and without GPU in Google colab
- Using distributed or parallel set-up in script?: No


Fix fast tokenizers too


[Docs] Incorrect links to models in the Summary of the Models page

### Issues with the Summary of the Models page
- Description:
	- Clicking on any model documentation located in https://huggingface.co/transformers/model_summary.html leads us to an empty page.
- Possible Fix:
	- I've noticed that the GPT link is https://huggingface.co/model_doc/gpt while it should be https://huggingface.co/transformers/model_doc/gpt.html
	- This fix seems to resolve all other issues in the page.

- Observations:
	- I've tried to clone this repo and fix this issue, but for some reason, when I build my own version of the documentation, it doesn't use the `.../transformers/...` path inside the url leading to a different issue from the current online documentation.

- Examples:

EXPECTED:

RESULT:


[Reformer] Adapt Reformer MaskedLM Attn mask

The official trax code added an `attention_mask` to the LSH Attention layer a couple of weeks ago: https://github.com/google/trax/commit/94d7d8643d9a6ea38539486dea9ad16da30ec897#diff-a022408d1029c5cbeac49f4589f1b713R1180 . This PR adapts the Hugging Face code to have equal outputs to the official trax code for masked self attention.

Integration tests are adapted and run on branch: `reformer_trax_tests`.


Fix #5507


Various tokenizers fixes

Fix https://github.com/huggingface/transformers/issues/5486
Fix https://github.com/huggingface/transformers/issues/5482
Fix https://github.com/huggingface/transformers/issues/5393
Fix https://github.com/huggingface/transformers/issues/5490


Roberta Large doesn't train for sentiment classification

The following is my classifier class. I trained it with appropriate code and same tokenizer(roberta-base). It trained well on SST2 2 class sentiment problem. But when i replaced base model with roberta-large (also the tokenizer and dim of linear layer)my classifier is stuck at 50% accuracy and not matter how many epoch i run it doesn't improve. Is the weight file faulty or am i doing something wrong?

```
class Model(nn.Module):
    def __init__(self):
        super(Model,self).__init__()
        self.Bert = transformers.RobertaModel.from_pretrained('roberta-base')
        self.fc0 = nn.Linear(768,512)
        self.d0 = nn.Dropout(0.5)
        self.d1 = nn.Dropout(0.5)
        self.fc1 = nn.Linear(512,1)
        nn.init.normal_(self.fc0.weight,std= 0.1)
        nn.init.normal_(self.fc0.bias ,0.)
        nn.init.normal_(self.fc1.weight,std =0.1)
        nn.init.normal_(self.fc1.bias, 0.)

    def forward(self,input_ids,attention_mask):
        hid= self.Bert(input_ids,attention_mask = attention_mask)
        hid= hid[0][:,0,:]
        x = self.d0(hid)
        x = self.fc0(x)
        x = torch.tanh(x)
        x = self.d1(x)
        x = self.fc1(x)
        return x

```
This the optimizer i used and i froze the Roberta Layers and only trained the final layers. I also trained with lr 3e-5. It didn't do any help either.
```
model = Model().to('cuda')
criterion = nn.CrossEntropyLoss(reduction='mean').to('cuda')
for params in model.Bert.parameters():
     params.requires_grad = False
optimizer = torch.optim.Adam(model.parameters(),lr=0.001)
```


[pl examples] add using_native_amp flag to support pl 0.8.4

running the example `finetune_bart_tiny.sh` in `seq2seq` fails with:
```
Traceback (most recent call last):
  File "finetune.py", line 344, in <module>
    main(args)
  File "finetune.py", line 322, in main
    logger=logger,
  File "/root/notebooks/analysis/transformers/examples/lightning_base.py", line 330, in generic_train
    trainer.fit(model)
  File "/apps/python3/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1020, in fit
    self.run_pretrain_routine(model)
  File "/apps/python3/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1156, in run_pretrain_routine
    self.train()
  File "/apps/python3/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 370, in train
    self.run_training_epoch()
  File "/apps/python3/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 452, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx)
  File "/apps/python3/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 659, in run_training_batch
    grad_norm_dic = self.run_batch_backward_pass(split_batch, batch_idx, opt_idx, optimizer)
  File "/apps/python3/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 711, in run_batch_backward_pass
    self.call_optimizer_step(optimizer, opt_idx, batch_idx, split_batch)
  File "/apps/python3/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 750, in call_optimizer_step
    using_native_amp=native_amp)
TypeError: optimizer_step() got an unexpected keyword argument 'using_native_amp'
```

seems like the issue is with the missing argument.


Training TFBertForSequenceClassification with DataFrame instead of tensorflow_datasets

I'm trying to fine tune `transformers` with my own dataset in the `csv` file. So, I found [an item in the docs which shows a basic usage example](https://huggingface.co/transformers/training.html#fine-tuning-in-native-tensorflow-2). But the main problem is that this example shows how to use transformers with the `tensorflow_datasets`, but not with the something more real, like with the `pandas` `Dataframe`. So, I have a problem with the `transformers` usage:
```
df = pd.DataFrame({'text': ['SOME ANGRY TEXT!!!', 'Some friendly text :)'], 'label': [1, 0]})

bert_model = transformers.TFBertForSequenceClassification.from_pretrained("bert-base-cased")
bert_tokenizer = transformers.BertTokenizer.from_pretrained("bert-base-cased")

x = bert_tokenizer.batch_encode_plus(
    df.text.values,
    max_length=10,
    add_special_tokens=True,
    pad_to_max_length=True,
)

bert_model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['Accuracy'])

bert_history = bert_model.fit(
 x=x,
 y=df.label.values.reshape(-1, 1),
)
```

Output:
```
ValueError: Failed to find data adapter that can handle input: (<class 'list'> containing values of types {'(<class \'list\'> containing values of types {"<class \'int\'>"})'}), <class 'numpy.ndarray'>
```
**So, how can I use my custom `pandas` `DataFrame` as x and y or as dataset?**

Also, you need to understand that I found a few questions on the SO which ask almost the same question and has upvotes, [here is one of the last question which I found](https://stackoverflow.com/questions/60463829/training-tfbertforsequenceclassification-with-custom-x-and-y-data). **So, maybe people will find MWE with the `np` or `pd` useful?**


huggingface optimizer cannot de-serialize

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

    >>> import transformers
    >>> opt, _ = transformers.optimization_tf.create_optimizer(init_lr=2e-5, num_train_steps=1000, num_warmup_steps=10)
    >>> opt.__class__.from_config(opt.get_config())
    Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "/home/ec2-user/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/optimizer_v2/optimizer_v2.py", line 741, in from_config
        config["learning_rate"], custom_objects=custom_objects)
        File "/home/ec2-user/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/optimizer_v2/learning_rate_schedule.py", line 992, in deserialize
        printable_module_name="decay")
        File "/home/ec2-user/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/utils/generic_utils.py", line 292, in deserialize_keras_object
        config, module_objects, custom_objects, printable_module_name)
        File "/home/ec2-user/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/utils/generic_utils.py", line 250, in class_and_config_for_serialized_keras_object
        raise ValueError('Unknown ' + printable_module_name + ': ' + class_name)
    ValueError: Unknown decay: WarmUp

## Expected behavior

One would expect that the `tf.keras.optimizers.Adam` object `opt` is successfully reconstructed from its config dictionary.

After inspecting `transformers.optimization_tf`, what I believe is happening here is that `create_optimizer` returns a regular `tf.keras.optimizers.Adam` object in the event that the `weight_decay_rate` parameter is zero:

    91 else:
    92    optimizer = tf.keras.optimizers.Adam(learning_rate=lr_schedule, epsilon=adam_epsilon)

However, if `num_warmup_steps` is nonzero, this object is instantiated with a _custom_ `tf.keras.optimizers.schedules.LearningRateSchedule` object of type `WarmUp`:

    77 if num_warmup_steps:
    78     lr_schedule = WarmUp(
    79         initial_learning_rate=init_lr, decay_schedule_fn=lr_schedule, warmup_steps=num_warmup_steps,
    80     )

In this case the `tf.keras.optimizers.Adam` implementation of `from_config` does not know to pass this class under the `custom_objects` keyword argument, causing deserialization to fail.

In my specific setting, this causes the method `horovod.create_distributed_optimizer(opt)` to fail, since it relies on serialization / deserialization to programmatically extend the passed optimizer class.

A solution that should work in my specific setting (and more generally) is to create an intermediate class such as

        class AdamWarmUp(tf.keras.optimizers.Adam):
            def __init__(self, *args, **kwargs):
                super().__init__(*args, **kwargs)
            @classmethod
            def from_config(cls, config):
                custom_objects = {"WarmUp":WarmUp}
                return super().from_config(config, custom_objects=custom_objects)

which is returned by `create_optimizer` and correctly implements `tf.keras.optimizers.Optimizer`, and to have the `AdamWeightDecay` optimizer extend this class.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.0
- Platform: Linux-4.14.181-108.257.amzn1.x86_64-x86_64-with-glibc2.9
- Python version: 3.6.10
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.1.0 (True)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: True


Customize widget text-generation inference with prepended input

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

Some models for text generation make use of special tokens or additional initialization text.
It would be useful to add the possibility to pass additional text to the input string (before or after) and also let people strip input text.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

It is not clear how the widget API generates text.
For example with my little demo huggingtweets, I see 2 limitations:

1. I always strip input with `text.strip` and add a white space at the beginning. Otherwise we get wrong tokenization of `"Paris"` vs `" Paris"`
2. With the way my model is trained, I always want to add `<|endoftext|>` at the beginning (training adds it between every tweet)

Input such as `"     this is my input   "` would become, after stripping, adding white space and special token `"<|endoftext|> this is my input"` and be then passed to tokenizer.

I can see other cases where people add their own special tokens for certain tasks.

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

I don't think there is any public access to the API so cannot contribute there.


[Don't merge] Reformer Trax Integration Tests


Fix #5544


Fix the tokenization warning noted in #5505

Fix https://github.com/huggingface/transformers/issues/5505 (unwanted warning in batch_encode_plus)


The `add_space_before_punct_symbol` is only for TransfoXL

Same fix as in the [pipelines](https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines.py#L626). Not the cleanest way to do a per-model change, but is bug-free for v3.0.2.

closes https://github.com/huggingface/transformers/issues/5525


Possibility to use WhitespaceSplit as pre_tokenizer instead of BPE/Sentencepiece?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I was customizing a tokenizer with vocab and merges from tokenizers module. However when running intended Language Modelling task with the customized tokenizer, the class in transformers repository(PretrainedTokenizer) is not the same. I was trying to run a roberta-for-masked-lm objective with whole word tokenization scheme but seems if I were to  RobertaTokenizer, it cannot support changing the 'pre_tokenizer' scheme. May I know in such case what shall I do


[Feature Request] Extract Predictions from Trainer

# 🚀 Feature request

Add the possibility to return predictions along with its example id in the new Trainer class.

## Motivation

When working with extractive QA (i.e.: SQuAD), you get back the best predictions, but, the current example for running squad uses the old, plain training/eval script, without the new Trainer class.

Additionally, there are other tasks where predictions can be extremely useful (i.e.: Multiple Choice).

Adding such functionality in the Trainer class could solve this and unify both question answering and multiple choice examples.

## Your contribution

I am familiarized with the code (both the Trainer class and the old train/eval script), so I could submit a PR with the new functionality.


GPT2 tokenizer should not output token type IDs

The GPT-2 and OpenAI GPT tokenizers should not output `token_type_ids` by default.

closes https://github.com/huggingface/transformers/issues/5517

code quality will pass when merged in master.

Edit (@thomwolf):
This will also fix #4922


Batching (TF)BertForQuestionAnswering deployment

# 🚀 Feature request

It may be already possible, but I couldn't figure out how, to call TFBertForQuestionAnswering on a batch on context-question pairs rather than just one by one.

## Motivation

I tried out the same code on a machine with 16 cores and a GPU. I could process 22 pairs/sec with the CPUs, but only 11 pairs/sec with the GPU. I assume it's not efficient due to I/O. However, I couldn't find in the example such a way to call the BERT model.

For example, when I do this:
```
input_ids = tokenizer.encode(question, context, add_special_tokens=True, max_length=512)
```

I would like to promote `question` and `context` to an array of pairs so that after calling
```
start_scores, end_scores = model({'input_ids': np.array([input_ids]), # The tokens representing our input text.
                                 'token_type_ids': np.array([segment_ids])}) # The segment IDs to differentiate question from context
```

the returned objects `start_scores` and `end_scores` should be a 2D array (or list).


incorrect typehint for PreTrainedTokenizer.convert_ids_to_tokens() return value

# 🐛 Bug

## Information

The return value of `tokenization_utils.PreTrainedTokenizer.convert_ids_to_tokens()` is declared as `Union[int, List[int]]` while the method clearly returns a `Union[str, List[str]]` object type.

Would you like me to open a PR? ;)

Thanks for the great work!
Andreas

PS: Excuse me for skipping my system specs since this is a rather small issue and I'm just a bit annoyed by PyCharm's inspection warning on this ;)


t5-base translation_en_to_de BLEU lower than the paper

I downloaded the "newstest2014.en" and "newstest2014.de" datasets. Then I used examples/translation/t5/evaluate_wmt.py to evaluate the BLEU value of en_to_de, and the BLEU finally obtained is equal to 22.15, which is much lower than the paper. I used the t5-base model and my transformers version is 2.11.0.
Is there something wrong with my operation? Is it necessary to fine-tune the t5 model to reproduce the results of the paper?


QA pipeline should mask CLS tokens after handling impossible answer

Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>


High F1 score. But poor accuracy during Inference due to tokenisation

# 🐛 Bug

## Information

I am using **Bert-Base-cased** model to train my custom Named entity recognition(NER) model with a sequence length of **512**.

Language I am using the model on: **English**

The problem arises when using:
* [ ] the official example scripts: **token-classification/run_ner.py**

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: **Named entity recognition**
* [ ] my own task or dataset: **Custom Dataset**

## To reproduce

Steps to reproduce the behavior:

1.Use the default NER Pipeline to load the custom trained model
`        self.model_prediction_pipeline = pipeline(
            "ner",
            model=model_path,
            tokenizer= model_path,
            grouped_entities=True
        )`
2. I've attached the Evaluation results of the model.
`eval_loss = 0.021479165139844086`
`eval_precision = 0.8725970149253731`
`eval_recall = 0.8868932038834951`
`eval_f1 = 0.8796870297923562`
`epoch = 5.0`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
1. Model should produce a good accuracy corresponding to the F1 score.
2. However during Inference, I am not getting an accuracy over **30%**
3. Not sure if the inconsistent tokenisation leads to poor results.

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.0
- Platform: Linux-4.15.0-109-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.4.0
- Tensorflow version (GPU?):NA
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


MobileBert embedding vectors values

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): MobileBert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

```py
def BERT_embeddings(data, max_seq_len):
    tokenizer = MobileBertTokenizer.from_pretrained('google/mobilebert-uncased')
    model = MobileBertModel.from_pretrained('google/mobilebert-uncased')
    batches = np.array_split(data, 5)
    encoded = None
    for batch in batches:
        encoded_text = tokenizer.batch_encode_plus(batch.values, max_length=max_seq_len, pad_to_max_length=True, truncation=True)
        input_ids = tf.constant(encoded_text['input_ids'])#[None, :]
        #attention_masks = encoded_data_train['attention_mask']
        outputs = model(input_ids)
        if encoded == None:
            encoded = outputs[1]
        else:
            encoded = np.concatenate((encoded, outputs[1]), axis=0)

    encoded_test_x = pd.DataFrame(data=encoded, index=test_x.index)
    return encoded_test_x

print(BERT_embeddings(data, max_seq_len=18)
```

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)
I would like to create sentence embeddings with MobileBert
## To reproduce

Steps to reproduce the behavior:

1. 'text' is one column pandas object
2. run above code
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
Embeddings have a form of vectors with large values like [-24888514.0  222398.593750...]

## Expected behavior
Should have [-1, 1] range

<!-- A clear and concise description of what you would expect to happen. -->
I would expect that elements of a vector will not be in range hundred of thousands but [-1, 1]

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.1
- Platform:
- Python version: 3.7.3
- PyTorch version (GPU?): -
- Tensorflow version (GPU?): 2.0.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Fine Tuning Using /question-answering/run_squad.py

How do I use `/question-answering/run_squad.py` to fine-tune my own custom dataset?

[StackOverflow][1]

[1]: https://stackoverflow.com/questions/62752709/how-to-fine-tune-bert-for-question-answering


Easier way to download pretrained model files to local

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarily intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Is there any why we can save vocab and models files to local without having to run the following code with cache_dir parameter. What i am asking is some utility or some endpoint from where we could get a tar of all model files.
```
from transformers import BartTokenizer, BartForSequenceClassification
tokenizer = BartTokenizer.from_pretrained('facebook/bart-large',cache_dir='path/to/local_dir')
model = BartForSequenceClassification.from_pretrained('facebook/bart-large',cache_dir='path/to/local_dir')
```
The reason for such a requirement is, we run our code in VMs where there is no internet access, so we have to run the code in our local for every model and save files. It would be helpful if there is a easier way to download all the files for pretrained models as a tar or zip file.

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->


Longformer - Compression

Hello, I have trained a longformer model on my custom question answering dataset, and now I wanted to know if there is a way to compress this trained model?

Thank you!


Create README

model card

Related to pretrained model cards


How i can set the special token <|endoftext|> to an other id ?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


How-to-fine-tune-bert-for-question-answering?

Help wanted

Extra attention is needed, help appreciated

I wish to train two domain-specific models:

Domain 1: Constitution and related Legal Documents
Domain 2: Technical and related documents.
For Domain 1, I've access to a text-corpus with texts from the constitution and no question-context-answer tuples. For Domain 2, I've access to Question-Answer pairs.

Is it possible to fine-tune a light-weight BERT model for Question-Answering using just the data mentioned above?

If yes, what are the resources to achieve this task?

Some examples, from the huggingface/models library would be mrm8488/bert-tiny-5-finetuned-squadv2, sshleifer/tiny-distilbert-base-cased-distilled-squad, /twmkn9/albert-base-v2-squad2.

[You can find the question here.](https://datascience.stackexchange.com/questions/77213/how-to-fine-tune-bert-for-question-answering)


[wip] Label smooth


Error in Loading bert-large-uncased-whole-word-masking-finetuned-squad

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
In order to use bert-large-uncased-whole-word-masking-finetuned-squad, I firstly download it from https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased-whole-word-masking-finetuned-squad-config.json and https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased-whole-word-masking-finetuned-squad-pytorch_model.bin. Then, as I used other pretrained model before, I change the files name into config.json and pytorch_model.bin. Finally, I run code bert = BertForQuestionAnswering.from_pretrained(root_path + "/bert-large-uncased-whole-word-masked-finetuned-squad/") and meat some errors.
The error info shows as follows:
  File "/home/dl/anaconda3/lib/python3.6/site-packages/transformers/modeling_utils.py", line 662, in from_pretrained
    "Unable to load weights from pytorch checkpoint file. "
OSError: Unable to load weights from pytorch checkpoint file. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
*** Error in `/home/dl/anaconda3/bin/python3': corrupted double-linked list: 0x000055e3278c88b0 ***

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


fixed ImportError: cannot import name 'hf_bucket_url' on convert_pytorch_checkpoint_to_tf2.py

So I installed the latest version of transformers on Google Colab

    !pip install transformers

When trying to invoke the conversion file using

    !python /usr/local/lib/python3.6/dist-packages/transformers/convert_pytorch_checkpoint_to_tf2.py .py --help

Or trying to use

    from transformers.file_utils import hf_bucket_url.                                 // works
    from transformers.convert_pytorch_checkpoint_to_tf2 import *.                      // fails

    convert_pytorch_checkpoint_to_tf("gpt2", pytorch_file, config_file, tf_file).

I get this error

     ImportError                               Traceback (most recent call last)

    <ipython-input-3-dadaf83ecea0> in <module>()
          1 from transformers.file_utils import hf_bucket_url
    ----> 2 from transformers.convert_pytorch_checkpoint_to_tf2 import *
          3
          4 convert_pytorch_checkpoint_to_tf("gpt2", pytorch_file, config_file, tf_file)
    /usr/local/lib/python3.6/dist-packages/transformers/convert_pytorch_checkpoint_to_tf2.py in <module>()
         20 import os
         21
    ---> 22 from transformers import (
         23     ALBERT_PRETRAINED_CONFIG_ARCHIVE_MAP,
         24     BERT_PRETRAINED_CONFIG_ARCHIVE_MAP,

    ImportError: cannot import name 'hf_bucket_url'

Turns out, there's a problem in the import of `hf_bucket_url`. Once fixed, the code ran properly.


Tabert

Feature request

Request for a new feature

# 🌟 New model addition

## Model description
a pre-trained language model for learning joint representations of natural language utterances and (semi-)structured tables for semantic parsing. TaBERT is pre-trained on a massive corpus of 26M Web tables and their associated natural language context, and could be used as a drop-in replacement of a semantic parsers original encoder to compute representations for utterances and table schemas (columns).
<!-- Important information -->

## Open source status

* [X] the model implementation is available: (give details)
https://github.com/facebookresearch/TaBERT
* [X] the model weights are available: (give details)
https://github.com/facebookresearch/TaBERT
* [ ] who are the authors: (mention them, if possible by @gh-username)


[fix] pin sacrebleu to fix CI ImportError


3.0.1: "unexpected keyword argument 'is_pretokenized'" when using batch_encode_plus() w/ Fast Tokenizers

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

See title. Does not occur with "slow" tokenizers.

## To reproduce

```python
from transformers import GPT2TokenizerFast

tokenizer = GPT2TokenizerFast.from_pretrained('gpt2')
texts = ['I am a teapot', 'Short and stout']
tokenizer.batch_encode_plus(texts)
```

## Trace

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-a99759a2bc25> in <module>
      1 texts = ['I am a teapot', 'Short and stout']
      2
----> 3 tokenizer.batch_encode_plus(texts)

/usr/local/lib/python3.8/site-packages/transformers-3.0.1-py3.8.egg/transformers/tokenization_utils_base.py in batch_encode_plus(self, batch_text_or_text_pairs, add_special_tokens, padding, truncation, max_length, stride, is_pretokenized, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
   1813         )
   1814
-> 1815         return self._batch_encode_plus(
   1816             batch_text_or_text_pairs=batch_text_or_text_pairs,
   1817             add_special_tokens=add_special_tokens,

/usr/local/lib/python3.8/site-packages/transformers-3.0.1-py3.8.egg/transformers/tokenization_gpt2.py in _batch_encode_plus(self, *args, **kwargs)
    365         )
    366
--> 367         return super()._batch_encode_plus(*args, **kwargs)
    368
    369     def _encode_plus(self, *args, **kwargs) -> BatchEncoding:

/usr/local/lib/python3.8/site-packages/transformers-3.0.1-py3.8.egg/transformers/tokenization_utils_fast.py in _batch_encode_plus(self, batch_text_or_text_pairs, add_special_tokens, padding_strategy, truncation_strategy, max_length, stride, is_pretokenized, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
    340             encodings = [encodings]
    341         else:
--> 342             encodings = self._tokenizer.encode_batch(
    343                 batch_text_or_text_pairs, add_special_tokens=add_special_tokens, is_pretokenized=is_pretokenized
    344             )

TypeError: encode_batch() got an unexpected keyword argument 'is_pretokenized'
```

## Environment info

- `transformers` version: 3.0.1
- `tokenizers` version: 0.8.0rc4
- Platform: macOS
- Python version: 3.8.2


Tf to pytorch

Hi, I have XLNet checkpoint file and want to convert it to pytorch. I have converted the file to pytorch and when I am using the same file with simpletramsformers for text classification. It says the number of labels is not matching. Where should I specify the number of classes while converting from tf to pytorch?


Fix `RobertaClassificationHead` style consistency.

There's a slight inconsistency in `RobertaClassificationHead` in that it takes in the whole sequence output from the `RobertaModel`, and extracts the pooled output inside its own forward method, seen [here](https://github.com/huggingface/transformers/blob/58cca47c16149e43d1b516623d59e3c5d97f695e/src/transformers/modeling_roberta.py#L561).

This is different from other models, where the pooled output is computed beforehand and directly expected as input in the classifier. E.g. in [`BertForSequenceClassification`](https://github.com/huggingface/transformers/blob/58cca47c16149e43d1b516623d59e3c5d97f695e/src/transformers/modeling_bert.py#L1270), [`DistilBertForSequenceClassification`](https://github.com/huggingface/transformers/blob/58cca47c16149e43d1b516623d59e3c5d97f695e/src/transformers/modeling_distilbert.py#L631), [`BartForSequenceClassification`](https://github.com/huggingface/transformers/blob/58cca47c16149e43d1b516623d59e3c5d97f695e/src/transformers/modeling_bart.py#L1138), etc.

This mainly addresses this issue in `modeling_roberta.py` and `modeling_tf_roberta.py`. Additionally, some minor aesthetic changes are made to these files in order to pass the black / sort code quality checks.

Note: This PR is a duplicate of #4107 with minor changes made to pass code quality checks. Closed that one since it was outdated.


WARNING:transformers.tokenization_utils:Keyword arguments {'add_space_before_punct_symbol': True} not recognized.

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
ctrl

Language I am using the model on (English, Chinese ...):
English
The problem arises when using:
* [ ] the official example scripts: (give details below)
```
python run_generation.py \
    --model_type ctrl \
    --model_name ctrl --length=100 --temperature 0.2 --num_return_sequences=5 --p=0.8 --seed=17 --repetition_penalty=1.2
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

```
- `transformers` version: 3.0.1
- Platform: Darwin-19.5.0-x86_64-i386-64bit
- Python version: 3.7.4
- PyTorch version (GPU?): 1.2.0 (False)
- Tensorflow version (GPU?): 2.1.0 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no
```


How to fine-tune tinyBERT for question-asnwering

# ❓ Questions & Help
How do I fine-tune tiny-BERT for question answering on my custom dataset. I've a list of questions and corresponding answers in a csv format, How do I fine-tune tiny-BERT on this dataset.
[You can find it at StackOverFlow here](https://stackoverflow.com/questions/62710931/huggingface-transformers-model-for-legal-question-answering)


Create model card

model card

Related to pretrained model cards