Loading DistilBertModel with AutoModel gives 12 layers

I am using ``AutoModel`` to load  ``distilbert-base-uncased`` and save the fine-tuned model after training using ``model.save_pretrained('path_to_save')``. However, when I load the fine-tuned model using ``AutoModel.from_pretrained('path_to_the_saved_model')``, it extracts 12 layers instead of 6 layers. I also checked the ``config.json`` file that was saved automatically and the number of layers is still 6. When I load the model with ``DistilBertModel.from_pretrained()`` it extracts 6 layers. In the following, I copied the ``config.json`` file. Does anyone know why this happens? am I lacking some packages or files when loading/saving the model?

> {
  "activation": "gelu",
  "architectures": [
    "DistilBertModel"
  ],
  "attention_dropout": 0.1,
  "bos_token_id": 0,
  "dim": 768,
  "do_sample": false,
  "dropout": 0.1,
  "eos_token_ids": 0,
  "finetuning_task": null,
  "hidden_dim": 3072,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1"
  },
  "initializer_range": 0.02,
  "is_decoder": false,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1
  },
  "length_penalty": 1.0,
  "max_length": 20,
  "max_position_embeddings": 512,
  "model_type": "distilbert",
  "n_heads": 12,
  "n_layers": 6,
  "num_beams": 1,
  "num_labels": 2,
  "num_return_sequences": 1,
  "output_attentions": true,
  "output_hidden_states": false,
  "output_past": true,
  "pad_token_id": 0,
  "pruned_heads": {},
  "qa_dropout": 0.1,
  "repetition_penalty": 1.0,
  "seq_classif_dropout": 0.2,
  "sinusoidal_pos_embds": false,
  "temperature": 1.0,
  "tie_weights_": true,
  "top_k": 50,
  "top_p": 1.0,
  "torchscript": false,
  "use_bfloat16": false,
  "vocab_size": 40000
}


Make sacremoses dependency optional due to GPL license.

Closes #2453.


Create README.md

model card

Related to pretrained model cards


Update examples/ner/run_ner.py to use AutoModel

This PR is updating `run_ner.py` to use AutoModel implementation. Refer to #3290 and is simpler than before.

Maybe @srush can review this.


Error in loading  albert-base-v2

![image](https://user-images.githubusercontent.com/25845940/76815952-f3be9700-6839-11ea-9ef9-53bc3655cefa.png)
![image](https://user-images.githubusercontent.com/25845940/76815974-020cb300-683a-11ea-830a-8a26b1eb6822.png)

Help~~
The above problems appear in loading the pre-trained Albert by transformers.


Error in loading Albert model

![image](https://user-images.githubusercontent.com/25845940/76815416-88c09080-6838-11ea-829a-5a5874692009.png)

Help~
The above problems appear in loading the pre-trained Albert by transformers. Albert pre-trained model download address is https://drive.google.com/file/d/1byZQmWDgyhrLpj8oXtxBG6AA52c8IHE-/view


[BART] Delete redundant unit test


Add model card for Google AI's BERT Miniatures

Related to pretrained model cards

This model card is intended to be shared among all models under google/bert_uncased_*
(We'll need some support from HuggingFace to get this card cross-linked from all models)


## Information

Hi, I am trying to use the BART model to sumamrize a text snippet.

The problem arises when using:
* from transformers import BartTokenizer, BartConfig, BartForConditionalGeneration

## To reproduce

Steps to reproduce the behavior:

1. Installed Tensorflow 2.0 and Pytorch
2. pip install transformers
3. from transformers import BartTokenizer, BartConfig, BartForConditionalGeneration

<!-- ImportError: cannot import name 'BartForConditionalGeneration'-->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: (2.5.1)
- Platform:
- Python version: Python 3.6.5 :: Anaconda, Inc.
- PyTorch version (GPU?): (1.3.1)
- Tensorflow version (GPU?): (2.0.0)


add camembert for Question answering for examples

This one might have been accidently deleted in PR #2700 I think.


[generate] do_sample default back to False

This somewhat reverts the commit:
https://github.com/huggingface/transformers/commit/6c1b23554f8bb5b5e1f6c80969acab764c755678

and the decision taken in #2696

and sets the default sampling behavior of `generate()` to greedy - / beam search.

Pros:
- `False` is the more natural default value
- Prettier API (especially for encoder_decoder models which will mostly only use beam search generate())

Cons:
- Some people might aleady be used to the `do_sample=True` default value and this commit might break the logic of their code (but would be trivial to change for them)

I'm somewhat indifferent whether this PR should be merged, but I think @thomwolf and @sshleifer  are in favor of it.

@LysandreJik @thomwolf @sshleifer


Getting output of any hidden layer

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

Is there a way to get output from any hidden layer of the model?

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

I'm working on  the ALBERT transformer (more specifically `AlbertForSequenceClassification`, and when I print the model, this is the model's architecture:

```py
AlbertForSequenceClassification(
  (albert): AlbertModel(
    (embeddings): AlbertEmbeddings(
      (word_embeddings): Embedding(30000, 128, padding_idx=0)
      (position_embeddings): Embedding(512, 128)
      (token_type_embeddings): Embedding(2, 128)
      (LayerNorm): LayerNorm((128,), eps=1e-12, elementwise_affine=True)
      (dropout): Dropout(p=0, inplace=False)
    )
    (encoder): AlbertTransformer(
      (embedding_hidden_mapping_in): Linear(in_features=128, out_features=768, bias=True)
      (albert_layer_groups): ModuleList(
        (0): AlbertLayerGroup(
          (albert_layers): ModuleList(
            (0): AlbertLayer(
              (full_layer_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (attention): AlbertAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0, inplace=False)
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              )
              (ffn): Linear(in_features=768, out_features=3072, bias=True)
              (ffn_output): Linear(in_features=3072, out_features=768, bias=True)
            )
          )
        )
      )
    )
    (pooler): Linear(in_features=768, out_features=768, bias=True)
    (pooler_activation): Tanh()
  )
  (dropout): Dropout(p=0, inplace=False)
  (classifier): Linear(in_features=768, out_features=2, bias=True)
)
```

I would like to get the outputs of middle / hidden layers, for example of layers `ffn_output` or `pooler`, but I'm not sure if that option exists. I've tried extracting `hidden_states` by setting `output_hidden_states` to True in AlbertConfig, but that doesn't bring me the result that I want.

I believe tf-hub has a method  or attribute for this called by `model.get_layer(layer_name)`.

Is there a way to extract hidden layers?


Installation error: can not find Rust compiler

I used pip to install transformers like this:
pip install transformers
in the end, I got the error:
   Can not find Rust compiler
while, I have installed rust on my apple computer, please tell me how to deal with the problem, thank you!


Create CodeBERTaJS model card

Related to pretrained model cards


BertForPreTraining should compute only <MASKED> prediction_scores

# 🚀 Feature request
In class transformers.BertForPreTraining, the forward compute all prediction_scores. In fact, we may only calculate the prediction_scores on the <MASKED> tokens to save some computational cost.

## Motivation
In source code of class BertForPreTraining:
```python
outputs = self.bert(input_ids, attention_mask, ...)
sequence_output, pooled_output = outputs[:2]
prediction_scores, seq_relationship_score = self.cls(sequence_output, pooled_output)
```
It compute `prediction_scores` using `sequence_output` as the input of function `self.cls()`. I was wondering if we can gather the MASKED index (not equal to -100) and only pass the MASKED sequence_output into function `self.cls` and return the MASKED prediction_scores. Then pass them into `CrossEntropy()` to compute the `masked_lm_loss`. In this way, we can save some computational cost and partially relief the OOM problem in GPU.

## My suggestion
We may change the code to
```python
outputs = self.bert(input_ids, attention_mask, ...)
sequence_output, pooled_output = outputs[:2]
# before gather_indexes, size of sequence_output: (batch_size, sequence_length, hidden_size)
# after gather_indexes, size of sequence_output: (batch_size, masked_lm_nums, hidden_size)
sequence_output = gather_indexes(sequence_output, masked_ln_labels)
prediction_scores, seq_relationship_score = self.cls(sequence_output, pooled_output)
```
Then pass prediction_scores and corresponding `masked_lm_labels` (also need to gather the indexes) into function CrossEntropy to compute the masked_lm_loss.

Note: if we adopt this method, the return of BertForPreTraining is changed since we will not compute prediction_scores on all tokens.


Create model card for spanbert-finetuned-squadv2

Related to pretrained model cards


NER Pipeline returns null

# 🐛 Bug

## Information

Model I am using (NER Pipeline):

Language I am using the model on (English):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```
from transformers import pipeline
# Allocate a pipeline for named entity recognition
nlp = pipeline('ner')
nlp(['We are very happy to include pipeline into the transformers repository.'])

```
**returns null.**

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Expect to have the named entity label for each token. But it returns null.

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.5.1
- Platform: Jupyter lab
- Python version: 3.6.1
- PyTorch version (GPU?): CPU-1.4.0
- Tensorflow version (GPU?): CPU-1.15.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


a lot of examples in doc can't run successful

I used example in doc, but a lot of example can't run successful, what's wrong?


[WIP] Lightning glue example

This PR adds an example of using Pytorch Lightning to run the GLUE benchmark. Additionally, I altered the `transformer_base.py` to use auto models and moved it to the example directory so it could be copied in by any script that wishes to use it.

Preferably, the base transformer would have subclasses for the different types of tasks, but I just used a dictionary with a key passed on init instead. (i.e. NER uses `AutoModelForTokenClassification` and GLUE uses `AutoModelForSequenceClassification`).


GPT-2 attention_mask reshaping uses input_ids first dimension

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT-2

## To reproduce
Use attention_mask in **GPT2LMHead**, while feeding **inputs_embeds** instead of **input_ids**
The code fails because line 427 of modeling_gpt2.py uses input_ids first dimension to reshape the mask
```
        if attention_mask is not None:
            batch_size = input_ids.shape[0]
            attention_mask = attention_mask.view(batch_size, -1)
```
I fix it by changing **input_ids.shape[0]** to **attention_mask.shape[0]**, but I think it would be more correct to obtain single batch_size from one of the available input formats.
**Update**
   I think `batch_size = input_shape[0]` is the best way
- `transformers` version: **master**
- Python version: 3.7
- PyTorch version (GPU?):1.4


Dockerhub images huggingface/transformers_cpu for version 2.5.1 has version 2.5.0 installed

# 🐛 Bug

## Information

Model I am using any model introduced in 2.5.1

The problem arises when using:
pulling `huggingface/transformers_cpu:2.5.1` from dockerhub

## To reproduce

Steps to reproduce the behavior:

1. pull docker image from dockerhub
2. run docker container
3. run `pip freeze` to see `transformers==2.5.0`

## Expected behavior
`pip freeze` in the docker container should show: `transformers==2.5.1`


Unexpected output from feature extraction pipeline

Hi everyone,

I'm not sure if it is a bug or if I simply overlooking something so I did not want to submit a bug report yet. I have the following example code:

```
from transformers import pipeline, AutoTokenizer
import numpy as np

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased',
add_special_tokens=False)
#initialize pipeline
nlp = pipeline('feature-extraction', model='bert-base-uncased', config='bert-base-uncased', tokenizer=tokenizer, device=1)
features = nlp("Why is Howard asking questions about the food after Leonard gives him a carton ?")
features = np.squeeze(features)
print(features.shape)
```

I expect the output: (15,768)
But I receive the output: (18,768)

I think there are only 15 tokens but somehow the shape is 18. What am I missing here?

Is this output expected and am I simply missing something or is there more to it?


Adding LM Head to Transfo-XL and first step to fixing problem with Adaptive Embeddings in TransfoXL

This PR adds LM generation capabilitiies to the TF transfo-xl model. The integration tests for language generation pass, so generation from a pretrained model works now in TF as well.

What does definitely not work yet is running the both PT and TF models with `self.sample_softmax > 0`:
- Transfo-XL uses adaptive word embeddings -> the word embeddings ale broken down into 4 Embeddings of different shapes: `[20000, 1024], [20000, 1024], [160000, 64]` and `[67735, 16]` . When `self.sample_softmax > 0` though, it seems like the model expects the `normal` word embeddings with just a single weight matrix. When trying to tie the weights then as done in line 831 (see comment below), the logic breaks.

This problem seems to be more complex though and I'd suggest to solve it in another PR (add possible have a call before to make things clear).


Is there a way to evaluate GPT-2 model during fine-tuning process for accuracy and fluency?

# ❓ Questions & Help

I'm trying to evaluate GPT-2 model during fine tuning  process, and I'm able to calculate the loss at each epoch, but do not know how accuracy can be calculated or how to give a score to the model. Would like to get some suggestions as help.
## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: https://stackoverflow.com/questions/60483956/how-to-perform-accuracy-testing-on-text-generation-task


Return token span from NerPipeline

# 🚀 Feature request

I would like to suggest that NerPipeline should return the span in the original text where the matched entity exists.

Instead of:
```python
[
    {"word": "New", "score": 0.9995751976966858, "entity": "LOC"},
    {"word": "York", "score": 0.9996403455734253, "entity": "LOC"}
]
```

I would like to see this:
```python
[
    {"word": "New", "score": 0.9995751976966858, "entity": "LOC", "span": (0, 3)},
    {"word": "York", "score": 0.9996403455734253, "entity": "LOC", "span": (4, 8)}
]
```

## Motivation

I'm trying to use transformers for NER, and I specifically want to return multi word entities as one phrase.

With the above example, I would like to return "New York". With spans added, I would be able to merge nearby tokens into one.

This makes it possible to differentiate the result of "A place called New, and a place called York" from "A place called New York". With the current scheme, they both return the same thing.

## Your contribution

I think I understand NerPipeline enough to make a PR, if this is something you would be open to.


What is the most effective way to use BERT , ROBERTA , GPT-2 architectures as frozen feature extractors ?

Discussion

Discussion on a topic (keep it focused or open a new issue though)

We use pretrained self-supervised learning (SSL) models for NLP as feature extractors for downstream tasks like sentiment analysis. In most of such cases, we add a simple new classification layer and **fine-tune the whole model**.  With the SSL models getting bigger and the amount of unsupervised training data is huge it would be nice if we can use the problem agnostic behavior of SSL embeddings. In other words if we use them as **Frozen Feature extractors**, we can save lot of time and computational cost.

**Have anyone seen a good review on using SSL networks as frozen feature extractors?**


Install error , Win10,anaconda3,python3.5,pytorch

 When I pip install the transformers,its not successfull,my environment is Win10,anaconda3,python3.5,the error is as follows,what is wrong with it,thank you!.

## Details
 Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\sjh\AppData\Local\Temp\pip-install-fu05dcfq\sentencepiece\setup.py", line 29, in <module>
        with codecs.open(os.path.join('..', 'VERSION'), 'r', 'utf-8') as f:
      File "C:\Users\sjh\Anaconda3\Lib\codecs.py", line 895, in open
        file = builtins.open(filename, mode, buffering)
    FileNotFoundError: [Errno 2] No such file or directory: '..\\VERSION'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\sjh\AppData\Local\Temp\pip-install-fu05dcfq\sentencepiece\


how to use TFBertModel to load a Bert, which the path is from own computer.

my model's path is on my computer, so, I want to load it, but when I use TFBertModel to load it, it appeared this error.
`model = TFBertModel.from_pretrained('/Users/maxiong/Workpace/Code/transformers/pre_model',config=config)`

error:
![image](https://user-images.githubusercontent.com/30991932/76696213-dcf63400-66c3-11ea-8345-2d7780470709.png)

this is my model files
![image](https://user-images.githubusercontent.com/30991932/76696214-e384ab80-66c3-11ea-9b97-6a0a685f67d1.png)


how to finetune with PreTrainedEncoderDecoder

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue --> I am trying to run seq2seq within the same language (say it is English). I was trying to use PreTrainedEncoderDecoder (BERT, BERT), was trying to use BERT and GPT2 however, looks like it does not support this combination yet.
I am trying to understand what the forward function does in the class PreTrainedEncoderDecoder, and how can we use it training my dataset.
Also how can we use it at prediction time since forward need to have both decode_input_ids and encode_input_ids. I do not think we will have decode_input_ids at prediction time. Thank you

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


[BART] Remove unused kwargs

This doesn't change anything,
- k_dim and v_dim kwargs are there for other models in fairseq but we don't need them.
- attention weights are returned by the AttentionModule (and ignored later) no matter what


[BART] generation_mode as a kwarg not a class attribute

Currently, we set it to `BartModel.decoder.generation_mode = True` and then never unset it, which is confusing in the the rare case where you try to finetune or extract features after generating.
We can encapsulate bart specific logic to modeling_bart.py by just using a kwarg.


Add missing token classification for XLM

The current [modeling_xlm.py](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_xlm.py) did not have alike `ForTokenClassification` class like others, which helps for NER task comparison across all existing models.

now `XLMForTokenClassification` can be called via:
```python
from transformers import XLMForTokenClassification

model = XLMForTokenClassification.from_pretrained('xlm-mlm-100-1280')

```


Model fail to revert to generation_mode=False after generation

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BART

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name): CNN/DM
* [ ] my own task or dataset: (give details below)

## To reproduce

Hi @sshleifer,

Thanks for the amazing model!
I found a bug when alternatively trying to use **forward** to train the BartForConditionalGeneration and use **generate** to inference and evaluate the trained model. As shown in the code below, if I first use generate function then call forward function, the generation_mode attribute of decoder is set to True, and the shape of the decoder_output seems incorrect.

```
model = BartForConditionalGeneration.from_pretrained('bart-large-cnn')
tokenizer = BartTokenizer.from_pretrained('bart-large-cnn')

# input sequence
input_seq = "Bart is a deep-learning pretrained model implemented in pytorch. It can smoothly handle summarization. It is a big model pretrained for generation tasks, especially summarizaition."
input_ids = torch.LongTensor([tokenizer.encode(input_seq)])

# expected output sequence
decoder_input_seq = "Bart is a big pretrained deep model in pytorch for summarization."
decoder_input_ids = torch.LongTensor([tokenizer.encode(decoder_input_seq)])

# using generate method to inference
with torch.no_grad():
    result = model.generate(input_ids=input_ids, eos_token_ids=tokenizer.eos_token_id, num_beams=4, max_length=20)
print(tokenizer.decode(result[0]))
# 'B. It is a big model pretrained for generation tasks, especially summarizaition. It'

# NOW use forward to train
result = model(input_ids, decoder_input_ids=decoder_input_ids)
# the shape of decoder_output and encoder_output
# what expected is: <1, 18, 50264> and <40, 1, 1024>
# but actual output is:  torch.Size([1, 1, 50264]) torch.Size([40, 1, 1024])
print(result[0].shape, result[2].shape)

```

Such issue can **seemingly** be addressed by mannually setting the generation_mode.

```
# mannually set the generation mode to False **seemingly** fix the issue
model.model.decoder.generation_mode=False
result = model(input_ids, decoder_input_ids=decoder_input_ids)
print(result[0].shape, result[2].shape)
# output is: torch.Size([1, 18, 50264]) torch.Size([40, 1, 1024]) and make sense
```

However, now the output of the forward function doesn't make sense, as shown below.

```
with torch.no_grad():
    result = model.generate(input_ids=input_ids, eos_token_ids=tokenizer.eos_token_id, num_beams=4, max_length=20)

model.model.decoder.generation_mode=False
new_result = model(input_ids, decoder_input_ids=result)

print(tokenizer.decode(torch.argmax(new_result[0][0], dim=1)))
# ' and and and and and and and and and and and and and and and and and and and'
```
In expectation, when we feed the generated decoder sequence to the decoder and remain the input of the encoder unchanged, the output of the decoder would at least resemble the decoder_input. However, the actual output of the model is `' and and and and and and and and and and and and and and and and and and and'`, which is definately not a reasonable output of a trained decoder.

To go deeper, I tested different samples, all feeding the generated output of a encoder_input to the decoder and inspect the decoder_output (which "should" resemble decoder_input, i.e. generated sequence), the same happens everytime for me (most output is the duplication of a single token for many times, like "the the the ...", "and and and...", ". . . . ").

Given all these results, I think there is a bug in either the implementation of **generate**  or that of **forward**, the revert of generation_mode is simple, but I don't think the output of forward is a reasonable result currently. Could you please look into the issue?

 @sshleifer If I'm not using those functions in a correct manner, any advise or instruction is welcomed! Many thanks for the help!

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: master-branch
- Platform: windows 10
- Python version: 3.7.0
- PyTorch version (GPU?):  1.4.0
- Tensorflow version (GPU?): /
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Cannot Achieve Reproducability with Tensorflow Transformer Models

I've been experimenting with the roberta-large model with both PyTorch and TensorFlow for a sentiment analysis task. With the PyTorch model, I am able to achieve 100% reproducability, however, this is not the case with the Tensorflow model.

I have set all the necessary seeds as follows:

```
seed_val = 3
os.environ['PYTHONHASHSEED'] = str(seed_val)
np.random.seed(seed_val)
random.seed(seed_val)
tf.random.set_seed(seed_val)
```
and I am even using the fix described at https://github.com/NVIDIA/tensorflow-determinism:

```os.environ['TF_DETERMINISTIC_OPS'] = '1'```

I am not sure if reproducability is fully supported with the TensorFlow transformer models yet, or if I am doing something wrong.

Here is the link to the **CoLab notebook** containing my code:
https://drive.google.com/open?id=1xPTYPl8LyRrMgkiXUNtSbxxJ7pAslD2x

Thanks in advance


Tremendous slowdown in multi-node distributed training

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Bert
Finetuning a bert-base model on language modeling for a particular domain

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Training is happening on Azure NC24s_v3 nodes (4 V100s each) with NCCL as the backend. I'm comparing the performance on a single node scenario vs a 2 node scenario. Note that there is no infiniband networking between the nodes, only 40Gbps ethernet.
2. Use torch.distributed.launch to launch `run_language_modeling.py` in single node (mult-gpu) and multi node (mult-gpu) scenarios

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
In the single node scenario, I'm getting about 2 iteration/sec during training. In the multi-node scenario, it drops to 4 sec/iteration.

Theorizing that the network was the issue, I reduced the model size significantly (down to 1 layer from 12 and the other hyperparameters also scaled down appropriately) and ran the test again. The same slowdown in performance was observed even then.

Am I missing something here? Is it possible to perform multi-node training of bert models without infiniband?

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.5.1
- Platform: Ubuntu 18.04
- Python version: 3.6.10
- PyTorch version (GPU?): 1.5.0
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes


add XLMForTokenClassification

Firstly, I want to experiment with NER task across all available architecture with Thai language pretrained model.

Turned out I found not TokenClassification class for XLM yet.


how can i distill xlm-roberta model , just like distill roberta model , any suggestion ?thanks a lot

# ❓ Questions & Help

how can i distill xlm-roberta model , maybe i can distill it ,just like distill roberta model , any suggestion ? thanks a lot


Finetuning before feature extraction

Hi,

Currently I am using the pipeline feature extraction to extract features with my own dataset as input. I was wondering if it is possible to finetune different models using my own dataset before using the pipeline for feature extraction and if so what will be the easiest way to do so?

In the past there used to be a lm_finetuning script example but this one is no longer available. I cannot find any examples or guides how to finetune different models on a personal dataset.

tl;dr is it possible to finetune different models on a personal dataset in just a few lines of code and if so how?

Thanks in advance,

A clueless person trying to learn more about the world of NLP.


train model from scratch with big data

# 🚀 Feature request
<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

I would like to train a new model from scratch with some big data sets and the tutorial suggests to load and tokenize examples on the fly. It would be great if that feature is readily integrated in the example scripts (e.g. `run_language_modeling.py`). It is not entirely clear to me how to implement this in the most efficient way. My data set does not fit into memory and I cannot train out-of-the box with the existing scripts.

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


when I install transformers, it appeared this error

ERROR: Could not build wheels for tokenizers which use PEP 517 and cannot be installed directly


add gpt2-xl for tf

TF GPT2-XL is now added to AWS and can be loaded via:

```
from transformers import TFGPT2LMHeadModel
model = TFGPT2LMHeadModel.from_pretrained('gpt2-xl')
```

Thanks @bkkaggle for pointing this out!


removing torch.cuda.empty_cache() from TF function

torch.cuda.empty_cache() was being called from a TF function (even when torch is unavailable)
not sure any replacement is needed if TF OOMs

simply running the benchmarks on a GPU with lower HBM will reproduce this error


[BART] FP16 testing fixes

closes #3249: fp16 forward pass failing when no `decoder_attention_mask` provided. Adds test coverage.

closes #3265: test_generate_fp16 was failing since #3140   (by sending proper kwargs to `BartForConditionalGenerate.generate`)


[BART] test_generate_fp16 fails after PR#3140

passes after `git checkout d6de6423` (commit preceding #3140)
Traceback:
```
                 unfinished_sents.mul_((~eos_in_sents).long())

            # stop when there is a </s> in each sentence, or if we exceed the maximul length
>           if unfinished_sents.max() == 0:
E           RuntimeError: cuda runtime error (716) : misaligned address at /pytorch/aten/src/THC/THCReduceAll.cuh:327

src/transformers/modeling_utils.py:992: RuntimeError
```
@thomwolf @patrickvonplaten


Clean special token init in modeling_....py

#### INTRO:
This PR is a follow-up from PR #3011.

After discussion with @thomwolf today, we decided that the variable `eos_token_ids` in all models causes more confusion and ugly code than it helps.

#### BACKGROUND:
All models now have `pad_token_id`, `bos_token_id` and `eos_token_id` as default values. The reasons are discussed and explained in #3011.

Originally, we had the `list` variable `eos_token_ids`. The idea behind was that a model could have multiple `eos_token_ids` if the user wants to finish at certain tokens besides the standard EOS token. But this caused a lot of unclean code AND is not consistent with `tokenizers` which all has a `tokenizer.eos_token_id` int variable. So, we return to `eos_token_id` for moders as well and might in the future have a variable `forbidden_tokens` or `special_stop_tokens`.

#### THIS PR DOES:
- Replace all list `eos_token_ids` with `eos_token_id`
- Add default `eos_token_id, pad_token_id, bos_token_id` to all models

#### TESTS:
I tested that the `pretrained Config` has now the same special tokens as the `pretrained Tokenizer` for all model identifier names (e.g. `gpt2-large`) with the following code:

```
for model_id_name in ALL_PRETRAINED_MODEL_ARCHIVE_MAP.keys():
    tok = AutoTokenizer.from_pretrained(model_id_name)
    conf = AutoConfig.from_pretrained(model_id_name)

    pad_equal = tok.pad_token_id == conf.pad_token_id
    eos_equal = tok.eos_token_id == conf.eos_token_id
    bos_equal = tok.bos_token_id == conf.bos_token_id

    if not pad_equal:
        print("PAD not equal for {}!".format(model_id_name))
        print("TOK: {} | CONF: {}".format(tok.pad_token_id, conf.pad_token_id))

    if not eos_equal:
        print("EOS not equal for {}!".format(model_id_name))
        print("TOK: {} | CONF: {}".format(tok.eos_token_id, conf.eos_token_id))

    if not bos_equal:
        print("BOS not equal for {}!".format(model_id_name))
        print("TOK: {} | CONF: {}".format(tok.bos_token_id, conf.bos_token_id))
```

which gives the following result:

```
PAD not equal for bert-base-dutch-cased!
TOK: 3 | CONF: 0
BOS not equal for distilbert-base-cased!
TOK: None | CONF: 0
BOS not equal for distilbert-base-cased-distilled-squad!
TOK: None | CONF: 0
```

This means that:
- `bert-base-dutch-cased` has a different `pad_token_id` in its tokenizer config than the `pad_token_id` in default Bert tokenizer, so that we will have to update the `bert-base-dutch-cased-config.json` file on AWS (Best option in my opinion).
- `distilbert-base-cased` and `distilbert-base-cased-distilled-squad` have hard coded `bos_token_id` in their config.json file on AWS (I checked), but the distilbert tokenizer doesn`t even have it -> is that correct? @VictorSanh

#### TODO:

- [x] Is the approach good for you? @thomwolf @julien-c @LysandreJik @mfuntowicz @sshleifer
- [ ] Should we also check all community models whether their tokenizer differs from the default one?
- [ ] I think the test I wrote is quite useful, but it uses Config and Tokenizer Classes in the same file, which is not in line with the current test files, which is why I didn't add it. Should we add a test like this? If yes, how?


Create camembert-base-README.md

model card

Related to pretrained model cards

First version of our model_card for the original uploaded CamemBERT.
@louismartin @pjox


TFAlbertMainLayer cannot be imported from the transformers library.

Unlike the TFBertMainLayer class which can be imported from transformers, TFAlbertMainLayer cannot be imported.

Locally I made the changes to init.py:

Import TFAlbertMainLayer from modeling_tf_albert

Similar to tensorflow version, pytorch version should also be implemented.


Update examples/ner/run_ner.py

Update the example file by changing the name of AlbertForTokenClassification to AlbertForSequenceClassification.


This code is ok:

`from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-german-cased")
model = AutoModel.from_pretrained("distilbert-base-german-cased")
`

But the following fails:

`from transformers import BertForSequenceClassification, AdamW, BertConfig

model = BertForSequenceClassification.from_pretrained(
    "distilbert-base-german-cased",
    num_labels = 2,
    output_attentions = False,
    output_hidden_states = False,
)
`

`OSError: Model name 'distilbert-base-german-cased' was not found in model name list. We assumed 'https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-german-cased/config.json' was a path, a model identifier, or url to a configuration file named config.json or a directory containing such a file but couldn't find any such file at this path or url.`


Great job advice

First of all this is a very perfect job，I shared it with multiple colleagues，also please provide an implementation of the ElMo model and more tensorflow examples.
thank you very much~


very slow performance on transformer 2.5.0 versus 2.3.0

Hi
I am running the run_glue version with last version of transformer 2.5.0 python=3.5 and this is at least 10 times slower than running the same code with transformer 2.3.0 with python 3.6.9. I use the BERT model. The difference between speed of these versions is extremely high, could you have a look and check why the performance of latest version of the transformer is very low. thank you.


ELECTRA

Adds ELECTRA to the library.

The script I'm using to compare the different models is this [Github gist](https://gist.github.com/LysandreJik/db4c948f6b4483960de5cbac598ad4ed), coupled to [a modified version of the ELECTRA repository](https://github.com/LysandreJik/electra).

- [x] add model/configuration/tokenization classes
- [x] add conversion scripts
- [x] add tests
- [x] finalize

Let's detail what should be done at each step

## Adding model/configuration/tokenization classes

Here is the workflow for adding model/configuration/tokenization classes:

- [x] copy the python files from the present folder to the main folder and rename them, replacing `xxx` with your model name,
- [x] edit the files to replace `XXX` (with various casing) with your model name
- [x] copy-paste or create a simple configuration class for your model in the `configuration_...` file
- [x] copy-paste or create the code for your model in the `modeling_...` files (PyTorch)
- [x] copy-paste or create the code for your model in the `modeling_...` files (TF 2.0)
- [x] copy-paste or create a tokenizer class for your model in the `tokenization_...` file

# Adding conversion scripts

Here is the workflow for the conversion scripts:

- [x] copy the conversion script (`convert_...`) from the present folder to the main folder.
- [x] edit this script to convert your original checkpoint weights to the current pytorch ones.

# Adding tests:

Here is the workflow for the adding tests:

- [x] copy the python files from the `tests` sub-folder of the present folder to the `tests` subfolder of the main folder and rename them, replacing `xxx` with your model name,
- [x] edit the tests files to replace `XXX` (with various casing) with your model name
- [x] edit the tests code as needed

# Final steps

You can then finish the addition step by adding imports for your classes in the common files:

- [x] add import for all the relevant classes in `__init__.py`
- [x] add your configuration in `configuration_auto.py`
- [x] add your PyTorch and TF 2.0 model respectively in `modeling_auto.py` and `modeling_tf_auto.py`
- [x] add your tokenizer in `tokenization_auto.py`
- [x] add your models and tokenizer to `pipeline.py`
- [x] add a link to your conversion script in the main conversion utility (in `commands/convert.py`)
- [x] edit the PyTorch to TF 2.0 conversion script to add your model in the `convert_pytorch_checkpoint_to_tf2.py` file
- [x] add a mention of your model in the doc: `README.md` and the documentation itself at `docs/source/pretrained_models.rst`.
- [x] upload the pretrained weigths, configurations and vocabulary files.


Implement Electra

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

https://github.com/google-research/electra

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

Getting better results for example in QA tasks.

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

I have no idea how to implement, caused by the experience with this library


add BART to README


Bump psutil from 5.6.3 to 5.6.6 in /examples/distillation

dependencies

Pull requests that update a dependency file

Bumps [psutil](https://github.com/giampaolo/psutil) from 5.6.3 to 5.6.6.

Changelog

Sourced from psutil's changelog.

5.6.6

2019-11-25

Bug fixes

1179_: [Linux] Process cmdline() now takes into account misbehaving processes
renaming the command line and using inappropriate chars to separate args.

1616_: use of Py_DECREF instead of Py_CLEAR will result in double free and
segfault
(CVE-2019-18874 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-18874>__).
(patch by Riccardo Schirone)

1619_: [OpenBSD] compilation fails due to C syntax error.  (patch by Nathan
Houghton)

5.6.5

2019-11-06

Bug fixes

1615_: remove pyproject.toml as it was causing installation issues.

5.6.4

2019-11-04

Enhancements

1527_: [Linux] added Process.cpu_times().iowait counter, which is the time
spent waiting for blocking I/O to complete.

1565_: add PEP 517/8 build backend and requirements specification for better
pip integration.  (patch by Bernát Gábor)

Bug fixes

875_: [Windows] Process' cmdline(), environ() or cwd() may occasionally fail
with ERROR_PARTIAL_COPY which now gets translated to AccessDenied.

1126_: [Linux] cpu_affinity() segfaults on CentOS 5 / manylinux.
cpu_affinity() support for CentOS 5 was removed.

1528_: [AIX] compilation error on AIX 7.2 due to 32 vs 64 bit differences.
(patch by Arnon Yaari)

1535_: 'type' and 'family' fields returned by net_connections() are not
always turned into enums.

1536_: [NetBSD] process cmdline() erroneously raise ZombieProcess error if
cmdline has non encodable chars.

1546_: usage percent may be rounded to 0 on Python 2.

 ... (truncated)

Commits

c6cd256 pre release

b2414b8 revert #1595

c63369e updat HISTORY

edb20f6 linux, cmdline(), fix for #1179, comment 552984549: sometimes string ends wit...

d739cbb use PROCESS_QUERY_LIMITED_INFORMATION

f7e898b #1595: use psutil_pid_is_running() instead of GetExitCodeProcess

72c84cb #fix #1595 / windows: kill() may not raise AccessDenied

1f8d432 Merge branch 'master' of github.com:giampaolo/psutil

e6faebc release gil around users()/BSD (#1425)

5cb1b0b Merge branch 'master' of github.com:giampaolo/psutil

Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=psutil&package-manager=pip&previous-version=5.6.3&new-version=5.6.6)](https://help.github.com/articles/configuring-automated-security-fixes)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/huggingface/transformers/network/alerts).


bug in run_glue

Hi I got this error when running the run_glue.py
    from transformers import glue_compute_metrics as compute_metrics
ImportError: cannot import name 'glue_compute_metrics'


batch_encode_plus cannot work properly

I have data like below:
```
>>> ds[0][0:3]
["John was writing lyrics for his new album.He started experiencing writer 's block.He tried to force himself to write but it would n't do anythingHe tried to force himself to write but it would n't do anything.He took a walk , hung out with some friends , and looked at natureHe took a walk , hung out with some friends , and looked at natureHe took a walk , hung out with some friends , and looked at nature", 'Franny did not particularly like all of the immigration happening.She thought immigrants were coming to cause social problemsShe thought immigrants were coming to cause social problems.Franny was upset when an immigrant moved in next door.The immigrant , Sal , was kind and became friends with FrannyThe immigrant , Sal , was kind and became friends with Franny', 'Ari spends $ 20 a day on pickles.He decides to make his own to save money.He puts the pickles in brine.Ari waits 2 weeks for his pickles to get sour']
>>> ds[1][0:3]
['He felt inspiration and then went back home to write', 'When he finished his paper he went to bedWhen he finished his paper he went to bed', 'Trudey hoped self-publishing would be more profitable']
```
I was trying to do next sentence prediction using BERT model, it is necessary to use text pair to finish this problem.However,when I was tring to encode ds[0] and ds[1] as a batched text pair,I have met following problem.
```
>>> input_wsrq1ids=tokenizer.batch_encode_plus([(ds[0][0:3],ds[1][0:3])],add_special_tokens=True,return_tensors='pt')
>>> input_wsrq1ids
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'inp_wsrq1ids' is not defined
>>> input_wsrq1ids
{'input_ids': tensor([[101, 100, 100, 100, 102, 100, 100, 100, 102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 1, 1, 1, 1]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1]])}
```
The answer is definitely not the right encoding fot this data. I did not quite understand this documentation of batch_encod_plus method,it says for text_pair,look for encode_plus for details.However,the encode_plus only works for non-batched data, and there is no clue and example code to show us how to use batch_enocde_plus properly.


Why is the seq_len dimension hard coded to be the first dimension of BERT's input?

According to your [code](https://github.com/huggingface/transformers/blob/2e81b9d8d76a4d41a13f74eb5e0f4a65d8143cab/src/transformers/modeling_bert.py#L164), the `seq_len` dimension always corresponds to the first dimension of the input tensor. I found this problem when I tried to feed a tensor of shape `(batch_size, num_seq, seq_len)` and error occurred. Is there any specific reason for you to adopt this setting? Given that normally a PyTorch` Embedding` layer  can take an input of any shape, it doesn't look so natural to me to have this setting. Any suggestions? Many thanks!


UnicodeDecodeError when loading BART from fairseq checkpoint

# 🐛 Bug

When trying to load a checpoint from the fariseq library I'm getting an UnicodeError

## Information

Model I am using (Bert, XLNet ...): BART

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Train a checkpoint with the fairseq library
2. Load it using BartForMaskedLM.from_pretrained

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Get the checpoint loaded into a BART model.
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: master
- Platform: linux
- Python version: 3.6.7
- PyTorch version (GPU?): 1.4.0 / Yes
- Tensorflow version (GPU?):
- Using GPU in script?: No
- Using distributed or parallel set-up in script?:
- `fairseq` version: master


Using FP16 on BartModel

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BART

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: CNN/DM
* [ ] my own task or dataset: (give details below)

## To reproduce

I've installed the master branch of transformers but I still encountered the same issue as #3117 when using FP16 BartModel. I just initialized the model without loading the pretarined weights, but I guess the model should still be able to correctly forward the input LongTensor(batch, seq_length). The code is shown below, simply initialize a model and forward an input:

```
model = BartModel(BartConfig())
model = model.cuda().half()
cur_inputs = torch.zeros(4,16,dtype=torch.long).cuda()
cur_res = model(cur_inputs)
```

The error is:

>~\Anaconda3\envs\pytorch\lib\site-packages\transformers\modeling_bart.py in forward(self, query, key, value, key_padding_mask, layer_state, need_weights, static_kv, attn_mask)
    assert v is not None
-->  attn_output = torch.bmm(attn_probs, v)
    assert attn_output.size() == (bsz * self.num_heads, tgt_len, self.head_dim)
    attn_output = attn_output.transpose(0, 1).contiguous().view(tgt_len, bsz, embed_dim)
 RuntimeError: Expected object of scalar type Float but got scalar type Half for argument #2 'mat2' in call to _th_bmm

@sshleifer The model is quite novel to me, so am I using it incorrectly or there's still a bug in BertModel class? Thanks in advance for the help!

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: master branch
- Platform: Windows
- Python version: 3.7.0
- PyTorch version (GPU?): 1.4.0
- Tensorflow version (GPU?): /
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


[model_cards] polbert: simplify usage example with pipelines

Co-Authored-By: Darek Kłeczek


Improved Error message when loading config/model with .from_pretrained()

Given the previous error message, it can be quite time-consuming to find out that the only problem was that the /path/to/model/dir was incorrect :D


How do you do inference in production?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I was wondering how do you guys do inference in production? I tried to convert this model to tensorflow model but failed.
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
This is what I tried:

```
tf_model = TFGPT2LMHeadModel.from_pretrained("tmp/", from_pt=True)
tf.saved_model.save(tf_model,"tmp/saved")
loaded = tf.saved_model.load("tmp/saved")
print(list(loaded.signatures.keys()))
```
And it returns an empty list

**A link to original question on Stack Overflow**: https://stackoverflow.com/questions/52826134/keras-model-subclassing-examples


pad error in BertTokenizer.batch_encode_plus

As the document writes,the value that bert do not attend should have attention value of 0.However, I was using the BertTokenizer,the result is different.
This is my code:
```
input_wsrq1ids=tokenizer.batch_encode_plus(d[0],text_pair=d[1],add_special_tokens=True,return_tensors='pt')
>>> input_wsrq1ids['input_ids'][0]
tensor([  101,  2198,  2001,  3015,  4581,  2005,  2010,  2047,  2201,  1012,
         2002,  2318, 13417,  3213,  1005,  1055,  3796,  1012,  2002,  2699,
         2000,  2486,  2370,  2000,  4339,  2021,  2009,  2052,  1050,  1005,
         1056,  2079,  2505,  5369,  2699,  2000,  2486,  2370,  2000,  4339,
         2021,  2009,  2052,  1050,  1005,  1056,  2079,  2505,  1012,  2002,
         2165,  1037,  3328,  1010,  5112,  2041,  2007,  2070,  2814,  1010,
         1998,  2246,  2012,  3267,  5369,  2165,  1037,  3328,  1010,  5112,
         2041,  2007,  2070,  2814,  1010,  1998,  2246,  2012,  3267,  5369,
         2165,  1037,  3328,  1010,  5112,  2041,  2007,  2070,  2814,  1010,
         1998,  2246,  2012,  3267,   102,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0])
>>> input_wsrq1ids['attention_mask'][0]
tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
```
The attention_mask are all ones,however, the input_ids are padded with zero,which seems do not match the attention.


Get word seperator char for tokenization

Core: Tokenization

Internals of the library; Tokenization.

# 🚀 Feature request
When using some of tokenization models (e.g. : CamemBERT) you can find the char used to separate words after some investigation ([start](https://github.com/huggingface/transformers/blob/a4c75f149269099a98613f51b76cd0b579a109ee/src/transformers/tokenization_camembert.py#L274), [first jump](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_camembert.py#L27), [last jump](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_xlnet.py#L43)). Making it a general rule seems interesting.

## Motivation

With this you can detect whether a token is a subword or the start of a word (and then use this info for masking). But this is inconsistent across models. After several hours I couldn't find this information for RoBERTa (and GPT2) in the code. We can obviously get this char experimentally but it is not robust to new models or even different versions of the model.


seems TFBertForSequenceClassification cannot load tf1.x model?

for TFPreTrainedModel
  model.load_weights(resolved_archive_file, by_name=True)

seems by_name is always true.while i see in "tensorflow_core/python/keras/engine/network.py"

 if save_format == 'tf':
      status = self._trackable_saver.restore(filepath)
      if by_name:
        raise NotImplementedError(
            'Weights may only be loaded based on topology into Models when '
            'loading TensorFlow-formatted weights (got by_name=True to '
            'load_weights).')

So, it will always be NotImplementedError


Update examples/ner/run_ner.py

Update the example file by changing the name of AlbertForTokenClassification to AlbertForSequenceClassification.


simplify polbert usage example with pipelines

model card

Related to pretrained model cards

Indeed this is much simpler now! Pipelines look great :)


Minor Bug Fix for Running Roberta on Glue

Since `RobertaTokenizer`  does not generate `return_type_ids`, running Glue with Roberta throws errors. This fix overwrites the default behaviour of the tokenizers, and forces them to generate `return_type_ids`.


add output_past option to BERT class

I need key-value of the present state like GPT class for BERT as well (I'm testing PPLM like architecture but with masked LM), and here I've added the option `output_past` to BERT to enable returning those statistics.


How to encode a batch of sequence?

Hi,I am trying to learn this transformers package.
I prepared the data as following format:
`(("Mary spends $20 on pizza"),("She likes eating it),("The pizza was great"))`
I saw methods like `tokenizer.encode`,`tokenizer.encode_plus`t and `tokenizer.batch_encode_plus`.However, the `tokenizer.encode` seems to only encode single sentence.
Because when I input the data below,the answer it gives are like this:
```
>>> d[0][0]
'John was writing lyrics for his new album'
>>> d[0][1]
'Franny did not particularly like all of the immigration happening'
>>> input_ids = torch.tensor(tokenizer.encode([d[0][0],d[0][1]]))
>>> input_ids
tensor([101, 100, 100, 102])
```
Obviously,this is not the rights answer for the encoding.
When I was try method tokenizer.encode_plust,it can't even work properly,as the document write

> "text (str or List[str]) – The first sequence to be encoded. This can be a string, a list of strings (tokenized string using the tokenize method) or a list of integers (tokenized string ids using the convert_tokens_to_ids method)"

It can't even work when I only input a single sentence:
```
>>> input_ids = torch.tensor(tokenizer.encode_plus(d[0][0]))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: Could not infer dtype of dict
```
And the method,tokenizer.batch_encode_plust gives the same error message.


[WIP] Add BART for summarization training with CNN/DM using pytorch-lightning

This pull request adds to the example for BART for summarization. I used the [example for NER](https://github.com/huggingface/transformers/tree/master/examples/ner) using pytorch-lightning as guidance. This example will train on CNN/DM and evaluate, and get decent results, though I haven't trained it on the full dataset just yet. I'm sure there are better defaults for the hyperparams but these seem to work.
I based this PR on the code I wrote in this [colab](https://colab.research.google.com/drive/1C4jEf0fnLiz6Xdx4TDz1OoO4BRCjCx1m).

This would hopefully close https://github.com/huggingface/transformers/issues/3004

## TODO
- [x] Be able to train the model on a GPU.
- [x] remove unused args
- [x] add test step and save results.

Happy to hear any feedback!


Directories not found when saving checkpoints

_rotate_checkpoints deletes some checkpoint directory which causes "directories not found" error


[model_cards] 🇹🇷 Add new (cased) DistilBERTurk model

Hi,

this PR adds a new distilled BERT model for Turkish: DistilBERTurk 🤗

It was trained with the official Hugging Face [implementation](https://github.com/huggingface/transformers/tree/master/examples/distillation) for model distillation. It uses 7GB of the original training data of BERTurk, and uses the cased BERTurk model as teacher model.

DistilBERTurk was trained for 5 days on 4 RTX 2080 TI.

Performance is really promising: for PoS tagging the model outperforms the 24-layer XLM-RoBERTa and is only 0.69% behind the teacher model. For NER there's a performance diff of 0.44% compared to mBERT and 1.68% compared to the teacher model.


Bart: update example for #3140 compatibility


[TorchHub]Repo's layout is not compatible with TorchHub anymore since 2.0

# 🐛 Bug

When I try loading a model/tokenizer from the [pytorch hub](https://pytorch.org/hub/huggingface_pytorch-transformers/) page, the hub loading code is not working anymore. Pre transformers 2.0.0, the same loading code is working.

On a quick look, I believe it's related to the repo's fold layout since 2.0 where the `transformers` module is moved inside `src` but `hub_conf` is still assuming `transformers` exists in the same level and we get a module not found error.  For context, torch hub insert the repo root directory into `sys.path` to enable import.

## To reproduce

Run
```
import torch
tokenizer = torch.hub.load('huggingface/transformers:v2.5.0', 'tokenizer', 'bert-base-cased')
```
Stack trace:

```
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-5-155f4aa294f1> in <module>
      1 import torch
----> 2 tokenizer = torch.hub.load('huggingface/transformers:v2.5.0', 'tokenizer', 'bert-base-cased')
      3
      4 text_1 = "Who was Jim Henson ?"
      5 text_2 = "Jim Henson was a puppeteer"

~/miniconda3/envs/poc/lib/python3.7/site-packages/torch/hub.py in load(github, model, *args, **kwargs)
    354     sys.path.insert(0, repo_dir)
    355
--> 356     hub_module = import_module(MODULE_HUBCONF, repo_dir + '/' + MODULE_HUBCONF)
    357
    358     entry = _load_entry_from_hubconf(hub_module, model)

~/miniconda3/envs/poc/lib/python3.7/site-packages/torch/hub.py in import_module(name, path)
     70         spec = importlib.util.spec_from_file_location(name, path)
     71         module = importlib.util.module_from_spec(spec)
---> 72         spec.loader.exec_module(module)
     73         return module
     74     elif sys.version_info >= (3, 0):

~/miniconda3/envs/poc/lib/python3.7/importlib/_bootstrap_external.py in exec_module(self, module)

~/miniconda3/envs/poc/lib/python3.7/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

~/.cache/torch/hub/huggingface_transformers_v2.5.0/hubconf.py in <module>
----> 1 from transformers import (
      2     AutoConfig,
      3     AutoModel,
      4     AutoModelForQuestionAnswering,
      5     AutoModelForSequenceClassification,

ModuleNotFoundError: No module named 'transformers'
```

## Expected behavior

This was working before 2.0.0.

```
import torch
tokenizer = torch.hub.load('huggingface/transformers:1.2.0', 'tokenizer', 'bert-base-cased')

```

## Environment info

- `transformers` version: >2.0.0
- Platform: all
- Python version: 3.7
- PyTorch version (GPU?): 1.3.1
- Tensorflow version (GPU?): N/A
- Using GPU in script?: N/A
- Using distributed or parallel set-up in script?: N/A


train dev test split with BERT

Does `run_multiple_choice.py` work on train dev test splits?
I need to run BERT on 3 labeled datasets. Train it on my training set, validate it on my validation set (tune hyperparameters and calculate loss), and evaluate it on my test set (report performance). I finally want to do prediction on a forth unlabeled dataset.
I am wondering which of the codes in your repository includes these 3 modes. Thank you.


Create README.md for bio+discharge summary BERT

Related to pretrained model cards

Add Bio+ Discharge Summary BERT from Publicly Available Clinical BERT Embeddings


Add Bio+ Clinical BERT model card

Related to pretrained model cards

Adding Bio+ Clinical BERT model from Publicly Available Clinical BERT Embeddings paper


Support T5 Generation

In this PR some first commits are added to make T5 work for generation.

the `T5WithLMHeadModel.forward()` has a special API due to its encoder-decoder nature.
This is why we need to add a `prepare_inputs_for_generation()` in `t5_modeling_utils.py` to
correctly prepare t5's inputs for generation.

Some easy translation seems to give alright results (same results for TF):

```
model = T5WithLMHeadModel.from_pretrained('t5-base')
tok = T5Tokenizer.from_pretrained('t5-base')

text = "translate English to German: How old are you?"

input_ids = tok.encode(text, return_tensors='pt')
outputs = model.generate(input_ids, bos_token_id=tok.pad_token_id, max_length=22, num_beams=4, do_sample=False, early_stopping=True)

print(tok.decode(outputs[0], skip_special_tokens=True))
# prints:
# Wie alt bist du?st du?st du?st
```

UPDATE:

Updated generate() in both TF and PT to compute the `encoder_outputs` only once for `encoder-decoder` models as discussed below.

Tests `RUN_SLOW=1` for `test_modeling_bart.py`, `test_modeling_gpt2.py` and `test_modeling_tf_gpt2.py` all pass.

#### **FUTURE PR**:
- [ ] add generation integration test for T5 in PT and TF (could be similar to what is done in
e.g.  OR better compare numbers to original T5 model numbers @craffel.

Good for me to merge!
@thomwolf @sshleifer @craffel


An Error report about pipeline

Core: Pipeline

Internals of the library; Pipeline.

# 🐛 Bug

## Information

This may be an easy question, but it has been bothering me all day.

When I run the code:
nlp = pipeline("question-answering")

It always tells me:
Couldn't reach server at 'https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-cased-distilled-squad-modelcard.json' to download model card file.
Creating an empty model card.

If I ignore it and continue to run the rest of the code:
nlp({
    'question': 'What is the name of the repository ?',
    'context': 'Pipeline have been included in the huggingface/transformers repository'
})

The error will appear:
KeyError: 'token_type_ids'


Strange behaviour after using BertTokenizer.add_tokens()

When a word is in origin Bert vocabulary, it would not be split.  But after adding several its subtokens, it would be split into pieces, which seems to inconsistent with the longest match first algorithm.
To reproduce:
```
import transformers
tokenizer = transformers.BertTokenizer.from_pretrained('bert-base-uncased)
tokenizer.tokenize('involve')
tokenizer.add_tokens(['inv','ol','ve'])
tokenizer.tokenize('involve')
```
Line 2 returns 'involve' , and line 4 returns ['inv','##ol','##ve'].
Is this behaviour expected or is it a bug?


Complete merge Seq-2-Seq generation into default generation

This is a follow-up PR to finalize #3140 .

There was still no conclusion on how to handle the fairseq tricks for generation. To summarize:

I think we have three options:

1. Remove all faiseq tricks. Here the ROUGE score is: **20.285**
2. Implement the fairseq tricks EXCEPT leaving the starting decoding_inputs_tokens to be the BOS token instead of EOS. Here the ROUGE score is: **19.369**
3. Add all fairseq tricks and maybe add a new argument to `generate()` which is called `decoder_start_token_id=bos_token_id` , but can be overriden to be the `eos_token_id` in the case of Bart. Here the ROUGE score is: **21.072**

ROUGE scores from @sshleifer

For comparison:

![76256460-5b675780-6226-11ea-8516-28d0427251ca](https://user-images.githubusercontent.com/23423619/76422619-cdb27600-63a5-11ea-82c7-f36addb7fefb.png)

UPDATE:

Given the above scores, option 1. was chosen for the moment to have the same scores as fairseq.
This means that we have to start the decoder ids with a EOS token (which might be weird and fairseq specific). Therefore, a new argument `decoder_start_token_id` was added to the generate function that defaults to the `bos_token_id`. When using Bart generate this argument should be set to the `eos_token_id` to have good results. To see how `Bart.generate()` should be used take a look at:
https://github.com/huggingface/transformers/blob/2e81b9d8d76a4d41a13f74eb5e0f4a65d8143cab/tests/test_modeling_bart.py#L470

At the moment option 2. is implemented which seems to give the worst results and is also not the cleanest option.
This PR implements option 3.

For me either option 1. or option 3. is fine. Up to discuss @thomwolf , @julien-c , @LysandreJik @sshleifer


Problem with PreTrainedTokenizerFast and return_offsets_mapping

Internals of the library; Pipeline.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Bert, bert-large-uncased-whole-word-masking-finetuned-squad

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts:
* [X] my own modified scripts:

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: SQUaD
* [ ] my own task or dataset: (give details below)

## To reproduce

Script to reproduce the behavior:

```python
# transformers v2.5.1 (https://github.com/huggingface/transformers/releases)
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch

model_name = "bert-large-uncased-whole-word-masking-finetuned-squad"
tokeniser = AutoTokenizer.from_pretrained(model_name, use_fast=True)
inputs = tokeniser.encode_plus("Who is Bert?", "Bert is a puppet by Jim Henson", add_special_tokens=True, return_tensors="pt", return_offsets_mapping=True)
```

This script produces the following wrong output:

```
Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm 2018.3.5\helpers\pydev\pydevd.py", line 1741, in <module>
    main()
  File "C:\Program Files\JetBrains\PyCharm 2018.3.5\helpers\pydev\pydevd.py", line 1735, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files\JetBrains\PyCharm 2018.3.5\helpers\pydev\pydevd.py", line 1135, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2018.3.5\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "D:/OneDrive/RC/Segovia/DEMO-11/Adam11/_QUESTION_ANSWERER.py", line 8, in <module>
    inputs = tokeniser.encode_plus("Who is Bert?", "Bert is a puppet by Jim Henson", add_special_tokens=True, return_tensors="pt", return_offsets_mapping=True)
  File "C:\Users\mary\.conda\envs\adam11\lib\site-packages\transformers\tokenization_utils.py", line 1889, in encode_plus
    **kwargs,
  File "C:\Users\mary\.conda\envs\adam11\lib\site-packages\transformers\tokenization_utils.py", line 1843, in batch_encode_plus
    stack = torch.stack(stack, dim=0)
TypeError: expected Tensor as element 0 in argument 0, but got list
```

## Expected behavior

According to the documentation that is available in file "tokenization_utils.py", the behaviour should be as follows:

>return_offsets_mapping:
>(optional) Set to True to return (char_start, char_end) for each token (default False).
>If using Python's tokenizer, this method will raise NotImplementedError. This one is only available on Rust-based tokenizers inheriting from PreTrainedTokenizerFast.

## Environment info

- `transformers` version: 2.5.1
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.7.6
- PyTorch version (GPU?): 1.4.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## My Patch

I have isolated the problem in file `tokenization_utils.py`.  Take a look at method `batch_encode_plus`.  It has the following statements:

```python
        # Sanitize the output to have dict[list] from list[dict]
        sanitized = {}
        for key in tokens[0].keys():
            stack = [e for item in tokens for e in item[key]]
            if return_tensors == "tf":
                stack = tf.stack(stack, axis=0)
            elif return_tensors == "pt":
                stack = torch.stack(stack, dim=0)
            elif not return_tensors and len(stack) == 1:
                stack = stack[0]

            sanitized[key] = stack
```
The problem is that `stack` may be a list with a toch.Tensor, but also a list of tuples with start/end offsets when `return_offsets_mapping=True`.  In such cases, `tf.stack` or `torch.stack` will fail because they expect a list of tensors as argument, not a list of tuples.

I have patched my local transformers installation as follows, and it seems to work well:

```python
            if return_tensors and len(stack) == 1 and isinstance(stack[0], torch.Tensor):
                if return_tensors == "tf":
                    stack = tf.stack(stack, axis=0)
                elif return_tensors == "pt":
                    stack = torch.stack(stack, dim=0)
            elif not return_tensors and len(stack) == 1:
                stack = stack[0]
```

Thanks!


torch.distributed.barrier() have NCCL error

## Details
<!-- Description of your issue -->
**singularity :** ```singularity build pytorch20.02.simg docker://nvcr.io/nvidia/pytorch:20.02-py3```

I use Slurm and singularity to run run_glue.py but have NCCL error on torch.distributed.barrier()
### test.sh
```
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks=2
#SBATCH --cpus-per-task=4
#SBATCH --gres=gpu:2
#SBATCH --mem=161920
module purge
module load compiler/gnu/7.3.0 openmpi3 singularity

singularity exec pytorch20.02.simg python -m torch.distributed.launch --nproc_per_node 2 run_glue.py --model_type bert --model_name_or_path bert-base-uncased --task_name MRPC --do_train --do_eval --do_lower_case --data_dir ./glue_data/MRPC --max_seq_length 128 --per_gpu_eval_batch_size=8 --per_gpu_train_batch_size=8 --learning_rate 2e-5 --num_train_epochs 3.0 --output_dir /tmp/MRPC/
```
### Output
```
Traceback (most recent call last):
Traceback (most recent call last):
  File "run_glue.py", line 701, in <module>
  File "run_glue.py", line 701, in <module>
    main()
  File "run_glue.py", line 618, in main
    main()
  File "run_glue.py", line 641, in main
    torch.distributed.barrier()  # Make sure only the first process in distributed training will download model & vocab
  File "/opt/conda/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 1489, in barrier
    torch.distributed.barrier()  # Make sure only the first process in distributed training will download model & vocab
  File "/opt/conda/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 1489, in barrier
    work = _default_pg.barrier()
RuntimeError: NCCL error in: ../torch/lib/c10d/ProcessGroupNCCL.cpp:450, unhandled system error, NCCL version 2.5.6
    work = _default_pg.barrier()
RuntimeError: NCCL error in: ../torch/lib/c10d/ProcessGroupNCCL.cpp:450, unhandled system error, NCCL version 2.5.6
[E ProcessGroupNCCL.cpp:284] NCCL watchdog thread terminated
[E ProcessGroupNCCL.cpp:284] NCCL watchdog thread terminated
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/conda/lib/python3.6/site-packages/torch/distributed/launch.py", line 263, in <module>
    main()
  File "/opt/conda/lib/python3.6/site-packages/torch/distributed/launch.py", line 259, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/opt/conda/bin/python', '-u', 'run_glue.py', '--local_rank=1', '--model_type', 'bert', '--model_name_or_path', 'bert-base-uncased', '--task_name', 'MRPC', '--do_train', '--do_eval', '--do_lower_case', '--data_dir', '.glue_data/MRPC', '--max_seq_length', '128', '--per_gpu_eval_batch_size=8', '--per_gpu_train_batch_size=8', '--learning_rate', '2e-5', '--num_train_epochs', '3.0', '--output_dir', '/tmp/MRPC/']' returned non-zero exit status 1.
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
```
Sloved
```
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks=2
#SBATCH --cpus-per-task=4
#SBATCH --gres=gpu:2
#SBATCH --mem=161920
module purge
module load compiler/gnu/7.3.0 openmpi3 singularity

singularity exec --nv pytorch20.02.simg python -m torch.distributed.launch --nproc_per_node 2 run_glue.py --model_type bert --model_name_or_path bert-base-uncased --task_name MRPC --do_train --do_eval --do_lower_case --data_dir ./glue_data/MRPC --max_seq_length 128 --per_gpu_eval_batch_size=8 --per_gpu_train_batch_size=8 --learning_rate 2e-5 --num_train_epochs 3.0 --output_dir /tmp/MRPC/
```


Why the pre-trained models will be downloaded each times?

I often use pre-trained models, but when I want to import it (after restart).
But each time, I have to download again. So is there a function that can automatically save the model you downloaded in a directory? In Gluon-NLP, it will save the model you used in `.mxnet` in your home directory.


Model card for dkleczek/bert-base-polish-uncased-v1

model card

Related to pretrained model cards


How to tokenize word to characeter

Core: Tokenization

Internals of the library; Tokenization.

I am studying machine reading comprehension on xlmroberta.
My data is korquad.

I need to tokenize all word to character.
e.g. by english
This is a dog
-> _T h i s _i s _a _d o g

please let me know.


Typo in warning message

`T5Tokenizer` instead of `XLNetTokenizer`


Create README.md

model card

Related to pretrained model cards


Create README.md


fix typo in docstring demonstrating invocation of PreTrainedEncoderDecoder.from_pretrained


Update README.md

Related to pretrained model cards

- Update title
- Remove metrics


Update README.md

Related to pretrained model cards

Change title to clarify the model description


Update README.md

Related to pretrained model cards

- Remove metrics until use other benchmarks to test the model


Update README.md

Related to pretrained model cards

- Remove metrics until tested on other xquad benchmarks


Error loading pretrained bert-base-multilingual-cased

# 🐛 Bug

## Information

Loading `bert-base-multilingual-cased` from pretrained gives an error:

```
Traceback (most recent call last):
  File "error.py", line 4, in <module>
    bert_model = transformers.AutoModel.from_pretrained('bert-base-multilingual-cased', config=config)
  File "/scratch/gobi1/bai/bai-conda/lib/python3.7/site-packages/transformers/modeling_auto.py", line 380, in from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
  File "/scratch/gobi1/bai/bai-conda/lib/python3.7/site-packages/transformers/modeling_utils.py", line 558, in from_pretrained
    model.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for BertModel:
        size mismatch for bert.embeddings.word_embeddings.weight: copying a param with shape torch.Size([119547, 768]) from checkpoint, the shape in current model is torch.Size([30522, 768]).
```

## To reproduce

The following snippet produces the error:

```
import transformers

config = transformers.BertConfig(output_hidden_states=True)
bert_model = transformers.AutoModel.from_pretrained('bert-base-multilingual-cased', config=config)
print(bert_model)
```

## Expected behavior

The model should load. Note that `bert-base-uncased` is able to load properly.

## Environment info

- `transformers` version: 2.5.1
- Platform: Linux
- Python version: 3.7.5
- PyTorch version (GPU?): 1.4.0 with GPU
- Tensorflow version (GPU?): N/A
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


Pipeline for Question Answering: How to return multiple correct answers output?

Core: Pipeline

Internals of the library; Pipeline.

Very simple question I'm using transformer pipelines question answering on a very long piece of text. And, it is the cases that the are are multiple cases for the correct answer. I want them all.

I was wondering if I could retrieve the 10 best scores of output instead of just one.

Many thanks.


More details about DistilBERT experiment setting.

NIPS workshop paper (http://arxiv.org/abs/1910.01108) does not provide loss weights and other distillation hyper-paramerters (temperature, learning rate, epoch, step... ). and also (https://medium.com/huggingface/distilbert-8cf3380435b5). So when I try to experiment with this scripts(/transformers/examples/distillation/), it's hard to set hyper parameters.

Actually, I tried to train my model with distilation/ scrpts (loss weights are set equally 0.33), and got undesirable result. :sob: I think it would be helpful for others to explicitly write experimental settings of DistilBERT in the readme file. Have a good day.


where is the position emdeddings in bert for training a new model from scratch ?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


UnboundLocalError: local variable 'tokenizer' referenced before assignment

I am runnnig the example code on the homepage.
However,I met this problem.
```
import torch
from transformers import *

MODELS = [(BertModel,       BertTokenizer,       'bert-base-uncased'),
          (OpenAIGPTModel,  OpenAIGPTTokenizer,  'openai-gpt'),
          (GPT2Model,       GPT2Tokenizer,       'gpt2'),
          (CTRLModel,       CTRLTokenizer,       'ctrl'),
          (TransfoXLModel,  TransfoXLTokenizer,  'transfo-xl-wt103'),
          (XLNetModel,      XLNetTokenizer,      'xlnet-base-cased'),
          (XLMModel,        XLMTokenizer,        'xlm-mlm-enfr-1024'),
          (DistilBertModel, DistilBertTokenizer, 'distilbert-base-cased'),
          (RobertaModel,    RobertaTokenizer,    'roberta-base'),
          (XLMRobertaModel, XLMRobertaTokenizer, 'xlm-roberta-base'),
         ]
for model_class, tokenizer_class, pretrained_weights in MODELS:
    tokenizer = tokenizer_class.from_pretrained(pretrained_weights)
    model = model_class.from_pretrained(pretrained_weights)
    input_ids = torch.tensor([tokenizer.encode("Here is some text to encode", add_special_tokens=True)])
    with torch.no_grad():
        last_hidden_states = model(input_ids)[0]
`UnboundLocalError: local variable 'tokenizer' referenced before assignmen
```

This happened when the model_class goes to the XLMModel.I do not quite understand why this happen,because this problem only occurs when the model is XLMModel.


Attention mask always returns array of ones for CamembertTokenizer.batch_encode_plus

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

The model i'm using is CamebertTokenizer for the French language

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

- First init the models with the following code:
```python
camembert_tokenizer = CamembertTokenizer.from_pretrained('../models/', cache_dir='./models')
camembert_tf_model = TFCamembertModel.from_pretrained('../models/', output_hidden_states=True,
                                                          cache_dir='./models'
                                                     )
camembert_tf_model.trainable = False
```
- Prepare the input data :
```python
text = ' '.join(['je' for i in range(25)])
texts = [ text, "je suis cool"]
input_ids = camembert_tokenizer.batch_encode_plus(texts,
                                             add_special_tokens=True,
                                             max_length=8,
                                            return_tensors='tf')
print(input_ids)
```
## Expected behavior

What should happen is that the padded tokens should have a mask value of 0 if I've correctly understood the doc. so the output of the snippet should be :
```
{'input_ids': <tf.Tensor: shape=(2, 8), dtype=int32, numpy=
 array([[   5,   50,   50,   50,   50,   50,   50,    6],
        [   5,   50,  146, 4261,    6,    1,    1,    1]], dtype=int32)>,
 'token_type_ids': <tf.Tensor: shape=(2, 8), dtype=int32, numpy=
 array([[0, 0, 0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 1, 1, 1]], dtype=int32)>,
 'attention_mask': <tf.Tensor: shape=(2, 8), dtype=int32, numpy=
 array([[1, 1, 1, 1, 1, 1, 1, 1],
        [1, 1, 1, 1, 1, 0, 0, 0]], dtype=int32)>}
```
Instead, i'm always getting an attention_mask full of ones like this :
```
{'input_ids': <tf.Tensor: shape=(2, 8), dtype=int32, numpy=
 array([[   5,   50,   50,   50,   50,   50,   50,    6],
        [   5,   50,  146, 4261,    6,    1,    1,    1]], dtype=int32)>,
 'token_type_ids': <tf.Tensor: shape=(2, 8), dtype=int32, numpy=
 array([[0, 0, 0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 1, 1, 1]], dtype=int32)>,
 'attention_mask': <tf.Tensor: shape=(2, 8), dtype=int32, numpy=
 array([[1, 1, 1, 1, 1, 1, 1, 1],
        [1, 1, 1, 1, 1, 1, 1, 1]], dtype=int32)>}
```

## Environment info
- `transformers` version: 2.5.1
- Platform: Linux-5.3.0-40-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?): 2.1.0 (False)
- Using GPU in script?: (False)
- Using distributed or parallel set-up in script?: (False)


Update README.md

model card

Related to pretrained model cards

- Clarify that the model is not trained on the evaluation dataset


Update README.md

Related to pretrained model cards

- Fix path of tokenizer
- Clarify that the model is not trained on the evaluation set


TF GPT2 Language model can't be created with from_pretrained() for specific shortcut name

Core: Modeling

Internals of the library; Models.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): TFGPT2LMHeadModel

The colab notebook works for all model sizes except for gpt2-xl, where it throws an error. It looks like it can't download the correct checkpoint from the model name (gpt2-xl)

I tried running the colab notebook with other gpt2-models and they all work.

Stack trace:

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-068b0d38bee3> in <module>()
      1 strategy = tf.distribute.experimental.TPUStrategy(resolver)
      2 with strategy.scope():
----> 3   model = create_model()
      4
      5   loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

2 frames
<ipython-input-7-f6b9ea32b94a> in create_model()
      1 def create_model():
----> 2   return TFGPT2LMHeadModel.from_pretrained('gpt2-xl')

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    401         model(model.dummy_inputs, training=False)  # build the network with dummy inputs
    402
--> 403         assert os.path.isfile(resolved_archive_file), "Error retrieving file {}".format(resolved_archive_file)
    404         # 'by_name' allow us to do transfer learning by skipping/adding layers
    405         # see https://github.com/tensorflow/tensorflow/blob/00fad90125b18b80fe054de1055770cfb8fe4ba3/tensorflow/python/keras/engine/network.py#L1339-L1357

/usr/lib/python3.6/genericpath.py in isfile(path)
     28     """Test whether a path is a regular file"""
     29     try:
---> 30         st = os.stat(path)
     31     except OSError:
     32         return False

TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
```

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
*  my own modified scripts: (give details below)

See colab: https://colab.research.google.com/drive/12gEGdxUjyVLBSUjkjngAWiE_ENIUIV8o

The tasks I am working on is:
* my own task or dataset: (give details below)

Finetuning gpt2-xl on wikitext2

## To reproduce

Run the colab notebook,

## Expected behavior

All gpt2 model sizes work except for gpt2-xl

## Environment info
- `transformers` version: master
- Platform: google colab
- Tensorflow version (GPU?): 2.1 (TPU)


Model card for albert-base-v2-squad2

model card

Related to pretrained model cards

Just creating model card for new community model!


XLM-R Tokenizer now passes common tests + Integration tests

XLM-R Tokenizer had a lot of issues that were not identified as no testing was done on it.

closes #2993
closes #2795
closes #2741
closes #2727
closes #2508

This fixes all the above issues, and works for all official checkpoints as well as other SPM files.

However, there are a few things I dislike about the way things stands, which I'm detailing in the comments below.


[model upload] Support for organizations


Create README.md

Related to pretrained model cards


Error reported when fine tuning on my dataset using ''run_language_modeling.py"

Ex: LM (Finetuning)

Related to language modeling fine-tuning

# 🐛 Bug

## Information

Model I am using (RoBerta):

Language I am using the model on (English):

The problem arises when using:
* [ ] the official example scripts: (give details below)
      Using the script provided in run_language_modeling.py I tried to fine tune the model on my
     own dataset. But it shows a "KeyError 1" when It is about to run the first epoch and first iteration
* [ ] my own modified scripts: (give details below)
 i only modified the script to read in my dataset using the LineByLineTextDataset class to read in my sentence which is in an excel file and convert them to a list like in the class

Please advice, if there a proper format my data-set should be in, before fine tuning it.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)
To Fine tuning the language model of Bert on our my own data.

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?): yes
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes


[fix] Bart CNN Example: model.to(device)


Where is the default download address for pre-trained weight

# ❓ Questions & Help

```
from transformers import DistilBertTokenizer, DistilBertModel

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = DistilBertModel.from_pretrained('distilbert-base-uncased')
```
I can't find the downloaded file.
Thanks for your help


Provide comprehensive guide & best-practices for run_language_modeling.py

Ex: LM (Pretraining)

Related to language modeling pre-training

# 🚀 Feature request

Provide comprehensive guide for running scripts included in the repository, especially `run_language_modeling.py` it's parameters and model configurations.

## Motivation

1. Current version has `argparse` powered help, from which a lot of parameters seem to be either mysterious or have variable runtime behaviour (i.e `tokenizer_name` is sometimes path and the value that user provides is expected to provide different data for different models, ie. for Roberta and BERT). Again, when it comes to `tokenizer_name` - it claims that `If both are None, initialize a new tokenizer.`, which does not work at all, i.e when you use RoBERTa model. It should handle the training of the new tokenizer on provided `train_data` right away.

1. There are bunch of parameters that are critical to run the script at all (!), which are not even mentioned here https://huggingface.co/blog/how-to-train or even in the notebook https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb, i.e:
for Roberta, without `"max_position_embeddings": 514,` in config, the script crashes with:
    ```
    CUDA error: device-side assert triggered
    ```
    I had to dig into github to see some unresolved issues around this case and try out a few solutions before the script finally executed (https://github.com/huggingface/transformers/issues/2877).

1. Models with LM heads will train even though the head output size is different than vocab size of the tokenizer - the script should warn the user or (better) raise an exception in such scenarios.

1. Describe how the input dataset should look like. Is it required to have one sentence per-line or one article per line or maybe one paragraph per line?

1. Using multi-GPU on single machine and parameter `--evaluate_during_training` crashes the script -why? It might be worth an explanation. It's probably also a bug (https://github.com/huggingface/transformers/issues/1801).

1. Those are just from the top of my head - I will update this issue once I come up with more or maybe someone else will also add something to this thread.

Given the number of issues currently open, I suspect that I'm not the only one that struggles with the example script. **The biggest problem here is that running it without proper configuration might really cost a lot, but the script will still execute, yielding garbage model.**

Moreover - by improving the docs and providing best practices guide, you can enable many people with even better toolkit for their research and business.


Add integration tests lm generate torch tf

Add integration tests for all LM models that are able to generate language.

- All integration tests use `do_sample=False` (greedy) generation and verify that TF 2.0 and PT yield the same results.
- Fixed a small bug with TFXLMModelWithLMHead


fix repetition penalty mask in tf

Fixed bug with TF 2.0 `repetition_penalty` when doing generation add make `early_stopping` an argument to the function `generate()`.


I want to import the model path on my owm computer?

hi, I want to import the model path from my own computer, how to write or change the code, can u give me a example?