[pegasus] Faster tokenizer tests

This PR implements #7354

* The suggested `fixtures/test_sentencepiece.model` couldn't be used since it has wrong special token ids: we need
   1. no bos
   2. eos_id is 1
   3. unk_id is 2

   added a script that builds a custom tokenizer model: `test_sentencepiece_no_bos.model`. Had to figure out how to match `"google/pegasus-large"` spm file. See the build script for nuances.
* switched pegasus common tests to use the newly added `test_sentencepiece_no_bos.model` - 2 custom tests still use the large tokenizer - remained untouched

And a few extra tweaks I made while sorting this PR out:

* removed `get_vocab` in `tokenization_pegasus.py` as it's identical to superclass's one
* a few minor prose edits in related files
* expanded `testing_utils.py`' s `get_tests_dir` to accept an optional `append_path` arg to remove clutter from tests. will probably rename it in the future to something else, works for now.

Fixes #7354

@sshleifer, @LysandreJik


fix nn.DataParallel compatibility with PyTorch 1.5

The same type of errors as in https://github.com/huggingface/transformers/pull/4300

# What does this PR do?
DataParallel replicate has a known issue in PyTorch 1.5: https://github.com/pytorch/pytorch/issues/40457

A similar PR proposes a work around by removing the `next(self.parameters().dtype)`: https://github.com/huggingface/transformers/pull/4300/files/7eef4f5a7575e05e822f8ef45d7f473a102671aa

I did the same in LXMERT

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@patrickvonplaten
 @julien-c


[s2s] Switch README urls to cdn

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Update XLM-RoBERTa pretrained model details


Default Model Licenses

Hi, thanks for the great library!

I've been trying to compile a list of licenses for the default models and wanted to share in case others were wondering about it. Here's what I have so far:

*Note: table has been updated based on this discussion.*

Task | Model | License | Model Card w/ License
--- | --- | --- | ---
feature-extraction | distilbert-base-cased | Apache-2.0 | ✓ (added)
sentiment-analysis | distilbert-base-uncased-finetuned-sst-2-english | Apache-2.0 | ✓ (added)
ner | dbmdz/bert-large-cased-finetuned-conll03-english | MIT* (added) |
question-answering | distilbert-base-cased-distilled-squad | Apache-2.0 | ✓ (added)
fill-mask | distilroberta-base | Apache-2.0 | ✓
text-generation | gpt2 | MIT* | ✓
summarization | sshleifer/distilbart-cnn-12-6 | Apache-2.0 | ✓
translation, text2text-generation | t5-base | Apache-2.0 | ✓
zero-shot-classification (PyTorch) | facebook/bart-large-mnli | MIT (added) | ✓ (added)
zero-shot-classification (TensorFlow) | roberta-large-mnli |  MIT | ✓
conversational | microsoft/DialoGPT-medium | MIT | ✓

Notes:

- `distil` models without a model card are listed as Apache 2.0 based on this comment: https://github.com/huggingface/transformers/issues/3357#issuecomment-614856396
- `gpt2` was changed from MIT to a custom license earlier this year: [history](https://github.com/openai/gpt-2/commits/master/LICENSE)
- Other `dbmdz` models use MIT (https://github.com/huggingface/transformers/pull/3492), but didn't find info on `dbmdz/bert-large-cased-finetuned-conll03-english`. If the model was fine-tuned from a pretrained BERT model, I imagine it would need to retain Apache 2.0 in addition to how the final model is licensed.

It'd be nice to get clarification on the two models that are missing licenses (and ideally ensure all default models have a clear license going forward).


Add multi-class processor to apply categorical classification

This PR adds multi-class processor to glue.py to support categorical classification.


Clear up confusing translation pipeline task naming

# 🚀 Feature request
Hello!
I am using the translation pipeline, and I noticed that even though I have to specify the language when I create the pipeline, the passed model overwrites that. So pipeline created as
`nlp = pipeline('translation_en_to_de', 'Helsinki-NLP/opus-mt-en-jap')`
would translate english to japanese, in contrary to the task name. Is this the intended way of translating other languages, will it change in the future?

Would it be possible to just add a single 'translation' task for pipelines, which would then resolve the languages based on the model (which it seems to do anyway now) ?

## Motivation
It would clear up the current confusion, and make the `pipeline` function singature less prone to change.
It could also possibly reduce code duplication in https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines.py
<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## My contribution
I'd love to help with a PR, though I'm confused: The `SUPPORTED_TASKS` dictionary in pipelines.py contains exactly the same entries for each translation pipeline, even the default model is the same, yet the specific pipelines actually translate to different languages 🤔


tokenizer_bert.py not call _clean_text?

for transformers/src/transformers/tokenization_bert.py, there is a function called _clean_text.
But seems this function is not be called at all?

In google bert(https://github.com/google-research/bert/blob/master/tokenization.py) there exists a same function and that function has been called at the beginning of the tokenization.


TF Slow test CI

Tests

Related to tests

I Don't think tf slow tests are run by circleci OR github actions.
Should they be @LysandreJik ?


2 slow TF T5 common tests failing on master

TensorFlow

Anything TensorFlow

These should probably be run on CI somewhere.
Didn't know whether to assign @patrickvonplaten or @jplu.

These fail in both tf 2.2 and tf 2.3.

#### Command
```bash
RUN_SLOW=1 pytest tests/test_modeling_tf_t5.py -k saved
```
#### Traceback
```
================================================================================= FAILURES =================================================================================
__________________________________________________________ TFT5ModelTest.test_saved_model_with_attentions_output ___________________________________________________________
tests/test_modeling_tf_common.py:223: in test_saved_model_with_attentions_output
    self.assertEqual(len(outputs), num_out)
E   AssertionError: 5 != 4
_________________________________________________________ TFT5ModelTest.test_saved_model_with_hidden_states_output _________________________________________________________
tests/test_modeling_tf_common.py:185: in test_saved_model_with_hidden_states_output
    self.assertEqual(len(outputs), num_out)
E   AssertionError: 5 != 4
---------------------------------
```


loss.backward() being called twice in Trainer._training_step()

**Setup**
pytorch: 1.5.1
huggingface transformers: 3.0.2
python: 3.7.6
OS: Pop!_OS 20.04 on VM

**Sample Code**
```python
from transformers import GPT2Tokenizer, GPT2LMHeadModel, TrainingArguments, Trainer
import torch
from torch.utils.data import Dataset
import sys
import pandas as pd

ZERO = sys.float_info.min
ZERO_PT = torch.tensor(ZERO)

class GPT2FinetunedWithNgrams(GPT2LMHeadModel):
    def __init__(self, config):
        super().__init__(config)
        self.tokenizer = GPT2Tokenizer.from_pretrained('gpt2', padding_side='right')
        self.tokenizer.pad_token = self.tokenizer.eos_token

    def eval_sentence(self, sent: str):
        vec = torch.tensor(sentence_vec(sent), dtype=torch.float, requires_grad=True) # remove punct, lower case, split on space, prepend "<s>", postpend "</s>" start and stop tokens. Returns tensor of ints of vocab.
        last_idx = min(max_ngram, len(vec)) #max_ngram is an int
        probs = [max(ZERO_PT, pkatz(vec[0:i])) for i in range(2, last_idx + 1)] #pkatz is katz backoff probability and returns a tensor with grad function set.
        for i in range(1, len(vec) - last_idx + 1):
            j = i + last_idx
            probs.append(max(ZERO_PT, pkatz(vec[i:j])))

        probs = torch.stack(probs)
        log_probs = torch.log(probs)
        log_prob = torch.sum(log_probs)
        len_tensor = torch.tensor(len(vec), dtype=float, requires_grad=True)
        final_prob = torch.true_divide(-log_prob, len_tensor)
        return final_prob

    def sentence_loss(self, sent: str):
        p, l = self.eval_sentence(sent)
        return -p

    def generate_text_while_finetuning(self,
                                       input_ids=None,
                                       past=None,
                                       attention_mask=None,
                                       token_type_ids=None,
                                       position_ids=None,
                                       head_mask=None,
                                       inputs_embeds=None,
                                       labels=None,
                                       use_cache=None,
                                       output_attentions=None,
                                       output_hidden_states=None, ):
        transformer_outputs = self.transformer(
            input_ids,
            past=past,
            attention_mask=attention_mask,
            token_type_ids=token_type_ids,
            position_ids=position_ids,
            head_mask=head_mask,
            inputs_embeds=inputs_embeds,
            use_cache=use_cache,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
        )
        hidden_states = transformer_outputs[0]
        lm_logits = self.lm_head(hidden_states)
        outputs = (lm_logits,) + transformer_outputs[1:]
        return outputs  # (loss), lm_logits, presents, (all hidden_states), (attentions)

    def forward(
            self,
            input_ids=None,
            past=None,
            attention_mask=None,
            token_type_ids=None,
            position_ids=None,
            head_mask=None,
            inputs_embeds=None,
            labels=None,
            use_cache=True,
    ):

        max_length = input_ids.shape[1] + 50
        full_generated_gpt2_ids = self.generate(input_ids=input_ids,
                                                max_length=max_length,
                                                is_finetuning_current_model=True,
                                                attention_mask=attention_mask,
                                                pad_token_id=50256,
                                                do_sample=True,
                                                top_k=50,
                                                top_p=0.95)

        decoded_gen_samples = self.tokenizer.batch_decode(full_generated_gpt2_ids, skip_special_tokens=True)
        tmp_losses = [self.sentence_loss(decoded_sample) for decoded_sample in decoded_gen_samples]
        losses = torch.stack(tmp_losses)
        loss = losses.mean()
        return (loss,)

##The code below is the run script.
class MyDataset(Dataset):
    def __init__(self, csv_file: str):
        self.df = pd.read_csv(csv_file, encoding='ISO-8859-1')

    def __len__(self):
        return len(self.df)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()
        text = self.df.iloc[idx, 1]
        return text

def my_data_collator(dataset_samples_list):
    tokenizer = GPT2Tokenizer.from_pretrained('gpt2', padding_side='right')
    tokenizer.pad_token = tokenizer.eos_token

    encoded_results = tokenizer(dataset_samples_list, padding=True, truncation=True, return_tensors='pt', return_attention_mask=True)

    batch = {}
    batch['input_ids'] = torch.stack([result for result in encoded_results['input_ids']])
    batch['past'] = None
    batch['attention_mask'] = torch.stack([result for result in encoded_results['attention_mask']])
    batch['position_ids'] = None
    batch['head_mask'] = None
    batch['inputs_embeds'] = None
    batch['labels'] = None
    batch['use_cache'] = True
    return batch

dataset_train = MyDataset('/path/to/train_dataset.csv')

training_args = TrainingArguments(
    output_dir='/path/to/out',
    do_train=True,
    per_device_train_batch_size=64,
    logging_dir='/path/to/dir',
    max_steps=300000
)

model = GPT2FinetunedWithNgrams.from_pretrained('gpt2')

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=my_data_collator,
    train_dataset=dataset_train
)
trainer.train()
trainer.save_model('/path/to/model_save_dir')

```

**Issue**
The above code will produce the following error for some training examples:
```python
Traceback (most recent call last):
  File "/home/aclifton/ric-2020/textgen/run_finetune_gpt2.py", line 221, in <module>
    testfinetune()
  File "/home/aclifton/ric-2020/textgen/run_finetune_gpt2.py", line 215, in testfinetune
    trainer.train()
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/transformers/trainer.py", line 499, in train
    tr_loss += self._training_step(model, inputs, optimizer)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/transformers/trainer.py", line 637, in _training_step
    loss.backward()
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/torch/tensor.py", line 198, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/torch/autograd/__init__.py", line 100, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

```

What I'm finding is that in `Trainer._training_step()`, some examples are causing `loss.backward()` to be called twice. So the error makes sense as the first call will compute the graph then clear it and the second call is what throws the error. I'm not sure what would cause this to happen and was wondering if others might have an idea?


[pseudo] Switch URLS to CDN

Switch s3 urls -> CDN urls.
cc @julien-c


[broken] tf generate: use model_kwargs

@patrickvonplaten , I started trying to get tf generation/cache to be consistent with pytorch, but got stuck trying to get T5 working. I figured I would share in case you see an easy fix/want to take over. Otherwise, feel free to ignore :)


[Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies

# What does this PR do?

Both the [SentencePiece](https://github.com/google/sentencepiece) and [Tokenizers](https://github.com/huggingface/tokenizers) libraries can limit the users:
- `sentencepiece` is not available on Conda on every plateform and one of the reason `transformers` is not on Conda
- `tokenizers` cannot be used inside some labs which need to build all from source and don't have a Rust tooling.

This PR aim at making both optional leveraging the addition of SentencePiece algorithms in Tokenizer.

Note: at least one of `sentencepiece` and `tokenizers` will be required to use the SentencePiece tokenizers. `tokenizers` is also required to use the Fast tokenizers.

Main changes in the library organization:
- fast tokenizers are now separated in `tokenization_XXX_fast.py` files
- a `convert_slow_tokenizer.py` file host conversion methods for a slow to a fast tokenizer but a direct path from a `tokenizers` serialization file is favored when such a file is available.
- the test suite for slow and fast tokenizers are now gathered in a single test suite.

Main new requirements for the tokenizers to pass the new test suite:
- at least one default vocabulary checkpoint (and max length) should be provided, it is used for the deep tests
- the fast tokenizer should have an explicit `tokenizer_file` keyword argument with a default to `None` (we check that to be sure all the fast tokenizer can accept the new serialization format.

To-add:
- when the documentation for `tokenizers` is ready: add a lot of link on how to build and add a fast tokenizer
- add a detailed explanation on how to add a fast tokenizer in the library

This PR also:
- add a `__repr__` for the tokenizers (finally...)
- add a `name_or_path` attribute to the models and tokenizers giving the shortcut name or the path of the pretrained checkpoint used for instantiation
- update the fast tokenizer to use (when possible) the new serialization format of the `tokenizers` library, falling back on the old diverse set of saving format if not available.
- clean up the tests for the fast tokenizers to bring them in the common tokenizer tests

Fixes #7402 #5100 (and maybe others)

## Before submitting
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


Green tests: update torch-hub test dependencies (add protobuf and pin tokenizer 0.9.0-RC2)

# What does this PR do?

Update the torch-hub CI test dependencies to add protobuf and pin tokenizer on 0.9.0-rc2 until final release.

## Who can review?

@sgugger @n1t0


SqueezBert link gives a 404 error

The main Readme.md file (https://github.com/huggingface/transformers/blob/master/README.md), the SqueezeBert link (https://huggingface.co/transformers/model_doc/squeezebert.html) gives a "404 - Not Found" Error.


T5 Beam search num_beans always equals 1

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Debian 10.6
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6
- Tensorflow version (GPU?): N.A.
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.  -->
TextGeneration: @TevenLeScao
 T5: @patrickvonplaten

## To reproduce

Steps to reproduce the behavior:

1. load T5 model and tokenizer
```
from transformers import T5ForConditionalGeneration, T5Tokenizer

# initialize the model architecture and weights
model = T5ForConditionalGeneration.from_pretrained("t5-base")

tokenizer = T5Tokenizer.from_pretrained("t5-base")
```
2. prepare input for summarization. i guess the error persists for any given generation task but did not try.
```
article = """article etc etc"""
inputs = tokenizer.encode("summarize: " + article,
                          return_tensors = "pt",
                          max_length = 512, truncation = True)
```
3. attempt beam search
```
model.config.update({"num_beans": 4})
print(model.config.num_beans)
# output is 4 as expected

outputs = model.generate(inputs,
                         max_length = 200,
                         min_length = 100,
                         length_penalty = 5,
                         num_return_sequences = 2,
                         early_stopping = True)
```
or
```
outputs = model.generate(inputs,
                         max_length = 200,
                         min_length = 100,
                         length_penalty = 5,
                         num_beams = 4,
                         num_return_sequences = 2,
                         early_stopping = True)
```

error:

> AssertionError: Greedy decoding will always produce the same output for num_beams == 1 and num_return_sequences > 1. Please set num_return_sequences = 1

as if num_beans == 1, but we set num_beans to 4.

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
the generate function should execute beam search with 4 beams without errors


Eval_loss in prediction is very high : transformers/examples/token-classification/run_ner.py

## Environment info
- `transformers` version: 3.3.1
- Platform: Linux-4.9.0-12-amd64-x86_64-with-debian-9.12
- Python version: 3.7.6
- PyTorch version (GPU?): 1.4.0
- Tensorflow version (GPU?): not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
 examples/token-classification: @stefan-it

## Information

 I am using NER "Emerging and Rare Entities task: WNUT’17 (English NER) dataset"
I am executing the steps as prescribed in https://github.com/huggingface/transformers/tree/08ba4b4902df5a18f5ad41d9490c50fe0a4c970f/examples/token-classification

The problem arises when using prediction:
* [ ] the official example script: wnut_17.json
{
"data_dir": "/home/priya/data_wnut_17",
"labels": "/home/priya/data_wnut_17/labels.txt",
"model_name_or_path": "bert-large-cased",
"output_dir": "wnut-17-model-1",
"max_seq_length": 128,
"num_train_epochs": 3,
"per_device_train_batch_size": 16,
"save_steps": 425,
"seed": 1,
"do_train": true,
"do_eval": true,
"do_predict": true,
"fp16": false
}
* [ ] my own modified scripts: wnut_17_mod.json
{
"data_dir": "/home/priya/data_wnut_17",
"labels": "/home/priya/data_wnut_17/labels.txt",
"model_name_or_path": "bert-large-cased",
"output_dir": "wnut-17-model-1",
"max_seq_length": 128,
"num_train_epochs": 3,
"per_device_train_batch_size": 16,
"save_steps": 425,
"seed": 1,
"do_train": **false,**
"do_eval": **false**,
"do_predict": true,
"fp16": false,
"overwrite_output_dir":false
}

The tasks I am working on is:
* [ ] re-run WNUT’17 dataset.

My end-goal is to identify abbreviation and explanation from sentences (labels B-abbr, I-abbr, B-expl, I-expl and O). For the example sentence

> Here GAAP stands for Generally accepted accounting principles

we should get token classified as

> Here O
> GAAP B-abbr
> stands O
> for O
> Generally B-expl
> accepted I-expl
> accounting I-expl
> principles I-expl

## To reproduce

Steps to reproduce the behavior:

1.python run_ner.py wnut_17.json
Prediction: 100%|████████████████████████████████████████████████████████| 162/162 [01:08<00:00,  2.38it/s]
10/06/2020 07:21:15 - INFO - __main__ -     eval_loss = 0.2851179020827574
10/06/2020 07:21:15 - INFO - __main__ -     eval_accuracy_score = 0.9511413182867402
10/06/2020 07:21:15 - INFO - __main__ -     eval_precision = 0.5997392438070405
10/06/2020 07:21:15 - INFO - __main__ -     eval_recall = 0.4263206672845227
10/06/2020 07:21:15 - INFO - __main__ -     eval_f1 = 0.49837486457204777

2.python run_ner.py wnut_17_mod.json
Prediction: 100%|████████████████████████████████████████████████████████| 162/162 [01:08<00:00,  2.38it/s]
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py:1175: FutureWarning: This method is deprecated, use `Trainer.is_world_process_zero()` instead.
  warnings.warn("This method is deprecated, use `Trainer.is_world_process_zero()` instead.", FutureWarning)
10/06/2020 08:30:41 - INFO - __main__ -     eval_loss = 2.827890293463151
10/06/2020 08:30:41 - INFO - __main__ -     eval_accuracy_score = 0.016072497221509788
10/06/2020 08:30:41 - INFO - __main__ -     eval_precision = 0.0065180614986565825
10/06/2020 08:30:41 - INFO - __main__ -     eval_recall = 0.12140871177015755
10/06/2020 08:30:41 - INFO - __main__ -     eval_f1 = 0.012371912924399112

## Expected behavior
I am seeing a 10-fold increase in eval-loss from 0.28 to 2.8.
Other than the changes in  wnut_17_mod.json , I have done no other changes. Please advice how to achieve the published eval_loss and performance.

Thanks,


# 🚀 Feature request

output probabilities of generated sequences in generate function (generation utils)

thank you so much! :)


[pseudolabels] cleanup markdown table


Fix 3 failing slow bart/blender tests

3 of these were simple fixes.
+ 1 typo in blenderbot
+ 2 BART failures changed by new `assert_tensors_close` helper fn checking shapes more aggressively. Output shapes have not changed.

The fourth failure is a bit harder to verify
Blenderbot 3b was OOMing
before fix: 11.4 GB
After: 6.4 GB
Why:  going to fp16 before going to cuda


Fix Failing Slow tests

```
FAILED tests/test_modeling_bart.py::BartHeadTests::test_tokenization - Assert...
FAILED tests/test_modeling_bart.py::BartModelIntegrationTests::test_mnli_inference
FAILED tests/test_modeling_blenderbot.py::Blenderbot3BIntegrationTests::test_generation_from_short_input_same_as_parlai_3B
FAILED tests/test_modeling_blenderbot.py::Blenderbot90MIntegrationTests::test_90_generation_from_long_input
```


Import integration libraries first

# What does this PR do?

This PR restores the order of importing 3rd-party integrations before other ML frameworks, and before any other transformer modules.

## Before PR:

* importing comet_ml later causes an error

## After PR:

* using comet_ml functionality is restored


setup of Trainer class for distributed trainning

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

I am running on the sample code and got confused on how to setup a distributed trainning, below are the code I used
from pathlib import Path

`from tokenizers import ByteLevelBPETokenizer

from tokenizers.implementations import ByteLevelBPETokenizer
from tokenizers.processors import BertProcessing

tokenizer = ByteLevelBPETokenizer(
    "./EsperBERTo/vocab.json",
    "./EsperBERTo/merges.txt",
)
tokenizer.enable_truncation(max_length=512)

import torch
torch.cuda.is_available()

from transformers import RobertaConfig

config = RobertaConfig(
    vocab_size=52_000,
    max_position_embeddings=514,
    num_attention_heads=12,
    num_hidden_layers=6,
    type_vocab_size=1,
)

from transformers import RobertaTokenizerFast

tokenizer = RobertaTokenizerFast.from_pretrained("./EsperBERTo", max_len=512)

from transformers import RobertaForMaskedLM

model = RobertaForMaskedLM(config=config)

from transformers import LineByLineTextDataset

dataset = LineByLineTextDataset(
    tokenizer=tokenizer,
    file_path="./oscar.eo.txt",
    block_size=128,
)

from transformers import DataCollatorForLanguageModeling

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer, mlm=True, mlm_probability=0.15
)

from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir="./EsperBERTo",
    overwrite_output_dir=True,
    num_train_epochs=1,
    per_device_train_batch_size=64,
    save_steps=10_000,
    save_total_limit=2,
    fp16=True,
    local_rank=3,
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=dataset,
    prediction_loss_only=True,
)

trainer.train()

trainer.save_model("./EsperBERTo")`

I want to know how to set the local_rank parameter in trainer class and what comman should I use.

python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=3 --master_addr="192.168.1.1" --master_port=1234 starttrans2.py

is above a correct way to run this script if I want to run on a single machine with 4 gpus?

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


does tokenizer support emoji?

Hi, I have the code below and it always encodes emoji as "unk". Can someone tell me what I should do? Thanks

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', do_lower_case=True)
s =" 😃 hello how are you"
tokenizer.tokenize(s)
['[UNK]', 'hello', 'how', 'are', 'you']


Project: Gather summarization datasets and try to replicate pegasus results on them

Help wanted

Extra attention is needed, help appreciated

Dear @stas00 and whoever else is willing to help!

So far I have only checked pegasus' rouge scores on 2/12 datasets for which we have checkpoints.
For the other 10 datasets I either haven't tried or have tried briefly and gotten stuck.

The full scope of the project is that:
for each dataset:

1) There is an automated way to download the data, either from S3 or source. (To the extent possible, much of the logic in this script should eventually live in the `datasets` package).
2) we know our pegasus implementation's rouge score
2b) if our score is very different than the authors', we know whether that difference is due to data preprocessing, and if it is, we can preprocess the dataset similarly to the pegasus authors.
3) Our rouge score is within 0.3 Rouge2 of the reported. (Authors) column below.

### Steps

#### Getting Data

By far the most difficult part of each project is getting the dataset. And giving up quickly if you can't and writing a github issue somewhere.
I tried 1 approach to getting data: [this script](https://gist.github.com/sshleifer/c4aed7bf4418b50caee731e94be05d9f)
It worked for gigaword, I just haven't done the evaluation, but it failed for `aeslc` and then I gave up.

Another complementary approach would be to try to directly use the [pegasus dataset code](https://github.com/google-research/pegasus/blob/master/pegasus/data/public_supervised_datasets.py)

This will likely push preprocessing issues towards the back of the project. (when we try to send PRs to the datasets repo), but might be better than using my script.

#### After you get data

When you have gotten a dataset you can sanity check

```bash
python -m torch.distributed.launch --nproc_per_node=2  run_distributed_eval.py \
	--model_name google/pegasus-large $\  # see note 1
    --save_dir xsum_generations \
    --data_dir xsum \
	--prefix test \
	--n_obs 100 \
```
Note 1: you can just keep running pegasus-large and expect a high single digits or better rouge2 score,to avoid downloading all the checkpoints or, you can change this to the relevant checkpoint.
Note 2: I am happy to run all the evals on newer hardware, very easy for me.
Note 3: We can do data sharing by getting you aws creds, or some other solution. Key is that I can download from command line, e.g. Google Drive + gdown.

### Misc thoughts:

+ arxiv and pubmed are listed under `scientific_papers` in the datasets package.
+ This is really 10 projects (1 each dataset, 2 of which I've started). If I were you I would ignore the started 2 and start on a few other ones.
+ If a dataset only has train/test or train/val or some other splits, see how the pegasus authors did the split.
+ Partial credit is valuable!
+ this could easily have been an issue for the datasets project rather than the transformers project.
+ There is no reason to merge PRs quickly for this project, but eventually we want a (much better) download_summ_dataset.py script or instructions for using other libs to accomplish the same outcome.
+ Will be good for both of us to learn the datasets internals.
+ Raw Billsum has multiple line articles, which breaks everything :( , (we could try to support raw nlp datasets in our `DataLoader`)

Here is a copy of the table we are trying to fill out in #6844 : (I made a new issue to avoid spamming that one)

| dataset | Authors| This Repo|
| ---- | ----|----|
| xsum | 47.60/24.83/39.64|  46.87/24.46/39.15|
| cnn_dailymail | 44.16/21.56/41.30| see 1|
| newsroom | 45.07/33.39/41.28 | have `.tar` file|
| multi_news | 47.65/18.75/24.95|
| gigaword | 39.65/20.47/36.76| 39.79/20.56/36.80|
| wikihow | 46.39/22.12/38.41 *| Asked Authors |
| reddit_tifu | 27.99/9.81/22.94|32.75/11.68/24.97|
| big_patent |52.29/33.08/41.66 *| |
| arxiv | 44.21/16.95/25.67| |
| pubmed | 45.97/20.15/28.25| |
| aeslc | 37.68/21.25/36.51|37.1/21.4/35.94|
| billsum | 59.67/41.58/47.59|54.99/37.43/43.07|

Originally from mixed & stochastic column of this [table](https://github.com/google-research/pegasus#results-update)

This was really long, and probably disorganized, so feel free to ask clarifying questions here or on slack!
cc @stas00

1) I got similar scores on cnn-dailymail by finetuning the authors' model on our dataset for a bit.
2) reddit_tifu: added `--min_length 32`


Openai gpt for classification

model card

Related to pretrained model cards

# What does this PR do?
Adds  sequence classification architecture for GPT-1,
Strongly based on modifications made in #7501

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7623 (issue) (Partially)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [✓] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [✓] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [✓] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [✓] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Fix integration tests of DeBERTa

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


NER pipeline documentation example failing

Hello,

I am running the code through your documentation for named entity recognition and am trying to save this "ner" model locally:

https://huggingface.co/transformers/usage.html#named-entity-recognition

```
nlp = pipeline("ner")

sequence = "Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, therefore very" \
           "close to the Manhattan Bridge which is visible from the window."

nlp.save_pretrained("path to folder")
```

When going to load this model up and make predictions, I am getting the error: "IndexError: list index out of range" pointing the very last line below:

```
model = AutoModelForTokenClassification.from_pretrained("path to folder")
tokenizer = AutoTokenizer.from_pretrained("path to folder")

label_list = [
    "O",       # Outside of a named entity
    "B-MISC",  # Beginning of a miscellaneous entity right after another miscellaneous entity
    "I-MISC",  # Miscellaneous entity
    "B-PER",   # Beginning of a person's name right after another person's name
    "I-PER",   # Person's name
    "B-ORG",   # Beginning of an organisation right after another organisation
    "I-ORG",   # Organisation
    "B-LOC",   # Beginning of a location right after another location
    "I-LOC"    # Location
]

sequence = "Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, therefore very" \
           "close to the Manhattan Bridge."

# Bit of a hack to get the tokens with the special tokens
tokens = tokenizer.tokenize(tokenizer.decode(tokenizer.encode(sequence)))
inputs = tokenizer.encode(sequence, return_tensors="pt")

outputs = model(inputs)[0]
predictions = torch.argmax(outputs, dim=2)

print([(token, label_list[prediction]) for token, prediction in zip(tokens, predictions[0].tolist())])
```

I would like to get the entity for each token. I believe that the error is with "label_list" portion of the code, and I ran the following this has the token along with the prediction represented as integers:

`print([(token,prediction) for token, prediction in zip(tokens, predictions[0].tolist())])`

I am unable to recreate the output shown on the website due to that error. Any help would be much appreciated.


quick question about `BertForMaskedLM`

Hello,
I have a question about the example code that can be found in the documentation for `BertForMaskedLM` model. The example from the documentation is shown below:
```python
from transformers import BertTokenizer, BertForMaskedLM
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForMaskedLM.from_pretrained('bert-base-uncased', return_dict=True)
input_ids = tokenizer("Hello, my dog is cute", return_tensors="pt")["input_ids"]
outputs = model(input_ids, labels=input_ids)
loss = outputs.loss
prediction_logits = outputs.logits
```
In this example, from the input string "Hello, my dog is cute", I don't see any `mask_token` in it. Also, the example code simply passes the label as `label=input_ids`.

So in this particular example, how exactly does the `BertForMaskedLM` model calculate the masked-LM loss (since the `mask_token` is not specified in the input string)? when I simply pass `labels = input_ids`, does `BertForMaskedLM` model automatically place the `mask_token` over the the first token of the input string (or something similar to this)?

I don't think that the code provided in the documentation is wrong, because when I run the code on my machine, it runs smoothly without generating any error.

Thank you,


Fix RobertaForCausalLM docs

`RobertaLMHeadModel` does not exist, and we can't pass the `return_dict` value if a config has already been passed during instantiation.

closes #7635


[s2s] configure lr_scheduler from command line

# What does this PR do?

This PR adds the ability to configure `lr_scheduler` from command line for `Seq2SeqTrainer`.

Fixes #7543

@sshleifer


Create README.md for IsRoBERTa language model

Related to pretrained model cards

# What does this PR do?

Adds a model card Readme for the IsRoBERTa language model


[s2s] release pseudolabel links and instructions

+ Release a bunch of summarization and translation pseudolabels with reasonably nice documentation.
+ Allow `make_student(teacher, 'student_000_baseline', 12, 3, d_layers_to_copy=[0,0,0])` for baseline purposes.


error AttributeError: 'tuple' object has no attribute 'logits'

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

AttributeError                            Traceback (most recent call last)
<ipython-input-4-594fed3b7299> in <module>()
      6 input = tokenizer.encode(sequence, return_tensors="pt")
      7 mask_token_index = torch.where(input == tokenizer.mask_token_id)[1]
----> 8 token_logits = model(input).logits
      9 mask_token_logits = token_logits[0, mask_token_index, :]
     10 top_5_tokens = torch.topk(mask_token_logits, 5, dim=1).indices[0].tolist()

AttributeError: 'tuple' object has no attribute 'logits'

- `transformers` version:
Successfully installed sacremoses-0.0.43 sentencepiece-0.1.91 tokenizers-0.8.1rc2 transformers-3.3.1
- Platform:
Masked Language Modeling - Colab PyTorch
- Python version:
Python 3.6.9
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [V ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Summary of the tasks Open Colab Pytorch
2. Masked Language Modeling example
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


ValueError("The training dataset must have an asserted cardinality") when running run_tf_text_classification.py

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1 (installed from master)
- Platform: Linux-4.15.0-118-generic-x86_64-with-debian-stretch-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@jplu

## Information

Model I am using (Bert, XLNet ...): Bert (bert-base-uncased)

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SST-2
* [x] my own task or dataset: (give details below)

This same problem happened to my custom dataset, as I described here in #7535 , and also using SST-2 from GLUE (which I did to confirm the error). The following steps are using SST-2 with bert-base-uncased.

## To reproduce

Steps to reproduce the behavior:

1. Created a new conda environment using conda env -n transformers python=3.7
2. Cloned transformers master, `cd` into it and installed using pip install --editable .  -r examples/requirements.txt
3. Installed tensorflow with `pip install tensorflow`
4. Updated datasets to version 1.1.1, as needed according to issue #7535
5. Ran `run_tf_text_classification.py` with the following parameters:

```
--train_file <DATASET_PATH>/train.csv \
--dev_file <DATASET_PATH>/dev.csv \
--test_file <DATASET_PATH>/dev.csv \
--label_column_id 1 \
--model_name_or_path bert-base-uncased \
--output_dir <OUTPUT_PATH> \
--num_train_epochs 4 \
--per_device_train_batch_size 4 \
--per_device_eval_batch_size 4 \
--do_train \
--do_eval \
--do_predict \
--logging_steps 1000 \
--evaluate_during_training \
--save_steps 1000 \
--overwrite_output_dir \
--overwrite_cache
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

Here is the stack trace:

```
10/07/2020 09:48:49 - INFO - __main__ -   Training/evaluation parameters TFTrainingArguments(output_dir='/media/discoD/models/datalawyer/pedidos/transformers_tf', overwrite_output_dir=True, do_train=True, do_eval=True, do_predict=True, evaluate_during_training=True, evaluation_strategy=<EvaluationStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=1, per_device_eval_batch_size=1, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=1, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=4.0, max_steps=-1, warmup_steps=0, logging_dir='runs/Oct07_09-48-45_user-XPS-8700', logging_first_step=False, logging_steps=10000, save_steps=10000, save_total_limit=None, no_cuda=False, seed=42, fp16=False, fp16_opt_level='O1', local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=10000, dataloader_num_workers=0, past_index=-1, run_name='/media/discoD/models/datalawyer/pedidos/transformers_tf', disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model=None, greater_is_better=False, tpu_name=None, xla=False)
10/07/2020 09:48:52 - INFO - filelock -   Lock 140079222710992 acquired on /home/user/.cache/huggingface/datasets/c19c3494c195b40ef4234cb533a8f3ce0bca75ffcf602cc246c390073e633c46.1d5301eeb143a6a4f6f3a2bf726921db0de85048303426a3810f96d735d50d8a.py.lock
10/07/2020 09:48:52 - INFO - filelock -   Lock 140079222710992 released on /home/user/.cache/huggingface/datasets/c19c3494c195b40ef4234cb533a8f3ce0bca75ffcf602cc246c390073e633c46.1d5301eeb143a6a4f6f3a2bf726921db0de85048303426a3810f96d735d50d8a.py.lock
Using custom data configuration default
10/07/2020 09:48:52 - INFO - filelock -   Lock 140084305595600 acquired on /home/user/.cache/huggingface/datasets/_home_user_.cache_huggingface_datasets_csv_default-477ee137eed7e5ae_0.0.0_49187751790fa4d820300fd4d0707896e5b941f1a9c644652645b866716a4ac4.lock
10/07/2020 09:48:52 - INFO - filelock -   Lock 140084305595600 released on /home/user/.cache/huggingface/datasets/_home_user_.cache_huggingface_datasets_csv_default-477ee137eed7e5ae_0.0.0_49187751790fa4d820300fd4d0707896e5b941f1a9c644652645b866716a4ac4.lock
10/07/2020 09:48:52 - INFO - filelock -   Lock 140080785346896 acquired on /home/user/.cache/huggingface/datasets/_home_user_.cache_huggingface_datasets_csv_default-477ee137eed7e5ae_0.0.0_49187751790fa4d820300fd4d0707896e5b941f1a9c644652645b866716a4ac4.lock
Reusing dataset csv (/home/user/.cache/huggingface/datasets/csv/default-477ee137eed7e5ae/0.0.0/49187751790fa4d820300fd4d0707896e5b941f1a9c644652645b866716a4ac4)
10/07/2020 09:48:52 - INFO - filelock -   Lock 140080785346896 released on /home/user/.cache/huggingface/datasets/_home_user_.cache_huggingface_datasets_csv_default-477ee137eed7e5ae_0.0.0_49187751790fa4d820300fd4d0707896e5b941f1a9c644652645b866716a4ac4.lock
100%|██████████| 68/68 [01:20<00:00,  1.18s/ba]
100%|██████████| 1/1 [00:01<00:00,  1.71s/ba]
100%|██████████| 1/1 [00:01<00:00,  1.44s/ba]
10/07/2020 09:50:23 - INFO - filelock -   Lock 140078150630032 acquired on /home/user/.cache/torch/transformers/336363d3718f8cc6432db4a768a053f96a9eae064c8c96aff2bc69fe73929770.4733ec82e81d40e9cf5fd04556267d8958fb150e9339390fc64206b7e5a79c83.h5.lock
Downloading: 100%|██████████| 536M/536M [04:08<00:00, 2.16MB/s]
10/07/2020 09:54:32 - INFO - filelock -   Lock 140078150630032 released on /home/user/.cache/torch/transformers/336363d3718f8cc6432db4a768a053f96a9eae064c8c96aff2bc69fe73929770.4733ec82e81d40e9cf5fd04556267d8958fb150e9339390fc64206b7e5a79c83.h5.lock
2020-10-07 09:54:46.214922: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 93763584 exceeds 10% of free system memory.
Some weights of the model checkpoint at bert-base-uncased were not used when initializing TFBertForSequenceClassification: ['nsp___cls', 'mlm___cls']
- This IS expected if you are initializing TFBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing TFBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of TFBertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['dropout_37', 'classifier']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
  File "/media/discoD/pycharm-community-2019.2/plugins/python-ce/helpers/pydev/pydevd.py", line 1448, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/media/discoD/pycharm-community-2019.2/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/media/discoD/repositorios/transformers_pedro/examples/text-classification/run_tf_text_classification.py", line 283, in <module>
    main()
  File "/media/discoD/repositorios/transformers_pedro/examples/text-classification/run_tf_text_classification.py", line 258, in main
    trainer.train()
  File "/media/discoD/repositorios/transformers_pedro/src/transformers/trainer_tf.py", line 474, in train
    train_ds = self.get_train_tfdataset()
  File "/media/discoD/repositorios/transformers_pedro/src/transformers/trainer_tf.py", line 140, in get_train_tfdataset
    raise ValueError("The training dataset must have an asserted cardinality")
ValueError: The training dataset must have an asserted cardinality
```

## Expected behavior

Should be able to run the text-classification example as described in [https://github.com/huggingface/transformers/tree/master/examples/text-classification#run-generic-text-classification-script-in-tensorflow](https://github.com/huggingface/transformers/tree/master/examples/text-classification#run-generic-text-classification-script-in-tensorflow)

## An additional info: For my own data, using our bert-portuguese model, we don't have a model based on tensorflow available. So I had to force `from_pt` in the code below to be True, otherwise I would get a different error. The [script which converts pytorch to tensorflow](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_bert_pytorch_checkpoint_to_original_tf.py) doesn't work with TF 2.0.

```
    with training_args.strategy.scope():
        model = TFAutoModelForSequenceClassification.from_pretrained(
            model_args.model_name_or_path,
            from_pt=bool(".bin" in model_args.model_name_or_path),
            config=config,
            cache_dir=model_args.cache_dir,
        )
```


Model templates

This PR adds a `cookiecutter`-based utility to generate configuration/modeling/tokenization files, the test suites and the statements across the library necessary for adding a new model.

This PR's goal is to make adding a new model way simpler, by having a simple CLI request information, generate files, that will then need to be edited to implement the changes relative to BERT. The test suites are implemented and run.

Left to do:
- [x] TensorFlow files
- [x] Tokenizer files
- [x] Remove the pooler from the base model
- [x] Ensure the documentation has the right format + .rst file
- [x] Clean-up/refactor the `add_new_model.py` file
- [x] Clarify and add comments to the "dark arts" parts of this PR, such as the `to_replace` file.
- [x] Add encoder-decoder models:
  - [x] Modeling PT file
  - [x] Configuration file
  - [x] Testing the modeling PT file
  - [x] Modeling TF file
  - [x] Testing the modeling TF file
  - [x] Update the RST with the appropriate files
  - [x] Add to all auto + init files
- [x] Run the CI on generated files
- [x] Update the LysBERT proposal to something better
- [x] Do a checklist of things left to do after running the script

Possible improvements:
- [ ] Ask the user whether they want to support `token_type_ids`

## For reviewers

If you review this PR, the simplest would be to review the `add_new_model.py` file, and to generate model files using the utility:
```
transformers-cli add_new_model
```
And review the generated files.
Reviewing the current files with `{{cookiecutter.lowercase_modelname}}` doesn't seem reviewer-friendly to me.


Hi all, I was just trying to run a text generation script for low-resource languages, and therefore experimented with XLM-R and initially with Roberta, using the documentation for RobertaForCausalLM:
here: https://huggingface.co/transformers/model_doc/roberta.html#robertaforcausallm

I am running into the import error shown in the title. See code snippet and error message below. I also experimented with different Tensorflow and transformers versions to no avail. I suspect that the model classes have changed (or the documentation may not be up to date with the current version). I also tried importing RobertaForCausalLM but it returned the same error.

## Environment info
- `transformers` version: 3.1.0
- Platform: Linux-4.15.0-112-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- Tensorflow version: 2.3.1

### Who can help
@LysandreJik , @TevenLeScao

Model I am using (**Roberta**, **XLM-Roberta**):

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce
Steps to reproduce the behavior:

1. Run:
```
from transformers import RobertaTokenizer, RobertaLMHeadModel, RobertaConfig
import torch
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
config = RobertaConfig.from_pretrained("roberta-base")
config.is_decoder = True
model = RobertaLMHeadModel.from_pretrained('roberta-base', config=config, return_dict=True)
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)
prediction_logits = outputs.logits
```

Error: `ImportError: cannot import name 'RobertaLMHeadModel'`

If this script runs succesfully, I 'd like to re-run it for XMLRoberta (changing the imports and model names of course).
Many thanks!


How to get cross attention for bert when config.add_cross_attention is True

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
As far as I can tell, when using BERT with cross attention and output_attention is True, the returned attention only contains self attention (i.e. a tuple of length num_hidden_layers with size (batch_size, num_head, seq_length, seq_length)). How can I get the cross attention weights in that case?

After digging a little bit, I saw that ModelOutput (and child classes) do not include cross attention as potential outputs. The cross attention is returned by [BertLayer](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L420) (at index 2) but then ignored in [BertEncoder](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L486). A quick look at the outputs for Encoder-Decoder models shows the same issue. Would it be possible to include cross attention in model outputs? And if yes, how can I help doing so?

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to similar question on the forum/Stack Overflow**: https://discuss.huggingface.co/t/how-to-get-cross-attention-values-of-t5/970


Unique names for dataset cache for each tokenizer

# 🚀 Feature request

Currently in the examples, dataset caches are named for the family of tokenizer used. For example, 'cached_train_BertTokenizer_128'. This may lead to unexpected behavior when running multiple models/tokenizers within the same model type, but different variations on the model/tokenizer.

In this colab notebook, NER training is used run using scibert and then bert-base-cased. Even though the old data files are removed, the code still uses the old cache, resulting in an indexing error due to the mismatched token indices.

https://colab.research.google.com/drive/1q4uBFm81yBWVNzG3Si2ByBh1nw8fk-Q5?usp=sharing

In this colab notebook, NER training is run on scibert-cased and then scibert-uncased. In this notebook, no explicit error occurs since there isn't an indexing error, but it seems that the wrong dataset is being used. In the output of scibert-uncased, there are many warnings of unpredicted tokens, and a lower than expected score. Both of these do not occur if scibert-cased is not run before scibert-cased

https://colab.research.google.com/drive/1pnpWfRqX4nknc0RRe9A2CArbVok3NbhC?usp=sharing

## Motivation

Prevent unexpected behavior when testing on multiple variations on the same transformer architecture.


Is there a fine-tuning script for DPR?

It would be nice to have a fine-tuning script for DPR.


Add GPT2 to sequence classification auto model

Add `GPT2ForSequenceClassification` to the `AutoModelForSequenceClassification` auto model.

closes #7493.


Update model card - Fix arxiv link

Related to pretrained model cards

Minor changes: Add arxiv link + Layout improvement + fix typos

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


The newly added config decoder_start_token_id for bart-base model is wrong?

The new config file for `bart-base` has been updated on October 5.th.  The new config file looks like the following:
```
{
  ...
  "bos_token_id": 0,
  ...
  "decoder_start_token_id": 2,
  ...
  "eos_token_id": 2,
  ...
}
```

the `decoder_start_token_id` was added newly, it wasn't there before. But as far I understand, the `decoder_start_token_id` should be `bos_token_id` as default . The newly added config-line changed the behavior for the `generate` function.


Added sampler'set_epoch when use distributed training

`run_squad.py` file is independent of `Trainer Class`(https://github.com/huggingface/transformers/issues/4398). Therefore, there is no method related to `set_epoch` in distributed training.


Unable to pass encoder_outputs to generate calls

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: Github Main branch

### Who can help
 TextGeneration: @TevenLeScao

## Information

Model I am using (Bert, XLNet ...): T5

I'm unable to pass preconputed encoder_outputs to the `.generate()` method.

I've tried defining:

```python
model_kwargs = {"encoder_outputs": encoder_outputs}
output = model.generate(model_kwargs)
```
But I've noticed some validation errors for `input_ids`. Even if I replace input_ids with dummy values, I've noticed the model_kwargs is always replaced here:

https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L448

I think it can be fixed (without optimization) by just replacing:

```python
model_kwargs["encoder_outputs"] = encoder_outputs
```

with:

```python
model_kwargs.setdefault("encoder_outputs", encoder_outputs)
```

If you agree I can try to open a PR to fix this.

Best,


Create README.md

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Free Inference API Not Accessible

Hi, I attempted to use the free version of the Model Hub's Inference API. However, it is not working for me anymore:

![image](https://user-images.githubusercontent.com/7613470/95273777-e7b81800-0811-11eb-8090-800188be768f.png)

I do have an account and I am signed in when I get the above message. I also have done my email verification as well. Also, when I try to send a POST request using curl with my API token, I get a 503 error.

I was able to successfully use the free version on October 4th, 2020 and so I was wondering, is this a bug or is the free version no longer available?


Implement PyTorch and/or TensorFlow sequence classification architectures for causal language models

# 🚀 Feature request

The architecture `GPT2ForSequenceClassification` was added in #7501 in PyTorch. It would be great to have it in TensorFlow (cf. issues #7622), but it would also be great to have it for other causal models: ~OpenAI GPT~, ~CTRL~ (PR opened @elk-cloner), ~TransfoXL~ (PR opened @spatil6)

Below is a list of items to follow to make sure the integration of such an architecture is complete:

- Implement `XXXForSequenceClassification` in `modeling_xxx.py` or `TFXXXForSequenceClassification` in `modeling_tf_xxx.py
- Test that architecture in `tests/test_modeling_xxx.py` or `tests/test_modeling_tf_xxx.py`
- Add that architecture to `__init__.py` and ` docs/source/model_doc/xxx.rst`.

Taking a look at the code changes in #7501 would be a good start.

A very good first issue to get acquainted with the library and its architectures!


Implement a TF2 version of `GPT2ForSequenceClassification`

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

The architecture `GPT2ForSequenceClassification` was added in #7501 in PyTorch. It would be great to have it in TensorFlow as well.

Below is a list of items to follow to make sure the integration is complete:

- Implement `TFGPT2ForSequenceClassification` in `modeling_tf_gpt2.py`
- Test that architecture in `tests/test_modeling_tf_gpt2.py`
- Add that architecture to `__init__.py` and ` docs/source/model_doc/gpt2.rst`.

Taking a look at the code changes in #7501 would be a good start, as this PR would essentially be a TF2 copy of it.

A very good first issue to get acquainted with the library and its architectures!


[No merge] TF integration testing

Adds integrationt tests for BERT, ELECTRA and Longformer to ensure that PR such as https://github.com/huggingface/transformers/pull/7605 do not impact the current state of models. RoBERTa not done because it's already done.

Patches a bug with the `ElectraForPreTraining` when batch size = 1


Downloading DPR model ('facebook/dpr-ctx_encoder-single-nq-base')

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!--Hi, I want to use model build on the natural questions. I have to question:
1.. I see an example of the above-mentioned model. here it is
```
from transformers import DPRReader, DPRReaderTokenizer
tokenizer = DPRReaderTokenizer.from_pretrained('facebook/dpr-reader-single-nq-base')
model = DPRReader.from_pretrained('facebook/dpr-reader-single-nq-base', return_dict=True)
encoded_inputs = tokenizer(
        questions=["What is love ?"],
        titles=["Haddaway"],
        texts=["'What Is Love' is a song recorded by the artist Haddaway"],
        return_tensors='pt'
    )
outputs = model(**encoded_inputs)
start_logits = outputs.stat_logits
end_logits = outputs.end_logits
relevance_logits = outputs.relevance_logits
```  So I want to run the command which gives me the answers here. It gives me just the start and end position of answer.
2.. Can I use my context (my document) where I want to search model the answer? is it possible here?
Thanks in advance-->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


Enhance TFTrainer.save_model()

# What does this PR do?

@jplu , could you close PR #7597, please? This PR is a clean one, without the file `modeling_tf_utils` being changed at all.

Currently, TFTrainer.save_model() raises errors if the model is not TFPreTrainedModel . However Trainer works fine with torch.nn.modules.Module.

This is a step to make TFTrainer work with usual tf.keras.models.Model models. The idea (from @sgugger) is that a user is building their own models that work like ours (e.g., return the loss as the first output) and can train them with Trainer.

Furthermore, a SavedModel is also saved using tf.saved_model.save().

For @jplu and @sgugger .


position_ids parameter cannot work with past parameter for GPT2Model during batch inference

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
Hi, @patrickvonplaten.  I just finished training a GPT-2 model by using the `GPT2Model` class, and I try to speed up the inference by using the batch inference (very similar to #3021 ). However, I found that the `position_ids` parameter cannot work with the `past` parameter, and it raises the Error: `RuntimeError: The size of tensor a (32) must match the size of tensor b (2592) at non-singleton dimension 0`.
![微信图片_20201007002211](https://user-images.githubusercontent.com/27548710/95229447-2b6f4980-0833-11eb-8cf4-07465f8935ec.png)
I found the exception happens in the `modeling_gpt2.py` line 471, so I check the original codes of the `GPT2Model` class. In `modeling_gpt2.py` line 426 to line 427 (line 558 to line 559 in the lastest original code):
```python
if position_ids is not None:
    position_ids = position_ids.view(-1, input_shape[-1])
```

Actually, during using the `past` parameter for speeding up the inference, the input_shape is `[batch_size, 1]`, but the `position_ids` is `[batch_size, seq_length]`. So, when we use the `past` and `position_ids` at the same time, the position_ids will be converted into a wrong shape `[batch_size*seq_length, 1]` (the shape we want should be `[batch_size, seq_length]`). For example, as shown in the figure, the `batch_size` is 32 and the `seq_length` is 81, and the generated position_ids shape is `[2592, 1]` (32*81=2592), but the correct position_ids shape should be `[32, 81]`.
So I think it may be a bug, but I am not so sure about it. Can you guys help me to figure it out?

Here are the environment variables in my system:
* transformers==2.11.0
* pytorch==1.5.1
* python==3.6.11

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


OSError: Can't load config for saved_model when deploying on EC2.

I was deploying a trained model on AWS EC2 instance (t3a.xlarge) using a dockerized image and Flask. The model was trained using [fast-bert](https://github.com/kaushaltrivedi) that implements transformers as a dependency.

When I passed a sentence on the rendered page, I received

`"In get_config_dict raise EnvironmentError OSError"`

and

```
OSError: Can't load config for 'model/final_model'. Make sure that:
'path/to/final_model' is a correct model identifier listed on 'https://huggingface.co/models'
or 'path/to/final_model' is the correct path to a directory containing a config.json file
```
As suggested in certain threads, I re-installed the image with the latest transformers==3.3.1 release.

However, I am unable to figure out the issue.

Kindly help.

Similar to #6267 #5803 #7412

![config_Error](https://user-images.githubusercontent.com/42617598/95229583-5e5b1280-081e-11eb-9a8b-e47a0f9131ab.png)


Fix wrong reference name/filename in docstring of `SquadProcessor`

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7613

Replace wrong filenames in docstring:  `train-v1.1.json`/`train-v2.0.json` -> `dev-v1.1.json`/`dev-v2.0.json`

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Feature Request: Support training/evaluation on Squad-format (json) files in SquadDataset for quick Squad fine-tuning

I am currently working on a project using [run_squad_trainer.py](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad_trainer.py)
to quickly fine-tune on new Squad-format (v2.0 json) files. However, the current SquadDataset class only allows training/evaluating on the original Squad jsons (train-v2.0.json, dev-v2.0.json). I have used a quick work around by using softlinks, to the actual training/evaluation files, but feel this is a little contrived.

I believe if the arguments `train_file` and `predict_file` are added in [SquadArguments](https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/src/transformers/data/datasets/squad.py#L36) and also line 152 in [SquadDataset](https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/src/transformers/data/datasets/squad.py#L152) is changed to

 `self.examples = self.processor.get_dev_examples(args.data_dir, filename=args.predict_file)`

and line 154 in [SquadDataset](https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/src/transformers/data/datasets/squad.py#L154)

`self.examples = self.processor.get_train_examples(args.data_dir, filename=arg.train_file)`

that may do the trick. At least this approach works in [run_squad.py](https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/examples/question-answering/run_squad.py#L444).

Thanks for your great work!


Feature Request: Support training and evaluating on Squad-format (json) files in SquadDataset for easy Squad fine-tuning


SquadProcessor: Wrong reference name/filename in docstring

As the docstring of the function `get_train_examples()` refers to  `train-v1.1.json`/`train-v2.0.json`, I guess `get_dev_examples()` should refer to `dev-v1.1.json`/`dev-v2.0.json` (but refers to `train-v1.1.json`/`train-v2.0.json`):

https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/src/transformers/data/processors/squad.py#L610-L617


updating modelcard with training dataset information.

Related to pretrained model cards


typo fix

Related to pretrained model cards

It should be T5-3B not T5-3M.
Fixes # (issue)

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

 Model Cards: @julien-c
 T5: @patrickvonplaten


Fix tokenizer UnboundLocalError when padding is set to PaddingStrategy.MAX_LENGTH

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7609

## Who can review?

 tokenizers: @mfuntowicz


Tokenizer: UnboundLocalError with PaddingStrategy MAX_LENGTH

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.1.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help

 tokenizers: @mfuntowicz

## Information

Model I am using: <transformers.tokenization_bert.BertTokenizer>

The problem arises when using:
* the official example scripts:
using the `encode_plus`

The tasks I am working on is:
* Tokenizing

## To reproduce

Steps to reproduce the behavior:

1. tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
2. tokenizer.encode_plus("hello word", max_length=128, padding=PaddingStrategy.MAX_LENGTH)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

`UnboundLocalError: local variable 'padding_strategy' referenced before assignment`

## Expected behavior

Return Tokenizer output


Ability to pre-train BART model

# 🚀 Feature request

Ability to pre-train BART model, same like there is an ability to pre-train BERT and other models.

## Motivation

I'm using a pre-trained BART model for sequence-to-sequence problem and trained it using my own data, using the examples here: https://github.com/huggingface/transformers/tree/master/examples/seq2seq

I was wondering if there is a chance to add an ability to continue the pre-training of the already pre-trained `facebook/bart-base` and `facebook/bart-large` models, with my own unsupervised data, in order to improve the results.

@sshleifer Can you please help? thanks in advance!


Create README.md (LEGAL-BERT Model card)

Related to pretrained model cards

Model description for all LEGAL-BERT models, published as part of  "LEGAL-BERT: The Muppets straight out of Law School". Chalkidis et al., 2018, In Findings of EMNLP 2020

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Add ProtT5-XL-BFD model card

Related to pretrained model cards

Fixes # (issue)
Create a new card for our ProtT5-XL-BFD model.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
Model Cards: @julien-c
T5: @patrickvonplaten


TensorFlow training/inference optimization

# What does this PR do?

This PR fixes a performance issue where some operation was done on CPU instead of GPU and would result to put the GPU in idle mode. This optimization is feasible thanks to the recent update we made on the way we load the TF weights.

@patrickvonplaten I have done few changes in the `TFLongformer` model but I'm sure it can be further optimized the same way (see `TFLongformerSelfAttention`) but as I don't know much on how works this model, can you take a look if the same optimization can be applied?

Fixes #6771


way to make inference Zero Shot pipeline faster?

Hi
Can you guys give me tips how to make Zero Shot pipeline inference faster?

My current approach right now is reducing the model size/parameter
(trying to train "base model" instead of "large model)

Is there another approach?

CCing @joeddav


Added model cards for Tagalog BERT models

Related to pretrained model cards

# What does this PR do?
Adds model cards for five Tagalog BERT models:
* jcblaise/bert-tagalog-base-cased
* jcblaise/bert-tagalog-base-uncased
* jcblaise/bert-tagalog-base-cased-WWM
* jcblaise/bert-tagalog-base-uncased-WWM
* jcblaise/distilbert-tagalog-base-uncased


RAG : Can we fine-tune RAG with update frequency method similar to Fairseq framework?

RAG fine-tune script needs 8 GPUs to train. Is there any chance that the training can be done with less number of GPUs using the update frequency?