Multiprocessing behavior change 3.1.0 -> 3.2.0

## Environment info

```
- `transformers` version: 3.2.0
- Platform: Linux-4.15.0-88-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes (Python multiprocessing)
```

## Information
I am writing a custom script that uses Python's multiprocessing. The goal of it is to have multiple child processes that run inference (using `torch.nn.Module`) **on separate GPUs**.

See below a minimal example of the issue. Please note that script contains pure `torch` code, however, it seems like importing `transformers` (**and not even using it afterwards**) changes some internal states.

```python
import multiprocessing as mp
import os

import torch
import transformers  # <-- Just imported, never used

def diagnostics(name):
    """Print diagnostics."""
    print(name)
    print(f"CUDA initialized: {torch.cuda.is_initialized()}")
    print(f"Is bad fork: {torch._C._cuda_isInBadFork()}")
    print(80 * "*")

def fun(gpu):
    current_process = mp.current_process()

    diagnostics(current_process.pid)
    os.environ["CUDA_VISIBLE_DEVICES"] = str(gpu)
    model = torch.nn.Linear(200, 300)

    model = model.to("cuda")  # Trouble maker

    while True:
        model(torch.ones(32, 200, device="cuda"))

if __name__ == "__main__":
    n_processes = 2
    gpus = [0, 1]
    start_method = "fork"  # fork, forkserver, spawn

    diagnostics("Parent")

    mp.set_start_method(start_method)

    processes = []
    for i in range(n_processes):
        p = mp.Process(name=str(i), target=fun, kwargs={"gpu": gpus[i]})
        p.start()
        processes.append(p)

    for p in processes:
        p.join()

```
The above script works as expected in `3.1.0` or when we do not import transformers at all. Each subprocess does inference on a separate GPU. See below the standard output.
```
Parent
CUDA initialized: False
Is bad fork: False
********************************************************************************
21091
CUDA initialized: False
Is bad fork: False
********************************************************************************
21092
CUDA initialized: False
Is bad fork: False
********************************************************************************
```

 However, for `3.2.0` and higher there is the following error.
```
Parent
CUDA initialized: False
Is bad fork: False
********************************************************************************
21236
CUDA initialized: False
Is bad fork: True
********************************************************************************
Process 0:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "git_example.py", line 21, in fun
    model = model.to("cuda")  # Trouble maker
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 612, in to
    return self._apply(convert)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 381, in _apply
    param_applied = fn(param)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 610, in convert
    return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
  File "/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py", line 164, in _lazy_init
    "Cannot re-initialize CUDA in forked subprocess. " + msg)
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
21237
CUDA initialized: False
Is bad fork: True
********************************************************************************
Process 1:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "git_example.py", line 21, in fun
    model = model.to("cuda")  # Trouble maker
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 612, in to
    return self._apply(convert)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 381, in _apply
    param_applied = fn(param)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 610, in convert
    return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
  File "/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py", line 164, in _lazy_init
    "Cannot re-initialize CUDA in forked subprocess. " + msg)
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
```

Changing the `start_method` to `"forkserver"` or `"spawn"` prevents the exception from being raised. However, only a single GPU for all child processes is used.


Problem with using custom tokenizers with run_mlm.py

Hi! I have an issue with running the `run_mlm.py` script with a tokenizer I myself trained. If I use pretrained tokenizers everything works.

Versions:

```
python: 3.8.3
transformers: 3.5.1
tokenizers: 0.9.4
torch: 1.7.0
```

This is how I train my tokenizer:

```
from tokenizers import BertWordPieceTokenizer

tokenizer = BertWordPieceTokenizer(lowercase=False, strip_accents=False, clean_text=True)
tokenizer.train(files=['/mounts/data/proj/antmarakis/wikipedia/wikipedia_en_1M.txt'], vocab_size=350, special_tokens=[
    "[PAD]",
    "[UNK]",
    "[CLS]",
    "[SEP]",
    "[MASK]",
])

tokenizer.save_model('wikipedia_en')
```

The above results in a vocab.txt file.

And this is how I try to train my model (using the `run_mlm.py` script):

```
python run_mlm.py \
--model_type bert \
--config_name bert_custom.json \
--train_file wikipedia_en_1M.txt \
--tokenizer_name wikipedia_en \
--output_dir lm_temp \
--do_train \
--num_train_epochs 1 \
--overwrite_output_dir
```

If I use a pretrained model/tokenizer, this script works (that is, I replace `config_name` and `tokenizer_name` with `model_name_or_path roberta-base` or something). But using the above code, I get the following error message:

```
Traceback (most recent call last):
  File "run_mlm.py", line 392, in <module>
    main()
  File "run_mlm.py", line 334, in main
    tokenized_datasets = tokenized_datasets.map(
  File "/mounts/Users/cisintern/antmarakis/.local/lib/python3.8/site-packages/datasets/dataset_dict.py", line 283, in map
    {
  File "/mounts/Users/cisintern/antmarakis/.local/lib/python3.8/site-packages/datasets/dataset_dict.py", line 284, in <dictcomp>
    k: dataset.map(
  File "/mounts/Users/cisintern/antmarakis/.local/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1240, in map
    return self._map_single(
  File "/mounts/Users/cisintern/antmarakis/.local/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 156, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/mounts/Users/cisintern/antmarakis/.local/lib/python3.8/site-packages/datasets/fingerprint.py", line 163, in wrapper
    out = func(self, *args, **kwargs)
  File "/mounts/Users/cisintern/antmarakis/.local/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1525, in _map_single
    writer.write_batch(batch)
  File "/mounts/Users/cisintern/antmarakis/.local/lib/python3.8/site-packages/datasets/arrow_writer.py", line 278, in write_batch
    pa_table = pa.Table.from_pydict(typed_sequence_examples)
  File "pyarrow/table.pxi", line 1531, in pyarrow.lib.Table.from_pydict
  File "pyarrow/array.pxi", line 295, in pyarrow.lib.asarray
  File "pyarrow/array.pxi", line 195, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 107, in pyarrow.lib._handle_arrow_array_protocol
  File "/mounts/Users/cisintern/antmarakis/.local/lib/python3.8/site-packages/datasets/arrow_writer.py", line 100, in __arrow_array__
    if trying_type and out[0].as_py() != self.data[0]:
  File "pyarrow/array.pxi", line 949, in pyarrow.lib.Array.__getitem__
  File "pyarrow/array.pxi", line 362, in pyarrow.lib._normalize_index
IndexError: index out of bounds
```

This approach used to work for previous versions, but after I upgraded to the latest releases this doesn't seem to work anymore and I do not know where it broke. Any help would be appreciated!


Warning about too long input for fast tokenizers too

# What does this PR do?

If truncation is not set in tokenizers, but the tokenization is too long
for the model (`model_max_length`), we used to trigger a warning that

The input would probably fail (which it most likely will).

This PR re-enables the warning for fast tokenizers too and uses common code
for the trigger to make sure it's consistent across.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@LysandreJik
@thomwolf

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Fix setup.py on Windows

# What does this PR do?

This PR fixes the target `deps_table_update` on Windows by forcing the newline to be LF.


Minor docs typo fixes

# What does this PR do?
Just a few typo fixes in the docs.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@sgugger


QARiB Arabic and dialects models

model card

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Use model.from_pretrained for DataParallel also

When training on multiple GPUs, the code wraps a model with torch.nn.DataParallel. However if the model has custom from_pretrained logic, it does not get applied during load_best_model_at_end.

This commit uses the underlying model during load_best_model_at_end, and re-wraps the loaded model with DataParallel.

If you choose to reject this change, then could you please move the this logic to a function, e.g. def load_best_model_checkpoint(best_model_checkpoint) or something, so that it can be overridden?

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Can I get logits for each sequence I acqired from model.generate()?

Hi, I’m currently stucked in getting logits from model.generate. I’m wondering if it is possible to get logits of each seqeucne returned by model.generate. (like logits for each token returned by model.logits)


Loss pooling layer parameters after Fine-tune.

According to the [code](https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/modeling_bert.py#L1005): if I want to fine-tune BERT with LM, we don't init pooling layer.
So we loss the original(pre-trained by Google) parameters if we save the fine-tune model and reload it.
Mostly, we use this model for downstream task( text classification), this (may) lead to a worse result.
This `add_pooling_layer` should be `true` for all time even if we don't update them in fine-tune.
@thomwolf  @LysandreJik


[finetune_trainer] --evaluate_during_training is no more

In `examples/seq2seq/builtin_trainer/` all scripts reference `--evaluate_during_training ` but it doesn't exist in pt trainer, but does exist in tf trainer:
```
grep -Ir evaluate_during
builtin_trainer/finetune.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
builtin_trainer/train_distil_marian_enro.sh:    --do_train --do_eval --do_predict --evaluate_during_training\
builtin_trainer/finetune_tpu.sh:    --do_train --do_eval --evaluate_during_training \
builtin_trainer/train_distilbart_cnn.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
builtin_trainer/train_distil_marian_enro_tpu.sh:    --do_train --do_eval --evaluate_during_training \
builtin_trainer/train_mbart_cc25_enro.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
```

```
Traceback (most recent call last):
  File "finetune_trainer.py", line 310, in <module>
    main()
  File "finetune_trainer.py", line 118, in main
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/transformers/hf_argparser.py", line 144, in parse_args_into_dataclasses
    raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
ValueError: Some specified arguments are not used by the HfArgumentParser: ['--evaluate_during_training']
```
Is this meant to be replaced by: `--evaluation_strategy` - this is the closest I found in `training_args.py`

If so which one? `steps` or `epoch`?

Also the help output is borked:

```
$ python finetune_trainer.py -h
...
 [--evaluation_strategy {EvaluationStrategy.NO,EvaluationStrategy.STEPS,EvaluationStrategy.EPOCH}]
```
probably this is not what what's intended, but
```
 [--evaluation_strategy {no, steps, epochs}
```

But perhaps it's a bigger issue - I see `trainer.args.evaluate_during_training`:
```
src/transformers/integrations.py:        ) and (not trainer.args.do_eval or not trainer.args.evaluate_during_training):
```
and also `--evaluate_during_training` in many other files under `examples/`.

Thank you.

@sgugger, @patrickvonplaten


[FlaxBert] Fix non-broadcastable attention mask for batched forward-passes

# What does this PR do?

Fixes #8790

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@mfuntowicz
@avital
@LysandreJik


[FlaxBert] Non-broadcastable attention mask in batched forward-pass

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Linux
- Python version: 3.8
- JAX version:
  - jax==0.2.6
  - jaxlib==0.1.57+cuda110
  - flax==0.2.2
- PyTorch version (GPU?): n/a
- Tensorflow version (GPU?): n/a
- Using GPU in script?: **yes** (cuda 11.0)
- Using distributed or parallel set-up in script?: **no**

### Who can help
@mfuntowicz
@avital
@LysandreJik

## Information

I ran the script from the recent Twitter [post](https://twitter.com/huggingface/status/1331255460033400834):

![](https://pbs.twimg.com/media/EnmRwGDW4AA0j26?format=jpg)

The only thing I changed was that I fed in multiple sentences:

```python
from transformers import FlaxBertModel, BertTokenizerFast, TensorType

tokenizer = BertTokenizerFast.from_pretrained('bert-base-cased')
model = FlaxBertModel.from_pretrained('bert-base-cased')
# apply_fn = jax.jit(model.model.apply)

sentences = ["this is an example sentence", "this is another", "and a third one"]
encodings = tokenizer(sentences, return_tensors=TensorType.JAX, padding=True, truncation=True)
tokens, pooled = model(**encodings)
```
> ValueError: Incompatible shapes for broadcasting: ((3, 12, 7, 7), (1, 1, 3, 7))

See full stack trace: https://pastebin.com/sPUSjGVi


KeyError: 'eval_loss' when fine-tuning gpt-2 with run_clm.py

## Environment info
- `transformers` version: 4.0.0-rc-1
- Platform: Linux-4.19.0-12-amd64-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: default option

### Who can help
 albert, bert, GPT2, XLM: @LysandreJik
 Trainer: @sgugger

## Information

Model I am using (Bert, XLNet ...): GPT2

The problem arises when using:
* [x] the official example scripts: (give details below)
Bug occurs when running run_clm.py file from  transformers/examples/language-modeling/ , the evaluation step (--do_eval) will crash with a python error related to missing KeyError 'eval_loss',

## To reproduce

Steps to reproduce the behavior:

1. Use run_clm.py file from  transformers/examples/language-modeling/
2. Try to fine-tune gpt-2 model, with your own train file and your own validation file
3. When you add "--do_eval" option in run_clm.py then an error will occur when the step "evaluation" is reached :

```
  File "run_clm.py", line 353, in <module>
    main()
  File "run_clm.py", line 333, in main
    perplexity = math.exp(eval_output["eval_loss"])
KeyError: 'eval_loss'
```
when I try to print the content of eval_output then there is just one key :  "epoch"

the way I execute run_clm.py  :

```
python run_clm.py \
    --model_name_or_path gpt2 \
    --train_file train.txt \
    --validation_file dev.txt \
    --do_train \
    --do_eval \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --output_dir results/test-clm
```

## Expected behavior
The evaluation step should run without problems.


Add QCRI Arabic and Dialectal BERT (QARiB) models

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


QA pipeline fails during convert_squad_examples_to_features

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.0.0-rc-1
- Platform: Linux-3.10.0-1062.9.1.el7.x86_64-x86_64-with-redhat-7.8-Maipo
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help
@LysandreJik @mfuntowicz

IDK who else can help, but in sort, I am looking for someone who can help me in QA tasks.

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: run_squad.py (modifying to run using jupyter notebook, using "HfArgumentParser")

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SQUaD
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. modified all argparse to HfArgumentParser
2. created "ModelArguments" dataclass function for HfArgumentParser (Ref: https://github.com/patil-suraj/Notebooks/blob/master/longformer_qa_training.ipynb)
3. need to small changes in the whole script.

The test fails with error `TypeError: TextInputSequence must be str`

Complete failure result:
```
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
  File "/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/site-packages/transformers/data/processors/squad.py", line 175, in squad_convert_example_to_features
    return_token_type_ids=True,
  File "/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2439, in encode_plus
    **kwargs,
  File "/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/site-packages/transformers/tokenization_utils_fast.py", line 463, in _encode_plus
    **kwargs,
  File "/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/site-packages/transformers/tokenization_utils_fast.py", line 378, in _batch_encode_plus
    is_pretokenized=is_split_into_words,
TypeError: TextInputSequence must be str
"""

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
<ipython-input-19-263240bbee7e> in <module>
----> 1 main()

<ipython-input-18-61d7f0eab618> in main()
    111     # Training
    112     if train_args.do_train:
--> 113         train_dataset = load_and_cache_examples((model_args, train_args), tokenizer, evaluate=False, output_examples=False)
    114         global_step, tr_loss = train(args, train_dataset, model, tokenizer)
    115         logger.info(" global_step = %s, average loss = %s", global_step, tr_loss)

<ipython-input-8-79eb3ed364c2> in load_and_cache_examples(args, tokenizer, evaluate, output_examples)
     54             max_query_length=model_args.max_query_length,
     55             is_training=not evaluate,
---> 56             return_dataset="pt",
     57 #             threads=model_args.threads,
     58         )

/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/site-packages/transformers/data/processors/squad.py in squad_convert_examples_to_features(examples, tokenizer, max_seq_length, doc_stride, max_query_length, is_training, padding_strategy, return_dataset, threads, tqdm_enabled)
    366                 total=len(examples),
    367                 desc="convert squad examples to features",
--> 368                 disable=not tqdm_enabled,
    369             )
    370         )

/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/site-packages/tqdm/std.py in __iter__(self)
   1131
   1132         try:
-> 1133             for obj in iterable:
   1134                 yield obj
   1135                 # Update and possibly print the progressbar.

/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/multiprocessing/pool.py in <genexpr>(.0)
    323                     result._set_length
    324                 ))
--> 325             return (item for chunk in result for item in chunk)
    326
    327     def imap_unordered(self, func, iterable, chunksize=1):

/data/user/tr27p/.conda/envs/DeepBioComp/lib/python3.7/multiprocessing/pool.py in next(self, timeout)
    746         if success:
    747             return value
--> 748         raise value
    749
    750     __next__ = next                    # XXX

TypeError: TextInputSequence must be str
```

## Expected behavior
#### for more details check here:
link: https://github.com/uabinf/nlp-group-project-fall-2020-deepbiocomp/blob/cancer_ask/scripts/qa_script/qa_squad_v1.ipynb
<!-- A clear and concise description of what you would expect to happen. -->


What would be the license of the model files available in Hugging face repository?

Dear Team,

Could you clarify what would be the license for different models pushed to hugging face repo like legal_bert, contracts_bert etc? Would the model file follows the same license i.e. Apache 2.0 like the hugging face library?

Regards
Gaurav


Update README.md

Related to pretrained model cards

Disable Hosted Inference API while output inconsistency is not solved.
"How to use" section.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Different ouputs from code and Hosted Inference API

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.5.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

## To reproduce

Steps to reproduce the behavior:

1. Run the following code:

```python
from transformers import BertForTokenClassification, DistilBertTokenizerFast, pipeline

model = BertForTokenClassification.from_pretrained('monilouise/ner_pt_br')
tokenizer = DistilBertTokenizerFast.from_pretrained('neuralmind/bert-base-portuguese-cased', model_max_length=512, do_lower_case=False)
nlp = pipeline('ner', model=model, tokenizer=tokenizer)
result = nlp("O Tribunal de Contas da União é localizado em Brasília e foi fundado por Rui Barbosa.  Fiscaliza contratos, por exemplo com empresas como a Veigamed e a Buyerbr.")
print(result)
````

It'll ouput:

[{'word': 'Tribunal', 'score': 0.9858521819114685, 'entity': 'B-PUB', 'index': 2}, {'word': 'de', 'score': 0.9954801201820374, 'entity': 'I-PUB', 'index': 3}, {'word': 'Contas', 'score': 0.9929609298706055, 'entity': 'I-PUB', 'index': 4}, {'word': 'da', 'score': 0.9949454665184021, 'entity': 'I-PUB', 'index': 5}, {'word': 'União', 'score': 0.9913719296455383, 'entity': 'L-PUB', 'index': 6}, {'word': 'Brasília', 'score': 0.9405767321586609, 'entity': 'B-LOC', 'index': 10}, {'word': 'Rui', 'score': 0.979736328125, 'entity': 'B-PESSOA', 'index': 15}, {'word': 'Barbosa', 'score': 0.988306999206543, 'entity': 'L-PESSOA', 'index': 16}, {'word': 'Veiga', 'score': 0.9748793244361877, 'entity': 'B-ORG', 'index': 29}, {'word': '##med', 'score': 0.9309309124946594, 'entity': 'L-ORG', 'index': 30}, {'word': 'Bu', 'score': 0.9679405689239502, 'entity': 'B-ORG', 'index': 33}, {'word': '##yer', 'score': 0.6654638051986694, 'entity': 'L-ORG', 'index': 34}, {'word': '##br', 'score': 0.9575732350349426, 'entity': 'L-ORG', 'index': 35}]

including all entity types (PUB, PESSOA, ORG and LOC)

2. In Hosted Inference API, the following result is returned for the same sentence, ignoring PUB entity type and giving incorrect and incomplete results:

```json
[
  {
    "entity_group": "LOC",
    "score": 0.8127626776695251,
    "word": "bras"
  },
  {
    "entity_group": "PESSOA",
    "score": 0.7101765692234039,
    "word": "rui barbosa"
  },
  {
    "entity_group": "ORG",
    "score": 0.7679458856582642,
    "word": "ve"
  },
  {
    "entity_group": "ORG",
    "score": 0.45047426223754883,
    "word": "##igamed"
  },
  {
    "entity_group": "ORG",
    "score": 0.8467527627944946,
    "word": "bu"
  },
  {
    "entity_group": "ORG",
    "score": 0.6024420410394669,
    "word": "##yerbr"
  }
]
```

## Expected behavior

How can it be possible the same model file give different results?(!) May I be missing anything?


MPNet: Masked and Permuted Pre-training for Natural Language Understanding

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Unexpected output from bart-large

I am looking this thread about generation, https://stackoverflow.com/questions/64904840/why-we-need-a-decoder-start-token-id-during-generation-in-huggingface-bart

I re-run his code,
Use ```facebook/bart-base``` model,
 ```
from transformers import *

import torch
model = BartForConditionalGeneration.from_pretrained('facebook/bart-base')
tokenizer = BartTokenizer.from_pretrained('facebook/bart-base')
input_ids = torch.LongTensor([[0, 894, 213, 7, 334, 479, 2]])
res = model.generate(input_ids, num_beams=1, max_length=100)

print(res)

preds = [tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True).strip() for g in res]
print(preds)
```
I get output:
```
tensor([[  2,   0, 894, 213,   7, 334, 479,   2]])
['He go to school.']
```
Then I just simply change the model to ```facebook/bart-large```  with everything kept the same, i.e.
 ```
from transformers import *

import torch
model = BartForConditionalGeneration.from_pretrained('facebook/bart-large')
tokenizer = BartTokenizer.from_pretrained('facebook/bart-large')
input_ids = torch.LongTensor([[0, 894, 213, 7, 334, 479, 2]])
res = model.generate(input_ids, num_beams=1, max_length=100)

print(res)

preds = [tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True).strip() for g in res]
print(preds)
```
Then I get output:
```
tensor([[  2, 894, 894, 213,   7, 334, 479,   2]])
['HeHe go to school.']
```
Is this normal?
Thanks.


NerPipeline (TokenClassification) now outputs offsets of words

# What does this PR do?

- It happens that the offsets are missing, it forces the user to pattern
match the "word" from his input, which is not always feasible.
For instance if a sentence contains the same word twice, then there
is no way to know which is which.
- This PR proposes to fix that by outputting 2 new keys for this
pipelines outputs, "start" and "end", which correspond to the string
offsets of the word. That means that we should always have the
invariant:

```python
input[entity["start"]: entity["end"]] == entity["entity_group"]
                                    # or entity["entity"] if not grouped
```

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->
Example of users that encounter problems:

https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessica+Parker+who+Jessica+lives+in+New+York
https://discuss.huggingface.co/t/token-positions-when-using-the-inference-api/2188

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Hi all! I'm getting this error when trying to run the example code:

Can't load tokenizer for 'facebook/rag-token-base/question_encoder_tokenizer'. Make sure that:

- 'facebook/rag-token-base/question_encoder_tokenizer' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'facebook/rag-token-base/question_encoder_tokenizer' is the correct path to a directory containing relevant tokenizer files


Fix PPLM

# What does this PR do?

API changes break PPLM example, this PR should fix it. However I haven't test it on 'run_pplm_discrim_train.py'.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Using the XLNet or Tranformer-XL as an EncoderDecoder

I want to train a long sequence dataset (MIDI text event representation like the one in [MuseNet](https://openai.com/blog/musenet/#dataset)) from scratch. Since, I can't split the sequence to "sentences" I am using XLNet (or Transformer-XL). I am modelling the task as a sequence2sequence task (with max input seq length of around 40k tokens and output length of 4k tokens) so I want to use an Encoder Decoder Framework.

Is it possible to use XLNet as the encoder and decoder, or just the encoder and use GPT-2 for example to do the decoding (because of the smaller output sequence length).

Thank you 🤗


Better booleans handling in the TF models

# What does this PR do?

This PR provides a better handling for the booleans. More precisely, the execution mode (eager or graph) is detected and the booleans are accordingly set to have a proper execution. Nevertheless, this brings a small breaking change in graph mode, it is not possible anymore to update the booleans with the model parameters but only with through the config and the `return_dict` is forced to be `True`.

Now to activate the `output_attentions` or `output_hidden_states` values in graph mode one has to create the model config like:
```
config = XConfig.from_pretrained("name", output_attentions=True, output_hidden_states=True)
```


Documentation and source for `RobertaClassificationHead`

The docstring for `RobertaForSequenceClassification` says
```
RoBERTa Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks
```

Looking at the code, this does not seem correct. Here, the RoBERTa output is fed into an instance of the class `RobertaClassificationHead`, which feeds the pooled output into a multilayer feedforward-network with one hidden layer and tanh activation. So this is more than only a simple linear layer.

I have two questions:
1. Should the documentation reflect this different classification head for RoBERTa?
2. Where does this classification head originally come from? I could not find a citable source where such a "deep" classification head is used. The original RoBERTa paper only seems to state that their task-specific fine-tuning procedure is the same as BERT uses (which is only a linear layer).

I would be glad if someone could shed light on this.


Converting all model Config classes to dataclasses

Feature request

Request for a new feature

It seems that we could save a lot of boilerplate code and potentially prevent some bugs if we migrated all of the model config classes over to being dataclasses. Already many of our classes (BaseModelOutput, TrainingArguments, etc.) are dataclasses, so we are already committed to having dataclasses as a dependency.

It's relatively low priority, but I would be willing to help implement the change since I'm kind of a neat freak about code.


Big model table

# What does this PR do?

This PR adds a big table in the first page of the doc, indicating whether each of our models has support for a slow/fast tokenizer, PyTorch, TensorFlow and Flax. Result can be found [here](https://125258-155220641-gh.circle-artifacts.com/0/docs/_build/html/index.html) (scroll a bit down).
It is updated automatically via `make fix-copies` and checked for updates in `make quality`, being built form the content of the auto models module.

There were a few issues with the imports on the flax side that I fixed in passing, and I renamed a constant to add the `FLAX` prefix. @mfuntowicz this doesn't really change anything but pinging you just so you're aware.


saving checkpoints on gs bucket

Hi
In case of running on cloud, the model does not work to save the checkpoints on gs bucket, could you help please?
thanks


Possible to add additional features as input to TFBertForSequenceClassification?

Say I have a binary classification problem, but in addition to the sentence I'd like to also input some scalar value as well. Is it possible to just tack on this scalar as input to the last linear layer of BERT?

For example, I'd like to detect if a particular sentence is from my source data or generated. And I know that many instances of a repeated word increases the likelihood that it is a generated sentence. So I'd like to pass the sentence itself into BERT as well as a scalar feature such as the number of unique words in the sentence.


Model Parallelism and Big Models

Model Parallel

Model Parallelilsm Implementations

# 🚀 Feature request

This is a discussion issue for training/fine-tuning very large transformer models. Recently, model parallelism was added for gpt2 and t5. The current implementation is for PyTorch only and requires manually modifying the model classes for each model. Possible routes (thanks to @stas00  for identifying these):
- `fairscale` to avoid individual model implementation
- `deepspeed` to possibly enable even larger models to be trained


Extend typing to path-like objects in `PretrainedConfig` and `PreTrainedModel`

# What does this PR do?

In my experience, I often call the `from_pretrained` and `save_pretrained` methods of models and configurations with a path-like variable rather than a string. Since the paths are then used by various `os` functions, this works just fine: however, the relevant variables are typed as strings only, raising warnings when using an IDE :anguished: .

This PR extends the typing to `Union[str, os.PathLike]` when relevant inside `PretrainedConfig` and `PreTrainedModel`   methods.
Since passing a path-like object is already tacitly supported in most cases, no significant changes to the code are necessary. In a few places, the relevant variable needs to be turned to a string in order to support functions such as `is_remote_url`.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

Maybe (documentation): @sgugger


LXMERT - Visual features don't match original implementation

## Environment info
- `transformers` version: 4.0.0-rc-1
- Platform: Linux-4.15.0-122-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@eltoto1219

## Information

Model I am using: **unc-nlp/lxmert-gqa-uncased**

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] **GQA**

## Details

I've tried to reproduce LXMERT results on GQA -
Using the [visual features from the original repo](https://github.com/airsplay/lxmert#gqa), I got an accuracy of 59.29% on the testdev split (which is a bit less than expected, but close enough).
However, when generating the visual features using the [extraction script](https://github.com/huggingface/transformers/blob/master/examples/lxmert/extracting_data.py) in the examples (which uses "unc-nlp/frcnn-vg-finetuned"), the accuracy is only ~33%.
I checked further, and the bounding boxes are also different.

Any idea to what could be the problem?


Attempt to get a better fix for QA


Allow to set truncation strategy for pipeline

# 🚀 Feature request

The highlevel pipeline function should allow to set the truncation strategy of the tokenizer in the pipeline.

## Motivation

Some models will crash if the input sequence has too many tokens and require truncation. Additionally available memory is limited and it is often useful to shorten the amount of tokens. Sadly this is currently not possible using the pipeline-API. One has to call the tokenizer manually to set the truncation strategy or hope that the task specific pipeline has truncation turned on by default (the summarization pipeline for example has not).

## Your contribution

I could potentially create a PR for this, but want to confirm first that the change is welcome.


[Error: PyTorch to tf]convert_pytorch_checkpoint_to_tf2: AttributeError: bert.pooler.dense.weight not found in PyTorch model

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.5.1
- Platform: Linux-5.4.0-1029-gcp-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help

I think:
 @patrickvonplaten @LysandreJik @VictorSanh

Anyone is welcome!

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Trying to convert my pytorch checkpoint to tf using the below code:

```python
from transformers import convert_pytorch_checkpoint_to_tf2
convert_pytorch_checkpoint_to_tf2.convert_pt_checkpoint_to_tf(
    model_type = "bert",
    pytorch_checkpoint_path="model/pytorch_model.bin",
    config_file="model/config.json",
    tf_dump_path="TFmodel",
    compare_with_pt_model=False,
    use_cached_models=False
)
```

my model folder has a tiny bert trained using HuggingFace: contents of `model` folder are:
`checkpoint-500            special_tokens_map.json
config.json               tokenizer_config.json
eval_results_mlm_wwm.txt  training_args.bin
pytorch_model.bin         vocab.txt`

## To reproduce

Error:

```
Loading PyTorch weights from /home/3551351/bert-mlm/model/pytorch_model.bin
PyTorch checkpoint contains 8,354,548 parameters
Traceback (most recent call last):
  File "pt2tf.py", line 7, in <module>
    convert_pytorch_checkpoint_to_tf2.convert_pt_checkpoint_to_tf(
  File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/convert_pytorch_checkpoint_to_tf2.py", line 283, in convert_pt_checkpoint_to_tf
    tf_model = load_pytorch_checkpoint_in_tf2_model(tf_model, pytorch_checkpoint_path)
  File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/modeling_tf_pytorch_utils.py", line 96, in load_pytorch_checkpoint_in_tf2_model
    return load_pytorch_weights_in_tf2_model(
  File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/modeling_tf_pytorch_utils.py", line 172, in load_pytorch_weights_in_tf2_model
    raise AttributeError("{} not found in PyTorch model".format(name))
AttributeError: bert.pooler.dense.weight not found in PyTorch model
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Fix QA argument handler

The QA argument handler does not handle multiple sequences at a time anymore. This was not tested, so I added it to the tests.

Fix https://github.com/huggingface/transformers/issues/8759

To test it out run the following on `master`:

```py
from transformers import pipeline

nlp = pipeline("question-answering")

context = r"""
Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
a model on a SQuAD task, you may leverage the `run_squad.py`.
"""

print(
    nlp(
        question=["What is extractive question answering?", "What is a good example of a question answering dataset?"],
        context=[context, context],
    )
)
```


Tokenizers - move from hardcoded configs and models to fully hosted

# What does this PR do?

Fixes #8125 #8117

Tokenizer checkpoint files are now handled the same way than model files.

Also:
- add a `tokenizer_class_name` field in the tokenization config file `tokenizer_config.json`. This field is used when possible by `AutoTokenizer` to disambiguate the tokenizer to instantiate
- concurrently configuration and vocabulary files are updated on the relevant model repo in the hub to wake them independant from the code-base.
- the max length of the models was always the same for all the variante in an architecture. Consequently we simply the `max_model_input_sizes` attribute to make it a single integer instead of a mapping from string (model names) to integers.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Migration guide from v3.x to v4.x

Additionally to the release notes, this completes the migration guide to showcase the expected breaking changes from v3.x to v4.x, and how to retrieve the original behavior.


## Environment info
- `transformers` version: 3.5.1
- Platform: Linux
- Python version: 3.7
- PyTorch version (GPU?): 1.6
- Tensorflow version (GPU?): -
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help

 Model Cards: @julien-c
 Text Generation: @patrickvonplaten @TevenLeScao
 T5: @patrickvonplaten
 examples/seq2seq: @patil-suraj

## Information

Hi
I would like to use T5 untrained here is the command I try:

` model = T5ForConditionalGeneration.from_config(config=config)
`
I am getting this error, could you assist me please? thank you Looks weird error to me, since from_config should work based on documentations. thanks

```
  File "finetune_t5_trainer.py", line 88, in main
    model = T5ForConditionalGeneration.from_config(config=config)
AttributeError: type object 'T5ForConditionalGeneration' has no attribute 'from_config'

```

## To reproduce
Please run the command given

## Expected behavior
load unpretrained model


Create README.md

model card

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Create README.md

Related to pretrained model cards


Version 3.5 broke the multi context/questions feature for the QuestionAnsweringPipeline

## Environment info

- `transformers` version: 3.5.1 (also in 3.5.0)
- Platform: Darwin-20.1.0-x86_64-i386-64bit
- Python version: 3.7.5
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help

 tokenizers: @mfuntowicz

## Information

Model I am using (Bert, XLNet ...): Default QuestionAnsweringPipeline

The problem arises when using:
* [x] my own modified scripts: (see below, modified from the example given here https://huggingface.co/transformers/usage.html#extractive-question-answering)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: Extractive Question Answering

## To reproduce

Steps to reproduce the behavior:

 1. Install transformers 3.5.1 (also in 3.5.0)
 2. Run the following:

```python
from transformers import pipeline

nlp = pipeline("question-answering")

context = r"""
Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
a model on a SQuAD task, you may leverage the `run_squad.py`.
"""

print(
    nlp(
        question=["What is extractive question answering?", "What is a good example of a question answering dataset?"],
        context=[context, context],
    )
)
```

In versions 3.5.0 and 3.5.1, I have this error:

```
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/Users/cytadel/.pyenv/versions/3.7.5/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/Users/cytadel/.pyenv/versions/3.7.5/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/Users/cytadel/Library/Caches/pypoetry/virtualenvs/feedly.ml-cyber-attacks-4LjjtgqO-py3.7/lib/python3.7/site-packages/transformers/data/processors/squad.py", line 110, in squad_convert_example_to_features
    for (i, token) in enumerate(example.doc_tokens):
AttributeError: 'list' object has no attribute 'doc_tokens'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/cytadel/feedly/ml/do_not_commit.py", line 14, in <module>
    context=[context, context],
  File "/Users/cytadel/Library/Caches/pypoetry/virtualenvs/feedly.ml-cyber-attacks-4LjjtgqO-py3.7/lib/python3.7/site-packages/transformers/pipelines.py", line 1787, in __call__
    for example in examples
  File "/Users/cytadel/Library/Caches/pypoetry/virtualenvs/feedly.ml-cyber-attacks-4LjjtgqO-py3.7/lib/python3.7/site-packages/transformers/pipelines.py", line 1787, in <listcomp>
    for example in examples
  File "/Users/cytadel/Library/Caches/pypoetry/virtualenvs/feedly.ml-cyber-attacks-4LjjtgqO-py3.7/lib/python3.7/site-packages/transformers/data/processors/squad.py", line 368, in squad_convert_examples_to_features
    disable=not tqdm_enabled,
  File "/Users/cytadel/Library/Caches/pypoetry/virtualenvs/feedly.ml-cyber-attacks-4LjjtgqO-py3.7/lib/python3.7/site-packages/tqdm/std.py", line 1171, in __iter__
    for obj in iterable:
  File "/Users/cytadel/.pyenv/versions/3.7.5/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 325, in <genexpr>
    return (item for chunk in result for item in chunk)
  File "/Users/cytadel/.pyenv/versions/3.7.5/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
AttributeError: 'list' object has no attribute 'doc_tokens'
```

## Expected behavior

Same result as in transformers version 3.4.0:

`[{'score': 0.6222442984580994, 'start': 34, 'end': 96, 'answer': 'the task of extracting an answer from a text given a question.'}, {'score': 0.5115318894386292, 'start': 147, 'end': 161, 'answer': 'SQuAD dataset,'}]`


[Help] GPU with query answering

I want to figure out some way to get faster results from a QA model. I did some tests on google cloud with different GPUs and got some results, those tests was made with different GPUs and same CPU using this code:

```
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
from transformers.pipelines import pipeline

tokenizer = AutoTokenizer.from_pretrained("deepset/bert-large-uncased-whole-word-masking-squad2")
model = AutoModelForQuestionAnswering.from_pretrained("deepset/bert-large-uncased-whole-word-masking-squad2")
nlp_qa = pipeline('question-answering', model=model, tokenizer=tokenizer)
X = nlp_qa(context = text, question=queryy, topk = 3, device = 0, max_answer_len = 50)
```
Where context is just a long string and the question a simple query, and I got those results:

```
TESTE 1: **********
4 vCPUs 15Gb RAM
NVIDIA TESLA P100X1
Tempo1: 1:45 min
Tempo2: 1:40 min
Tempo3: 1:45 min
***************
***************
TESTE 2: **********
4 vCPUs 15Gb RAM
NVIDIA TESLA V100X1
Tempo1: 1:58 min
Tempo2: 1:58 min
Tempo3: 1:55 min
***************
***************
TESTE 3: **********
4 vCPUs 15Gb RAM
NVIDIA TESLA K80X1
Tempo1: 2:06 min
Tempo2: 2:18 min
Tempo3: 2:20 min
***************
***************
TESTE 4: **********
4 vCPUs 15Gb RAM
NVIDIA TESLA T4X1
Tempo1: 1:45 min
Tempo2: 1:50 min
Tempo3: 1:50 min
***************
***************
TESTE 5: **********
4 vCPUs 15Gb RAM
NVIDIA NONE
Tempo1: 2:22 min
Tempo2: 1:57 min
Tempo3: 1:57 min
```

I want to know if I am  using GPU wrong, or is it normal to get almost same results with and without GPU on this set?.
Is there anyway to get faster results?


## Environment info

- `transformers` version: 3.5.1
- Platform: Linux-5.4.0-1029-gcp-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
I think:
 @patrickvonplaten @LysandreJik @VictorSanh

Anyone is welcome!

## Information

I am using `examples/language-modeling/run_mlm_wwm.py` to train my own Tiny BERT model.

## To reproduce
Using Tiny BERT from Google [https://github.com/google-research/bert/blob/master/README.md](https://github.com/google-research/bert/blob/master/README.md)
Using `examples/language-modeling/run_mlm_wwm.py` from HuggingFace to train a language model on raw text.

files in my `google-bert-tiny` are `bert_config.json  bert_model.ckpt.data-00000-of-00001  bert_model.ckpt.index  vocab.txt`

Steps to reproduce the behavior:

1. install transformers torch and Tensorflow using pip
2. Get `examples/language-modeling/run_mlm_wwm.py` from HuggingFace>Transformers [Link](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm_wwm.py)
3. Running the following command:
```shell
python run_mlm_wwm.py \
    --model_name_or_path google-bert-tiny/bert_model.ckpt.index \
    --config_name google-bert-tiny/bert_config.json \
    --train_file train.txt \
    --validation_file val.txt \
    --do_train \
    --do_eval \
    --output_dir test-mlm-wwm \
    --cache_dir cache
```

Error:
```
Traceback (most recent call last):
  File "run_mlm_wwm.py", line 340, in <module>
    main()
  File "run_mlm_wwm.py", line 236, in main
    tokenizer = AutoTokenizer.from_pretrained(
  File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/tokenization_auto.py", line 306, in from_pretrained
    config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
  File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/configuration_auto.py", line 333, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/configuration_utils.py", line 391, in get_config_dict
    config_dict = cls._dict_from_json_file(resolved_config_file)
  File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/configuration_utils.py", line 474, in _dict_from_json_file
    text = reader.read()
  File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 49: invalid start byte
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Want it to train

<!-- A clear and concise description of what you would expect to happen. -->


Continued training of the original BERT models (not to PyTorch)

# 🚀 Feature request

I am looking for continued training of original tiny-bert with my own raw data using masked language modelling.
But I want the final model in TF.

## Motivation

I tried this [LM](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm_wwm.py), but this only works from pytorch to pytorch.

I tried with using original ckpt and from_tf, it always results in a .h5 error

Please let me know I can explain more OR help in any way.

Basically, we should be able to use tf weights >> masked language modelling >> and have a domain specific pre-trained Tensorflow language model.


Why there are no such 'cls/' layers in roberta pytorch checkpoints

In pytorch checkpoints of roberta in huggingface transformers, the last two layers are the "pooler layers":
pooler.dense.weight
pooler.dense.bias

However, In original roberta tensorflow checkpoints,  the last few layers are not the pooler layers, instead, they are:
cls/predictions/output_bias (DT_FLOAT) [21128]
cls/predictions/transform/LayerNorm/beta (DT_FLOAT) [768]
cls/predictions/transform/LayerNorm/gamma (DT_FLOAT) [768]
cls/predictions/transform/dense/bias (DT_FLOAT) [768]
cls/predictions/transform/dense/kernel (DT_FLOAT) [768,768]
cls/seq_relationship/output_bias (DT_FLOAT) [2]
cls/seq_relationship/output_weights (DT_FLOAT) [2,768]

these 'cls/' layers came after the pooler layers.
I converted the pytorch checkpoints into tensorflow checkpoints. Then when I try to load the weights,  all I was told was:
tensorflow.python.framework.errors_impl.NotFoundError: Key cls/predictions/transform/dense/kernel not found in checkpoint

which means the 'cls/' layers do not exist at all! so why these layers are gone in pytorch checkpoints provided by huggingface transformers? What should I do to get the weights of these 'cls/' layers? I am trying to use a roberta checkpoint that is trained by someone else using huggingface transformers, however, I have to convert it to a tensorflow version for my code is in tensorflow version , but this problem occurs. how can I correctly convert the checkpoints?


Allow to provide specific params in WandbCallback

# 🚀 Feature request

It would be nice to be able to track additional params in wandb when using the Trainer interface. For example, I need to track down how many layers were frozen in each experiment. I'm currently using a custom WandCallback class to do this.

## Your contribution

I can work on a PR.


update README.txt

Related to pretrained model cards


Create README.md

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Fix minor bug to handle dynamic sequence length

WIP

Label your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

This PR is to fix a minor bug for handling dynamic sequence length, e.g., during building the serving model(s).


[core] transformers version number normalization

It looks that we need to either normalize the `transformers` version number to one of the accepted formats:

```
x.y.z
x.y.z.dev0
x.y.z.rc1
```
or silence the warning.

Currently, `setuptools` doesn't like `-rc-1` and `-dev`, as you can see from:

```
python setup.py --name
.../python3.8/site-packages/setuptools/dist.py:452: UserWarning: Normalizing '4.0.0-rc-1' to '4.0.0rc1'
```
with `4.0.0-dev`
```
python3.8/site-packages/setuptools/dist.py:452: UserWarning: Normalizing '4.0.0-dev' to '4.0.0.dev0'
```

Otherwise, this warning will be showing up all the time during `make style` and friends once https://github.com/huggingface/transformers/pull/8645 is merged.

@sgugger, @LysandreJik


"AutoTokenizer.from_pretrained" does not work when loading a pretrained Albert model

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: 5.4.0-53-generic #59~18.04.1-Ubuntu SMP Wed Oct 21 12:14:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.0
- Tensorflow version (GPU?): N/A
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Install PyTorch from the official website as well as the transformers via pip.
2. Using the following pre-trained model:
```
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("ckiplab/albert-tiny-chinese")

model = AutoModelForMaskedLM.from_pretrained("ckiplab/albert-tiny-chinese")
```
3. Error:
```
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 683/683 [00:00<00:00, 1.32MB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 215kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 174/174 [00:00<00:00, 334kB/s]
Traceback (most recent call last):
  File "/home/faith/torch_tutorials/torch_chatbot.py", line 30, in <module>
    tokenizer = AutoTokenizer.from_pretrained("ckiplab/albert-tiny-chinese")
  File "/home/faith/miniconda3/envs/torch/lib/python3.7/site-packages/transformers/tokenization_auto.py", line 341, in from_pretrained
    return tokenizer_class_py.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/home/faith/miniconda3/envs/torch/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1653, in from_pretrained
    resolved_vocab_files, pretrained_model_name_or_path, init_configuration, *init_inputs, **kwargs
  File "/home/faith/miniconda3/envs/torch/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1725, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/faith/miniconda3/envs/torch/lib/python3.7/site-packages/transformers/tokenization_albert.py", line 149, in __init__
    self.sp_model.Load(vocab_file)
  File "/home/faith/miniconda3/envs/torch/lib/python3.7/site-packages/sentencepiece.py", line 367, in Load
    return self.LoadFromFile(model_file)
  File "/home/faith/miniconda3/envs/torch/lib/python3.7/site-packages/sentencepiece.py", line 177, in LoadFromFile
    return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
TypeError: not a string
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
Expect to download this model correctly with error prompting.


Return correct Bart hidden state tensors

# What does this PR do?

Fixes #8601. When `output_hidden_states=True`, hidden states transposing is done _before_ being fed through the next layer. This ensures that the returned hidden state tensors lie upstream in the graph from the model outputs (allowing their gradients to be computed).


Fix slow tests v2

Fix a few tests that were broken in recent PRs.


added instructions for syncing upstream master with forked master via PR

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #8742

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [commit](https://github.com/lucidworks/transformers/commit/46b17d206529206848116fd6219643446bac938c#commitcomment-44356945)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->
@stas00


Added instructions for syncing forked masters to avoid references


MT5 should have an autotokenizer

MT5 should have an auto-tokenizer.

Currently fails a lot of slow tests.


Add instructions for syncing forked masters to avoid references

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->
Ticket created based on discussion https://github.com/lucidworks/transformers/commit/46b17d206529206848116fd6219643446bac938c#commitcomment-44356945
The problem is when someone on a forked repository decides to sync up their masters with upstream (HF master) using a branch and a PR, all the PR and issue references on the upstream will make their way into the forked PR's commit history, if that user creates a merge commit.
Since GitHub autolinks issues and PRs on public forks, this will end up pinging the devs responsible for the referenced PRs creating unnecessary noise. The solution is to use a squashed merge.

One way to educate users with forked repos about this potential issue is to add instructions on how to do so to the `CONTRIBUTING.md` file.

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->
A PR with instructions to avoid this situation will be up shortly.

cc @stas00