Add DALL-E: Zero-Shot Text-to-Image Generation

# 🚀 Feature request

Please add DALLE-E model to huggingface's Transformers library.

1. [Announcement](https://openai.com/blog/dall-e/)
2. [Abstract](https://arxiv.org/abs/2102.12092v2)
3. [Paper](https://arxiv.org/pdf/2102.12092v2.pdf)
4. Code:
    - [openai/DALL-E](https://github.com/openai/DALL-E) (official)
    - [lucidrains/DALLE-pytorch](https://github.com/lucidrains/DALLE-pytorch) ([Colab](https://colab.research.google.com/drive/1dWvA54k4fH8zAmiix3VXbg95uEIMfqQM?usp=sharing))

## Motivation

> DALL·E is a 12-billion parameter version of [GPT-3](https://huggingface.co/transformers/model_doc/gpt.html) trained to generate images from text descriptions, using a dataset of text–image pairs
>
> We (Open AI) decided to name our model using a portmanteau of the artist Salvador Dalí and Pixar’s WALL·E.


Add `examples/multiple-choice/run_swag_no_trainer.py`

This PR adds an example of a multiple-choice task on the SWAG dataset to show the functionalities of the new `accelerate` library.

<hr>

**Reviewers:** @sgugger


Can't download the facebook/bart-large-mnli tensorflow model

Hello! When I try to create a pipeline with the model specified as "facebook/bart-large-mnli" I get a 404 Client Error: Not Found for url: https://huggingface.co/facebook/bart-large-mnli/resolve/main/tf_model.h5
And when I try to go directly to that url I do also notice it throws a 404 error. Any ideas on how to fix this would be greatly appreciated! Thanks!

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.2
- Python version: 3.8.6
- Tensorflow version: 2.2.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
Models:
blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj

The code I tried running is:
```python:
from transformers import pipeline
classifier = pipeline('zero-shot-classification', model='facebook/bart-large-mnli')
```

The full error message is:
```
404 Client Error: Not Found for url: https://huggingface.co/facebook/bart-large-mnli/resolve/main/tf_model.h5
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
~/Documents/github/econ2355/version2env/lib/python3.8/site-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1219                 # Load from URL or cache if already cached
-> 1220                 resolved_archive_file = cached_path(
   1221                     archive_file,

~/Documents/github/econ2355/version2env/lib/python3.8/site-packages/transformers/file_utils.py in cached_path(url_or_filename, cache_dir, force_download, proxies, resume_download, user_agent, extract_compressed_file, force_extract, use_auth_token, local_files_only)
   1133         # URL, so get it from the cache (downloading if necessary)
-> 1134         output_path = get_from_cache(
   1135             url_or_filename,

~/Documents/github/econ2355/version2env/lib/python3.8/site-packages/transformers/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, use_auth_token, local_files_only)
   1299             r = requests.head(url, headers=headers, allow_redirects=False, proxies=proxies, timeout=etag_timeout)
-> 1300             r.raise_for_status()
   1301             etag = r.headers.get("X-Linked-Etag") or r.headers.get("ETag")

~/Documents/github/econ2355/version2env/lib/python3.8/site-packages/requests/models.py in raise_for_status(self)
    942         if http_error_msg:
--> 943             raise HTTPError(http_error_msg, response=self)
    944

HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/facebook/bart-large-mnli/resolve/main/tf_model.h5

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-2-7aad78410119> in <module>
     14
     15 from transformers import pipeline
---> 16 classifier = pipeline('zero-shot-classification',
     17                       model='facebook/bart-large-mnli')
     18

~/Documents/github/econ2355/version2env/lib/python3.8/site-packages/transformers/pipelines/__init__.py in pipeline(task, model, config, tokenizer, framework, revision, use_fast, model_kwargs, **kwargs)
    342         model = get_default_model(targeted_task, framework, task_options)
    343
--> 344     framework = framework or get_framework(model)
    345
    346     task_class, model_class = targeted_task["impl"], targeted_task[framework]

~/Documents/github/econ2355/version2env/lib/python3.8/site-packages/transformers/pipelines/base.py in get_framework(model, revision)
     66             model = AutoModel.from_pretrained(model, revision=revision)
     67         elif is_tf_available() and not is_torch_available():
---> 68             model = TFAutoModel.from_pretrained(model, revision=revision)
     69         else:
     70             try:

~/Documents/github/econ2355/version2env/lib/python3.8/site-packages/transformers/models/auto/modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    616
    617         if type(config) in TF_MODEL_MAPPING.keys():
--> 618             return TF_MODEL_MAPPING[type(config)].from_pretrained(
    619                 pretrained_model_name_or_path, *model_args, config=config, **kwargs
    620             )

~/Documents/github/econ2355/version2env/lib/python3.8/site-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1234                     f"- or '{pretrained_model_name_or_path}' is the correct path to a directory containing a file named one of {TF2_WEIGHTS_NAME}, {WEIGHTS_NAME}.\n\n"
   1235                 )
-> 1236                 raise EnvironmentError(msg)
   1237             if resolved_archive_file == archive_file:
   1238                 logger.info("loading weights file {}".format(archive_file))

OSError: Can't load weights for 'facebook/bart-large-mnli'. Make sure that:

- 'facebook/bart-large-mnli' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'facebook/bart-large-mnli' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.
```


Updated colab links in readme of examples

# What does this PR do?
Updated the Google Colab links for the The Big Table of Tasks in the examples folder readme.
Google Colab links were replace to the appropriate examples found [here](https://github.com/huggingface/notebooks/tree/master/examples)

text_generation.ipynb is not present in the [notebook repo](https://github.com/huggingface/notebooks/tree/master/examples). Will text_generation.ipynb be added in the future?

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@sgugger


Another way to express masked_index = torch.nonzero(input_ids == self.tokenizer.mask_token_id, as_tuple=False)

Dear,

We use PyTorch 1.1.0 to run a masked word completion with BERT. However, we found an error
`TypeError: nonzero() got an unexpected keyword argument 'as_tuple'`

The error refers to this:
`masked_index = torch.nonzero(input_ids == self.tokenizer.mask_token_id, as_tuple=False)`

Is there another way to express the syntax above with keep using PyTorch 1.1.0?

Best regards,

Mohammad YANI


Error while predicting on single sentence for token classification task

Hi

I have fine-tuned BERT for NER task. I am predicting on the fine-tuned model as:

`output = self.tokenizer(text)
trainer = Trainer(model=self.model, tokenizer=self.tokenizer)
trainer.predict(output)`

This code snippet is throwing the following error:

  File "run_ner_test_3.py", line 486, in <module>
    obj.predict(text="i require to send 9330793.33 by account")
  File "run_ner_test_3.py", line 430, in predict
    trainer.predict(output)
  File "/home/dev01/python_3/lib/python3.6/site-packages/transformers/trainer.py", line 1596, in predict
    test_dataloader, description="Prediction", ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix
  File "/home/dev01/python_3/lib/python3.6/site-packages/transformers/trainer.py", line 1658, in prediction_loop
    for step, inputs in enumerate(dataloader):
  File "/home/dev01/python_3/lib64/python3.6/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/home/dev01/python_3/lib64/python3.6/site-packages/torch/utils/data/dataloader.py", line 475, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/dev01/python_3/lib64/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/dev01/python_3/lib64/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/dev01/python_3/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 305, in __getitem__
    return self._encodings[item]
IndexError: list index out of range

Can you please suggest how to predict on a single sentence?


Training with DeepSpeed takes more GPU memory than without DeepSpeed

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.5.0.dev0
- deepspeed version: 0.3.13
- Platform: Linux-4.15.0-66-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.8
- PyTorch version (GPU?): 1.8.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
 @stas00

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

I'm interested in training the large T5 models with deepspeed and huggingface. More specifically, I'm interested in fine-tuning a T5-11B model on one RTX-8000 48 GB GPU (similarly to https://huggingface.co/blog/zero-deepspeed-fairscale, https://github.com/huggingface/transformers/issues/9996).

However, when I try to use deepspeed the amount of memory on the GPU increases. For example, running the example seq2seq/run_summarization.py script with T5-Small and without deepspeed takes ~6GB, and running it with deepspeed takes ~8GB.

Model I am using: T5

The problem arises when using: The official  examples/seq2seq/run_summarization.py script.

Without deepspeed:
python examples/seq2seq/run_summarization.py --model_name_or_path t5-small --do_train  --do_eval --dataset_name cnn_dailymail --dataset_config "3.0.0"  --source_prefix "summarize: " --output_dir /tmp/tst-summarization --per_device_train_batch_size=4 --per_device_eval_batch_size=4 --overwrite_output_dir  --predict_with_genera

With deepspeed:
deepspeed examples/seq2seq/run_summarization.py --model_name_or_path t5-small --do_train  --do_eval --dataset_name cnn_dailymail --dataset_config "3.0.0"  --source_prefix "summarize: " --output_dir /tmp/tst-summarization --per_device_train_batch_size=4 --per_device_eval_batch_size=4 --overwrite_output_dir  --predict_with_generate --deepspeed examples/tests/deepspeed/ds_config.json

The tasks I am working on is:
Sequence to sequence generation.

## To reproduce

Steps to reproduce the behavior:

1. Clone transformers repo
2. Install requirements (including deepspeed: pip install deepspeed)
3. Run summarization example without deeepspeed:
python examples/seq2seq/run_summarization.py --model_name_or_path t5-small --do_train  --do_eval --dataset_name cnn_dailymail --dataset_config "3.0.0"  --source_prefix "summarize: " --output_dir /tmp/tst-summarization --per_device_train_batch_size=4 --per_device_eval_batch_size=4 --overwrite_output_dir  --predict_with_genera
4. Run summarization example with deepspeed:
deepspeed examples/seq2seq/run_summarization.py --model_name_or_path t5-small --do_train  --do_eval --dataset_name cnn_dailymail --dataset_config "3.0.0"  --source_prefix "summarize: " --output_dir /tmp/tst-summarization --per_device_train_batch_size=4 --per_device_eval_batch_size=4 --overwrite_output_dir  --predict_with_generate --deepspeed examples/tests/deepspeed/ds_config.json

## Expected behavior

I would expect using deepspeed would reduce the amount of memory being used by the GPU.


Add example for registering callbacks with trainers

# What does this PR do?

Fixes the issue addressed in #9036 by adding an example for registering a custom callback with the Trainer.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Fixes: https://github.com/huggingface/transformers/issues/9036

## Who can review?

Anyone in the community is free to review the PR. But @sgugger seems the most appropriate.


Add Pooler to DistilBERT

# 🚀 Feature request
Hi, I'd like to add a Pooler class to the DistilBERT model, whose interface is similar to BertPooler [here](https://github.com/huggingface/transformers/blob/7da995c00c025c4180c7fb0357256b7f83d342ef/src/transformers/models/bert/modeling_bert.py#L610)

## Motivation
I was using your DistilBERT model and discovered that I needed a pooler, so I wrote my own class. Thought I would add it to the repo in case others would like it.

## Your contribution
If this is something you're interested in, I can submit a PR
<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Typo in examples/text-classification README

In the examples/text-classification README, the example scripts for "PyTorch version, no Trainer" are slightly incorrect. They should be adjusted as:
```diff
export TASK_NAME=mrpc

python run_glue_no_trainer.py \
  --model_name_or_path bert-base-cased \
  --task_name $TASK_NAME \
-   --max_seq_length 128 \
+   --max_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir /tmp/$TASK_NAME/
```
Thanks for your great repo!
Luke


Sagemaker test

# What does this PR do?

This PR creates tests for `SageMaker`. This PR creates tests for the `Pytorch` and `tensorflow` DLC. I added a documentation `README.md` which explains when the Tests need to be run and how. Currently, not all tests are leveraging our `examples/` due to limitations like `SageMakerTrainer` not being integrated into `Trainer` and missing implementation for `keras` for SageMaker specific libraries for data/model parallel. In the near future all scripts in `tests/sagemaker/scripts` are going to be removed and copy the scripts from `examples/` before executing the tests.

## Current Tests

| ID                                  | description                                                       | plattform                   | #GPUS | collected & evaluated metrics            |
|-------------------------------------|-------------------------------------------------------------------|-----------------------------|-------|------------------------------------------|
| pytorch-transfromers-test-single    | test bert finetuning using BERT fromtransformerlib+PT             | SageMaker createTrainingJob | 1     | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-2-ddp     | test bert finetuning using BERT from transformer lib+ PT DPP      | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-2-smd     | test bert finetuning using BERT from transformer lib+ PT SM DDP   | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-1-smp     | test roberta finetuning using BERT from transformer lib+ PT SM MP | SageMaker createTrainingJob | 8     | train_runtime, eval_accuracy & eval_loss |
| tensorflow-transfromers-test-single | Test bert finetuning using BERT from transformer lib+TF           | SageMaker createTrainingJob | 1     | train_runtime, eval_accuracy & eval_loss |
| tensorflow-transfromers-test-2-smd  | test bert finetuning using BERT from transformer lib+ TF SM DDP   | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |


Models not able to run when packed with PyInstaller

## Environment info
- `transformers` version: 4.1.1
- Platform: Linux-5.8.0-48-generic-x86_64-with-glibc2.27
- Python version: 3.8.8
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
-

## Information
I am trying to create an executable for a flask application that uses [haystack](https://github.com/deepset-ai/haystack/) to serve a QA System. Haystack uses transformers.
If I run my API normally with python with `python api.py` it works fine.
When I run `pyinstaller main.spec --distpath distAPI` the executable gets created fine (note that I will post main.spec down in the **To reproduce** section). However, when I run it with `./distAPI/main/main` I get the following error:
```
03/26/2021 16:45:35 - INFO - faiss -   Loading faiss with AVX2 support.
03/26/2021 16:45:35 - INFO - faiss -   Loading faiss.
Traceback (most recent call last):
  File "torch/_utils_internal.py", line 49, in get_source_lines_and_file
  File "inspect.py", line 979, in getsourcelines
  File "inspect.py", line 798, in findsource
OSError: could not get source code

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "main.py", line 8, in <module>
    from haystack.preprocessor.cleaning import clean_wiki_text
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/__init__.py", line 5, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/finder.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/retriever/base.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/document_store/base.py", line 6, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "haystack/preprocessor/utils.py", line 11, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "farm/data_handler/utils.py", line 18, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "farm/file_utils.py", line 26, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/__init__.py", line 91, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/modelcard.py", line 31, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/auto/__init__.py", line 20, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/auto/configuration_auto.py", line 28, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/deberta/__init__.py", line 25, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "transformers/models/deberta/modeling_deberta.py", line 462, in <module>
  File "torch/jit/_script.py", line 936, in script
  File "torch/jit/frontend.py", line 197, in get_jit_def
  File "torch/_utils_internal.py", line 56, in get_source_lines_and_file
OSError: Can't get source for <function c2p_dynamic_expand at 0x7f45cc4d85e0>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available.
[104072] Failed to execute script main

```
It seems that it cannot get the source code for the `function c2p_dynamic_expand` in `transformers/models/deberta/modeling_deberta.py`.

## Additional information
This is a problem that already happened in the past when using Pyinstaller and Torch.
See this issue [here](https://github.com/pyinstaller/pyinstaller/issues/4926) for example.

## To reproduce

Steps to reproduce the behavior:

1.Make a main.py :
```
from haystack.preprocessor.cleaning import clean_wiki_text

if __name__=='__main__':
    print('Hello World')

```
2. Install haystack 'pip install haystack`
3. See if it runs with `python main.py' (it should)
4. Install pyinstaller with `pip install pyinstaller`
5. Create a hooks/ folder containing the following files:
**hook-justext.py hook-packaging.py hook-requests.py hook-tokenizers.py hook-tqdm.py hook-transformers.py hook-filelock.py hook-numpy.py hook-regex.py hook-sacremoses.py hook-torch.py**
Each of these files should provide a hook for pyinstaller to to download that module. So, for example the hook-numpy.py file should be:
```
from PyInstaller.utils.hooks import collect_all

datas, binaries, hiddenimports = collect_all('numpy')
```
And so the rest of them.
6. create a main.spec file:
```
# -*- mode: python ; coding: utf-8 -*-
block_cipher = None
a = Analysis(['main.py'],
             pathex=['/Wexond/QandA/api'],
             binaries=[],
             datas=[],
             hiddenimports=['justext'],
             hookspath=['./hooks/'],    ## <-------------- Specifying the hooks
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data, source_files_toc,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='main',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               upx_exclude=[],
               name='main')
```
7. Pack the app with `pyinstaller main.spec --distpath distAPI`
8. Try to run it with `./distAPI/main/main` .

You should now get the `OsError` metioned above..


## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.2
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.0+cu101 (False)
- Tensorflow version (GPU?):
- Using GPU in script?: TPU
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

@patrickvonplaten @sgugger

## Information

I am using LongformerForSequenceClassification and LongformerTokenizerFast for a simple text classification problem on Google Colab TPU:

The problem arises when using:
* [ ] my own modified scripts: (Script shared) If I replace the LongformerForSequenceClassification model with the DistilBertForSequenceClassification model, the same code works perfectly fine and the training starts without any issues. However, with LongformerForSequenceClassification, I start getting weird errors on TPU.

```
from pathlib import Path

def read_imdb_split(split_dir):
    split_dir = Path(split_dir)
    texts = []
    labels = []
    for label_dir in ["pos", "neg"]:
        for text_file in (split_dir/label_dir).iterdir():
            texts.append(text_file.read_text())
            labels.append(0 if label_dir is "neg" else 1)

    return texts, labels

train_texts, train_labels = read_imdb_split('aclImdb/train')
test_texts, test_labels = read_imdb_split('aclImdb/test')

from sklearn.model_selection import train_test_split
train_texts, val_texts, train_labels, val_labels = train_test_split(train_texts, train_labels, test_size=.2)

from transformers import DistilBertTokenizerFast, LongformerTokenizerFast
# tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-uncased')
tokenizer = LongformerTokenizerFast.from_pretrained('allenai/longformer-base-4096', max_length = 8)

train_encodings = tokenizer(train_texts, truncation=True, padding=True)
val_encodings = tokenizer(val_texts, truncation=True, padding=True)
test_encodings = tokenizer(test_texts, truncation=True, padding=True)

import torch

class IMDbDataset(torch.utils.data.Dataset):
    def __init__(self, encodings, labels):
        self.encodings = encodings
        self.labels = labels

    def __getitem__(self, idx):
        item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
        item['labels'] = torch.tensor(self.labels[idx])
        return item

    def __len__(self):
        return len(self.labels)

train_dataset = IMDbDataset(train_encodings, train_labels)
val_dataset = IMDbDataset(val_encodings, val_labels)
test_dataset = IMDbDataset(test_encodings, test_labels)

from transformers import DistilBertForSequenceClassification, Trainer, TrainingArguments, LongformerForSequenceClassification
import torch_xla.distributed.xla_multiprocessing as xmp
import torch_xla.core.xla_model as xm

def _mp_fn(index):
  training_args = TrainingArguments(
      output_dir='./results',          # output directory
      num_train_epochs=3,              # total number of training epochs
      per_device_train_batch_size=16,  # batch size per device during training
      per_device_eval_batch_size=64,   # batch size for evaluation
      warmup_steps=500,                # number of warmup steps for learning rate scheduler
      weight_decay=0.01,               # strength of weight decay
      logging_dir='./logs',            # directory for storing logs
      logging_steps=10,
  )

  # model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased')
  model = LongformerForSequenceClassification.from_pretrained("allenai/longformer-base-4096", attention_window = 2)

  trainer = Trainer(
      model=model,                         # the instantiated 🤗 Transformers model to be trained
      args=training_args,                  # training arguments, defined above
      train_dataset=train_dataset,         # training dataset
      eval_dataset=val_dataset             # evaluation dataset
  )

  trainer.train()

xmp.spawn(_mp_fn, args=(), nprocs=1, start_method='fork')
```

The tasks I am working on is:
* [ ] my own task or dataset: Using the IMDB Dataset for Text Classification

## To reproduce

Steps to reproduce the behavior:

1. Setup TPU-client on google Colab: !pip install cloud-tpu-client==0.10 https://storage.googleapis.com/tpu-pytorch/wheels/torch_xla-1.8-cp37-cp37m-linux_x86_64.whl
2. Download the dataset:
    a. !wget http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz
    b. !tar -xf aclImdb_v1.tar.gz
3. Execute the given script

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

```
RuntimeError: /pytorch/xla/torch_xla/csrc/helpers.h:100 : Check failed: scalar_value.isIntegral()
*** Begin stack trace ***
	tensorflow::CurrentStackTrace()
	torch_xla::XlaHelpers::ScalarValue(c10::Scalar, xla::PrimitiveType, xla::XlaBuilder*)

	torch_xla::ir::ops::InferOutputShape(absl::lts_2020_02_25::Span<xla::Shape const>, std::function<xla::XlaOp (absl::lts_2020_02_25::Span<xla::XlaOp const>)> const&)

	torch_xla::ir::Node::GetOpShape(std::function<xla::Shape ()> const&) const
	torch_xla::ir::Node::Node(torch_xla::ir::OpKind, absl::lts_2020_02_25::Span<torch_xla::ir::Value const>, std::function<xla::Shape ()> const&, unsigned long, absl::lts_2020_02_25::uint128)
	torch_xla::ir::ops::ConstantPadNd::ConstantPadNd(torch_xla::ir::Value const&, std::vector<long, std::allocator<long> >, c10::Scalar)
	void __gnu_cxx::new_allocator<torch_xla::ir::ops::ConstantPadNd>::construct<torch_xla::ir::ops::ConstantPadNd, torch_xla::ir::Value, std::vector<long, std::allocator<long> >&, c10::Scalar&>(torch_xla::ir::ops::ConstantPadNd*, torch_xla::ir::Value&&, std::vector<long, std::allocator<long> >&, c10::Scalar&)
	torch_xla::XLATensor::constant_pad_nd(torch_xla::XLATensor const&, absl::lts_2020_02_25::Span<long const>, c10::Scalar)
	torch_xla::AtenXlaType::constant_pad_nd(at::Tensor const&, c10::ArrayRef<long>, c10::Scalar)
	c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoRuntimeFunctor_<at::Tensor (*)(at::Tensor const&, c10::ArrayRef<long>, c10::Scalar), at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::ArrayRef<long>, c10::Scalar> >, at::Tensor (at::Tensor const&, c10::ArrayRef<long>, c10::Scalar)>::call(c10::OperatorKernel*, at::Tensor const&, c10::ArrayRef<long>, c10::Scalar)
	at::constant_pad_nd(at::Tensor const&, c10::ArrayRef<long>, c10::Scalar)

	at::constant_pad_nd(at::Tensor const&, c10::ArrayRef<long>, c10::Scalar)

	_PyMethodDef_RawFastCallKeywords
	_PyCFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyObject_Call_Prepend
	PyObject_Call
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyObject_Call_Prepend

	_PyObject_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyObject_Call_Prepend
	PyObject_Call
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyObject_Call_Prepend

	PyObject_Call
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallDict
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	PyEval_EvalCode

	_PyMethodDef_RawFastCallKeywords
	_PyCFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyObject_Call_Prepend
	PyObject_Call
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallDict
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallDict
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallDict
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyObject_FastCallDict

	_PyObject_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyObject_Call_Prepend
	_PyObject_FastCallKeywords

	_PyMethodDef_RawFastCallDict
	PyCFunction_Call
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	_PyFunction_FastCallKeywords
	_PyEval_EvalFrameDefault
	_PyEval_EvalCodeWithName
	PyEval_EvalCode

*** End stack trace ***
Scalar type not supported
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
Model training should have started but instead got the error


Use reformer in down stream task meet problem

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:4.2.2
- Platform:CentOS
- Python version:3.7
- PyTorch version (GPU?):1.5.1 cpu only
- Tensorflow version (GPU?):
- Using GPU in script?:no
- Using distributed or parallel set-up in script?:no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts:
### sequence classification task under glue
- bug
```
Traceback (most recent call last):
  File "examples/text-classification/run_glue.py", line 584, in <module>
    main()
  File "examples/text-classification/run_glue.py", line 410, in main
    datasets = datasets.map(preprocess_function, batched=True, load_from_cache_file=not data_args.overwrite_cache)
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/datasets/dataset_dict.py", line 386, in map
    for k, dataset in self.items()
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/datasets/dataset_dict.py", line 386, in <dictcomp>
    for k, dataset in self.items()
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1120, in map
    update_data = does_function_return_dict(test_inputs, test_indices)
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1091, in does_function_return_dict
    function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
  File "examples/text-classification/run_glue.py", line 403, in preprocess_function
    result = tokenizer(*args, padding=padding, max_length=data_args.max_seq_length, truncation=True)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/tokenization_utils_base.py", line 2335, in __call__
    **kwargs,
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/tokenization_utils_base.py", line 2500, in batch_encode_plus
    **kwargs,
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/tokenization_utils_base.py", line 2217, in _get_padding_truncation_strategies
    "Asking to pad but the tokenizer does not have a padding token. "
ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as `pad_token` `(tokenizer.pad_token = tokenizer.eos_token e.g.)` or add a new pad token via `tokenizer.add_special_tokens({'pad_token': '[PAD]'})`.
```
- shell code
```python
python examples/text-classification/run_glue.py --model_type reformer --model_name_or_path google/reformer-crime-and-punishment --task_name $TASK_NAME --do_train --do_eval --max_seq_length 512 --per_gpu_eval_batch_size=32 --per_gpu_train_batch_size=32 --learning_rate 2e-5 --num_train_epochs 3.0 --output_dir /home2/zhenggo1/checkpoint/reformer_mrpc
```
### translation task under wmt_en_ro
- bug
```
Traceback (most recent call last):
  File "examples/seq2seq/finetune_trainer.py", line 451, in <module>
    main()
  File "examples/seq2seq/finetune_trainer.py", line 215, in main
    cache_dir=model_args.cache_dir,
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/auto/modeling_auto.py", line 1226, in from_pretrained
    ", ".join(c.__name__ for c in MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING.keys()),
ValueError: Unrecognized configuration class <class 'transformers.models.reformer.configuration_reformer.ReformerConfig'> for this kind of AutoModel: AutoModelForSeq2SeqLM.
Model type should be one of LEDConfig, BlenderbotSmallConfig, MT5Config, T5Config, PegasusConfig, MarianConfig, MBartConfig, BlenderbotConfig, BartConfig, FSMTConfig, EncoderDecoderConfig, XLMProphetNetConfig, ProphetNetConfig.
```
- shell code
```python
python examples/seq2seq/finetune_trainer.py --model_name_or_path google/reformer-crime-and-punishment --do_train --do_eval --task translation_en_to_ro --data_dir examples/seq2seq/test_data/wmt_en_ro/ --output_dir /home2/zhenggo1/checkpoint/reformer_translation --per_device_train_batch_size=4 --per_device_eval_batch_size=4 --overwrite_output_dir --predict_with_generate
```
### clm task under wikitext
- bug
```
Traceback (most recent call last):
  File "examples/language-modeling/run_clm.py", line 472, in <module>
    main()
  File "examples/language-modeling/run_clm.py", line 365, in main
    train_result = trainer.train(model_path=model_path)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/trainer.py", line 888, in train
    tr_loss += self.training_step(model, inputs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/trainer.py", line 1250, in training_step
    loss = self.compute_loss(model, inputs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/trainer.py", line 1277, in compute_loss
    outputs = model(**inputs)
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/reformer/modeling_reformer.py", line 2244, in forward
    return_dict=return_dict,
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/reformer/modeling_reformer.py", line 2090, in forward
    start_idx_pos_encodings=start_idx_pos_encodings,
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/reformer/modeling_reformer.py", line 264, in forward
    position_embeddings = self.position_embeddings(position_ids)
  File "/home2/zhenggo1/anaconda3/envs/lpot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/zhenggo1/LowPrecisionInferenceTool/examples/pytorch/huggingface_transformers/src/transformers/models/reformer/modeling_reformer.py", line 158, in forward
    self.axial_pos_shape, self.axial_pos_shape, sequence_length, reduce(mul, self.axial_pos_shape)
AssertionError: If training, make sure that config.axial_pos_shape factors: (512, 1024) multiply to sequence length. Got prod((512, 1024)) != sequence_length: 1024. You might want to consider padding your sequence length to 524288 or changing config.axial_pos_shape.
```
- shell code
```python
python examples/language-modeling/run_clm.py --model_name_or_path google/reformer-crime-and-punishment --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 --do_train --do_eval --output_dir /home2/zhenggo1/checkpoint/reformer_clm
```
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. run shell code as shown as above(translation dataset may not use the local)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

I just need a task to do evaluation and can compute a metrics.
Thks a lot if you can help me to give just a task that I can do evaluation!!!
<!-- A clear and concise description of what you would expect to happen. -->


Tokenizer is adding ## to every word from the second.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.2
- Platform: Linux-5.8.0-44-generic-x86_64-with-Ubuntu-20.04-focal
- Python version: 3.6.13
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help

 tokenizers: @LysandreJik

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ O ] the official example scripts: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. The tokenizer is adding ## to every words from the second.

For example, the code is:
text = 'テレビでサッカーの試合を見る。'
tokenized_text = tokenizer.tokenize(text)

Output is expected to be:['テレビ', 'で', 'サッカー', 'の', '試合', 'を', '見る', '。']
But I get  ['テレビ', '##で', '##サッカー', '##の', '##試', '##合', '##を', '##見', '##る', '。']

I dont know why it add ## at the start of the words...

```
import torch
from transformers import BertJapaneseTokenizer, BertForMaskedLM

# Model path
def sel_model(pre_model='32d'):
    if pre_model == '32d':
        sel = '/home/Xu_Zhenyu/cl-tohoku/BERT-base_mecab-ipadic-bpe-32k_do-whole-word-mask/'
    elif pre_model == '4d':
        sel = '/home/Xu_Zhenyu/cl-tohoku/BERT-base_mecab-ipadic-char-4k_do-whole-word-mask/'
    elif pre_model == '32n':
        sel = '/home/Xu_Zhenyu/cl-tohoku/BERT-base_mecab-ipadic-bpe-32k_no-whole-word-mask/'
    elif pre_model == '4n':
        sel = '/home/Xu_Zhenyu/cl-tohoku/BERT-base_mecab-ipadic-char-4k_no-whole-word-mask/'
    else:
        sel = '/home/Xu_Zhenyu/cl-tohoku/BERT-base_mecab-ipadic-bpe-32k_do-whole-word-mask/'
    return sel

# Load pre-trained tokenizer
tokenizer = BertJapaneseTokenizer.from_pretrained(sel_model())

# Tokenize input
text = 'テレビでサッカーの試合を見る。'
tokenized_text = tokenizer.tokenize(text)
# ['テレビ', 'で', 'サッカー', 'の', '試合', 'を', '見る', '。']

# Mask a token that we will try to predict back with `BertForMaskedLM`
masked_index = 2
tokenized_text[masked_index] = '[MASK]'
print(tokenized_text)
# ['テレビ', 'で', '[MASK]', 'の', '試合', 'を', '見る', '。']

# Convert token to vocabulary indices
indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_text)
# [571, 12, 4, 5, 608, 11, 2867, 8]

# Convert inputs to PyTorch tensors
tokens_tensor = torch.tensor([indexed_tokens])
# tensor([[ 571,   12,    4,    5,  608,   11, 2867,    8]])

# Load pre-trained model
model = BertForMaskedLM.from_pretrained(sel_model())
model.eval()

# Predict
with torch.no_grad():
    outputs = model(tokens_tensor)
    predictions = outputs[0][0, masked_index].topk(5)  # 予測結果の上位5件を抽出

# Show results
for i, index_t in enumerate(predictions.indices):
    index = index_t.item()
    token = tokenizer.convert_ids_to_tokens([index])[0]
    print(i, token)

```


Rename NLP library to Datasets library

# What does this PR do?

Fixes #10897

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.


GPT2 on TPU, training is so slow.

When training GPT2 on TPU from scratch, training loss is constant & evaluation loss is decreasing very small amount.

> [INFO|trainer.py:1776] 2021-03-26 04:06:22,551 >>   Num examples = 100000
> [INFO|trainer.py:1777] 2021-03-26 04:06:22,551 >>   Batch size = 2
> {'eval_loss': 4.687133312225342, 'eval_runtime': 736.3302, 'eval_samples_per_second': 135.809, 'epoch': 0.05}
>   1%|#                                                                                | 22000/2080235 [22:38:08<2499:54:52,  4.37s/it]                                                [INFO|trainer.py:1528] 2021-03-26 04:18:38,885 >> Saving model checkpoint to outputs/line_by_line/checkpoint-22000
> [INFO|configuration_utils.py:314] 2021-03-26 04:18:38,912 >> Configuration saved in outputs/line_by_line/checkpoint-22000/config.json
> [INFO|modeling_utils.py:837] 2021-03-26 04:18:56,125 >> Model weights saved in outputs/line_by_line/checkpoint-22000/pytorch_model.bin
> [INFO|tokenization_utils_base.py:1896] 2021-03-26 04:18:56,130 >> tokenizer config file saved in outputs/line_by_line/checkpoint-22000/tokenizer_config.json
> [INFO|tokenization_utils_base.py:1902] 2021-03-26 04:18:56,131 >> Special tokens file saved in outputs/line_by_line/checkpoint-22000/special_tokens_map.json
> {'loss': 2.56, 'learning_rate': 0.0004963706023598295, 'epoch': 0.05}
> {'loss': 2.56, 'learning_rate': 0.0004963465666138682, 'epoch': 0.05}
> {'loss': 2.56, 'learning_rate': 0.0004963225308679067, 'epoch': 0.05}
> {'loss': 2.56, 'learning_rate': 0.0004962984951219453, 'epoch': 0.05}
> {'loss': 2.56, 'learning_rate': 0.000496274459375984, 'epoch': 0.05}
> {'loss': 2.56, 'learning_rate': 0.0004962504236300226, 'epoch': 0.05}
> {'loss': 2.56, 'learning_rate': 0.0004962263878840611, 'epoch': 0.05}
> {'loss': 2.56, 'learning_rate': 0.0004962023521380998, 'epoch': 0.05}
> {'loss': 2.56, 'learning_rate': 0.0004961783163921384, 'epoch': 0.06}
> {'loss': 2.56, 'learning_rate': 0.000496154280646177, 'epoch': 0.06}
>   1%|#1                                                                                | 23000/2080235 [23:52:03<2524:57:42,  4.42s/it][INFO|trainer.py:1775] 2021-03-26 05:32:34,207 >> ***** Running Evaluation *****
> [INFO|trainer.py:1776] 2021-03-26 05:32:34,317 >>   Num examples = 100000
> [INFO|trainer.py:1777] 2021-03-26 05:32:34,317 >>   Batch size = 2
> {'eval_loss': 4.667241096496582, 'eval_runtime': 739.6907, 'eval_samples_per_second': 135.192, 'epoch': 0.06}`


## Environment info

- `transformers` version: 4.4.2
- Python version: 3.6
- PyTorch version (GPU?): 1.8.0 (Tesla V100)

## Information

The problem arises when using:
```
from transformers import BertModel
model = BertModel.from_pretrained('bert-base-uncased')
```

Error Info (Some personal info has been replaced by ---)
```
file bert-base-uncased/config.json not found
Traceback (most recent call last):
  File "---/anaconda3/envs/attn/lib/python3.6/site-packages/transformers-4.2.2-py3.8.egg/transformers/configuration_utils.py", line 420, in get_config_dict
  File "---/anaconda3/envs/attn/lib/python3.6/site-packages/transformers-4.2.2-py3.8.egg/transformers/file_utils.py", line 1063, in cached_path
OSError: file bert-base-uncased/config.json not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "---.py", line 107, in <module>
    from_pretrained_input()
  File "---.py", line 96, in from_pretrained_input
    model = BertModel.from_pretrained('bert-base-uncased')
  File "---/anaconda3/envs/attn/lib/python3.6/site-packages/transformers-4.2.2-py3.8.egg/transformers/modeling_utils.py", line 962, in from_pretrained
  File "---/anaconda3/envs/attn/lib/python3.6/site-packages/transformers-4.2.2-py3.8.egg/transformers/configuration_utils.py", line 372, in from_pretrained
  File "---/anaconda3/envs/attn/lib/python3.6/site-packages/transformers-4.2.2-py3.8.egg/transformers/configuration_utils.py", line 432, in get_config_dict
OSError: Can't load config for 'bert-base-uncased'. Make sure that:

- 'bert-base-uncased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'bert-base-uncased' is the correct path to a directory containing a config.json file
```

#### what I have read:

https://github.com/huggingface/transformers/issues/353

#### what I have tried:

1. loading from a downloaded model file works well

```
wget https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz
```
unzip the file and rename ```bert_config.json``` as ```config.json```, then
```
model = BertModel.from_pretrained(BERT_BASE_UNCASED_CACHE)
```

2. enough disk space, enough memory, free GPU

3. open internet connection, no proxy

4.
```
import pytorch_pretrained_bert as ppb
assert 'bert-large-cased' in ppb.modeling.PRETRAINED_MODEL_ARCHIVE_MAP
```

5. The following models work well
```
model = BertModel.from_pretrained('bert-base-cased')

model = RobertaModel.from_pretrained('roberta-base')
```

6. working well in server cmd but not in local pycharm (remote deployment to server)

Observation:
- Pycharm can found the ```transfromers``` installed with pip, but that will trigger this problem
- Pycharm cannot find the current  ```transformers```  installed with conda
```conda install transformers=4.4 -n env -c huggingface```


longformer speed compared to bert model

We are trying to use a LongFormer and Bert model for multi-label classification of different documents.

When we use the BERT model (BertForSequenceClassification) with max length 512 (batch size 8) each epoch takes approximately 30 minutes.

When we use LongFormer (LongformerForSequenceClassification with the 'allenai/longformer-base-4096' and gradient_checkpointing=True) with max length 4096 (batch size 1, Gradient Accumulation step 8) each epoch takes approximately 12 hours.

Is this reasonable or are we missing something?
Is there anything that we can try to make the training faster?


I try to fine tune the distilbert-base-uncased on my own dataset, which are csv made up of each news line by line.
here is my command:

```
nohup python run_mlm.py \
    --model_name_or_path distilbert-base-uncased \
    --train_file df_finetune_train.csv \
    --validation_file df_finetune_test.csv  \
    --do_train \
    --do_eval \
    --preprocessing_num_workers 72 \
    --output_dir ./finetuned_bert \
    --overwrite_cache True \
    --max_seq_length 256 \
    --line_by_line True > log_fintune_mlm &
```
Here is the error.

> {'loss': 1.7847, 'learning_rate': 3.264263411864888e-07, 'epoch': 2.98}
> {'loss': 1.7906, 'learning_rate': 1.7858832434478192e-07, 'epoch': 2.99}
> {'loss': 1.7839, 'learning_rate': 3.075030750307503e-08, 'epoch': 3.0}
> {'train_runtime': 65966.5445, 'train_samples_per_second': 2.563, 'epoch': 3.0}
> Traceback (most recent call last):
>   File "run_mlm.py", line 487, in <module>
>     main()
>   File "run_mlm.py", line 462, in main
>     trainer.log_metrics("train", metrics)
> AttributeError: 'Trainer' object has no attribute 'log_metrics'

transformers version: 4.3.3
torch version: 1.5.0+cu101


Bump pyyaml from 5.3.1 to 5.4 in /examples/research_projects/lxmert

dependencies

Pull requests that update a dependency file

Bumps [pyyaml](https://github.com/yaml/pyyaml) from 5.3.1 to 5.4.

Changelog

Sourced from pyyaml's changelog.

5.4 (2021-01-19)

yaml/pyyaml#407 -- Build modernization, remove distutils, fix metadata, build wheels, CI to GHA

yaml/pyyaml#472 -- Fix for CVE-2020-14343, moves arbitrary python tags to UnsafeLoader

yaml/pyyaml#441 -- Fix memory leak in implicit resolver setup

yaml/pyyaml#392 -- Fix py2 copy support for timezone objects

yaml/pyyaml#378 -- Fix compatibility with Jython

Commits

58d0cb7 5.4 release

a60f7a1 Fix compatibility with Jython

ee98abd Run CI on PR base branch changes

ddf2033 constructor.timezone: _copy & deepcopy

fc914d5 Avoid repeatedly appending to yaml_implicit_resolvers

a001f27 Fix for CVE-2020-14343

fe15062 Add 3.9 to appveyor file for completeness sake

1e1c7fb Add a newline character to end of pyproject.toml

0b6b7d6 Start sentences and phrases for capital letters

c976915 Shell code improvements

Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pyyaml&package-manager=pip&previous-version=5.3.1&new-version=5.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/huggingface/transformers/network/alerts).


[vulnerability] fix dependency

this PR fixes https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/PyYAML/open

@LysandreJik


pegasus xsum won't train on xsum dataset

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 5.0dev
- Platform: linux
- Python version: 3.6.9
- PyTorch version (GPU?): pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
- Tensorflow version (GPU?): 2.4.1
- Using GPU in script?: Both -
- Using distributed or parallel set-up in script?: no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
@sgugger  @patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): Pegasus XSUM

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

I am using run_summarization.py to retrain a fine-tuned model (before I try it on my own data).  I first fine-tuned on gigaword for a few thousand iterations, tested it on gigaword, then switched to evaluate on the xsum dataset.  The xsum eval dataset produces the following error on the CPU (similar error on GPU, just with a lot of extra fluff)

```
 File "run_summarization.py", line 593, in <module>
    main()
  File "run_summarization.py", line 550, in main
    max_length=data_args.val_max_target_length, num_beams=data_args.num_beams, metric_key_prefix="eval"
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/transformers/trainer_seq2seq.py", line 74, in evaluate
    return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/transformers/trainer.py", line 1707, in evaluate
    metric_key_prefix=metric_key_prefix,
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/transformers/trainer.py", line 1838, in prediction_loop
    loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/transformers/trainer_seq2seq.py", line 167, in prediction_step
    **gen_kwargs,
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/transformers/generation_utils.py", line 927, in generate
    model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(input_ids, model_kwargs)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/transformers/generation_utils.py", line 412, in _prepare_encoder_decoder_kwargs_for_generation
    model_kwargs["encoder_outputs"]: ModelOutput = encoder(input_ids, return_dict=True, **encoder_kwargs)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 725, in forward
    embed_pos = self.embed_positions(input_shape)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 139, in forward
    return super().forward(positions)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 147, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/disk1/work/marc/pegasus/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 1913, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Summarization length not controlled by max_length, min_length

I am using the pertained ctrlsum-cnndm model from transformers. I noticed that summarization text length is not exactly controlled by max_length, min_length arguments of model.generate(). Not sure why. It appears that empty spaces are included, but not sure. Please help. Thanks.

```
text1="The tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building, and the tallest structure in Paris. Its base is square, measuring 125 metres (410 ft) on each side. During its construction, the Eiffel Tower surpassed the Washington Monument to become the tallest man-made structure in the world, a title it held for 41 years until the Chrysler Building in New York City was finished in 1930. It was the first structure to reach a height of 300 metres. Due to the addition of a broadcasting aerial at the top of the tower in 1957, it is now taller than the Chrysler Building by 5.2 metres (17 ft). Excluding transmitters, the Eiffel Tower is the second tallest free-standing structure in France after the Millau Viaduct."

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("hyunwoongko/ctrlsum-cnndm")
model = AutoModelForSeq2SeqLM.from_pretrained("hyunwoongko/ctrlsum-cnndm")

inputs = tokenizer.encode(text1, return_tensors="pt", max_length=1024)#16
outputs = model.generate(inputs, max_length=100, min_length=50, num_beams=5, early_stopping=True)
print(tokenizer.decode(outputs[0]))
```

Results:
max_length=100, min_length=50, actually 36 words
`</s> The Eiffel Tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building. It is the tallest structure in Paris and the second tallest free-standing structure in France after the Millau Viaduct.</s>
`

max_length=200, min_length=100, actually 83 words
`</s> The Eiffel Tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building, and the tallest structure in Paris. It was the tallest man-made structure in the world for 41 years until the Chrysler Building in New York City was finished in 1930. It is the second tallest free-standing structure in France after the Millau Viaduct, which measures 125 metres (410 ft) on each side. The tower is now taller than the Chrysler building by 5.2 metres (17 ft)</s>
`


Add nvidia megatron models

# What does this PR do?

Add the megatron_gpt2 model. That model reuses the existing GPT2 model. This
commit includes a script to convert a Megatron-GPT2 checkpoint downloaded
from NVIDIA GPU Cloud. See examples/megatron-models/README.md for details.

Add the megatron_bert model. That model is implemented as a modification of
the existing BERT model in Transformers. This commit includes a script to
convert a Megatron-BERT checkpoint downloaded from NVIDIA GPU Cloud. See
examples/megatron-models/README.md for details.

@LysandreJik


Wav2Vec2 CommonVoice training - Save the processor before training starts

# What does this PR do?

Currently, the Wav2Vec2 processor is saved at the end of training. However, the vocabulary is non-deterministic and varies between runs. Thus, if the training is killed before it's done, the processor is not saved, meaning that the checkpoints do not contain the processor configuration files, making them unusable for resuming training or for evaluating on the checkpoint. Hence, this PR saves the processor before the training begins.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@patrickvonplaten @patil-suraj


LengthGroupedSampler slowly iterates over dataset

https://github.com/huggingface/transformers/blob/86c6f8a8b1f2bfc6c6d175590efc95a5e6facb51/src/transformers/trainer_pt_utils.py#L506

When using training arg "group_by_length", the sampler has to get the length of every input. it does so by iterating over the dataset in a simple for loop, without the use of a dataloader.

In my case, this led to long delays before the training starts, as reading the dataset without the worker parallelization from a dataloader took a long time.

I believe this can potentially improved, although one could also precompute the lengths and pass them to the sampler (but this can only happen if the user is aware of it being a potential bottleneck).
Maybe it is a good idea to add a dataloader + num_workers option here?

Thanks to Pere-Lluís and Pedro for being my co-detectives in spotting this caveat.


Improve the documentation for TrainingArguments.label_names, and if possible raise an error if users misinterpret this attribute like I did

### Original Issue Title: Possible typo in trainer.py: prediction_step(), forgetting to exclude loss item of outputs dict when assigning logits

_**Update**: I determined the root cause of my error to stem from an incorrect assignment of `TrainingArgument.label_names`. **There is not a typo** in `Trainer.prediction_step()`, as I've suggested below. However there is still an issue: see my comment for elaboration._

I was using the `Trainer` trying to fine-tune [KB-Bert-Base-Swedish-Cased](https://huggingface.co/KB/bert-base-swedish-cased) for multi-class SequenceClassification, when I got a `IndexError: tuple index out of range` during the evaluation stage (I set up `Trainer` to evaluate after each Epoch).

I started PDB and paused at this line in the evaluation phase:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1805

With the debugger, I saw that `loss=None`, `labels=None`, and `logits` is actually `tuple` with two items. The first item is the prediction loss as, and the second element is the actual output logits from the models forward pass.

I think this strange assignment of the local `logits` variable is coming from here, inside `prediction_step`:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1933

As the `outputs `dict includes the loss, and "loss" is not in ignore_keys, the loss value in outputs gets baked into `logits`.

I'm pretty sure it's a typo, as when I'm comparing it to a few lines above, (which is executed when has_labels=True), the similar line is:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1922

The above links are all from Version 4.4.2, but this possible typo is still present in master:

https://github.com/huggingface/transformers/blob/9856c9213dfe9f8355fe00dd6cd0fa1ceae4fa5a/src/transformers/trainer.py#L1966

I haven't been able to read and grasp the code too much, but it looks to me like either we're forgetting to ignore the "loss" key in outputs, or the return statement of `prediction_step` should be somehaw unpacking the logits tuple, so the two variables in "logits" tuple are unpacked into `loss` and` logits`:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1947

**For clarity, this is the stacktrace of how I encounter the tuple index error from the above typo:**

In the evaluation phase, `prediction_loop` runs over all the batches in my dev dataset. It gets the model output/prediction of each dev batch here:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1805

Later in` prediction_loop`, we, concatenate each prediction batch with the previous predictions here, calling the function `nested_concat`:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1810

Inside `nested_concat`, in the line below, `new_tensors` is the above mentioned "logits" tuple.
https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer_pt_utils.py#L95

The above line does a recursive call to `nested_concat`, and we arrive in the line below.

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer_pt_utils.py#L97

Which calls this:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer_pt_utils.py#L58

And I get a index error, as it's trying to index into what is actually the `loss` tensor.


Exception: cannot import name 'Regex' from 'tokenizers'

From: site-packages/transformers/convert_slow_tokenizer.py
When first-time call:
from transformers import XLMRobertaTokenizer
tokenizers-0.10.1 transformers-4.4.2


Return global attentions (see #7514)

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # 7514 (see discussion of March 22, 2021 with @patrickvonplaten )

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Add ImageFeatureExtractionMixin

# What does this PR do?

This PR adds a new `ImageFeatureExtractionMixin` to implement the common functionality needed for images (conversion to PIL Image / NumPy array /normalize/ resize) in a framework agnostic way. While it only adds support for torch (not tf) tensors as input, support for TF tensors is easy to add in the design and will be done when we have a TF model with a vision modality.

Along the way, this PR adds a new `is_vision_available` check (depends only on PIL for now, but we can add other dependencies later on if we fill we need them. It could for instance check for torchvision when torch is installed) and the "dummy" vision objects.

I will work on adding tests tomorrow, but the general design can already be reviewed to check if it has everything needed.
cc @NielsRogge


ONNX export: move sample input to same device as model when inferring shapes

# What does this PR do?

Training a model on a GPU and exporting it afterwards to ONNX has raised a `RuntimeError`, because the model and the [sample input](https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/convert_graph_to_onnx.py#L196) in [`transformers.convert_graph_to_onnx.infer_shapes()`](https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/convert_graph_to_onnx.py#L161-L222) were not on the same device. This PR moves the output to the same device as the model.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests) Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@mfuntowicz (according to `git blame`)


[WIP] Add 3D attention mask to T5 model (#9643)

# What does this PR do?
It allows for 3D attention mask in T5 model (modeling_t5.py).

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #9643

This  is a solution for allowing the 3D attention mask in the T5 model by making it broadcastable. It is based on what is used in BERT.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Add `examples/run_ner_no_trainer.py`

This PR adds an example of token classification tasks (`"ner", "pos", "chunk"`) to show the functionalities of the new `accelerate` library.

<hr>

**Reviewers:** @sgugger


Error with detecting cached files when running without Internet connection (related to #10067)

## Environment info
- `transformers` version: 4.5.0.dev0
- Platform: Linux-3.10.0-957.5.1.el7.x86_64-x86_64-with-centos-7.6.1810-Core
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
@LysandreJik (related to #10235 and #10067)

## Information
I'm trying to run
```
from transformers import BertTokenizer
BertTokenizer.from_pretrained("bert-large-uncased-whole-word-masking")
```
from an environment without Internet access. It crashes even though I have all files downloaded and cached. The uncaught exception:
https://github.com/huggingface/transformers/blob/5f1491d3b366d19cc08832d09bcfe007a2643089/src/transformers/file_utils.py#L1347-L1350

When `file_id == 'added_tokens_file'` `file_path` equals https://huggingface.co/bert-large-uncased-whole-word-masking/resolve/main/added_tokens.json which does not exist. (https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/tokenization_utils_base.py#L1653)
This results in line
https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/file_utils.py#L1294 throwing `ConnectTimeout` which is caught in https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/file_utils.py#L1313
and further ignored until another exception in https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/tokenization_utils_base.py#L1672
which is not caught enywhere.

When trying to get the same file with the internet is on the code work differently: line https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/file_utils.py#L1295 throws `requests.exceptions.HTTPError`, which is caught and processed here https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/tokenization_utils_base.py#L1674-L1677
The rest of the code works just fine after `resolved_vocab_files[file_id] = None`

Using `BertTokenizer.from_pretrained(bert_version, local_files_only=True)` works just fine because of this condition:
https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/tokenization_utils_base.py#L1668-L1672

The current workaround is to use `BertTokenizer.from_pretrained(bert_version, local_files_only=True)` but this does not allow to use same code with and without Internet.

## To reproduce

Steps to reproduce the behavior:

Run
```
from transformers import BertTokenizer
BertTokenizer.from_pretrained("bert-large-uncased-whole-word-masking")
```
from env without internet but all the required cache files pre-downloaded.

## Expected behavior

Works exactly as
```
from transformers import BertTokenizer
BertTokenizer.from_pretrained("bert-large-uncased-whole-word-masking", local_files_only=True)
```


Getting a model to work on a system with no internet access

## Environment info
We are trying to use the model from https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment on a system that has no connection to the internet. Normally one can do:
`pipeline = Pipeline(model='model_name")`
and huggingface will fetch everything it needs from the internet. Unfortunately, some datasets reside within a highly protected environment that does not allow for any internet connection and we can't fetch our models. (Hell, we can't even copy/paste errors).

Uploading files to that environment is really cumbersome and every file needs to go through a review process. Through trial and error, i have gotten the model and tokenizer to load, but it is now missing a "vocabulary". Before I got a submit a request for extra files to be uploaded, could someone just confirm for me what files do I need so that the model could be loaded offline. As far as I understand i have to have every single file from here: https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment/tree/main
and my script should look like:
`pipeline = Pipeline(model='/path/to/dir")`

### Who can help

Models:
- pipelines: @LysandreJik

## Information
https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment

The problem arises when using:
Running in a restricted environment

## To reproduce
Use the example script from https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment and try to manually input the files

## Expected behavior
The model should work just as if it were in a system connected to the internet.


updates sagemaker documentation

# What does this PR do?

Extends **local environment** configuration and adds import to make it more clear. Also, replaced two links.


run_glue_no_trainer: datasets -> raw_datasets

# What does this PR do?

Use the correct variable (raw_datasets) instead of the module (datasets)
where appropriate. The script will otherwise fail with "module object is not subscriptable".

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@sgugger


[doc] Custom datasets page reference dataset library as NLP library

### Who can help

@sgugger

## Information

The page [Fine-tuning with custom datasets](https://huggingface.co/transformers/custom_datasets.html) is referencing dataset library a lot, but in the old name (NLP library), and I've noticed that this still holds true in the [source .rst](https://github.com/huggingface/transformers/blob/master/docs/source/custom_datasets.rst) file.

If it wasn't left unmodified intended, I'm willing to help submitting a PR, just filling the issue to confirm.


# 🚀 Feature request

In the Trainer - Enable an option to save only the best performing checkpoints (rather than the newsest)

## Motivation

Usually when we train a model we would like to keep only the best performing checkpoints (on the dev set according to the specified metric) rather than the newest checkpoints.


Add missing global_attentions into the return_dict of Longformer models

The `global_attentions` is missing in the return_dict of `LongformerForSequenceClassification`, `LongformerForMaskedLM`, and `LongformerForTokenClassification` classes in `modeling_longformer.py`.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@patrickvonplaten

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->