# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on English

The problem arises when using:
* [ ] the official example scripts: [`run_language_modeling.py`](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py).

The tasks I am working on is:
* [ ] Continue Training XLNet on domain-specific dataset / finetuning XLNet LM

## To reproduce

Steps to reproduce the behavior:

1. Install transformers 3.0
2. run the following command as mentioned in the readme file from examples :

```
python run_language_modeling.py \
    --output_dir=output \
    --model_type=xlnet \
    --model_name_or_path=xlnet-base-cased \
    --do_train \
    --train_data_file=$TRAIN_FILE
```
Error message :
```
2020-07-20 10:47:29.463663: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
Traceback (most recent call last):
  File "run_language_modeling.py", line 29, in <module>
    from transformers import (
ImportError: cannot import name 'DataCollatorForPermutationLanguageModeling'
```

## Expected behavior
I expect not to have this import error since I'm using the latest release of the library
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0
- Platform: Google Colab
- Python version: Python 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101
- Tensorflow version (GPU?): 2.2.0

- Using GPU in script?: Tesla K80
- Using distributed or parallel set-up in script?: No


ModuleNotFoundError: No module named 'torch_xla'

from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-wikiSQL-sql-to-en")

model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-wikiSQL-sql-to-en")


Word frequencies in TransfoXLTokenizer

I was wondering if it is still possible to access word frequencies through the populated counter of TransfoXLTokenizer? For example, `tokenizer = TransfoXLTokenizer.from_pretrained('transfo-xl-wt103')` seems to have an empty counter. This is referring to:

"So p_M(S) is just the output of the model right?

For p_u(S), I think the easiest is probably to use the empirical probabilities.
`TransfoXLTokenizer` has a counter to store words frequencies [here](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/tokenization_transfo_xl.py#L98) which should be populated in the "pretrained" tokenizer so I would use and normalize this to get unconditional probabilities for each word and then compute SLOR."

_Originally posted by @thomwolf in https://github.com/huggingface/transformers/issues/477#issuecomment-483973033_


# ❓ Questions & Help

## Details
Hello,
I am wondering is it possible to reuse or retrain a fine-tuned model with a new set of labels(the set of labels contain new labels or the new set of labels is a subset of the labels used to fine-tune the model)?
What I try to do is fine-tune pre-trained models for a task (e.g. NER) in the domain free dataset, then reuse/retrain this fine-tuned model to do a similar task but in a more specific domain (e.g. NER for healthcare), thus in this specific domain, the set of labels may not be the same.

I already try to fine-tune a BERT model to do NER on WNUT17 data based on token classification example in Transformers GitHub. After that, I try to retrain the fine-tuned model by adding a new label and provide train data that has this label, the train failed with error:
```
RuntimeError: Error(s) in loading state_dict for BertForTokenClassification:
size mismatch for classifier.weight: copying a param with shape torch.Size([13, 1024]) from checkpoint, the shape in current model is torch.Size([15, 1024]).
size mismatch for classifier.bias: copying a param with shape torch.Size([13]) from checkpoint, the shape in current model is torch.Size([15]).
```

Is it possible to do this with Transformers and if so how? Maybe there is a method that can do something like [this](https://spacy.io/api/entityrecognizer#add_label)(the method is from spaCy). Thank you in advance!

I already post this in the forum
[Retrain/reuse fine-tuned models on a different set of labels](https://discuss.huggingface.co/t/retrain-reuse-fine-tuned-models-on-different-set-of-labels/346)


RobertaTokenizerFast unexpectedly quits when creating a TextDataset

# 🐛 Bug

When creating a `TextDataset` using `RobertaTokenizerFast` my program unexpectedly dies. (Not so with `RobertaTokenizer`).

## Information

Model I am using: RoBERTa

Language I am using the model on: English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: [language modelling](https://github.com/huggingface/transformers/blob/33d3072e1c54bcd235447b98c6dea1b4cb71234c/examples/run_lm_finetuning.py)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```
from transformers import AutoTokenizer, TextDataset

tokenizer = AutoTokenizer.from_pretrained("roberta-base", use_fast=True)

train_dataset = TextDataset(
    tokenizer=tokenizer,
    file_path="/home/ubuntu/data/wikitext-103-raw/wiki.train.raw",
    block_size=-1,
    overwrite_cache=False,
)
print(train_dataset)
```
## Expected behavior

Creation of the training dataset, not having the process killed. eg:

```
<transformers.data.datasets.language_modeling.TextDataset object at 0x7f138a1fd2b0>
```

## Environment info

- `transformers` version: 2.11.0
- Platform: Linux-5.3.0-1030-aws-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


[WIP] Add Theseus Compression

WIP

Label your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

`transformers.theseus` provides the implementation for BERT-of-Theseus, LayerDrop and Mixout.

Original BERT-of-Theseus authors: @JetRunner @MichaelZhouwang


🐛 BART : Same representations for different `<s>` tokens

Help wanted

Extra attention is needed, help appreciated

# 🐛 Bug

## Context

I'm trying to use BART for a sentence classification task. So I encode the input with the following format :

```
<s> Sen1 </s> <s> Sen2 </s> <s> Sen3 </s> ...
```

And use `<s>` as sentence representation (from the encoder, not decoder). Then I classify these representations.

---

I trained my model but the classification give random choice. After debugging, I noticed that the encoder produce always the same representation for `<s>` token.

## Bug

**The representation of `<s>` token is always the same, no matter where they appear in the input.**

Here is a notebook reproducing the issue : [Colab](https://colab.research.google.com/drive/1mqKKFAEGEwa5XbkJtm7_VrmQ8L3_0Bnt?usp=sharing)

In this notebook I simply encode an input, modify the input to add an additional `<s>` token, forward it through BART and compare the encoder representation of `<s>`. It gives me :

```
tensor([-0.0097,  0.0075,  0.0086,  ...,  0.0041, -0.0085, -0.0011],
       grad_fn=<SelectBackward>)
tensor([-0.0097,  0.0075,  0.0086,  ...,  0.0041, -0.0085, -0.0011],
       grad_fn=<SelectBackward>)
```

---

Now if I do the same with `</s>` token, their representation is different :

```
tensor([ 0.0658,  0.0161, -0.0062,  ..., -0.0536, -0.0515,  0.1837],
       grad_fn=<SelectBackward>)
tensor([ 0.0627,  0.0576,  0.0408,  ..., -0.0406, -0.0765,  0.1689],
       grad_fn=<SelectBackward>)
```

Which makes sense because they represent different thing...

## Note 1

I think this is a bug because I remember a few months ago, I tried to do the same (use `<s>` representation for classification) with fairseq version of BART, and it worked...

## Note 2

I'm aware the example in the Notebook is using just the pre-trained version of BART, so the `<s>` representation does not represent sentence. But even after training the model, the exact same behavior arise.

@sshleifer


How can I check the loss during pretraing huggingface/transformers

 thanks in advance.
I trained roberta model from scratch.
But I cant  check the training loss during pretraining.
I did it by refering below link.
https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb
in above link, loss is printed every 500 steps,
but when I did, there is no loss print.

Iteration: 100%|█████████▉| 20703/20711 [4:42:54<00:07,  1.14it/s]

Iteration: 100%|█████████▉| 20704/20711 [4:42:54<00:05,  1.24it/s]

Iteration: 100%|█████████▉| 20705/20711 [4:42:55<00:05,  1.20it/s]

Iteration: 100%|█████████▉| 20706/20711 [4:42:56<00:04,  1.18it/s]

Iteration: 100%|█████████▉| 20707/20711 [4:42:57<00:03,  1.19it/s]

Iteration: 100%|█████████▉| 20708/20711 [4:42:58<00:02,  1.16it/s]

Iteration: 100%|█████████▉| 20709/20711 [4:42:59<00:01,  1.14it/s]

Iteration: 100%|█████████▉| 20710/20711 [4:43:00<00:00,  1.13it/s]

Iteration: 100%|██████████| 20711/20711 [4:43:00<00:00,  1.45it/s]
Iteration: 100%|██████████| 20711/20711 [4:43:00<00:00,  1.22it/s]

Epoch: 100%|██████████| 13/13 [61:14:16<00:00, 16952.06s/it]
Epoch: 100%|██████████| 13/13 [61:14:16<00:00, 16958.16s/it]
compress roberta.20200717.zip on ./pretrained
save roberta.20200717.zip on minio(petcharts)

stackoverflow link
https://stackoverflow.com/questions/62988081/checking-pretraining-loss-in-huggingface-transformers


Is there any api for intermediate layer outputs?

I would like to get all 12 Bertlayer outputs. Can you please help?


Smaller output vocabulary for GPT-2

# ❓ Questions & Help
I noticed that by default, GPT2LMHeadModel returns prediction scores of shape (batch_size, sequence_length, config.vocab_size) ([docs link](https://huggingface.co/transformers/model_doc/gpt2.html#gpt2lmheadmodel)). Is there any way for me to limit the output vocabulary to only a subset of words?

I want to take the existing weights from GPT-2, but re-train a new top linear layer with a smaller vocabulary. I suppose I could mask the logits at the end, but then it feels like a waste of computational power to even predict them.

**A link to [original question on the forum](https://discuss.huggingface.co/t/smaller-output-vocabulary-for-gpt-2/366)**


Fix pack_dataset.py

right now it just sorts the examples and prints an incorrect logger message!


examples/seq2seq: add label_smoothing cross entropy option

Started!


MT: automate/experiment with pruning embeddings

This isn't really a github issue, but follow pytorch/fairseq#2120 and see if it works. If it does, make it easy to use!


examples/seq2seq/finetune.py and BART supports TPU

- [ ] test the code on tpu
- [ ] if it doesn't work well: change code as little as possible to get it working.
- [ ] add a script/command that works


fix typo in training_args_tf.py


fix typo in training_args.py


Benchmark: traceback does not describe real problem

To reproduce:
- execute a command that results in 'N/A'
- long traceback describes tuple unpacking. The real problem is what caused the N/A and is far above in a small logger.error statement.

Example Command:
```bash
python examples/benchmarking/run_benchmark.py --models facebook/mbart-large-en-ro
```

Traceback:
```python
1 / 1
ERROR:transformers.benchmark.benchmark_utils:<class 'transformers.configuration_bart.MBartConfig'>
<class 'transformers.configuration_bart.MBartConfig'>
Traceback (most recent call last):
  File "examples/benchmarking/run_benchmark.py", line 29, in <module>
    main()
  File "examples/benchmarking/run_benchmark.py", line 25, in main
    benchmark.run()
  File "/home/shleifer/transformers_fork/src/transformers/benchmark/benchmark_utils.py", line 665, in run
    memory, inference_summary = self.inference_memory(model_name, batch_size, sequence_length)
ValueError: too many values to unpack (expected 2)
```

This may not be worth a fix, and probably won't be fixed soon, but posting for others who might run into this or have ideas for a fix.


[WIP] Fix mbart benchmark

- [ ] undo benchmarking changes
- [ ] publish working command + results


How to finetune distillbart from distilbart-cnn-12-6 checkpoint using cnn_daily mail or gigawords dataset?

I would like to ask about how to finetune distillbart on gigaword and cnn dailymail with the starting checkpoint distilbart-cnn-12-6.
I did use the gigaword dataset provided by tensorflow but it replaces numbers by this character: "#", as a result, my summaries have # instead of numbers, is it normal that it has those # ?
Also is it really possible to finetune distillbart from the checkpoint distilbart-cnn-12-6 with cnn daily mail?


# ❓ Questions & Help

when I run run_generation.py（I use TF1.14 fine tuned gpt2）,the program is broken with the error message AttributeError: 'GPT2LMHeadModel' object has no attribute 'h'

Environment:
WIN10
pytorch 1.3.1
python 3.6

Part of the code has been modified
![image](https://user-images.githubusercontent.com/15426714/87869491-e6294180-c9d2-11ea-8b45-be83742b1045.png)


Reading transformer package from local codes and NOT the pip installed version

Hi @sshleifer ,

There seem to be a problem of python package structure. I am getting an error similar to a previous issue (https://github.com/huggingface/transformers/issues/5303) but with the token_classification/run_ner.py file.

File "examples/token-classification/run_ner.py", line 42, in
from modeling_auto import AutoModelForTokenClassification
File "transformers/src/transformers/modeling_auto.py", line 22, in
from .configuration_auto import (
ImportError: attempted relative import with no known parent package

I have not installed transformers library using pip because I want to use the local codes (cloned from transformers library). After reading various stackoverflow suggestions (https://stackoverflow.com/questions/16981921/relative-imports-in-python-3 and https://napuzba.com/a/import-error-relative-no-parent), I believe that when I am importing the transformer package locally from my own directory, then it is not able read+load transformer as a package.

I am using python3.7

Can you please suggest how to read transformer as a package from local codes.

Thanks...


Seq2Seq: same MultiGPU test failing twice!

This is not run by CI, but if you go on a machine with multiple GPU, you will see an interesting unittest failure:

```bash
pytest examples -k multigpu
```

```
===================================== FAILURES ======================================
_____________________ TestSummarizationDistiller.test_multigpu ______________________

self = <seq2seq.test_seq2seq_examples.TestSummarizationDistiller testMethod=test_multigpu>

    @require_multigpu
    def test_multigpu(self):
        updates = dict(no_teacher=True, freeze_encoder=True, gpus=2, sortish_sampler=False,)
>       self._test_distiller_cli(updates)

examples/seq2seq/test_seq2seq_examples.py:117:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
examples/seq2seq/test_seq2seq_examples.py:182: in _test_distiller_cli
    self.assertIn(ckpt_name, contents)
E   AssertionError: 'val_avg_rouge2=0.0000-step_count=2.ckpt' not found in {'metrics.json', 'hparams.pk
l', 'git_log.json'}
============================== short test summary info ==============================
FAILED examples/seq2seq/test_seq2seq_examples.py::TestSummarizationDistiller::test_multigpu
========================= 1 failed, 25 deselected in 9.75s ==========================
...
 'step_count': 4,
 'test_avg_gen_time': 0.9470310211181641,
 'test_avg_loss': 10.737613677978516,
 'test_avg_rouge1': 0.0,
 'test_avg_rouge2': 0.0,
 'test_avg_rougeL': 0.0,
 'test_avg_summ_len': 141.0,
 'test_loss': tensor(10.7376, device='cuda:0'),
 'test_rouge2': tensor(0., device='cuda:0')}
--------------------------------------------------------------------------------
Testing: 100%|█████████████████████████████████████████| 1/1 [00:01<00:00,  1.49s/it]
F

===================================== FAILURES ======================================
_____________________ TestSummarizationDistiller.test_multigpu ______________________

self = <seq2seq.test_seq2seq_examples.TestSummarizationDistiller testMethod=test_multigpu>

    @require_multigpu
    def test_multigpu(self):
        updates = dict(no_teacher=True, freeze_encoder=True, gpus=2, sortish_sampler=False,)
>       self._test_distiller_cli(updates)

examples/seq2seq/test_seq2seq_examples.py:117:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
examples/seq2seq/test_seq2seq_examples.py:193: in _test_distiller_cli
    self.assertEqual(len(metrics["val"]), desired_n_evals)
E   AssertionError: 3 != 5
============================== short test summary info ==============================
FAILED examples/seq2seq/test_seq2seq_examples.py::TestSummarizationDistiller::test_multigpu
========================= 1 failed, 25 deselected in 47.48s =========================
```

The same `SummarizationDistiller.test_multigpu` fails twice. I think we need to use `pl.Trainer.argparse.from_args` in the unittests.
How do you guys do it @nateraw ?


DataCollatorForLanguageModeling - Shift labels for left-to-right LM?

Hi,

Shouldn't the labels be shifted 1 step right in the data collation step at -https://github.com/huggingface/transformers/blob/09a2f40684f77e62d0fd8485fe9d2d610390453f/src/transformers/data/data_collator.py#L88


feat: allow prefix for any generative model

Allow `padding_text` for any generative model.

### Motivations

* some models require some text at the beginning such as GPT-2 which should always have a white space. GPT-2 model could now have `config.padding_text` set to `" "` to ensure this is done
* this argument adds more customization to huggingface inference widget (related issue #5553). For huggingtweets models I can set `model.config.task_specific_params['text-generation']['padding_text'] = '<|endoftext|> '`

See [example](https://gist.github.com/borisdayma/89aaf1587390340add44c3c7081bffcd) of use of `padding_text` to create negative/positive sentences.

*Side note*: this is not completely obvious that a leading space should always be added for GPT-2 as `" \n This"` is not tokenized the same as `" \nThis"` which GPT-2 may have seen more in its training data (unless white spaces are added after new lines as well)

### Future improvements

In the future, we should also add `prefix` and `suffix`. This could let us do things similar to GPT-3 with the inference widget and specific tasks:
* `padding_text` would be used to initialize tasks (ex: 10 examples conversation with a bot). This is always cropped out and never displayed in the output
* `prefix` would add required text (ex: `" Human: "`). This is displayed in output.
* `suffix` would add required text after prompt (ex: `" \n AI: "`). This is displayed in output.


Add ComVE model cards

model card

Related to pretrained model cards

Add `aliosm/ComVE*` model cards. These models were part of this work: https://sentic.net/transformer-models-for-commonsense-reasoning.pdf.


Xlnet outputs

Reopening #5770 again


Revert "Xlnet outputs"

This seems to break a test that isn't a CI. Think I've fixed it, will revert and re-open.


Xlnet outputs

Reopening #5770 since push failed.


How to get probabilities from MarianMT models?

Hey,

I try to use MarianMT models for distillation, but I didn't found way to get raw probabilities from model.

I use model as in examples:
```
from transformers import MarianMTModel, MarianTokenizer
class Translator:
    def __init__(self, src, dst):
        model_name = f'Helsinki-NLP/opus-mt-{src}-{dst}'
        self.tokenizer = MarianTokenizer.from_pretrained(model_name)
        self.model = MarianMTModel.from_pretrained(model_name)

    def __call__(self, text):
        if not isinstance(text, list):
            text = [text]
        preds = self.model.generate(**self.tokenizer.prepare_translation_batch(text))
        print(preds)
        translation = [self.tokenizer.decode(t, skip_special_tokens=True) for t in preds]
        return translation
```

How can I get translation and probs via Huggingface API?
Thanks in advance!


Create README.md

Related to pretrained model cards


[cleanup] Less aggressive warnings about checkpoint mismatches

These are often spurious, so, in general, I made them shorter, and more about what happened than how to fix it.

Also, removed warnings for missing keys in seq2seq checkpoints. LM head is often made on the fly (to make the download cheaper).

Some examples:
- key 'encoder.version' in `bart.large` should not be trained more
- no keys in `T5ForConditionalGeneration` should be trained more.


Silenced error while downloading pretrained model

# 🐛 Bug

Currently, if user failed to download a model, they will receive uninformative and confusing message (see #5787). This is most hurting for users behind a proxy. This happens because of this [line](https://github.com/huggingface/transformers/blob/0533cf470659b97c6279bd04f65536a1ec88404a/src/transformers/file_utils.py#L681). My opinion is that there should be at least a warning, and if `force_download` is `True`, then error should be raised. I can make a PR if you agree with that.


[cleanup] squad processor


Update tokenizers to 0.8.1.rc to fix Mac OS X issues

As mentioned in huggingface/tokenizers#321 tokenizers and thus transformers fails on macOS 10.11+. The issue was fixed in `0.8.1.rc2` so this PR updates the dependency to the new version.

I ran all tests on my mac with Mac OS High Sierra 10.13.6 as desribed in the [readme](https://github.com/huggingface/transformers/blob/master/README.md#tests), but not the examples:

```
1607 passed, 304 skipped, 15294 warnings in 465.74s (0:07:45)
```


T5Tokenizer adds EOS token if not already added

T5 Tokenizer should add `</s>` to the end of sequences. Since some users are doing this on their own, this PR only adds `</s>` if it has already been added.

On my machine, this makes zero shot validation BLEU go from 27.87 -> 27.65. Since this change is needed for finetuning, and the empirical difference is small and doesn't happen on Stas' machine, I would recommend merging this.

If others want to test, the command takes about 3 mins to run on brutasse.

### Zero Shot BLEU Scores
For english -> romanian
I grabbed the WMT english-romanian dataset:
```bash
wget https://s3.amazonaws.com/datasets.huggingface.co/translation/wmt_en_ro.tar.gz
tar -xzvf wmt_en_ro.tar.gz
```
Then ran evaluation (without finetuning) on the validation split:

```bash
export DATA_DIR=wmt_en_ro
python run_eval.py t5-base \
    $DATA_DIR/val.source t5_val_generations.txt \
    --reference_path $DATA_DIR/val.target \
    --score_path t5_enro_bleu_eos.json \
    --task translation_en_to_ro \
    --device cuda \
    --fp16 \
    --bs 32
```
(this branch) (with EOS):27.65
master (no EOS):  27.87
```
sacrebleu==1.4.3
torch==1.5.1
```
Will merge and fix tests if others have positive results.


[seq2seq] distillation.py accepts trainer arguments

cc @nateraw


not able to reproduce accuracy at the end of same epoch

# 🐛 Bug

update: Found what is causing this bug

Your optimization [code](https://github.com/huggingface/transformers/blob/master/src/transformers/optimization.py#L70) uses a linear scheduler for learning rate with warmup which is **dependent on the total number of training steps** (== no of batches*total epochs).  This injects randomness (as the total epoch count changes) during every scheduler.step(), which affects the weights of the trained model, which in turn reflects in accuracy....

The effect of this is very minimal in the in-domain setting. However, since I was evaluating on a cross-domain-setting it ended up being heavily amplified. Can you please make the warm up dependent on the current epoch and not on total epochs/step size.

----original post
When am running say for 25 epochs, I wanted to run an eval on dev and test partition at the end of every epoch. So I moved the evaluate() code in trainery.py to the end of all batches. However the accuracy I get at the end of say epoch 1 (in the big 25 epoch run)  is different than what I get when I run the entire code for just 1 epoch. This might be the same answer to [this](https://github.com/huggingface/transformers/issues/5264) issue also
## Information

Model I am using (Bert, XLNet ...):Bert (base-uncased)

Language I am using the model on (English, Chinese ...):English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x ] my own modified scripts: (give details below): the only modification is I call the same evaluate function to after epochs. You can see that [here](https://github.com/mithunpaul08/transformers/blob/master/src/transformers/trainer.py#L1592).

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ x] my own task or dataset: (give details below)
I am training and testing on [FEVER](https://s3-eu-west-1.amazonaws.com/fever.public/wiki-pages.zip) . My code should download it automatically though.

## To reproduce

Steps to reproduce the behavior:

- Kindly please clone my [repo](https://github.com/mithunpaul08/transformers)
- conda create --name huggingface python=3
- source activate huggingface
- pip install -r ./examples/requirements.txt
from home folder (i.e /huggingface/) do:
- cd mithun_scripts/
- ./run_all.sh
(should run i think)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Potential security vulnerability regarding Hosted Interface API?

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): distilbert-base-cased-distilled-squad with huggingface.co's API

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: Question and answering

## To reproduce

Steps to reproduce the behavior:

Run this code on a notebook
```
answer = requests.post('https://api-inference.huggingface.co/models/distilbert-base-cased-distilled-squad',
                       headers = {"Content-Type": "application/json"},
                       data = {"question":'What is my name?' ,
                              "context" :"My name is Batik"})
```

Result : 500 Error . With the following content when I executed `print(answer.content)`

```
b'{"error":"JSONDecodeError(\'Expecting value: line 1 column 1 (char 0)\')","traceback":"  File \\"/home/hf/api-inference/server.py\\", line 251, in model_forward\\n    inputs = await request.json()\\n  File \\"/home/hf/api-inference/.env/lib/python3.8/site-packages/starlette/requests.py\\", line 227, in json\\n    self._json = json.loads(body)\\n  File \\"/usr/lib/python3.8/json/__init__.py\\", line 357, in loads\\n    return _default_decoder.decode(s)\\n  File \\"/usr/lib/python3.8/json/decoder.py\\", line 337, in decode\\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\\n  File \\"/usr/lib/python3.8/json/decoder.py\\", line 355, in raw_decode\\n    raise JSONDecodeError(\\"Expecting value\\", s, err.value) from None\\n"}'
```

## Expected behavior

I imagine we don't want to give the Error traceback for security reasons

## Environment info

I tried using Python's request and Postman and have the same outcome. Furthermore, I'm wondering why I have a Error 500.  Was my code wrong?


[seq2seq] add back clargs


issue with loading pretrained model - xlnet

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->
I ran the following code:
from transformers import XLNetTokenizer, XLNetForSequenceClassification
import torch
tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')
model = XLNetForSequenceClassification.from_pretrained('xlnet-base-cased')

## Details
<!-- Description of your issue -->
and I have the following error:

> OSError: Can't load config for 'xlnet-base-cased'. Make sure that:

> 'xlnet-base-cased' is a correct model identifier listed on 'https://huggingface.co/models'

> or 'xlnet-base-cased' is the correct path to a directory containing a config.json file

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
The error didn't appear when I ran the same piece of code two days ago so I think it might be caused by recent update.


[seq2seq] organize commands into scripts/ subdir

whoever takes this, please test them after!


wrong args name: n_gpu -> gpus

The correct argument name should be `gpus`.

The change fixed the following error:
```bash
  warnings.warn(*args, **kwargs)
Traceback (most recent call last):
  File "/home/.conda/envs/hf/lib/python3.6/site-packages/pytorch_lightning/utilities/parsing.py", line 114, in __getattr__
    return self[key]
KeyError: 'n_gpu'
```


Update README.md

Related to pretrained model cards

Add nlp dataset used


Update README.md

Related to pretrained model cards

Add dataset used as it is now part of nlp package


docs: fix model sharing file names

Documentation references `vocab.txt` while I think it should be `vocab.json`.

Additional notes:
* I also have `training_args.bin` and `merges.txt` which I understand are not needed to be uploaded
* I didn't get `added_tokens.json` but I understand it's not always present


Revert "XLNet `use_cache` refactor"

Reverts huggingface/transformers#5770


[BartModel] Question for BartModel Output Shape when I pass the 'decoder_input_ids'

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->

Hi I'm trying to using model named 'BartForConditionalGeneration' for text generation,
The main question and problem is, when I pass the 'decoder_input_ids'  to the model as the input of forward in model,
the first index of model's output shows the tensor shape as [batch_size, 1 , vocab_size] all the time.
I might misunderstood the model and Docs of Bart overall,
when I only pass the Input_ids without decoder_input_ids, it worked right.
but It never make the shape of tensor like [batch_size, target_seq_len, vocab_size] when I put 'decoder_input_ids'

I think when I put the decoder_input_ids as [batch_size, target_seq_len], the Model's output should be like
[batch_size, target_seq_len, vocab_size] (for prediction scores)

the reason why I tried to pass the decoder_input_ids was , I think Bart is seq2seq model, and their output logits will be computed just like the basic transformer, so I wanted to pretrain this model.

Please help me to solve this problem.  If I misunderstood the concept of Bart at all, just comment me.
I need a clean view for my situation

I used mBartTokenizer and also 'facebook/mbart-large-en-ro'  pretrained weights,
I used Model BartForConditionalGeneration.from_pretrained('facebook/mbart-large-en-ro')
It also didn't worked for normal BartModel (with same pretrained config)

I'm really confused about it so the whole question looks like a mass, Sorry for that
Thank you for reading my questions. Please Help Me.
![1](https://user-images.githubusercontent.com/51171232/87811751-1a4d1700-c89a-11ea-8e47-0ae2987dfe29.PNG)


Exception in device=TPU:1: 'ascii' codec can't decode byte 0xc2 in position 37: ordinal not in range(128)

I get the error message when trying to run the seq2seq example on 8 TPU cores on XSUM and bart-large.

Run this from `examples/seq2seq`:
```
export PYTHONPATH="../":"${PYTHONPATH}"

python finetune.py \
    --learning_rate=3e-5 \
    --gpus 0 \
    --n_tpu_cores 8 \
    --do_train \
    --do_predict \
    --n_val 1000 \
    --val_check_interval 1.0 \
    --sortish_sampler \
    --data_dir ${PWD}/xsum \
    --train_batch_size=1 \
    --eval_batch_size=1 \
    --output_dir=xsum_results \
    --num_train_epochs 1 \
    --model_name_or_path facebook/bart-large
```

BTW there is also a bug in `lightning_base.py`. The `gpus` argument is given twice if `n_tpu_cores` is specified (second time through `**train_params`) so I replaced Trainer creation lines with these:
```
    n_gpus = train_params.pop("gpus", args.gpus)

    trainer = pl.Trainer(
        logger=logger,
        accumulate_grad_batches=args.gradient_accumulation_steps,
        gpus=n_gpus,
        max_epochs=args.num_train_epochs,
        early_stop_callback=early_stopping_callback,
        gradient_clip_val=args.max_grad_norm,
        checkpoint_callback=checkpoint_callback,
        callbacks=[logging_callback] + extra_callbacks,
        fast_dev_run=args.fast_dev_run,
        val_check_interval=args.val_check_interval,
        weights_summary=None,
        resume_from_checkpoint=args.resume_from_checkpoint,
        **train_params,
    )
```