noamaanMulla-03/transformers-issues huggingface dataset:
Problem with automatic best model loading.
Author: PhilipMay
Pull request: no

When I provide `load_best_model_at_end=True`, `metric_for_best_model='eval_f1_macro` and `greater_is_better=True` together with `save_total_limit=2` this happens:

```
Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/hf/lib/python3.7/site-packages/optuna/study.py", line 778, in _run_trial
    result = func(trial)
  File "train_aws.py", line 174, in opt
    trainer.train()
  File "/home/ubuntu/miniconda3/envs/hf/lib/python3.7/site-packages/transformers/trainer.py", line 810, in train
    and self.global_step % self.args.eval_steps == 0
ZeroDivisionError: integer division or modulo by zero
Traceback (most recent call last):
  File "train_aws.py", line 197, in <module>
    study.optimize(opt)
  File "/home/ubuntu/miniconda3/envs/hf/lib/python3.7/site-packages/optuna/study.py", line 328, in optimize
    func, n_trials, timeout, catch, callbacks, gc_after_trial, None
  File "/home/ubuntu/miniconda3/envs/hf/lib/python3.7/site-packages/optuna/study.py", line 726, in _optimize_sequential
    self._run_trial_and_callbacks(func, catch, callbacks, gc_after_trial)
  File "/home/ubuntu/miniconda3/envs/hf/lib/python3.7/site-packages/optuna/study.py", line 755, in _run_trial_and_callbacks
    trial = self._run_trial(func, catch, gc_after_trial)
  File "/home/ubuntu/miniconda3/envs/hf/lib/python3.7/site-packages/optuna/study.py", line 778, in _run_trial
    result = func(trial)
  File "train_aws.py", line 174, in opt
    trainer.train()
  File "/home/ubuntu/miniconda3/envs/hf/lib/python3.7/site-packages/transformers/trainer.py", line 810, in train
    and self.global_step % self.args.eval_steps == 0
ZeroDivisionError: integer division or modulo by zero
```

Comments:
It's hard to know what's going on without seeing your code. The error indicates `args.eval_steps = 0` and this argument is not modified by the `Trainer` itself, so you should make sure you did set it to something >0.

You are right. I had `eval_steps = 0`. Thanks for the feedback!
Allow soft dependencies in the namespace with ImportErrors at use
Author: sgugger
Pull request: yes

# What does this PR do?

This PR aims at making errors due to soft dependency (like datasets) easier to understand for users. It aims at making everything available in the namespace and raising an import error at `init` or `from_pretained`

<!-- Remove if not applicable -->

Fixes #7536


Comments:
> FAISS_IMPORT_ERROR here ;)

Good catch ;-)

(we should have this for TF/PyTorch as well)
Check and update model list in index.rst automatically
Author: sgugger
Pull request: yes

# What does this PR do?

We currently have two lists of the models to maintain (in the main README and in the `index.rst` for the docs) which is painful. This script checks the mode list in the index.rst is a proper converted copy of the model list in the README and can also fix it with the command `make fix-copies` (same API as for the copies of parts of the model). It also enforces our maximum character per line so that the `index.rst` is still readable in an editor.



Comments:
I'll let you rebase!
Is this realy a list or a Dict[str, int]? I think the docstring might be wrong because in the model json file it is stored as a dict.
Author: PhilipMay
Pull request: no

https://github.com/huggingface/transformers/blob/9bdce3a4f91c6d53873582b0210e61c92bba8fd3/src/transformers/configuration_utils.py#L117

See here for example: https://s3.amazonaws.com/models.huggingface.co/bert/oliverguhr/german-sentiment-bert/config.json

Comments:
Documentation fixes
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes two issues with the documentation:
- wrong type annotation in the configurations (see #7559)
- wrong example for masked LM models (see this [froum post](https://discuss.huggingface.co/t/questions-on-the-bertmodellmheadmodel/1317/6))

Fixes #7559


Comments:
Documentation framework toggle should stick
Author: LysandreJik
Pull request: yes

# What does this PR do?

This PR adds the following feature: when clicking on a `PyTorch` or `TensorFlow` button in the documentation in order to show the corresponding framework code sample, the toggle takes effect on all the current pages' code samples.

TensorFlow users won't need to click on every code sample to convert it to TensorFlow anymore!

Comments:
[model_cards]: use MIT license for all dbmdz models
Author: stefan-it
Pull request: yes
Labels: name: model card

Hi,

this PR adds MIT license tag for all dbmdz models 🤗

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3492?src=pr&el=h1) Report
> Merging [#3492](https://codecov.io/gh/huggingface/transformers/pull/3492?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/17dceae7a1de5577cd0c07a97dcd5821a08af07c&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3492/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3492?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3492      +/-   ##
==========================================
- Coverage   77.80%   77.79%   -0.01%     
==========================================
  Files         100      100              
  Lines       17051    17051              
==========================================
- Hits        13266    13265       -1     
- Misses       3785     3786       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3492?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3492/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.81% <0.00%> (-0.14%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3492?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3492?src=pr&el=footer). Last update [17dceae...70cab16](https://codecov.io/gh/huggingface/transformers/pull/3492?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Awesome! One more data point for #3357 

Hey @stefan-it, thanks for creating some great models. It looks like a few [dbmdz models](https://huggingface.co/dbmdz) are missing model cards, including the default model for NER (`dbmdz/bert-large-cased-finetuned-conll03-english`). Are these licensed as MIT as well?
Support T5 Distillation w/hidden state supervision
Author: sshleifer
Pull request: yes

Support distilling t5 for summarization and translation with hidden state supervision.

cc @patil-suraj @patrickvonplaten 


Here are some very simple commands that work for now:

### Yes Teacher/Traditional Distillation
```bash
python distillation.py --teacher t5-small --data_dir cnn_dm \
--student_decoder_layers 3 --student_encoder_layers 6 --tokenizer_name t5-small \
--learning_rate=3e-4 --freeze_encoder --no_teacher --freeze_embeds \
--do_train --train_batch_size 32 \
--do_predict \
--model_name_or_path t5-small --eval_beams 2 --eval_max_gen_length 142 \
--val_check_interval 0.25 --n_val 1000 \
--output_dir distilt5 --gpus 1 --logger_name wandb

```

### No teacher

```bash
python make_student.py t5-small t5_small_6_3 6 3

python finetune.py --model_name_or_path t5_small_6_3 --data_dir cnn_dm \
--learning_rate=3e-4 --freeze_encoder --freeze_embeds \
--do_train --train_batch_size 32 \
--do_predict \
--model_name_or_path t5_small_6_3 --eval_beams 2 --eval_max_gen_length 142 \
--val_check_interval 0.25 --n_val 1000 \
--output_dir distilt5 --gpus 1 --logger_name wandb
```

Comments:
Enable debug with TF2 and eager execution
Author: Neptune-Trojans
Pull request: yes

TF2 with eager execution  and tf.gradients not supported anymore.
In order to run the code with eager execution and not collapse instead of tf.gradients should be tf.GradientTape. 

[how-to-compute-gradient-of-output-wrt-input-in-tensorflow-2-0](https://stackoverflow.com/questions/59145221/how-to-compute-gradient-of-output-wrt-input-in-tensorflow-2-0)

Comments:
Hello!

Thanks for you PR. The reason we used `tf.gradients` is because it handles `None` gradients while `tape.gradients` don't, so for now unless we can find a better way to handle the `None` values for gradients, we will keep it like this.

Also the training is forced to be done in graph compilation with `tf.function` so the eager mode is anyway disactivated.

Thanks for detailed answer.
Another reason I did the change is to deal with following error that I getting after upgrading to the laters version of transformers library.
`ValueError: distributed_training_steps() should not modify its Python input arguments. Check if it modifies any lists or dicts passed as arguments. Modifying a copy is allowed.`

This error disappears after my change.
tensorflow==2.3.1
transformers==3.3.1

Ok, can you open an issue with the details on how to reproduce the error please.

I am closing this pull request as I found workaround regarding my issue issue with **parameter modification** .
make modified_only_fixup complains about non .py files
Author: sshleifer
Pull request: no

easy fix @stas00 ?

Comments:
yes, the fix is trivial. Which tool is complaining?

I was running `make modified_only_fixup` before merging master and `black` was complaining.
But after merging master, there is no complaining.
And I should be using `make fixup`, so this might just be a UserError.

Should we merge the linked PR anyways or wait to see if I run into this again?


Indeed, `black` isn't smooth - it picks .py files when you give it a dir, but doesn't do the same if you give it explicit files:
```
$ black Makefile
error: cannot format Makefile: Cannot parse: 1:1: .PHONY: modified_only_fixup extra_quality_checks quality style fixup fix-copies test test-examples docs
Oh no! 💥 💔 💥
1 file failed to reformat.
```
So yes, please merge the linked PR.

`make fixup` is just `make modified_only_fixup` + `make extra_quality_checks` so no user error
Add ProtT5-XL-BFD model card
Author: agemagician
Pull request: yes
Labels: name: model card

Fixes # (issue)
Create a new card for our ProtT5-XL-BFD model.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?
Model Cards: @julien-c
T5: @patrickvonplaten


Comments:
[TF generation] Fix typo
Author: SidJain1412
Pull request: yes

# What does this PR do?

Typo + Parameter Assertion Fix

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
I see that tests are failing, but shouldn't `min_length` and `top_k` simply not be allowed to go to zero?

`min_length` defaults to 0 which is expteced behavior. `top_k` is 0 if it is not used => so I don't think we should do these changes.

We can fix the typo though ;-) 

My bad, that makes sense 😄
Does tokenizer.from_pretrained tokenize text on CPU even a GPU is available?
Author: BaoshengHeTR
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
For a tokenizer from `tokenizer = AutoTokenizer.from_pretrained(model_name)`, if `tokenizer.encode_plus(text)` works on CPUs even a GPU is available. I tried to run such code on a AWS GPU machine instance, but found GPUs are totally not used.
Thanks.



Comments:
Hi, indeed GPUs are not used when doing tokenization. There are no matrix operations and there's no need for heavy parallelization, so no need to rely on GPUs for this operation.
typo fix
Author: agemagician
Pull request: yes
Labels: name: model card

It should be T5-3B not T5-3M.
Fixes # (issue)


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

 Model Cards: @julien-c
 T5: @patrickvonplaten


Comments:
way to make inference Zero Shot pipeline faster?
Author: acul3
Pull request: no

Hi 
Can you guys give me tips how to make Zero Shot pipeline inference faster?

My current approach right now is reducing the model size/parameter 
(trying to train "base model" instead of "large model)


Is there another approach?

CCing @joeddav

Comments:
Closing this and moving the conversation (w/ my answer) to [the forums](https://discuss.huggingface.co/t/way-to-make-inference-zero-shot-pipeline-faster/1384/2?u=joeddav).
Feature Request: Support training and evaluating on Squad-format (json) files in SquadDataset for easy Squad fine-tuning
Author: raperry
Pull request: no

I am currently working on a project using [run_squad_trainer.py](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad_trainer.py)
to quickly fine-tune on new Squad-format (v2.0 json) files. However, the current SquadDataset class only allows training/evaluating on the original Squad jsons (train-v2.0.json, dev-v2.0.json). I have used a quick work around by using softlinks, to the actual training/evaluation files, but feel this is a little contrived.

I believe if the arguments `train_file` and `predict_file` are added in [SquadArguments](https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/src/transformers/data/datasets/squad.py#L36) and also line 152 in [SquadDataset](https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/src/transformers/data/datasets/squad.py#L152) is changed to 
                   
 `self.examples = self.processor.get_dev_examples(args.data_dir, filename=args.predict_file)`

and line 154 in [SquadDataset](https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/src/transformers/data/datasets/squad.py#L154)



`self.examples = self.processor.get_train_examples(args.data_dir, filename=arg.train_file)`

that may do the trick. At least this approach works in [run_squad.py](https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/examples/question-answering/run_squad.py#L444).

Thanks for your great work!


Comments:
Sorry duplicated my request.
BartConfig saving and loading inconsistency
Author: Liyang90
Pull request: no

## Environment info

### Who can help

 Bart: @sshleifer
 
## Information

Model I am using (Bert, XLNet ...): Bart

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x ] my own modified scripts: (give details below)

## To reproduce

Steps to reproduce the behavior:

```
from transformers import BartConfig
config_to_save = BartConfig.from_pretrained('facebook/bart-base', classif_dropout=0.42)
config_to_save.save_pretrained('./')
config_loaded = BartConfig.from_pretrained('./')
assert config_to_save.classif_dropout == config_loaded.classif_dropout, "what?"
```

## Expected behavior

Should raise no error.


Comments:
The input argument for `BartConfig.__init__()` should be named `classif_dropout ` instead of `classifier_dropout`

Great catch, thanks!
[bart] fix config.classif_dropout
Author: sshleifer
Pull request: yes


Comments:
This breaks backwards compatibility on saved `classif_dropout`, but from my checks this is always set to 0 (so incorrect) anyways and will stay 0, so I'm not too concerned.

```python
from transformers import BartConfig
config_to_save = BartConfig.from_pretrained('facebook/bart-base', classif_dropout=0.42)
config_to_save.classif_dropout # AttributeError
```
Docker GPU Images: Add NVIDIA/apex to the cuda images with pytorch
Author: AdrienDS
Pull request: yes

# What does this PR do?
- Use cuda:10.2 image instead of 10.1 (to address version mismatch  warning with pytorch)
- Use `devel` version that is built on the `runtime` and includes headers and development tools (it was otherwise failing to build apex). For a description of the different flavors, see:  https://hub.docker.com/r/nvidia/cuda -> Overview of Images
- Download and build `apex` for pytorch. https://github.com/NVIDIA/apex#quick-start

## Docs
- https://github.com/NVIDIA/apex
- https://nvidia.github.io/apex/
- https://hub.docker.com/r/nvidia/cuda

## Who can review?
- @mfuntowicz co-authored the Dockerfiles in 71c87119

Comments:
Hi @AdrienDS, 

Thanks for suggesting the changes.

Did you try building the image locally? I totally understand the motivation behind the use of the `-devel` layer parent, but I have concern regarding the final image size. Would it be possible for you to include the resulting size for th `devel` based image?

Otherwise look good for me!


Hi @mfuntowicz It does increase the size, from 4.46GB (v3.3.1) to 6.53GB for `transformers-pytorch-gpu`. 

If it's too large, could we create a separate image ? (like: `transformers-pytorch-gpu-apex`)

Ok, that shouldn't hurt too much, let's go!

Thanks for the contribution 👍
Add GPT2ForSequenceClassification based on DialogRPT
Author: LysandreJik
Pull request: yes

# What does this PR do?

This PR implements `GPT2ForSequenceClassification` in order to support DialogRPT.
Closes https://github.com/huggingface/transformers/issues/7493.

`GPT2ForSequenceClassification` uses the last token in order to do the classification, as other causal models (e.g. GPT-1) do.

Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a pad token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the pad tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch).

Here's how to replicate the results shown on the [original implementation](https://github.com/golsun/DialogRPT#use-rankers-only):

```py
from transformers import GPT2Tokenizer, GPT2ForSequenceClassification
import torch

tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2ForSequenceClassification.from_pretrained("directory_where_pth_config_are_saved")

model_input = tokenizer.encode("I love NLP!
Here’s a free textbook (URL) in case anyone needs it.", return_tensors="pt")
result = model(model_input, return_dict=True)
final_output = torch.sigmoid(result.logits)

print(final_output)
# tensor([[0.6129]], grad_fn=<SigmoidBackward>)
```

Once this PR is merged I'll open two "Good first issues":

- Implement `GPT2ForSequenceClassification` in TF2
- Implement sequence classification models for other causal transformers

Comments: