noamaanMulla-03/transformers-issues huggingface dataset:
[T5, examples] replace heavy t5 models with tiny random models
Author: patrickvonplaten
Pull request: yes

As first done by @sshleifer in #3488 , this PR puts a tiny T5 model on S3 to save testing time.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3556?src=pr&el=h1) Report
> Merging [#3556](https://codecov.io/gh/huggingface/transformers/pull/3556?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ae6834e028ecdf7fdbe886c1f86d0e02d5fef6f0&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3556/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3556?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3556   +/-   ##
=======================================
  Coverage   77.80%   77.80%           
=======================================
  Files         100      100           
  Lines       17064    17064           
=======================================
  Hits        13277    13277           
  Misses       3787     3787           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3556?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3556?src=pr&el=footer). Last update [ae6834e...2e14442](https://codecov.io/gh/huggingface/transformers/pull/3556?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[T5, TF 2.2] change tf t5 argument naming
Author: patrickvonplaten
Pull request: yes

**Problem**: 
As shown in #3539, in TF 2.2 errors occur due to the naming of the first argument in the `keras.layer.__call__` function of TF T5.
Previously `decoder_input_ids` was used as the first argument - which did not produce any errors in TF <= 2.1. In TF 2.2, it produces the error: 

```
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    797     else:
    798       raise ValueError(
--> 799           'The first argument to `Layer.call` must always be passed.')
    800 
    801     call_context = base_layer_utils.call_context()

ValueError: The first argument to `Layer.call` must always be passed.
```

**Conclusion**
It seems that we have to change to a consistent naming, being `inputs` for the first argument of every `keras.layer.__call__` function

Comments:
[T5, Testst] Add extensive hard-coded integration tests and make sure PT and TF give equal results
Author: patrickvonplaten
Pull request: yes

A direct comparison to google's official model seems quite hard - not sure if that's absolutely needed @thomwolf, @craffel

But some integration tests for T5 would be very nice, to be sure that changes in the future will not break T5.
This PR adds hard-coded integration tests, where the input for summarization is copied from Bart's summarization tests and the input for translation is taken from Appendix D of the official [paper](https://arxiv.org/pdf/1910.10683.pdf)

Checking the expected output for PT, one can see that the output looks quite good! 

- [x] Add PyTorch integration tests
- [x] Verify quality (subjectively for the moment)
- [x] Add TF integration tests
- [x] Same output for PT and TF 

UPDATE:
- Found a big bug in TF Beam Search generation (see comment) -> this PR fixes it 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3550?src=pr&el=h1) Report
> Merging [#3550](https://codecov.io/gh/huggingface/transformers/pull/3550?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6f5a12a5833d1e3783e4b8a42cb556b64085745e&el=desc) will **decrease** coverage by `0.49%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3550/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3550?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3550      +/-   ##
==========================================
- Coverage   78.30%   77.80%   -0.50%     
==========================================
  Files         100      100              
  Lines       17062    17062              
==========================================
- Hits        13360    13275      -85     
- Misses       3702     3787      +85     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3550?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/data/processors/squad.py](https://codecov.io/gh/huggingface/transformers/pull/3550/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Byb2Nlc3NvcnMvc3F1YWQucHk=) | `28.89% <0.00%> (-27.60%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3550?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3550?src=pr&el=footer). Last update [6f5a12a...c3ce2fe](https://codecov.io/gh/huggingface/transformers/pull/3550?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
No grad feature in model parameters
Author: lalalapotter
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I wonder if parameters in model.named_parameters() have the feature grad? If none, how can I add to it ?

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
This is more of a general PyTorch question than a Transformers-question. Have you tried asking on StackOverflow or the PyTorch forums?
Fix for continuing training
Author: xeb
Pull request: yes

If `args.should_continue` is true, then there is no way in the current example to reload a checkpoint since it is assumed it exists within the output_dir. Checking here fixes everything as expected.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3509?src=pr&el=h1) Report
> Merging [#3509](https://codecov.io/gh/huggingface/transformers/pull/3509?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/601ac5b1dc1438f00d09696588f2deb0f045ae3b&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3509/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3509?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3509   +/-   ##
=======================================
  Coverage   77.79%   77.79%           
=======================================
  Files         100      100           
  Lines       17051    17051           
=======================================
  Hits        13265    13265           
  Misses       3786     3786           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3509?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3509?src=pr&el=footer). Last update [601ac5b...43fda01](https://codecov.io/gh/huggingface/transformers/pull/3509?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Resizing embedding matrix before sending it to the optimizer.
Author: ngarneau
Pull request: yes

Hi there,

This is a minor bug when fine-tuning a pre-trained language model with a larger vocabulary using the run_lm_finetuning.py script.
Nothing major but I spent a couple of hours trying to figure out why my token embeddings were not being updated with their corresponding gradient :)

This bug will rise when you add new token types in the Tokenizer.
Since the resizing was done after passing the params to the optimizer, the wrong set of params for the embedding table were optimized.

Cheers

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3532?src=pr&el=h1) Report
> Merging [#3532](https://codecov.io/gh/huggingface/transformers/pull/3532?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d38bbb225f7b847e8be4e969cb9b40e7e4d798a6&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3532/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3532?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3532      +/-   ##
==========================================
- Coverage   77.81%   77.80%   -0.01%     
==========================================
  Files         100      100              
  Lines       17062    17062              
==========================================
- Hits        13276    13275       -1     
- Misses       3786     3787       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3532?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3532/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.81% <0.00%> (-0.14%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3532?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3532?src=pr&el=footer). Last update [d38bbb2...0475459](https://codecov.io/gh/huggingface/transformers/pull/3532?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Why is there not a SequenceClassification model for GPT-2?
Author: c-flaherty
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Why isn't there a SequenceClassificationModel (like there is for BERT) for GPT-2? I was able to implement this pretty easily by adding a "[CLS]" token to the vocabulary (like in the GPT2DoubleHeadsModel), appending sequences with "[CLS]", and then adding a linear layer that maps from the embedding of "[CLS]" to a vector of logits corresponding to the classes. After training, this model worked comparably to BertSequenceClassificationModel for my use-case. It would be nice to have this model in the transformers library and not have to code it up from scratch. 

If this sounds like a good idea, I can make a pull request with a GPT2SequenceClassificationModel added. If not, why is it not a good idea? 

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
My apologies, my computer glitched and posted twice. Please close this issue and refer to: https://github.com/huggingface/transformers/issues/3600
Update README.md
Author: ahotrod
Pull request: yes
Labels: name: model card

Update AutoModel & AutoTokenizer loading.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3603?src=pr&el=h1) Report
> Merging [#3603](https://codecov.io/gh/huggingface/transformers/pull/3603?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/81484b447b7d8504ff5e1cfff38ec35918383963&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3603/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3603?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3603   +/-   ##
=======================================
  Coverage   78.06%   78.06%           
=======================================
  Files         100      100           
  Lines       17134    17134           
=======================================
  Hits        13375    13375           
  Misses       3759     3759           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3603?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3603?src=pr&el=footer). Last update [81484b4...32340ca](https://codecov.io/gh/huggingface/transformers/pull/3603?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Update README.md
Author: ahotrod
Pull request: yes
Labels: name: model card

Update AutoModel & AutoTokernizer loading.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3604?src=pr&el=h1) Report
> Merging [#3604](https://codecov.io/gh/huggingface/transformers/pull/3604?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/81484b447b7d8504ff5e1cfff38ec35918383963&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3604/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3604?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3604   +/-   ##
=======================================
  Coverage   78.06%   78.06%           
=======================================
  Files         100      100           
  Lines       17134    17134           
=======================================
  Hits        13375    13375           
  Misses       3759     3759           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3604?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3604/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `93.45% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3604/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `92.10% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3604?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3604?src=pr&el=footer). Last update [81484b4...8845212](https://codecov.io/gh/huggingface/transformers/pull/3604?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Feature Request: Fill Mask more than 1 token
Author: p-christ
Pull request: no

At the moment you can use hugging face's mask filling pipeline to predict 1 masked token in a sentence using the below:

```
!pip install -q transformers
from __future__ import print_function
import ipywidgets as widgets
from transformers import pipeline

nlp_fill = pipeline('fill-mask')
nlp_fill("I am going to guess <mask> in this sentence")
```

The request is that you also add the ability to predict N masked tokens rather than only 1 masked token.  So that for example if the sentence is `"I am going to make <mask> <mask> for breakfast"` then the model might predict "fried eggs" for the 2 masked tokens

Comments:
Duplicate of #3609 

Not currently supported but we welcome a PR
Using Transformers for a Sequence with Multiple Variables at Each Step
Author: mnitin73
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

I have sequence data which I want to classify and predict future sequence. However, I   know that there are a few additional features which aso affect the subsequent values, each to a different extent. So I have multiple features for each step of input sequence. However, the output sequence can be just one feature for each step.

Input - [(x1, y1, z1), (x2,y2,z2), (x3,y3,z3)]
Output - [x4, x5, x6] or classification (a,b or c)

I do not want to concatenate x,y, and z as they have varying effects on the label and I think concatenating would give an erroneous result.

Need advise on whether Transformers, and specifically Huggingface Transformers can be used for this use case?

Thanks.
<!-- -->


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
cannot import name 'MODEL_FOR_TOKEN_CLASSIFICATION_MAPPING'
Author: vanh17
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BERT

Language I am using the model on (English, Chinese ...): ENglish

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
Run the run_ner.py script on examples/ner/


<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
This should be fixed on master since a8e3336a850e856188350a93e67d77c07c85b8af.

Feel free to re-open if that's not the case.

You might want to upgrade your repo.
`pip install --upgrade .`
How to increase model saving checkpoint from 50 to 1000?
Author: snijesh
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
When I run the script `run_squad.py` , it creates model check points every 50th iteration. How to increase model saving checkpoint from 50 to 1000. Where should I edit the code ?


Comments:
**You haven't to modify the source code in this script**. When you call `run_squad.py` script, you have to pass the `--save_steps` parameter and set its value to 1000 (as you can see [here](https://github.com/huggingface/transformers/blob/master/examples/run_squad.py#L429).) So, the entire command would be something like that: `python run_squad.py ... --save_steps 1000`

> ##  Questions & Help
> When I run the script `run_squad.py` , it creates model check points every 50th iteration. How to increase model saving checkpoint from 50 to 1000. Where should I edit the code ?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Related to this question, how about making the default value (50) bigger (e.g., 1000) in scripts such as `run_squad.py` and `run_ner.py`?
If a `--save_steps` option is not specified, and the default value is used, many checkpoints are saved.

@tomohideshibata you're right that 50 is too low, I bumped it to 500 in 335dd5e.

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to fine tune T5 like for translation tasks?
Author: prabalbansal
Pull request: no

# ❓ Questions & Help


## Details
<!-- Description of your issue -->
I want to pass 100000's of training instances to this syntax, but it says its limit is only 512. I am passing list of strings.
input_ids = tokenizer.encode('translate English to German: The house is wonderful. 
', return_tensors='pt')
lm_labels = tokenizer.encode('Das Haus ist wunderbar. 
', return_tensors='pt')
model(input_ids=input_ids, lm_labels=lm_labels)

And for 512 instances, it is not training, it is finishing in seconds without loading weights and training.

Could you explain how to fine-tune it in the correct way and use the fine-tuned model for the generation?
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
see https://github.com/huggingface/transformers/issues/3576