noamaanMulla-03/transformers-issues huggingface dataset:
Update shortcut name for reformer in pretrained_models.srt
Author: SamuelCahyawijaya
Pull request: yes

Adding `google/` prefix on shortcut name for reformer in `pretrained_models.srt` according to mapping in `configuration_reformer.py`

Comments:
I agree with the change but I think you probably broke the rst

> I agree with the change but I think you probably broke the rst

Ah, my bad, I have updated the rst file

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5849?src=pr&el=h1) Report
> Merging [#5849](https://codecov.io/gh/huggingface/transformers/pull/5849?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9d37c56bab8f7f1f1aa0b65be039516072254e77&el=desc) will **decrease** coverage by `1.19%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5849/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5849?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5849      +/-   ##
==========================================
- Coverage   78.48%   77.28%   -1.20%     
==========================================
  Files         146      146              
  Lines       26200    26200              
==========================================
- Hits        20563    20249     -314     
- Misses       5637     5951     +314     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5849?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/5849/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `23.38% <0.00%> (-73.39%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/5849/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `93.36% <0.00%> (+49.37%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5849?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5849?src=pr&el=footer). Last update [9d37c56...1103b9e](https://codecov.io/gh/huggingface/transformers/pull/5849?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I check that this update has been included in the main repo directly, closing this  PR
docs(pretrained_models): fix num parameters
Author: amineabdaoui
Pull request: yes

Correct the number of parameters of BERT based models.

Sometimes the difference is important. For instance:
`bert-base-uncased` has **110M parameters** but `bert-base-multilingual-cased` has more than **178M parameters**, even if both models share the same architecture (12-layer, 768-hidden, 12-heads).

The difference is due to the vocabulary size:
`bert-base-uncased` uses a vocabulary of 30k entries while `bert-base-multilingual-cased` uses a vocabulary of 119k entries.

Comments:
I created a new PR with the correct formatting: https://github.com/huggingface/transformers/pull/6043
add a summary report flag for run_examples on CI
Author: stas00
Pull request: yes

Currently, it's hard to derive which example tests were run on CI, and which weren't. Adding `-rA` flag to `pytest`, will now include a summary like:

```
==================================================================== short test summary info =====================================================================
PASSED examples/test_examples.py::ExamplesTests::test_generation
PASSED examples/test_examples.py::ExamplesTests::test_run_glue
PASSED examples/test_examples.py::ExamplesTests::test_run_language_modeling
PASSED examples/test_examples.py::ExamplesTests::test_run_squad
FAILED examples/test_examples.py::ExamplesTests::test_run_pl_glue - AttributeError: 'Namespace' object has no attribute 'gpus'
============================================================ 1 failed, 4 passed, 8 warnings in 42.96s ============================================================
```
which makes it easier to validate whether some example is being covered by CI or not.

The PR came about following on the discussion at https://github.com/huggingface/transformers/pull/6027#issuecomment-663894148

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6035?src=pr&el=h1) Report
> Merging [#6035](https://codecov.io/gh/huggingface/transformers/pull/6035?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/c69ea5efc4eac65b183e8d07b1bf91d20bbe0c8c&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6035/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6035?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6035      +/-   ##
==========================================
+ Coverage   78.50%   78.52%   +0.01%     
==========================================
  Files         146      146              
  Lines       26249    26249              
==========================================
+ Hits        20606    20611       +5     
+ Misses       5643     5638       -5     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6035?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6035/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.71% <0.00%> (+1.25%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6035?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6035?src=pr&el=footer). Last update [c69ea5e...9bca1bc](https://codecov.io/gh/huggingface/transformers/pull/6035?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
For Roberta pretraining, how to enable large batch training using gradient accumulation?
Author: quincyliang
Pull request: no

In the example code, where can I enable gradient accumulation for large batch size training. Thanks. https://github.com/huggingface/transformers/tree/master/examples/language-modeling

Comments:
Hi! You can use the `--gradient_accumulation_steps=N_STEPS` argument to the `run_language_modeling.py` script for that.

You can see all available flags by doing `python run_language_modeling.py --help`
Model card for Vamsi/T5_Paraphrase_Paws
Author: Vamsi995
Pull request: yes
Labels: name: model card

Creating a model card for my uploaded model on the transformers hub.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6037?src=pr&el=h1) Report
> Merging [#6037](https://codecov.io/gh/huggingface/transformers/pull/6037?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/c69ea5efc4eac65b183e8d07b1bf91d20bbe0c8c&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6037/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6037?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6037      +/-   ##
==========================================
+ Coverage   78.50%   78.51%   +0.01%     
==========================================
  Files         146      146              
  Lines       26249    26249              
==========================================
+ Hits        20606    20610       +4     
+ Misses       5643     5639       -4     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6037?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6037/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (+1.00%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6037?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6037?src=pr&el=footer). Last update [c69ea5e...4cf27e8](https://codecov.io/gh/huggingface/transformers/pull/6037?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks!
Update model_summary.rst
Author: xwen99
Pull request: yes

Add '-' to make the reference of Transformer-XL more accurate and formal.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5737?src=pr&el=h1) Report
> Merging [#5737](https://codecov.io/gh/huggingface/transformers/pull/5737?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/cd30f98fd24837f285cfc221b91cfa66b1b38c32&el=desc) will **decrease** coverage by `0.77%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5737/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5737?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5737      +/-   ##
==========================================
- Coverage   78.02%   77.25%   -0.78%     
==========================================
  Files         146      146              
  Lines       26055    26055              
==========================================
- Hits        20329    20128     -201     
- Misses       5726     5927     +201     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5737?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/5737/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `23.38% <0.00%> (-73.39%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5737/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (+0.25%)` | :arrow_up: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5737/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `82.31% <0.00%> (+1.28%)` | :arrow_up: |
| [src/transformers/tokenization\_xlm\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/5737/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxtX3JvYmVydGEucHk=) | `95.23% <0.00%> (+10.71%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5737/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `95.18% <0.00%> (+74.91%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5737?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5737?src=pr&el=footer). Last update [cd30f98...5347731](https://codecov.io/gh/huggingface/transformers/pull/5737?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
How I can  predict missing letters in a sentence, like " I want to b _ _  the car because it is cheap."
Author: taicaile
Pull request: no

Hi,

I am new on NLP, I want to predict missing letters in a sentence. Here is an example,

```text
I want to b _ _  the car because it is cheap.
```


Comments:
I am not sure how to predict letters, but you can use BERT to predict words. 

I'd try to train a character-level model. Some of the Reformer models are pretrained in a char-level setting, if I remember correctly: https://huggingface.co/models?search=reformer

In the future however, this question is more suited to [discuss.huggingface.co](https://discuss.huggingface.co)
Draft Etalab QA model
Author: psorianom
Pull request: yes
Labels: name: model card

Card for the model https://huggingface.co/etalab-ia/camembert-base-squadFR-fquad-piaf

Comments:
Is this ready to merge? I'll merge now and feel free to update later (we'll make it way easier to iterate on model cards in the medium-term future)

(You can input model-specific examples to the inference QA widget if you need to, see https://huggingface.co/docs#how-can-i-control-my-models-widgets-example-inputs)

Top ! Thanks @julien-c  ! Yes, I will update it with model-specific examples and fixing some typos also.
Remove unused file
Author: sgugger
Pull request: yes

This seems to be the old script to deploy the docs, new one is `.circleci/deploy.sh`.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6023?src=pr&el=h1) Report
> Merging [#6023](https://codecov.io/gh/huggingface/transformers/pull/6023?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a884b7fa38de9082a6f3f7889b9f7348a8dadbf5&el=desc) will **decrease** coverage by `1.37%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6023/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6023?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6023      +/-   ##
==========================================
- Coverage   78.68%   77.31%   -1.38%     
==========================================
  Files         146      146              
  Lines       26249    26249              
==========================================
- Hits        20655    20295     -360     
- Misses       5594     5954     +360     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6023?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/6023/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `23.38% <0.00%> (-73.39%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/6023/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `26.02% <0.00%> (-69.52%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/6023/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `98.79% <0.00%> (+33.89%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/6023/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `93.36% <0.00%> (+49.37%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6023?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6023?src=pr&el=footer). Last update [a884b7f...0cb4788](https://codecov.io/gh/huggingface/transformers/pull/6023?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[docs] AutoModelWithLMHead(model_name, **kwargs)
Author: sshleifer
Pull request: no
Labels: name: Documentation

`AutoModelWithLMHead.from_pretrained` cannot accept `output_attentions=True`

This [docstring](https://huggingface.co/transformers/model_doc/auto.html?highlight=transformers%20automodelwithlmhead#transformers.AutoModelWithLMHead) suggests that it can.

Thanks @jrvc for discovering!

```python
import transformers
modelname='Helsinki-NLP/opus-mt-en-de'
config_overrider={'output_attentions':True, 'output_hidden_states':True}
self.model = transformers.AutoModelWithLMHead.from_pretrained(modelname, **config_overrider)
=>
*** TypeError: __init__() got an unexpected keyword argument 'output_attentions'
```


I will dig deeper!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


unstale?

This was fixed with https://github.com/huggingface/transformers/pull/5665!
Possible bug in preparing deocder_input_ids for T5 in seq2seq finetune.y
Author: patil-suraj
Pull request: no

In finetune.py in `_step` method ([link](https://github.com/huggingface/transformers/blob/master/examples/seq2seq/finetune.py#L131)) `decoder_input_ids` are prepared like this
```python3
source_ids, source_mask, target_ids = batch["input_ids"], batch["attention_mask"], batch["decoder_input_ids"]
decoder_input_ids = target_ids[:, :-1].contiguous()  # Why this line?
lm_labels = target_ids[:, 1:].clone()  # why clone?
```

The `T5ForConditionalGeneration` automatically prepares `decoder_input_ids` using `lables` when they are not passed. It uses the _shift_right [method](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_t5.py#L620). 
```python3
decoder_start_token_id = self.config.decoder_start_token_id
pad_token_id = self.config.pad_token_id
# shift inputs to the right
shifted_input_ids = input_ids.new_zeros(input_ids.shape)
shifted_input_ids[..., 1:] = input_ids[..., :-1].clone()
shifted_input_ids[..., 0] = decoder_start_token_id
```

so finetune.py doesn't add decoder start token id as required by T5 model. This works for bart because for bart input starts with `bos` (`
`) which the tokenizer adds automatically. So for T5 model this removes the first token from `lm_labels` and adds no start token in `decoder_input_ids`.

### To reproduce
```python3
from transformers import T5ForConditionalGeneration, T5Tokenizer, T5Config
tokenizer = T5Tokenizer.from_pretrained("t5-base")
config = T5Config.from_pretrained("t5-base")

batch = tokenizer(["simple is better than complex 
"], return_tensors="pt")

# from finetune.py
pad_token_id = tokenizer.pad_token_id
target_ids = batch["input_ids"]
decoder_input_ids = target_ids[:, :-1].contiguous()  # Why this line?
lm_labels = target_ids[:, 1:].clone()  # why clone?

print(decoder_input_ids[0])
# => tensor([ 650,   19,  394,  145, 1561])
print(tokenizer.convert_ids_to_tokens(decoder_input_ids[0]))
# => ['▁simple', '▁is', '▁better', '▁than', '▁complex']
print(tokenizer.decode(lm_labels[0]))
# => is better than complex


# from T5PreTrainedModel._shift_right
decoder_start_token_id = config.decoder_start_token_id
input_ids = batch["input_ids"]
shifted_input_ids = input_ids.new_zeros(input_ids.shape)
shifted_input_ids[..., 1:] = input_ids[..., :-1].clone()
shifted_input_ids[..., 0] = decoder_start_token_id

print(shifted_input_ids[0])
# => tensor([   0,  650,   19,  394,  145, 1561])
print(tokenizer.convert_ids_to_tokens(shifted_input_ids[0]))
# => ['<pad>', '▁simple', '▁is', '▁better', '▁than', '▁complex']
print(tokenizer.decode(input_ids[0]))
# => simple is better than complex
```
@sshleifer 

Comments:
This could be one of the reasons for strange T5 behaviour.

Yes this is an excellent catch @patil-suraj !
Fix typo (model saving TF)
Author: astariul
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5734?src=pr&el=h1) Report
> Merging [#5734](https://codecov.io/gh/huggingface/transformers/pull/5734?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f0bda06f43a0d5e4ef80ad0f1812027b658b724d&el=desc) will **increase** coverage by `0.26%`.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5734/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5734?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5734      +/-   ##
==========================================
+ Coverage   77.05%   77.31%   +0.26%     
==========================================
  Files         146      146              
  Lines       26012    26012              
==========================================
+ Hits        20043    20111      +68     
+ Misses       5969     5901      -68     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5734?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/trainer\_tf.py](https://codecov.io/gh/huggingface/transformers/pull/5734/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyX3RmLnB5) | `16.53% <0.00%> (ø)` | |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/5734/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `43.98% <0.00%> (-49.38%)` | :arrow_down: |
| [src/transformers/generation\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5734/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3V0aWxzLnB5) | `96.82% <0.00%> (-0.29%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5734/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (+0.25%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/5734/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `95.53% <0.00%> (+69.51%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5734?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5734?src=pr&el=footer). Last update [f0bda06...74755d7](https://codecov.io/gh/huggingface/transformers/pull/5734?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
new AutoModel classes in pipeline
Author: patil-suraj
Pull request: no

The `translation`, `text-generation` and `fill-mask` pipelines still use the `AutoModelWithLMHead` class but from the warning it seems that it's going to be deprecated. Is there any specific reason for this or can we use the new `AutoModelForSeq2SeqLM` for translation, `AutoModelCausalLM` for text-generation and `AutoModelForMaskedLM` for fill-mask pipeline ? If yes then I'll be happy to open a PR.

@sgugger 

Comments: