noamaanMulla-03/transformers-issues huggingface dataset:
[T5] Automatic setting of decoder_input_ids is misleading and does not correspond to the expected behavior of T5
Author: patrickvonplaten
Pull request: no

These lines: https://github.com/huggingface/transformers/blob/f62f2ffdcc2df75cf01438bebc7ae281d921d21d/src/transformers/modeling_t5.py#L1020 were added in this PR: https://github.com/huggingface/transformers/pull/5518 . 

@mfuntowicz - do we need this hack to make onnx work? I would prefer to revert this change.

The lines do not make much sense IMO and should be deleted. Also the T5 error message when only `input_ids` are passed should be updated to be clearer.

And the docs should be cleaned as well: https://huggingface.co/transformers/model_doc/t5.html#t5model.

Also see:https://github.com/huggingface/transformers/issues/7358

Comments:
[examples/s2s] clean up finetune_trainer
Author: patil-suraj
Pull request: yes

This PR 
1. moves the `build_compute_metrics_fn` to `utils.py` because we need to be able to import it for `hparam` search with `Seq2SeqTrainer`. Could have made it top level but since rest of the helpers are in `utils.py`, moved it there.
2. Also moves the `Seq2SeqDataCollator` to `utils` as the dataset is also there.

@sshleifer 


Comments:
Also 
https://github.com/huggingface/transformers/blob/a42f62d34f7b2acdb7298e586adbe9f0f28864ea/examples/seq2seq/seq2seq_trainer.py#L56

we are using `model.config` here, this breaks on multi-gpu, right ?
and IMO we might not need this `assert` here 

```
self.pad_token_id = self.model.config.pad_token_id
```
This will also break.
Maybe we could just stuff the config or (`pad_token_id`) on `data_args`?





I may start `metrics.py` and move rouge/bleu funcs and their helpers in there as well. 

> ```
> self.pad_token_id = self.model.config.pad_token_id
> ```
> 
> This will also break.
> Maybe we could just stuff the config or (`pad_token_id`) on `data_args`?

we could pass `config` directly to `init`

works for me! <3 config
Clean the Trainer state
Author: sgugger
Pull request: yes

# What does this PR do?

This PRs clean the fields used inside the `Trainer` to store state and gather them all in a clear class named `TrainerState` that is typed so the user knows exactly what they can access when subclassing and overriding methods (and for my next step, when writing callbacks).

As a result, the `log_history` does not need to be saved and loaded separately and the hack to get the steps we were at from the checkpoint folder name is removed (the user can copy all the checkpoint saved by the `Trainer` in a previous training in any folder and use it for resuming training). This PRs adds a test of the full reproducibility of a training resumed from a checkpoint.

There is a tiny breaking change that would affect users that trained a model using an earlier version of transformers and would like to resume it with a version obtained after this commit but I don't think this matters much.

It also enforces that the `TrainingArguments` passed to a `Trainer` are not changed during training, to avoid subtle bugs when launching several trainings in a row. This is verified by a new test.

Comments:
Tests pass in a multi-GPU environment and the specific `test_distributed_trainer` passes too. There is just one test that requires a batch size not too big so I manually skip it if there are more than 2 GPUs.

Nice PR!!!

I think it is a nice addition. Only the `global_step` won't be necessary as it is already integrated by default into the TF checkpoints. This is perfect!

I plan also to add Keras callbacks into the TF Trainer that can handles few more arguments that are in this "state" class such as best checkpoint, and metrics.

It won't change much in the TF Trainer and could be very easily integrated as most of the "state" arguments work the same way for both Trainers. We can clearly imagine this state class for a much broader usage.

Checked the tests on TPU are passing, so can safely merge this.
Fix seq2seq example test
Author: sgugger
Pull request: yes

# What does this PR do?

#7490 removed the log_history save since it is now saved along inside the `TrainerState`. I didn't catch it was used in the seq2seq examples (must be due to a more recent PR because the tests were passing) so this PR adapts the part that loads `log_history` in those tests to fix them.

Comments:
[s2s] trainer scripts: Remove --run_name, thanks sylvain!
Author: sshleifer
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
You're welcome ;-)

cc @patil-suraj
MultiGPU Trainer: each processes uses more memory than 1 GPU job
Author: sshleifer
Pull request: no
Labels: name: Distributed Training / Models

I tried to run an 8 GPU training job, and it OOM'd, so I investigated whether it could run on 1 GPU. It could!

So here are two commands, the first one says that it is using 14814MiB on GPU 0.
The second says it is using 
`15594MiB` on each.

This doesn't happen in PL, which leads me to believe that `distributed_scalars` is to blame, but I am not sure. Has anyone run into this?
cc @sgugger @patil-suraj 


### Delta between two commands

```
- CUDA_VISIBLE_DEVICES=0 python
+ CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2
```


### Command 1
```bash
CUDA_VISIBLE_DEVICES=0 python finetune_trainer.py \
    --model_name_or_path student_pegasus_cnn_12_2 \
    --data_dir cnn_dm \
    --output_dir dpx_cnn_12_2_pl_comb_noDO --overwrite_output_dir --freeze_embeds \
    --learning_rate=3e-5 \
    --warmup_steps 500 --sortish_sampler \
    --gradient_accumulation_steps=4 \
    --per_device_train_batch_size=4 --per_device_eval_batch_size=8 --eval_beams 2 \
    --num_train_epochs=5 \
    --save_steps 3000 --eval_steps 3000 \
    --logging_first_step \
    --max_target_length 56 --val_max_target_length 142 --test_max_target_length 142 \
    --do_train --do_eval --do_predict --evaluate_during_training \
    --predict_with_generate --load_best_model_at_end
```


### Command 2
```bash
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 fine
tune_trainer.py \
    --model_name_or_path student_pegasus_cnn_12_2 \
    --data_dir cnn_dm \
    --output_dir dpx_cnn_12_2_pl_comb_noDO --overwrite_output_dir --freeze_embeds \
    --learning_rate=3e-5 \
    --warmup_steps 500 --sortish_sampler \
    --gradient_accumulation_steps=4 \
    --per_device_train_batch_size=4 --per_device_eval_batch_size=8 --eval_beams 2 \
    --num_train_epochs=5 \
    --save_steps 3000 --eval_steps 3000 \
    --logging_first_step \
    --max_target_length 56 --val_max_target_length 142 --test_max_target_length 142 \
    --do_train --do_eval --do_predict --evaluate_during_training \
    --predict_with_generate --load_best_model_at_end
```

Comments:
This seems to be a duplicate of #7169, finishing something and will investigate now that I have a proper multi-GPU steup.

Yes, most likely a duplicate.
One clue might be the call to `DataParallel` right before the call to `DistributedDataParallel`. 

Investigation seems to lead to: this is normal to have slightly more memory use per GPU in distributed mode since PyTorch keeps two copies in the gradients in that case. See [this issue](https://github.com/pytorch/pytorch/issues/37030).

You are correct, thank you for investigating. The difference was using `--adafactor` which is saving about 3GB.
I added that option to Seq2SeqTrainer. Take it if you want it :)
Enable pegasus fp16 by clamping large activations
Author: sshleifer
Pull request: yes

<!-- This line specifies which issue to close after the pull request is merged. -->
+ If encoder layer is about to return inf, clamp to a float **near** the boundary of fp16 range
+ This improves rouge for distil-pegasus-xsum-12-12 in fp16 by 0.1 ROUGE 2
+ I cannot think of how this change could make things worse.
+ Other models' metrics are unaffected!

Comments:
Hi,

Any chance of implementing the same for the "pegasus-cnn_dailymail" (distilled) model?

Regards,
Karthik

@karthikgali this PR implements for all models that inherit from Bart. This includes pretty much every `sshleifer/` checkpoint and every `pegasus/` checkpoint.

Pegasus variants with 16 encoder layers still do not work well in fp16, but I will push a 12 layer distilled version soon that does!
german distilbert not available?
Author: datistiquo
Pull request: no

I try using the distilbert-base-german-cased

which is also listed here

https://huggingface.co/transformers/pretrained_models.html

but I get the message that this model is not avaiable. Seems also not here listed?

https://huggingface.co/models?filter=tf,de

Is this just currently the case?

Comments:
Or is this model only available for pytorch and not TF?

https://huggingface.co/distilbert-base-german-cased

^^ currently only the Pytorch weights, but you can load into TF pretty easily (follow the README/doc). If needed we can upload the converted TF weights (cc @stefan-it who I believe trained this model? This predates our user/organization namespaces)

And I've added a `de` language tag so that the model is discoverable via https://huggingface.co/models?filter=de&search=distilbert

Thanks for reporting!

Oh, we've an open issue regarding to the TF checkpoint 😅

https://github.com/dbmdz/berts/issues/8

@julien-c could you convert the model and upload it (not sure if I've access to root S3), thanks :heart: 



> but you can load into TF pretty easily (follow the README/doc).

COuld you show me where I can read about htis? Cab just find from the getting stared tour:

`bert_model = TFDistilBertModel.from_pretrained('distilbert-base-german-cased', from_tf=False)
`
...but this does not work...

`from_pt=True`?

Ok, great. So easy.^^
Weird Behavior on XLNetTokenizer after new tokens added
Author: riven314
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: ubuntu
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.1 (GPU)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@mfuntowicz
 
## Information
I use XLNetTokenizer (pretrained) and added a new token in it. After that, the output from ```tokenizer.tokenize``` looks weird.
(I am not sure if the problem comes from ```transformers``` or ```tokenizers```, but I post here anyway.)

## To reproduce
```
from transformers import XLNetTokenizer
tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')

test = 'This is so awesome!!@username!'
out = tokenizer.tokenize(test)

# without new tokens added, @username is broken down as expected
out
>> ['▁This', '▁is', '▁so', '▁awesome', '!!', '@', 'user', 'name', '!']
```

```
from tokenizers import AddedToken

# introduce new tokens (with white-space on right only)
new_tokens = AddedToken(
    '@username', single_word = True, 
    lstrip = False, rstrip = True
)
tokenizer.add_tokens(new_tokens)

out = tokenizer.tokenize(test)

# weird result about the white-space around new tokens
out
>> ['▁This', '▁is', '▁so', '▁awesome', '!!', '@username', '▁', '!']
```

In here, there are two places weird:
1. new tokens "@username" has a white-space on the left, so ""!!@username" should not be broken down into "!!", "@username" (I think it should be broken down into "!!", "@", "user", "name", "!")
2. I am a bit confused on why there is a white-space produced after "@username" token. (i.e. '@username', '▁', '!')

And oddly, when I encode and decode the sentence back, the white-space token after "@username" is not translated to actual whitespace. (Also note there is a white space added before "@username" which mean the new token is correctly identified to have a white-space on the left):
```
enc = tokenizer.encode(test, add_special_tokens = False)
dec = tokenizer.decode(enc)

# in encoding stage, the 2nd last token is whitespace
enc 
>> [122, 27, 102, 8729, 7675, 32000, 17, 136]
# in decoding stage, the white-space disappear

dec
>> This is so awesome!! @username!
```



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Errors while creating a subclass of BertForTokenClassification in run_ner.py file
Author: vikas95
Pull request: no
Labels: name: wontfix


 Question 1) I want to skip calling AutoModelForTokenClassification ( https://github.com/huggingface/transformers/blob/f7f03b22dc15543317635770f312adf4513303d0/examples/token-classification/run_ner.py#L158 ) and directly use  BertForTokenClassification. When I am directly using BertForTokenClassification in run_ner.py using the same parameters which were fed to AutoModelForTokenClassification, it does not load the weights from pretrained BERT checkpoint. 
Can you suggest what changes I have to make in the input parameters (most likely in the "config") such that it loads the weights from pretrained BERT checkpoint. 


Question 2) I also tried to create a subclass of BertForTokenClassification and call it within AutoModelForTokenClassification. In this case, it throws an error - AttributeError: 'BertConfig' object has no attribute 'use_return_tuple'   .
Can anyone please suggest something to fix this error which is caused while just using a subclass of BertForTokenClassification instead of the original BertForTokenClassification class. Please note that the subclass is exactly same as BertForTokenClassification.

Thanks.



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Error in the RobertaTokenizer?
Author: h56cho
Pull request: no
Labels: name: wontfix

Hello, 

I think I spotted an error in the `RobertaTokenizer` (`AutoTokenizer`).

I tested the `tokenizer` function by using the code below:

```python
import torch
from torch.nn import CrossEntropyLoss
from matplotlib import pyplot as plt
from transformers import RobertaTokenizer, RobertaForMultipleChoice, AdamW, get_constant_schedule
from transformers import AutoTokenizer
import numpy as np
import pandas as pd
import pickle
import dill
from matplotlib.pyplot import plot, savefig, xlim, figure, ylim, legend, boxplot, setp, axes, xlabel, ylabel, xticks
import gc 
import math
import time
from random import seed
from random import randint
import sys
import statistics
from numpy import nan
import scipy.stats as ss
from statistics import mode
from pylab import *
from numpy import arange

# import the pre-trained HuggingFace RobertaTokenizer
tokenizer = AutoTokenizer.from_pretrained('roberta-base')

# get the pre-trained HuggingFace RobertaForMultipleChoice and resize the token embeddings after adding the special token
best_model_roberta = RobertaForMultipleChoice.from_pretrained('roberta-base', output_hidden_states = True)

sequence_a = 'china is a very large country .'
sequence_b = 'indeed'

tokenizer(sequence_a, sequence_b, padding = True, return_token_type_ids=True, return_tensors="pt")
```
and below is the corresponding output:

```
{
'input_ids': tensor([[ 0,  611,  1243,  16,  10,  182,  739,  247,  479, 2,  2, 2028, 10247, 2]]),
 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]), 
'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])
}
```

The output above is not what I was expecting to get from the `tokenizer` function. Since I am inputting `sequence_a` along with `sequence_b`, I was expecting my `token_type_ids` to be: `tensor([[0,0,0,0,0,0,0,0,0,0,0,1,1,1]])`, but the output does not include any 1's...everything is 0. Perhaps this is the error within the `RobertaTokenizer`?

Thank you,

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
lightning_base: new clarg: adam_betas
Author: sshleifer
Pull request: no
Labels: name: wontfix

should be able to pass `adam_betas` through command line like this [command](https://github.com/pytorch/fairseq/blob/master/examples/mbart/README.md#finetune-on-en-ro)


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
model.roberta.from_pretrained() fails to change the parameters
Author: wyin-Salesforce
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Hi all, my problem is "how to use a pretrained 3-way sequence classification model to fine-tune on a 2-way classification task". I am using "https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_glue.py"; 
in the code I change

```
    model_args.model_name_or_path = 'roberta-large'
    model = AutoModelForSequenceClassification.from_pretrained(
        model_args.model_name_or_path,
        from_tf=bool(".ckpt" in model_args.model_name_or_path),
        config=config,
        cache_dir=model_args.cache_dir,
    )
    model.roberta.from_pretrained(path_to_my_3way_pretrained_model)
```
This is what I tried. Pls note that I can not used the "AutoModelForSequenceClassification.from_pretrained()" to load "path_to_my_3way_pretrained_model" directly because there will be class mismatch (i.e., 3-way model does not apply to 2-way task); but I think no matter it is 3-way or 2-way sequence classification, what their architectures share is the roberta part; so I used the model.roberta to load the parameters from my pretrained model. 

However, I found the roberta parameters do not change, I checked as follows
```
    for name, param in model.named_parameters():
        if param.requires_grad and name == 'roberta.encoder.layer.16.attention.self.value.weight':
            print('new:', name, param.data)
```

Any clue why it doesn't work? Or any solution how to use a fine-tune a pretrained 3-way model on a 2-way task? Thanks

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.