noamaanMulla-03/transformers-issues huggingface dataset:
Add TFGPT2ForSequenceClassification based on DialogRPT
Author: spatil6
Pull request: yes

# What does this PR do?

This PR implements TFGPT2ForSequenceClassification in order to support DialogRPT.
Strongly based on modifications made in #7501
Fixes #7622 


## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@LysandreJik Please review this PR, let me know if there is anything that should be changed =)

Comments:
Thanks for review @jplu . I'll update my code with review comments and new input processing.


> Thank you very much for this very nice addition!!
> 
> I left few comments on it. Also can you run the following piece of code and tell me if it works properly:
> 
> ```
> import tensorflow as tf
> from transformers import GPT2Tokenizer, TFGPT2ForSequenceClassification
> 
> model = tf.function(TFGPT2ForSequenceClassification.from_pretrained("microsoft/dialogrpt"))
> tokenizer = GPT2Tokenizer.from_pretrained("microsoft/dialogrpt")
> inputs = tokenizer("Hello", return_tensors="tf")
> model(inputs)
> ```
> 
> @LysandreJik I would recommend as well to wait a bit that the new input processing to be merged.

<img width="1176" alt="output" src="https://user-images.githubusercontent.com/6419011/100481765-2c2da900-311b-11eb-8fdd-15762f7d43df.png">


Hello @jplu and @LysandreJik ,
I have refactored code as per review comments and added new input processing as well.

Kindly review.

> Much better!! Thanks for the updates.
> 
> There is still one comment to be addressed and the tests to fix.

@jplu tests are also fixed now.

@spatil6 we have merged today a PR that updates the way the booleans are processed. You can see an example in the TF BERT file for example, can you rebase and proceed to the same changes please. It would be awesome if you could do it!

> @spatil6 we have merged today a PR that updates the way the booleans are processed. You can see an example in the TF BERT file for example, can you rebase and proceed to the same changes please. It would be awesome if you could do it!

Sure, will do that.
checkpoint callbacks 
Author: rabeehk
Pull request: no

Hi
I need to save more things during checkpoint and I am using finetune_trainer.py not the pytorch lightening one, could you provide me with example on how I can modify the save part of the checkpoints? thanks 

Comments:
In particular I need to define a callback with on_save function, for this I need to be able to acces "is_world_process_zero" and I am not sure how to do it. If I want to pass trainer to callback, how can I make sure this is passed? thanks

solved with understanding is_world_process_zero is accessible via  state.is_world_process_zero
How to get the word embedding  after pre-training ? for example, a embedding matrix
Author: AI678
Pull request: no
Labels: name: wontfix

I am excited on this great model. And I want to get the word embedding . Where shold I find the file from output or should I change to code to do this?

Comments:
It depends of what you understand as "embedding", as it can be ambiguous with transformer models. 

Embeddings can be the embedding matrix, which returns context-less embedding of tokens, that you can obtain with `model.get_input_embeddings()`.

Embeddings can also be understood as the features generated by the base model, which are token embeddings with context (depends on the tokens surrounding the token you're studying). You can simply do a forward pass through the base models (e.g., `BertModel`, `GPT2Model`, etc.) to get these embeddings.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
modeling_xlnet.py:283: UserWarning: Mixed memory format inputs detected while calling the operator.
Author: fhamborg
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help

@TevenLeScao 

## Information

Model I am using (Bert, XLNet ...): xlnet-base-uncased

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
*  [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Simply invoke XLNet using the following

``` 
last_hidden_state, mems = lm(
                input_ids=input_ids,
                token_type_ids=token_type_ids,
                output_hidden_states=True,
            )
```

I receive a warning as follows:

```
...\anaconda3\envs\newstsc2\lib\site-packages\transformers\modeling_xlnet.py:283: UserWarning: Mixed memory format inputs detected while calling the operator. The operator will output contiguous tensor even if some of the inputs are in channels_last format. (Triggered internally at  ..\aten\src\ATen\native\TensorIterator.cpp:918.)
  attn_score = (ac + bd + ef) * self.scale
```

## Expected behavior

no warning

Comments:
Hey! I haven't been able to reproduce this with what you've given, maybe it is linked to your 'token_type_ids' ? I don't really have enough info on what you're trying to do with only this line. In any case:

 - The warning text seems innocuous.
 - I don't see this line in the current (master) codebase, so I'd suggest upgrading and seeing if this issue still crops up.

@TevenLeScao  FYI:  I have the same warning message, on Colab,using Xlnet-base-cased (custom script , custom dataset)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Can we resize embedding with embedding weighted initialized differently??
Author: guoxuxu
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help


When we add new tokens, this method automatically adds embedding using torch nn.Embedding. 
https://huggingface.co/transformers/_modules/transformers/modeling_utils.html#PreTrainedModel.resize_token_embeddings

The documentation says the resized embeddings are nn. Embedding, which said they by default initialize weights from N(0, 1) (https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html). But I have checked that the resized embedding weights are almost N(0, 0.01) or N(0, 0.02)? Can I check **the true distribution of resized embedding weights** ?

## If I want the embedding weights initialized differently, how can I achieve that efficiently?

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
One way is to use `config.initializer_range`

#6562

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
T5: Finetuning the language modelling objective on a  new dataset
Author: sb1992
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

I was wondering if there is a way I could fine tune T5 model on my own dataset. There are scripts for fine tuning help for GPT2, BERT and XLNET in language modelling examples, so was thinking if that could be extended to T5 as well?

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
have a look on: https://github.com/huggingface/transformers/tree/master/examples

for conditional generation in the folder seq2seq is an example for finetuning

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
what is the perplexity of distilbert-base-uncased ? 
Author: OleNet
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help


## Details

In the [readme](https://github.com/huggingface/transformers/tree/master/examples/distillation) , it is said that distilbert-base-uncased is pretraind on the same data used to pretrain Bert, so I wonder what is the final perplexity or cross entropy of the pretrain?


Comments:
Hey @OleNet ,

You probably have a better chance of getting an anwser by posting your question on the discussion forum here: `http://discuss.huggingface.co/`.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Help with finetuning mBART on an unseen language
Author: laibamehnaz
Pull request: no
Labels: name: wontfix

 I wanted to know how could we finetune mBART on summarization task on a different language than that of English. Also, how can we finetune mBART on translation task where one of the languages is not present in the language code list that mBART has been trained on. 
Appreciate any help!! Thank you.



Comments:
Hey @laibamehnaz , that's a great question for forum
https://discuss.huggingface.co/.

Could you post it there, someone might have tried it and the forum is better to discuss such questions :) 

Great. I will post it there. Thanks a lot.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[finetune trainer] better logging and help
Author: stas00
Pull request: yes

As a follow up to this [thread](https://discuss.huggingface.co/t/summarization-is-finetune-trainer-py-accepting-length-arguments-correctly/2879/) this PR:

* documents that `--val_max_target_length` is also used during `generate`
* disambiguates `use_task_specific_params` logger so that it's clear that it dumps just the initial params and that those could be overridden by user's cl args

@sgugger 

Comments:
n_gpu=1 in case of using distributed pytorch launcher 
Author: rabeehkarimimahabadi
Pull request: no

Hi
I am using transformer 4.1.1 on multiple gpus with distributed launch script below, the n_gpu set in this case is 1 instead of actual n_gpus, could you tell me how I can access to the total number of ranks calling the launcher with? Is there any variable in huggingface library which could show the number of gpus the script is called with? Here is the command I run: 

`export BS=4; CUDA_VISIBLE_DEVICES=0,1  USE_TF=0   python -m torch.distributed.launch --nproc_per_node=2 --master_port=9915  finetune_trainer.py --model_name_or_path t5-small --output_dir output_dir --adam_eps 1e-06 --data_dir wmt_en_ro --do_train --freeze_embeds --label_smoothing 0.1 --learning_rate 3e-5 --logging_first_step --logging_steps 1000 --max_source_length 128 --max_target_length 128 --num_train_epochs 1 --overwrite_output_dir --per_device_train_batch_size $BS --sortish_sampler  --task translation --val_max_target_length 128 --warmup_steps 500 --n_train 500
`

thanks 

Comments:
Duplicate of #9226
n_gpus is set to 1 in case of distributed training on multiple gpus, how to access to the correct n_gpus
Author: rabeehkarimimahabadi
Pull request: no

Hi
I am using transformer 4.1.1 on multiple gpus with distributed launch script below, the n_gpu set in this case is 1 instead of actual n_gpus, could you tell me how I can access to the total number of ranks calling the launcher with? Is there any variable in huggingface library which could show the number of gpus the script is called with? Here is the command I run: 

`export BS=4; CUDA_VISIBLE_DEVICES=0,1  USE_TF=0   python -m torch.distributed.launch --nproc_per_node=2 --master_port=9915  finetune_trainer.py --model_name_or_path t5-small --output_dir output_dir --adam_eps 1e-06 --data_dir wmt_en_ro --do_train --freeze_embeds --label_smoothing 0.1 --learning_rate 3e-5 --logging_first_step --logging_steps 1000 --max_source_length 128 --max_target_length 128 --num_train_epochs 1 --overwrite_output_dir --per_device_train_batch_size $BS --sortish_sampler  --task translation --val_max_target_length 128 --warmup_steps 500 --n_train 500
`

thanks 

Comments:
Please do not spam the issues: this is a duplicate of #9225. You can edit the title and the description, there is no need to send a new notification to everyone watching the repo.
Also, please use the [forum](https://discuss.huggingface.co/) for questions like this, we keep the issues for bugs and features requests only. In your case, this is a PyTorch command: `toch.distributed.get_world_size()` will return you the number of GPUs used.

Hi Slvain, thank you, sorry I did not really realized the duplicate, sure, thanks
[T5] Fix warning for changed EncDec Attention Bias weight
Author: patrickvonplaten
Pull request: yes

In this PR: https://github.com/huggingface/transformers/pull/8518 a bug was fixed that removed an unnecessary weight from the T5 Cross Attention layer. 
In the following, this layer was added to the wrong "ignore_weight" list. This weight will never be missing since it doesn't exist in the model anymore it can only be "not used" since it's still present in saved checkpoints. This PR fixes the incorrect warning by placing the regex in the correct list.

Comments:
GPT2 eval with attention_mask not returning expected result
Author: dan-i
Pull request: no

is this not correct use of attention_mask with padding (trying to batch sentence eval) or is it a bug? Shouldn't the return result be the same in each case because of the mask?

import torch
from transformers import GPT2LMHeadModel
model = GPT2LMHeadModel.from_pretrained('gpt2', return_dict=True)

```
with torch.no_grad():
    model.eval()
    loss,logits = model(input_ids=torch.tensor([15496,11,616]), attention_mask=torch.tensor([1,1,1]), labels=torch.tensor([15496,11,616]))[:2]
    print(loss)  # 3.0698 correct answer
    loss,logits = model(input_ids=torch.tensor([50256,15496,11,616]), attention_mask=torch.tensor([0,1,1,1]), labels=torch.tensor([50256,15496,11,616]))[:2]
    print(loss)  # 8.96 - shouldn't it be 3.0698 as above?
```

Comments:
Hey @dan-i, 

1) You should mask the labels of the first token it's not enough to set the attention_mask to 0 -> `labels=torch.tensor([-100, ...])`
2) You have to change the `position_ids` to ensure the expected behavior in GPT2, *i.e.* make sure that in the second case the position_ids are [0, 0, 1, 2] instead of [0, 1, 2, 3]

thank patrick.  seems to work if you just do labels to -100 where the pads are without mask and position. at least it returns same result as individual sentence scores. for anybody else that needs gpt2 batch sentence perplexity scores.

```
def calc_loss(sentences,inputs,logits):
    # calc loss from logits returned from model
    lines_len = torch.sum(inputs['attention_mask'], dim=1)                  # quick way to get sent len. mask isn't used for anything else
    for line_ind in range(len(sentences)):                                  
        log_prob = 0.0
        for token_ind in range(lines_len[line_ind] - 1):                    
            token_prob = F.softmax(logits[line_ind, token_ind], dim=0)
            token_id = inputs['input_ids'][line_ind, token_ind + 1]
            log_prob += torch.log(token_prob[token_id])
        sentloss=abs(log_prob)
        toks=lines_len[line_ind]
        loss=sentloss/(toks-1)
        print(sentences[line_ind],' loss=',loss.item(),'toks=',toks.item(),'sentloss=',sentloss.item())

sentences=["Hello, my","Hello, my dog","Hello, my dog is a dog"]
tokenizer.pad_token = tokenizer.eos_token # set pad to eos
inputs = tokenizer(sentences, return_tensors="pt", padding=True)  # right side padding as usual
# mask of label_id's
label_ids=inputs['input_ids'].clone()
label_ids[label_ids==tokenizer.encode(tokenizer.pad_token)[0]] = -100
# one shot score the batch
with torch.no_grad():
    model.eval()
    loss, logits  = model(input_ids=inputs['input_ids'], labels=label_ids)[:2]
# calc loss per sentence
calc_loss(sentences,inputs,logits)
```
outputs:
Hello, my  loss= 3.069786548614502 toks= 3 sentloss= 6.139573097229004
Hello, my dog  loss= 4.247798442840576 toks= 4 sentloss= 12.74339485168457
Hello, my dog is a dog  loss= 3.671175003051758 toks= 7 sentloss= 22.027050018310547
Fix beam search generation for GPT2 and T5 on model parallelism
Author: TobiasNorlund
Pull request: yes

# What does this PR do?

This PR fixes beam search generation crash when the model layers are distributed on several devices (model parallelism). I have also added a test which showcase the bug on master and pass with the provided fix. Fixes issue #9200 

This is my first contribution to Transformers, so please let me know if something seems wrong or can be improved! The added test doesn't actually test anything at the moment, just raises an error on master. Looking forward to some feedback on how you'd like to see that.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Issue #9200 
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@LysandreJik
@alexorona
@patrickvonplaten


Comments:
@sgugger can you maybe review as well and merge if ok?
Improve BERT-like models performance with better self attention
Author: jplu
Pull request: yes

# What does this PR do?

This PR updates the way we implement the self attention layers in order to be aligned on the original BERT performance. Small breaking change, this improvement needs at least TF 2.3. This change has already been discussed with @thomwolf, and he agreed. But still needs the approval of @LysandreJik @patrickvonplaten and @sgugger 

@patrickvonplaten I have removed the comment for `check_copies` in the Longformer model because I don't know enough this model to apply the proper changes, I will apply this update to one model by one model for the ones I know but can you take this one?

@jlei2 as I'm on Windows, unfortunately the GPU profiling is not yet available in WSL, can you clone this branch and be sure that everythings works like expected with your benchmark? Thanks!!

Fixes # (issue)

#6771 


Comments:
A Python profiling call gives the following improvements:
```
model = TFBertModel.from_pretrained("bert-base-cased")

# With the improvements
cProfile.run("model(model.dummy_inputs)") 
54591 function calls (53774 primitive calls) in 0.064 seconds

# Currently on master
cProfile.run("model(model.dummy_inputs)")
76166 function calls (75204 primitive calls) in 0.095 seconds
```

Thanks @patrickvonplaten !!

1. Slow tests are passing for these models
2. I confirm that "Old" pre-trained models `tf_model.h5` files that were saved with tf < 2.3 can be loaded into the new layer design

I haven't tested the tf1 models, you mean testing the `load_tf_weights_in_bert` in the `modeling_bert.py` file?

@jlei2 has confirmed that now everything works as expected in the profiler and benchmark 👍 https://github.com/huggingface/transformers/issues/6771#issuecomment-745786314

> 2\. "Old" pre-trained models `tf_model.h5` files that were saved with tf < 2.3 can be loaded into the new layer des

Yeah I mean loading a  tf `.ckpt` file using the `from_pretrained(...)` method. The `from_pretrained(...)` method automatically uses the correct functions to load `.ckpt`. I think the easiest way would be to download one of the zips of the official google bert: https://github.com/google-research/bert#bert and quickly check that it can be loaded and that the output on this branch and on master is the same.

> > 2. "Old" pre-trained models `tf_model.h5` files that were saved with tf < 2.3 can be loaded into the new layer des
> 
> Yeah I mean loading a tf `.ckpt` file using the `from_pretrained(...)` method. The `from_pretrained(...)` method automatically uses the correct functions to load `.ckpt`. I think the easiest way would be to download one of the zips of the official google bert: https://github.com/google-research/bert#bert and quickly check that it can be loaded and that the output on this branch and on master is the same.

Ok as discussed offline TF1 checkpoints cannot even be loaded into TF2 at the moment (only if one goes through PT), so this PR is good to go for me!
add base model classes to  bart subclassed models
Author: patil-suraj
Pull request: yes

# What does this PR do?

This PR adds base model classes for `MBart`, `Pegasus` and `Blenderbot`, and adds these in `MODEL_MAPPING` `dict`.
This will enable to load these models using the `AutoModel` class and `pipelines`.

Right now these models can't be loaded using `pipeline` since pipeline relies on the `AutoModel` class.
https://github.com/huggingface/transformers/blob/a4b21cdd20328f71448123ce7c962a78a5d75612/src/transformers/pipelines.py#L105-L110



Comments: