noamaanMulla-03/transformers-issues huggingface dataset:
Visualize self-attention for GLUE task
Author: Nickil21
Pull request: no

Is there a way to visualize the self-attention weights for different spans in a sentence, for instance, a sequence classification task inside [`run_glue.py`](https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_glue.py)?

[See here for a sample](https://imgur.com/a/7gAJvCJ)

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
ModuleAttributeError: 'GPT2LMHeadModel' object has no attribute 'backward'
Author: Octopirate1
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: 
- Python version: 3.7
- PyTorch version (GPU?):
- Tensorflow version (GPU?): None
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No?

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 Trainer: @sgugger

## To reproduce

Steps to reproduce the behavior:

1. Set up a TrainingArguments for a GPT2LMHeadModel with the following deepspeed config:

`{
    "fp16": {
        "enabled": true,
        "loss_scale": 0,
        "loss_scale_window": 1000,
        "hysteresis": 2,
        "min_loss_scale": 1
    },

    "zero_optimization": {
        "stage": 2,
       "allgather_partitions": true,
       "allgather_bucket_size": 2e8,
       "reduce_scatter": true,
       "reduce_bucket_size": 2e8,
        "overlap_comm": true,
        "contiguous_gradients": true,
        "cpu_offload": false
    },

    "optimizer": {
        "type": "Adam",
        "params": {
            "adam_w_mode": true,
            "lr": 3e-5,
            "betas": [ 0.9, 0.999 ],
            "eps": 1e-8,
            "weight_decay": 3e-7
        }
    },

    "scheduler": {
        "type": "WarmupLR",
        "params": {
            "warmup_min_lr": 0,
            "warmup_max_lr": 3e-5,
            "warmup_num_steps": 500
        }
    }
}`

2. Attempt to train.

## Expected behavior

Training should begin as expected.

## Believed bug location

It would appear that [line 1286 in trainer.py](https://github.com/huggingface/transformers/blob/76f36e183a825b8e5576256f4e057869b2e2df29/src/transformers/trainer.py#L1286) actually calls the `backward` method on the *model*, not the loss object. I will try rebuilding after fixing that line and seeing if it helps.


Comments:
https://github.com/huggingface/transformers/issues/9694
DPRReaderTokenizer does not generate the attention_mask properly
Author: mkserge
Pull request: no

Hello,

It seems like the DPRReaderTokenizer does not generate the `attention_mask` properly.

Steps to reproduce on the master branch 

```bash
(venv) sergey_mkrtchyan test (master) $ python
Python 3.8.6 (v3.8.6:db455296be, Sep 23 2020, 13:31:39)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers import DPRReaderTokenizer, DPRReader
>>> tokenizer = DPRReaderTokenizer.from_pretrained('facebook/dpr-reader-single-nq-base')
>>> model = DPRReader.from_pretrained('facebook/dpr-reader-single-nq-base')
>>> encoded_inputs = tokenizer(questions="What is love ?",
...                            titles="Haddaway",
...                            texts="What Is Love is a song recorded by the artist Haddaway",
...                            padding=True,
...                            return_tensors='pt')
>>> encoded_inputs
{'input_ids': tensor([[ 101, 2054, 2003, 2293, 1029,  102, 2018, 2850, 4576,  102, 2054, 2003,
         2293, 2003, 1037, 2299, 2680, 2011, 1996, 3063, 2018, 2850, 4576]]), 'attention_mask': tensor([True])}
```
Notice the `attention_mask` above is incorrect. It should have the same shape as the `input_ids` tensor. 

## Environment info
- `transformers` version: 4.2.0dev0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.6
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

Git blame says @lhoestq and @LysandreJik might be able to help :)

I believe the issue is in this part of the code

https://github.com/huggingface/transformers/blob/5f6721032af46cf491fe69c010805f8786bf63a1/src/transformers/models/dpr/tokenization_dpr.py#L254

(same thing for the fast tokenizer)

I fixed it locally by replacing the above line with

```Python
attention_mask = []
    for input_ids in encoded_inputs["input_ids"]:
        attention_mask.append([int(input_id != self.pad_token_id) for input_id in input_ids])
```

I am happy to submit a PR if that looks reasonable to you.



Comments:
Indeed, it doesn't! We would gladly welcome a PR!

Closed by #9663 :)
Fix DPRReaderTokenizer's attention_mask
Author: mkserge
Pull request: yes

# What does this PR do?

This PR fixes an issue with the `attention_mask` not being properly generated by the DPRReaderTokenizer. Please see  issue #9555 for more details.

I added an integration test that checks the DPRReader following a similar example in the test file.

I have some test failures due to 
```
"AttributeError: Can't pickle local object 'measure_peak_memory_cpu.<locals>.MemoryMeasureProcess'"
```
and 
```
"AttributeError: module 'wandb' has no attribute 'ensure_configured'"
```
which seem to be unrelated to my code changes.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@LysandreJik and @lhoestq would probably be best positioned to review.


Comments:
Thank you both. Should I close the corresponding issue?

Just did! Thanks!
The model learns nothing after 3 epochs of training
Author: geminiwenxu
Pull request: no

I have trained a multilingual Bert model on 3 different input data configurations ( imbalanced, partial balanced, and full balanced) for the sentiment classification task. Everything works fine so far, except the zero-shot model being trainined on the full balanced dataset (training data: label balanced data; val/test data: label balanced data). however, the result is very weird:
<img width="638" alt="Screen Shot 2021-01-20 at 11 45 47 AM" src="https://user-images.githubusercontent.com/41744366/105165475-afb1b800-5b16-11eb-9d8f-d775fa9a07ee.png">

As you can see, the model has not learned anything, and it classifies everything into neutral in the testing phase.

Could anyone helps please?


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead? You'll get more answers over there.

Thanks!
input one model's output to another one
Author: omerarshad
Pull request: no

Hello,

I want to create a model which generates text and the generated text is input to other model. So basically two models are trained together. How can i achieve this using hugging face?

Thanks

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead? You'll get more answers there!

Thanks!
Add t5 convert to transformers-cli
Author: acul3
Pull request: yes

# What does this PR do?

add t5 model convert to transformers-cli

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@patrickvonplaten @sgugger @LysandreJik 
and
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

Comments:
Can we trigger the CI again to make sure all tests are passing? Think you can add an empty git commit with 

```
git commit --allow-empty -m "Trigger notification"
```

@patrickvonplaten done it!..it seems failing 1 check only

@patrickvonplaten all seems good now...kindly check
Add DeBERTa head models
Author: NielsRogge
Pull request: yes

This PR adds 3 head models on top of the DeBERTa base model: `DebertaForMaskedLM`, `DebertaForTokenClassification`, `DebertaForQuestionAnswering`. These are mostly copied from `modeling_bert.py` with bert->deberta.

## Who can review?

@LysandreJik 

Also tagging original DeBERTa author: @BigBird01 

Fixes #9689 

Comments:
Thanks for the review @LysandreJik, the test did fail because of the pooler. Is fixed now!
MLM training for DeBERTa not supported: configuration class is missing
Author: xiaolin-cheng
Pull request: no

When I ran the example script run_mlm.py to fine tune the pretrained deberta model on a customized dataset, I got the following error. The same command worked for roberta-base.

The command:
python run_mlm.py --model_name_or_path 'microsoft/deberta-base' --train_file slogans/train.txt --validation_file slogans/test.txt --do_train --do_eval --per_device_train_batch_size 64 --per_device_eval_batch_size 64 --learning_rate 1e-3 --num_train_epochs 10 --output_dir /home/jovyan/share2/xiaolin/models/mlm/temp --save_steps 5000 --logging_steps 100

The terminal error:
Traceback (most recent call last):
  File "run_mlm.py", line 409, in <module>
    main()
  File "run_mlm.py", line 264, in main
    cache_dir=model_args.cache_dir,
  File "/home/jovyan/.local/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1093, in from_pretrained
    config.__class__, cls.__name__, ", ".join(c.__name__ for c in MODEL_FOR_MASKED_LM_MAPPING.keys())
ValueError: Unrecognized configuration class <class 'transformers.models.deberta.configuration_deberta.DebertaConfig'> for this kind of AutoModel: AutoModelForMaskedLM.
Model type should be one of LayoutLMConfig, DistilBertConfig, AlbertConfig, BartConfig, CamembertConfig, XLMRobertaConfig, LongformerConfig, RobertaConfig, SqueezeBertConfig, BertConfig, MobileBertConfig, FlaubertConfig, XLMConfig, ElectraConfig, ReformerConfig, FunnelConfig, MPNetConfig, TapasConfig. 

Comments:
Looking at the [docs](https://huggingface.co/transformers/model_doc/deberta.html), it seems like there's currently no `DeBERTaForMaskedLM` defined. I will make a PR that adds this.
Issue with TrainingArguments docs.
Author: gchhablani
Pull request: no

Hi Team,

This is a minor issue but on this [link](https://huggingface.co/transformers/main_classes/trainer.html#transformers.TrainingArguments), the default optimizer mentioned is Adam in the docs. However, the `Trainer` uses AdamW by default.

This is slightly misleading.

Thanks,
Gunjan


Comments:
Hi! Do you want to open a PR to fix it? Thanks!
how to run pegasus finetune on multiple gpus
Author: cheop-byeon
Pull request: no

## Environment Information
- transformers version: 4.2.0dev0
- Platform: Linux-3.10.0-1062.18.1.el7.x86_64-x86_64-with-centos-7.7.1908-Core
- Python version: 3.7.3
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): 2.4.0 (False)
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

## Who might help
@sgugger
@patrickvonplaten 
@patil-suraj

## Information

The fine-tune process is taking really long time, so I want to do it parallel on multiple gpus.

The problem arises when using:
 I do not have found the instructions for training on multiple gpus for the arguments, are there configurations for something like nodes, etc. or should I implement it in my own script?

## To reproduce
```
python finetune.py \
    --gpus 0 \
    --learning_rate=1e-4 \
    --do_train \
    --do_predict \
    --n_val 1000 \
    --val_check_interval 0.25 \
    --max_source_length 512 --max_target_length 56 \
    --freeze_embeds --label_smoothing 0.1 --adafactor --task summarization_xsum \
    --model_name_or_path google/pegasus-xsum \
    --output_dir=xsum_results \
    --data_dir xsum \
    --tokenizer_name google/pegasus-large \
    "$@"
```
and which of the belowings are correct? I saw both in other posts:

--model_name_or_path google/pegasus-xsum
--tokenizer_name google/pegasus-large \

or
--model_name_or_path google/pegasus-large
--tokenizer_name google/pegasus-xum \

I think it should be the second one but I am not sure. 

## Expected behavior
1. Enable the finetune of pegasus model on multiple gpus.
2. Inject the correct arguments.

Comments:
Please use the [forums](https://discuss.huggingface.co/) to ask questions like this. Also note that there is no `finetune` script in the example folder anymore, so you should probably be using `finetune_trainer` or `run_seq2seq`.
Let Trainer provide the device to perform training
Author: earlyr1
Pull request: no

# 🚀 Feature request

Training_args object [chooses](https://github.com/huggingface/transformers/blob/7acfa95afb8194f8f9c1f4d2c6028224dbed35a2/src/transformers/training_args.py#L477) the training device by itself(cuda:0 by default). I request a possibility for a user to be able to choose it :)  

## Motivation

Imagine a situation when we have a cluster with several gpus and cuda:0 memory is full(I have it right now :)) So user cannot use Trainer object for training.


Comments:
fix typo
Author: Muennighoff
Pull request: yes

# What does this PR do?
fix typo
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
Allow text generation for ProphetNetForCausalLM 
Author: guillaume-be
Pull request: yes

# What does this PR do?

The configuration for ProphetNetForCausalLM is overwritten at initialization to ensure that it is used as a decoder (and not as an encoder_decoder) for text generation.

The initialization of the parent class for ProphetNetForCausalLM is done before this overwrite, causing the `model.config.is_encoder_decoder` to remain possibly True. This leads to an error if the generate method of the model is later called as the non-existing method `get_encoder` is called.

Fixes https://github.com/huggingface/transformers/issues/9702


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@patrickvonplaten


Comments:
Thanks a lot for fixing it @guillaume-be
Fix mixed precision in TF models
Author: jplu
Pull request: yes

# What does this PR do?

This PR aims to fix the mixed precision issues when `tf.keras.mixed_precision.experimental.set_policy()` is set to something else than `tf.float32`.  In the same page, this PR aims to fix some TFLite quantization issues.

Before to further continue this PR, the PR #9418 has to be merged.

Fixes # (issue)

#7052 


Comments:
Fix Funnel Transformer conversion script
Author: sgugger
Pull request: yes

# What does this PR do?

The conversion script was using the wrong kind of model, so wasn't working. I've also added the option to convert the base models.

Fixes #9644


Comments:
Do not miss ```transformers/cammand/convert.py```  for ```transformer-cli``` user.
Need ```base_model``` arg for ```convert_tf_checkpoint_to_pytorch(self._tf_checkpoint, self._config, self._pytorch_dump_output)```.
fix T5 head mask in model_parallel
Author: patil-suraj
Pull request: yes

# What does this PR do?

`head_mask` in T5 is not parallelized correctly in model parallel, each layer's head mask should be put on that layer's device if it's not `None`. 

Fixes #9718 

Comments:
That's a better solution actually! Thanks @patil-suraj 

Also, there is a whole bunch of issues including this one I believe fixed in this PR: https://github.com/huggingface/transformers/pull/9323 where we no longer do it one by one.
Fix memory regression in Seq2Seq example
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes the memory regression introduced when putting the `Seq2SeqTrainer` inside the main library. The root of the memory regression comes from the fact that when doing label smoothing, we ended up computing the log softmax of the logits twice, once in the cross entropy loss, and a second time inside the label smoother.

To fix this, the loss computation needs to be entirely done inside the label smoother, so the labels must be extracted from the batch before being passed to the model. As a result, the `decoder_input_ids` must be computed in the `Seq2SeqDataCollator` and not the model for this to work. I've just reverted the code from #9343, I don't know if it actually matches what happens inside the models. Maybe we should have a method to compute those `decoder_input_ids` accessible from inside of those models, or a flag to tell them whether to compute the loss or not (in this case, computing the loss will not only be slower, it will also trigger back the memory regression).

The same fix will need to be applied to the `Seq2SeqDataCollator` now inside the library as well as the new `run_seq2seq` script, but I will do it once we have agreed on a long-term solution for the decoder input ids above.

Fixes #9261

Comments:
I agree with Patrick here, the collator was added to encode the text and to prepare the `decoder_input_ids` and `labels`, replace pad with 100 etc. Now we could encode and prepare `labels` in datasets.map(...) so collator won't be needed anymore. 

The only thing we need IMO is to be able to prepare `decoder_input_ids` outside of the model for label smoothing as Sylvain said. Could we maybe make the add `shift_right` method to every s2s model to able to prepare the `decoder_input_ids` outside of the model ?

Note that this is fixing the old script with the old data collator. The new one will be fixed with the proper fix (once we agree on it and there seems to be a consensus on having a model with a `shift_right` method) but is still necessary to do dynamic padding. The `Dataset.map` method is very nice for static things but when you want to pad to the length of the biggest sample in the batch, you need a special data collator, especially if it has to pad special keys like `"labels"`, `"decoder_input_ids"`...

The old `Seq2SeqDataCollator` in the utils file will be removed in a couple of weeks when the new seq2seq example is perfectly running, so I think it's fine to merge the quick hack in the meantime :-)