noamaanMulla-03/transformers-issues huggingface dataset:
How to train encoder decoder for explicit negation generation
Author: thak123
Pull request: no

Hi
I am trying to generate negations out of non-negated sentences.
I used a simple “I have tea” => “I don’t have tea” formatted dataset for training an XLMR encoder-decoder model using the example provided in the collab.

```
# set special tokens

roberta_shared.config.decoder_start_token_id = tokenizer.bos_token_id                                             

roberta_shared.config.eos_token_id = tokenizer.eos_token_id

# sensible parameters for beam search

# set decoding params                               

roberta_shared.config.max_length = 64

roberta_shared.config.early_stopping = True

roberta_shared.config.no_repeat_ngram_size = 3

roberta_shared.config.length_penalty = 2.0

roberta_shared.config.num_beams = 4

roberta_shared.config.vocab_size = roberta_shared.config.encoder.vocab_size

```

But the test set produces different tokens than the source. How can I preserve the source tokens when generating the output.
[“I have it.”, “I love tea”, “I can have coffee.”] =>
[‘I have no it.’, “I’ll not love.” “I can’t have food.”]
Where the model modifies the words in the sentence.



Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
Return global attentions (see #7514)
Author: gui11aume
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # 7514 (see discussion of March 22, 2021 with @patrickvonplaten )


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
Updated colab links in readme of examples
Author: WybeKoper
Pull request: yes

# What does this PR do?
Updated the Google Colab links for the The Big Table of Tasks in the examples folder readme.
Google Colab links were replace to the appropriate examples found [here](https://github.com/huggingface/notebooks/tree/master/examples)

text_generation.ipynb is not present in the [notebook repo](https://github.com/huggingface/notebooks/tree/master/examples). Will text_generation.ipynb be added in the future?


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger



Comments:
Fix initializing BertJapaneseTokenizer with AutoTokenizers
Author: singletongue
Pull request: yes

# What does this PR do?

This PR fixes a bug in loading some kinds of tokenizers using `AutoTokenizers.from_pretrained()`.
This issue is discussed in https://github.com/cl-tohoku/bert-japanese/issues/25.

When `sentencepiece` is not installed, the initialization of several tokenizers such as `BertJapaneseTokenizer`, `BarthezTokenizer`, and `MBart50Tokenizer` fails.
The exception is raised in `tokenizer_class_from_name()` when iterating over tokenizer classes which are `NoneType` objects. 
Such tokenizer classes are set to `None` in `is_sentencepiece_available()` if `sentencepiece` is not available.

This error affects the initialization of `BertJapaneseTokenizer` even though it does not depend on `sentencepiece`.

```sh
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("cl-tohoku/bert-base-japanese")
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 479/479 [00:00<00:00, 143kB/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/m-suzuki/.pyenv/versions/py3.7/lib/python3.7/site-packages/transformers/models/auto/tokenization_auto.py", line 370, in from_pretrained
    tokenizer_class = tokenizer_class_from_name(tokenizer_class_candidate)
  File "/Users/m-suzuki/.pyenv/versions/py3.7/lib/python3.7/site-packages/transformers/models/auto/tokenization_auto.py", line 273, in tokenizer_class_from_name
    if c.__name__ == class_name:
AttributeError: 'NoneType' object has no attribute '__name__'
```

## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@LysandreJik


Comments:
Instantiate model only once in pipeline
Author: sgugger
Pull request: yes

# What does this PR do?

The current implementation of `pipeline` is inefficient in the sense it instantiates the model twice just to guess the proper framework. This PR does not add any breaking change but reworks the function that infers the framework from the model to:
1. instantiate the proper class of the model (this avoids getting weird warnings about missing weights)
2. return the model instantiated so it's not re-instantiated later on.

cc @mfuntowicz  and @Narsil 

Comments:
Checked all the slow tests run. I have no idea on how to implement a test that checks the model is only loaded once, so I'm going merge this and if anyone wants to tackle that, it can be done in a separate PR.
[Trainer] possible DDP memory regression
Author: stas00
Pull request: no

I think we may have created a memory regression somewhere recently.

I tried with pt-1.7 and pt-1.8 with the same results.

memory limit on this setup is 8gb

on `transformers` master:

This takes about 5.5GB/gpu:
```
PYTHONPATH=src USE_TF=0 CUDA_VISIBLE_DEVICES=0,1 python examples/seq2seq/run_translation.py --model_name_or_path google/mt5-small --do_train --do_eval --source_lang en --target_lang ro --dataset_name wmt16 --dataset_config_name ro-en --output_dir /tmp/test --per_device_train_batch_size=4 --per_device_eval_batch_size=4 --overwrite_output_dir --predict_with_generate --logging_step 10
```

(no need to run more than a few secs, we are just trying to see that the job can start training)

switching to DDP immediately OOMs:
```
PYTHONPATH=src USE_TF=0 CUDA_VISIBLE_DEVICES=0,1 python  -m torch.distributed.launch --nproc_per_node=2  examples/seq2seq/run_translation.py --model_name_or_path google/mt5-small --do_train --do_eval --source_lang en --target_lang ro --dataset_name wmt16 --dataset_config_name ro-en --output_dir /tmp/test --per_device_train_batch_size=4 --per_device_eval_batch_size=4 --overwrite_output_dir --predict_with_generate --logging_step 10
```
even if I reduce the bs from 4 to 1 it still goes over 8GB.

@sgugger, could you please confirm if you're seeing the same?


Comments:
I don't have a setup with 8Gb so I have to rely on nvidia-smi numbers. First command is 13.2Gb on GPU0, 6.5Gb on GPU1, second command is 11.2GB on GPU0 and 10.1GB on GPU1.

Thank you for the sanity check, @sgugger 

This is very odd that we get such a discrepancy in memory allocation between the 2 gpus on DP! 2x gpu ram on card0.

But this explains why it works for me since I have precisely 24gb + 8gb, so this discrepancy fits just right. So it's unclear if it's a problem in DP or DDP.

I will investigate.

With DP the gradients and optimizer states are only on one GPU, I think that is why we have the big difference. With DDP they are copied over the two.

Oh wow, that's a huge difference. Clearly DP wins here for those with lopsided setups like mine! 

OK, then it's by design then. Closing this.

This is a bit of a problem with our memory metrics reporting as we only report gpu0, but I guess since most users will have symmetrical setups (cards of the same size) and gpu0 consumes the biggest amount of memory in DP/DDP then it's OK.

Will have to think how to extend the metrics for setups where it's critical to know each gpu's allocations - e.g. pipeline or model parallel.
Add `examples/run_ner_no_trainer.py`
Author: stancld
Pull request: yes

This PR adds an example of token classification tasks (`"ner", "pos", "chunk"`) to show the functionalities of the new `accelerate` library.

<hr>

**Reviewers:** @sgugger 

Comments:
Ok, added the missing part in Accelerate. In your code, before gathering the labels and predictions, you should pad them across the processes running if `pad_to_max_length` is False:
```python
if not args.pad_to_max_length:
    predictions = accelerator.pad_across_processes(predictions, dim=1, pad_index=-100)
    labels = accelerator.pad_across_processes(batch["labels"], dim=1, pad_index=-100)
```

This should solve the issue when `pad_to_max_length` is left at `False`!

Thanks for your comment, @sgugger. I fix the bugs and add documentation to README.

Thank you for spotting the mistake with unintentionally deleted `--label_all_tokens` from argarser. I added that argument back.

Just checked on TPU for completeness and it runs perfectly fine there, so we're all good!

Thanks for your contribution!!!
Use pre-computed lengths, if available, when grouping by length
Author: pcuenca
Pull request: yes

# What does this PR do?

A new argument `length_column_name` has been added to
`TrainingArguments`, with default value `"length"`. If this column
exists and `group_by_length` is `True`, the train sampler will use
it for grouping rather than computing all lengths before training starts.

This is an optimization that allows the user to prepare data for fast
processing, preventing sequential access to the dataset as described in
issue #10909.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? [Discussion](https://discuss.huggingface.co/t/spanish-asr-fine-tuning-wav2vec2/4586/6), related issue #10909.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@sgugger, this is what we discussed during the fine-tuning week.

Comments:
LengthGroupedSampler slowly iterates over dataset
Author: maxidl
Pull request: no

https://github.com/huggingface/transformers/blob/86c6f8a8b1f2bfc6c6d175590efc95a5e6facb51/src/transformers/trainer_pt_utils.py#L506

When using training arg "group_by_length", the sampler has to get the length of every input. it does so by iterating over the dataset in a simple for loop, without the use of a dataloader.

In my case, this led to long delays before the training starts, as reading the dataset without the worker parallelization from a dataloader took a long time.

I believe this can potentially improved, although one could also precompute the lengths and pass them to the sampler (but this can only happen if the user is aware of it being a potential bottleneck).
Maybe it is a good idea to add a dataloader + num_workers option here?


Thanks to Pere-Lluís and Pedro for being my co-detectives in spotting this caveat.


Comments:
I think the solution you suggested on the forums (leaving the length computation up to the user in the dataset) is a bit better. Will investigate a bit more.

> I think the solution you suggested on the forums (leaving the length computation up to the user in the dataset) is a bit better. Will investigate a bit more.

I agree, that surely is a good solution if the user is aware of it. It just took us a while to find the source of that delay before training. Maybe we can have some sort of user hint? 

Fixed by #10953
Add `examples/multiple-choice/run_swag_no_trainer.py`
Author: stancld
Pull request: yes

This PR adds an example of a multiple-choice task on the SWAG dataset to show the functionalities of the new `accelerate` library.

<hr>

**Reviewers:** @sgugger 

Comments:
Tested on one GPU, two GPUs and TPUs, this runs fine everywhere. So just waiting for the small adjustments and it should be good to be merged :-) 

Thanks a lot!
[vulnerability] dep fix
Author: stas00
Pull request: yes

Fixes https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/Pygments/open

@LysandreJik


Comments:
Oh, is this process somehow automated and I didn't know about it? All I see is the file which it complains about and no suggestion to automate the fix.

Yeah, but usually it's not as good as your suggestion as it offers `==` while you offer `>=`.

I think I mentioned it here: https://github.com/huggingface/transformers/pull/10817

But thank you nonetheless, these are helpful!

Hmm, I have been copying exactly what the vulnerability bot suggested - which is always `>=` - so it's probably the dependabot that could use a bit of an update to match the vulnerability report.

But it's good to know that this is already automated, I will know not to make a PR next time.
Sagemaker test
Author: philschmid
Pull request: yes

# What does this PR do?

This PR creates tests for `SageMaker`. This PR creates tests for the `Pytorch` and `tensorflow` DLC. I added a documentation `README.md` which explains when the Tests need to be run and how. Currently, not all tests are leveraging our `examples/` due to limitations like `SageMakerTrainer` not being integrated into `Trainer` and missing implementation for `keras` for SageMaker specific libraries for data/model parallel. In the near future all scripts in `tests/sagemaker/scripts` are going to be removed and copy the scripts from `examples/` before executing the tests. 

## Current Tests

| ID                                  | description                                                       | plattform                   | #GPUS | collected & evaluated metrics            |
|-------------------------------------|-------------------------------------------------------------------|-----------------------------|-------|------------------------------------------|
| pytorch-transfromers-test-single    | test bert finetuning using BERT fromtransformerlib+PT             | SageMaker createTrainingJob | 1     | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-2-ddp     | test bert finetuning using BERT from transformer lib+ PT DPP      | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-2-smd     | test bert finetuning using BERT from transformer lib+ PT SM DDP   | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-1-smp     | test roberta finetuning using BERT from transformer lib+ PT SM MP | SageMaker createTrainingJob | 8     | train_runtime, eval_accuracy & eval_loss |
| tensorflow-transfromers-test-single | Test bert finetuning using BERT from transformer lib+TF           | SageMaker createTrainingJob | 1     | train_runtime, eval_accuracy & eval_loss |
| tensorflow-transfromers-test-2-smd  | test bert finetuning using BERT from transformer lib+ TF SM DDP   | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |

Comments:
> LGTM! Should the images be added before merge?

Are you referring to container images? they will be added after the release of `transformers`. If you are referring to the `TODO: Add a screenshot of PR + Text template to make it easy to open.` nope I would add them as soon as we went through the process. To make screenshots while doing it.
[Marian] Run predictions on GPU? RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select`  
Author: mykhailoslukvin
Pull request: no

Is it possible to run the Marian NMT model predictions on GPU, should not it be faster? Once I try to move the model to GPU I get the error described below. Is there any quick fix to the code to achieve this?
```
torch.__version__
Out[15]: '1.5.0'

```

The code to reproduce:
```
from transformers import MarianMTModel, MarianTokenizer
model_name = 'Helsinki-NLP/opus-mt-fr-en'
model = MarianMTModel.from_pretrained(model_name).cuda()
tokenizer = MarianTokenizer.from_pretrained(model_name)
```
Then I check the current device:
```
model.device
Out[33]: device(type='cuda', index=0)
```
Then I try to predict following the examples from documentation:
```
batch = tokenizer.prepare_translation_batch(list_of_sents)
gen = model.generate(**batch, num_beam=1, early_stopping = True, no_repeat_ngram_size = 3) 


  File "<ipython-input-9-54f44d46fb14>", line 1, in <module>
    gen = model.generate(**batch, num_beam=1, early_stopping = True, no_repeat_ngram_size = 3)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1146, in generate
    encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/transformers/modeling_bart.py", line 292, in forward
    inputs_embeds = self.embed_tokens(input_ids) * self.embed_scale

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/sparse.py", line 114, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/functional.py", line 1724, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select`
 
```

Comments:
Is your `batch` on the correct device too?

Thank you, I moved `batch` to the GPU and it works.
`batch = tokenizer.prepare_translation_batch(sentences).to('cuda')`

More generally: `batch.to(model.device)`.
compute perplexity using a custom metric function
Author: TheAzouz
Pull request: no

Hello,

I am trying to replicate the "On the Cross-lingual Transferability of Monolingual Representations" paper from Artetxe et al. and I am using the code you're providing, specifically [run_mlm.py]
(https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py) 

I wanted to log the perplexity to tensorboard during the evaluation step. I found out that the best option is to add a custom compute_metrics function in the trainer that uses the evaluation results (predictions and target) to compute perplexity. However, I didn't manage to do that because I couldn't understand what the output of predictions represents.

I am really new to NLP and your help is very much appreciated.

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

cc @sgugger 

Thanks!

The predictions are the logits of your model, so in the case of a language model, it will be a big array `num_samples x seq_length x vocab_size`. Your labels will be a big array of `num_samples x seq_length` with the tokens corresponding to something not masked at -100 (index that is ignored).

First, I would like to apologize for writing to you here, I wasn't aware that there is a dedicated forum.
Thank you @sgugger, I appreciate your help, my function is working now.