noamaanMulla-03/transformers-issues huggingface dataset:
DeepSpeedEngine object has no attribute 'no_sync'
Author: ayubSubhaniya
Pull request: no

## Environment info

- `transformers` version: 4.3.0
- Platform: Linux-4.14.209-160.339.amzn2.x86_64-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.8
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: distributed

### Who can help
Library:
- deepspeed: @stas00

## Information
Deepspeed with single node, multi gpus breaking

```
Traceback (most recent call last):
  File "training/run_training.py", line 273, in <module>
    raise e
  File "training/run_training.py", line 270, in <module>
    remaining_args=remaining_args)
  File "training/run_training.py", line 186, in main
    trainer.train(model_path=model_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 937, in train
    with model.no_sync():
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'DeepSpeedEngine' object has no attribute 'no_sync'
```

```
deepspeed \
        --num_gpus ${GPUS_ALLOWED} \
        training/run_training.py \
        --deepspeed ds_config.json \
        --output_dir ${OUTPUT_BASE_PATH} \
        --model_name_or_path ${MODEL_NAME_OR_PATH} \
        --per_device_train_batch_size ${TRAIN_BATCH_SIZE} \
        --per_device_eval_batch_size ${TEST_BATCH_SIZE} \
        --gradient_accumulation_steps ${GRAD_STEP} \
        --evaluation_strategy steps \
        --eval_steps ${EVAL_STEP} \
        --num_train_epochs ${EPOCH} \
        --save_steps ${SAVE_STEP} \
        --logging_steps ${LOG_STEP} \
        --dataloader_num_workers ${DATALOADER_NUM_WORKERS} \
        --load_best_model_at_end true \
        --do_train true \
        --do_eval true \
        --fp16 true \
        --dataloader_drop_last true \
        --overwrite_output_dir true \
        --use_lazy true \
        --logging_first_step true || { echo 'training failed' ; exit 0; }

echo 'training successful'
```


```
ds_config.json
{
  "zero_optimization": {
     "stage": 2,
     "allgather_partitions": true,
     "reduce_scatter": true,
     "overlap_comm": true,
     "contiguous_gradients": true,
     "cpu_offload": true,
     "allgather_bucket_size": 2e8,
     "reduce_bucket_size": 2e8,
  }
}
```


Comments:
Thank you for your report. This issue has already been fixed in `transformers` master.
Question about the `decoder_input_ids`  in `LEDForConditionalGeneration` forward method
Author: yww211
Pull request: no

https://github.com/huggingface/transformers/blob/17b6e0d474b797cdddf5225b0f51bf0e928091b9/src/transformers/models/led/modeling_led.py#L2337

Hi, 
I have a question about the `LEDForConditionalGeneration` forward args. 
The `decoder_input_ids`  has a comment that `decoder_input_ids (torch.LongTensor of shape (batch_size, target_sequence_length), optional) – Provide for translation and summarization training. By default, the model will create this tensor by shifting the input_ids to the right, following the paper.`. 
Form the forward method in  `LEDForConditionalGeneration`, i can see that when not  assigning the `decoder_input_ids` in the forward method of  `LEDForConditionalGeneration` object , the `decoder_input_ids` will be generated by [shifting  the `labels`  value one token to right in the  forward method](https://github.com/huggingface/transformers/blob/17b6e0d474b797cdddf5225b0f51bf0e928091b9/src/transformers/models/led/modeling_led.py#L2337). 

So my question is  if i want to explictly pass the `decoder_input_ids` to the forward method, do i need to explictly shift it one token as the [code](https://github.com/huggingface/transformers/blob/17b6e0d474b797cdddf5225b0f51bf0e928091b9/src/transformers/models/led/modeling_led.py#L2337) shows before the forward pass?


Comments:
Hey @yww211, 

actually there was a bug in the docstring, that I found thanks to you :-) The attached PR corrects this mistake.

To answer your question, you should not shift the `decoder_input_ids` explicitly by one when passing them & in fact you have to pass the `decoder_input_ids` if you want to use the forward method.
Add Ray Tune hyperparameter search integration test
Author: krfricke
Pull request: yes

# What does this PR do?

Currently, only Optuna HP search is tested in integration tests. This PR duplicates and adjusts the test for the Ray backend.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@amogkam 
@sgugger 



Comments:
Oh, that makes sense. Does this happen in the repo? Is this something I can help with, or do you have to configure it?

I'm currently working on these scheduled tests, I'll enable these while I do so. Thanks!
[Deepspeed] getting multiple prints of: Avoid using `tokenizers` before the fork if possible
Author: stas00
Pull request: no
Labels: name: DeepSpeed

on master when running with DeepSpeed I started getting multiple dumps of:
```
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
```
This script:
```
export BS=16; rm -r output_dir; PYTHONPATH=src USE_TF=0 CUDA_VISIBLE_DEVICES=0,1 deepspeed --num_gpus=2 examples/seq2seq/run_seq2seq.py --model_name_or_path t5-small --output_dir output_dir --adam_eps 1e-06 --do_eval --do_train --do_predict --evaluation_strategy=steps  --label_smoothing 0.1 --learning_rate 3e-5 --logging_first_step --logging_steps 1000 --max_source_length 128 --max_target_length 128 --num_train_epochs 1 --overwrite_output_dir --per_device_eval_batch_size $BS --per_device_train_batch_size $BS --predict_with_generate --eval_steps 25000  --sortish_sampler --task translation_en_to_ro  --val_max_target_length 128 --warmup_steps 500 --max_train_samples 100 --max_val_samples 100 --max_test_samples 100 --dataset_name wmt16 --dataset_config ro-en  --source_prefix "translate English to Romanian: " --deepspeed examples/tests/deepspeed/ds_config.json
```
prints it 15 times.

There are no 15 forks, it probably gets triggered by threads. The problem doesn't happen with DDP or DP.

Thank you.

@LysandreJik, @n1t0 


Comments:
@chrissyjsartt, you probably accidentally subscribed/set to "Watching" the transformers repository which will now send you every comment on every Issue or PR. 

So urgently go to https://github.com/watching and "Unwatch" this or any other repositories you may have set to Watch. Then you will stop getting these notifications.

@LysandreJik replied elsewhere to set `TOKENIZERS_PARALLELISM=false` and to read https://github.com/huggingface/tokenizers/issues/187#issuecomment-635692450 for the explanation of why this is needed.

But this could make things slow, so trying `=true` first is a better idea - if it doesn't hang then all is good.

Also Anthony shared:
> If the `tokenizer` wasn't used to encode before forking the process, it shouldn't happen. So just a new `encode_batch` somewhere before the fork happens can be enough to trigger this.
RAG and retrieved documents
Author: calderma
Pull request: no

I pretrained a rag model using the "finetune_rag.py" script and it generates pretty good results for my (knowledge intensive) use case, certainly better than the straight finetuned BART model i was using before. I am using my own custom datasource generated from use_own_knowledge_dataset.py. One curious thing that is happening is that when i try to find what documents were retrieved during the generation process, i always get the same documents. I'm using the basic code snippet from issue#8104 and no matter what input i give, it returns the same few documents no matter how unrelated they are to the input. The generated results are still very good, so I'm not sure if there's an issue with how i'm retrieving the documents or if it really is always grabbing the same few docs for some reason, potentially an issue with my data. Any help or pointers with this would be greatly appreciated. Thank you.

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
[examples] better model example
Author: stas00
Pull request: yes

As a continued effort to make examples easy to read and synchronizing them all to use the same look and feel, this PR tries to improve `run_seq2seq.py` as a model and then future PRs will sync other examples with it.

* [x] makes the helper methods work for rank0 internally - simplifying the caller
* [x] abstracts the helper `trainer.state.save_to_json` into a simple method
* [x] automatically aggregates all metrics into `all_metrics.json` w/o requiring any extra code on the caller side 

Anything else?

@sgugger 

Comments:
[run_seq2seq.py] restore functionality: saving to test_generations.txt
Author: stas00
Pull request: yes

This PR restores the original functionality that for some reason was modified.

Fixes: https://github.com/huggingface/transformers/issues/10381

@sgugger


Comments:
Ah, I re-read it closer, you're correct. I just remembered the part about the `test_generations.txt` but didn't bother to check out the full story. My bad. I will study it and follow up once I understand it better. 

I went back to `finetune_trainer.py` from December and checked that it was just saving `test_generations.txt` at the very end once. I can't find any code where it did generate this at every checkpoint. @kingpalethe, please correct me if I'm wrong.

If there were to be a `save_checkpoint` callback then it could generate one for each saved check point. So it probably needs to be requested via a feature request Issue.

So the current PR is still is a good idea to support those who relied on this particular filename. But I'm not attached to it.

I don't think it matters much, we just have to be quick if people using the new script start to rely on the new name.

OK, let's restore the original name.
Add adapter support
Author: salimmj
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Add [adapter](https://arxiv.org/abs/1902.00751) support to transformers.

## Motivation

Adapters are great time-and-memory-savers for multitask use cases and would be a great addition to this library. Some very kind folks added support for them ([AdapterHub](https://adapterhub.ml/)) on top of transformers library but unfortunately in order to use it one needs to use their [fork](https://github.com/Adapter-Hub/adapter-transformers) which is slightly inconvenient.

## Your contribution

They've done the integration already so I hope it's straightforward. I've posted [an issue](https://github.com/Adapter-Hub/adapter-transformers/issues/65) on their end as well and would be happy to help in any way I can.


Comments:
That'd be great!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Just voicing my support of this too. Not sure yet if adapter-hub will end up having the resources to merge back into `transformers`. If they become inactive, me (and a lot of the community I'm sure) will want to try to take up that effort.
BART for generating sequence of length more than 1024 tokens
Author: silentghoul-spec
Pull request: no

I was using pretraining code given in transformers/examples/seq2seq to finetune on my custom dataset containing summaries of the text of greater than 1024 tokens. But I am getting an error regarding index out of bounds error. Is it possible to fine-tune BART to generate summaries of more than 1024 tokens? I have added log file for reference.

[v100job.txt](https://github.com/huggingface/transformers/files/6059178/v100job.txt)


Comments:
Hi @silentghoul-spec 

for BART the maximum sequence length is 1024, so it can't process text larger than 1024 tokens.
You could use the `LED` model for long document summarization, here's a notebook which demonstrates how to use LED https://github.com/patrickvonplaten/notebooks/blob/master/Fine_tune_Longformer_Encoder_Decoder_(LED)_for_Summarization_on_pubmed.ipynb 

Also please use the forum https://discuss.huggingface.co/ to ask such questions first

Thanks, @patil-suraj ,
I wondered why it has a token limit of 1024 as the original paper https://arxiv.org/pdf/1910.13461.pdf didn't have any mentioned limit as such. I guess it's because BART model cards currently available were trained with encoder having a limit of 1024 tokens. 
Btw thanks for pointing me to discussion forums; I will use them for further discussions.
RobertaTokenizerFast does not add special tokens
Author: marrrcin
Pull request: no

I'm not sure whether this should be a part of tokenizers or transformers, because it uses both. Classes that don't work are from `transformers` so I'm posting it here.


## Environment info

- `transformers` version: 4.3.3
- Platform: Colab
- PyTorch version (GPU?): n/a
- Tensorflow version (GPU?): n/a
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
- tokenizers: @n1t0, @LysandreJik 

## Information

### Reproduction code
https://colab.research.google.com/drive/1iYLBLzXRkQpdPyVlIdi_qNCzfbD1uwGs?usp=sharing

When loading tokenizer trained using `tokenizers` from transformers, e.g.
```python
tfast = RobertaTokenizerFast.from_pretrained("./workdir/tokenizer", model_max_length=10)
```
it does not add special tokens
```python
tfast("asd", add_special_tokens=True)
```

```
{'input_ids': [400, 72], 'attention_mask': [1, 1]}
```

"Slow" version behaves correctly:
```python
tslow = RobertaTokenizer.from_pretrained("./workdir/tokenizer", model_max_length=10)
tslow("asd", add_special_tokens=True)
```

```
{'input_ids': [0, 400, 72, 2], 'attention_mask': [1, 1, 1, 1]}
```

## Expected behavior

Both tokenizers produce the same output.


Comments:
Hi, thanks for opening an issue. Indeed, I can reproduce. The conversion to IDs happens in the `encode_batch` method in `tokenizers` directly, which doesn't return the special tokens even though it correctly receives `add_special_tokens=True`. The tokenizer object in transformers also seems to have the correct `special_tokens_map`.

@n1t0, is this an issue from `tokenizers` side? If we're not correctly passing something to the Rust tokenizer when instantiating a it from files, happy to look into it.

Discussed the issue with @n1t0 and the issue comes from the fact that the special tokens must be added to the tokenizer via a [post-processor](https://huggingface.co/docs/tokenizers/python/latest/api/reference.html?highlight=post#module-tokenizers.processors).

If it isn't done, then tokenizers cannot have their special tokens. The slow tokenizers having them anyway is linked to their initialization and not to the tokenizer you generated using the `tokenizers` library.

Here is a gist made from your colab showing how the post-processor should be used: https://gist.github.com/LysandreJik/04c7cfe3d2656ae1c4c388ce9cdd3ea4

Thanks @LysandreJik for the reply!
So it's more like a misconfiguration of the training pipeline on my side, not a bug per se?

Yes, I believe that is so. Tokenizers created with `tokenizers` need to have their post-processors/pre-tokenizers and other components defined to work correctly, otherwise it yields unexpected results as we have just seen!

Closing, but still seems odd that the behaviour for exact same files is different between those tokenizers...
Wav2Vec2 finetune
Author: idanmoradarthas
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

Can you please share code to how to finetune transformers.Wav2Vec2ForCTC, or maybe on how to give labels to the model in order to get loss?

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
Patrick is working on it, see #10145
Metric calculation across batches in seq2seq examples
Author: ZhaofengWu
Pull request: no

### Who can help
@sshleifer, @patil-suraj

## Information

Currently, the seq2seq finetune script calculates the final metrics (bleu or rouge) per batch, and then simply averages these numbers across batches. Is it correct? For example, I just took a brief look at sacrebleu (which the script uses), and it seems that `corpus_bleu(preds, targets)` isn't necessarily equal to `(corpus_bleu(preds[:n], targets[:n] + corpus_bleu(preds[n:], targets[n:])/2`.

Comments:
Definitely not correct. Additionally, the final batch might be much smaller (and still weighted evenly). To get "perfect" bleu scores use `run_distributed_eval.py`


Right, that's what I thought as well. If we think saving all the intermediate output and computing the bleu score once at the end of each validation epoch is too memory-intensive, one solution would be to accumulate intermediate BLEU-internal metrics, and compute the BLEU score at the end. This way the asymptotic memory complexity wouldn't depend on the validation set size. C.f. the AllenNLP implementation. At the very least, I think we should give a warning or something so that people realize this is not a precise number and wouldn't use it, say, in a paper.

Hey @ZhaofengWu 

This is now fixed in the new `run_seq2seq.py` script.
Add Fine-Tuning for Wav2Vec2
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

This PR adds the possibility to finetune Wav2Vec2 on a downstream task. I ran a couple of experiments and I think the training is pretty stable now, see this training run *e.g.*:

https://wandb.ai/patrickvonplaten/huggingface/reports/Project-Dashboard--Vmlldzo0OTI0OTc?accessToken=8azw8iyxnbiqd4ytxcgm4hbnfh3x1b2c9l2eyfqfzdqw7l0icreljc9qpx0rkl6f

Once this is merged, I will make a nice forum post and link 1,2 notebooks.

## Who can review?

Would be great if @sgugger @LysandreJik and @patil-suraj could review.

Comments:
This is really nice, and the piece that will make the wav2vec 2.0 stuff awesome and more readily available! let me know if I can assist in testing/whatnot :)
[WIP] Temp work on pipelines.
Author: Narsil
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->

Comments: