noamaanMulla-03/transformers-issues huggingface dataset:
[PR/Issue templates] normalize, group, sort + add myself for deepspeed
Author: stas00
Pull request: yes

This PR:
* case-normalizes, groups and sorts the tagging entries
* removes one duplicate
* adds myself for deepspeed
* adds/removes/moves others based on their suggestions through this PR

@LysandreJik, @sgugger, @patrickvonplaten 


Comments:
Once the PR template is complete and everybody is happy I will sync with the Issue template. So please only review the former if you're just joining in.

should we add bullets? As in:

```

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder, longformer, reformer, t5, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0
- trainer: @sgugger

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh
```

I like bullets.

someone to tag for ONNX issues? @mfuntowicz?
Fix broken links in the converting tf ckpt document
Author: forest1988
Pull request: yes

# What does this PR do?

This PR is to fix broken links in ["Converting TensorFlow Checkpoints"](https://huggingface.co/transformers/converting_tensorflow_models.html).

Advised by @LysandreJik in issue #9656, I updated the links.
I also refer to issue #8720 and it seems the issue is solved by this PR.

I think there are some outdated explanations.
As I discussed in issue #9657, I think it should be better to explain `from_pretrained()` instead of `torch.save()`.
Hence, I think the explanation below should be updated.

```
You can then disregard the TensorFlow
checkpoint (the three files starting with ``bert_model.ckpt``\ ) but be sure to keep the configuration file (\
``bert_config.json``\ ) and the vocabulary file (\ ``vocab.txt``\ ) as these are needed for the PyTorch model too.
```

I would be happy to get some advice on how to proceed with this PR.

Fixes #9656
Fixes #8720 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

 albert, bert, XLM: @LysandreJik

Comments:
I think the following comment of my own can be left as a PR for the future.
This change seems to require a modification of the `convert` code, and I think it may be necessary to separate it from this broken link issue.

I'm sorry for saying such a thing, even though the following is what I wrote.

> I think there are some outdated explanations.
> As I discussed in issue #9657, I think it should be better to explain `from_pretrained()` instead of `torch.save()`.
> Hence, I think the explanation below should be updated.
> 
> ```
> You can then disregard the TensorFlow
> checkpoint (the three files starting with ``bert_model.ckpt``\ ) but be sure to keep the configuration file (\
> ``bert_config.json``\ ) and the vocabulary file (\ ``vocab.txt``\ ) as these are needed for the PyTorch model too.
> ```
> 

I'll remove the WIP for now, but if you think this matter should be worked in this PR, please let me know.
Broken links in example for torch.load() after. converting tensorflow checkpoint to pytorch save model
Author: apurvaasf
Pull request: no
Labels: name: wontfix

name: Migration

The links for  run_bert_extract_features.py, run_bert_classifier.py, and run_bert_squad.py are all broken [here](https://huggingface.co/transformers/v2.4.0/converting_tensorflow_models.html). Could someone point me to a notebook where I can find examples for loading from a PyTorch save file pytorch_model.bin?


Comments:
If I understand correctly, you're trying to load a pytorch model from a `pytorch_model.bin`? If so, have you taken a look at the [quickstart](https://huggingface.co/transformers/v2.4.0/quickstart.html#main-concepts)? The `from_pretrained` method is probably what you're looking for.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
"Converting Tensorflow Checkpoints" document has wrong link in v4.2.0+
Author: forest1988
Pull request: no

## Environment info

- `transformers` version: v4.2.0+

### Who can help
 documentation: @sgugger

## Information

I'd like to convert BERT ckpt to PyTorch, and read [Converting Tensorflow Checkpoints](https://huggingface.co/transformers/converting_tensorflow_models.html) document.

It seems that the link to `convert_bert_original_tf_checkpoint_to_pytorch.py` is outdated.

It is linked to https://huggingface.co/transformers/converting_tensorflow_models.html, but `convert_bert_original_tf_checkpoint_to_pytorch.py` is now placed in https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/convert_bert_original_tf_checkpoint_to_pytorch.py (I found the information in https://github.com/huggingface/transformers/issues/9556).

It seems that in https://github.com/huggingface/transformers/pull/9217 the document is updated to use a prefix to get the `release` variable. 
However, perhaps the document is not yet changed to match the folder structure in v4?

Sorry if I misunderstand something.

Comments:
Hi! Indeed this link needs to be updated. Do you want to open a PR to fix it?

@LysandreJik 
Thank you for your comment! I'd love to open a PR to fix it.
I would like to open a PR by the end of this week.

Should I devise a way to change the link destination from the document before and after the change in the folder structure (version 3 to 4)?

Excuse me my opening a PR is delayed even though I said: "by the end of this week".

I haven't been able to find the time to do this, but your advice on another issue has helped me understand `convert` better, so I'm going to work on it.

I'll try to update the documentation on how to explain the differences between version 3 and 4, and I'd be happy to receive your comments in the PR (of course, any advice in advance would be greatly appreciated).


I apologize for the delay in getting the work done later than I said it would be.
I opened PR #9791.
If you have time, I would appreciate it if you could take a look.

Merged, thanks! No worries for the delay!

@LysandreJik 
Thank you for merging and giving me your kind words!
index mismatch in "offset_mapping" with TokenizerFast and pre-tokenized input
Author: simoneorlando
Pull request: no

## Environment info

- `transformers` version: 4.2.2
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0+cu101 (False)
- Tensorflow version (GPU?): 2.4.0 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

## Information

I am not sure if this is the expected behavior or not, but when i use BertTokenizerFast with pre-tokenized input (so i set the parameter "is_split_into_words" to True) i have a mismatch in the offsets_mapping. It considers every token standalone and restart the start index from zero.

## To reproduce

Steps to reproduce the behavior:

```
from transformers import BertTokenizerFast
    
tokenizer = BertTokenizerFast.from_pretrained('bert-base-cased') 

tokenized_one_string = tokenizer(
    ["This is not what i was expecting"],
    return_offsets_mapping=True,
)

tokenized_with_pre_tokenized_input = tokenizer(
    [["This", "is", "not", "what", "i", "was", "expecting"]],
    is_split_into_words=True,
    return_offsets_mapping=True,
)
print(tokenized_one_string["offset_mapping"])
print(tokenized_with_pre_tokenized_input["offset_mapping"])
```

and this is the output:
```
[[(0, 0), (0, 4), (5, 7), (8, 11), (12, 16), (17, 18), (19, 22), (23, 32), (0, 0)]]

[[(0, 0), (0, 4), (0, 2), (0, 3), (0, 4), (0, 1), (0, 3), (0, 9), (0, 0)]]
```
## Expected behavior
I was expected to get the same "offset_mapping" even from the pre-tokenized input.




Comments:
Hi @simoneorlando

This is indeed intended behavior. The values in `offset_mapping` return a mapping to the original input, and when you provide pre-tokenized input, each of them is treated individually. In this case, you can use the word mapping to know where you should do your extraction:
```python
from transformers import BertTokenizerFast

tokenizer = BertTokenizerFast.from_pretrained("bert-base-cased")

# Pre-tokenized input
labels = ["This", "is", "not", "what", "i", "was", "expecting"]
tokenized_with_pre_tokenized_input = tokenizer(
    labels,
    is_split_into_words=True,
    return_offsets_mapping=True,
)
print(tokenized_with_pre_tokenized_input["offset_mapping"])
for token_id, offsets in enumerate(tokenized_with_pre_tokenized_input["offset_mapping"]):
    word_id = tokenized_with_pre_tokenized_input.token_to_word(token_id)
    if word_id is not None:
        print(offsets, labels[word_id][offsets[0] : offsets[1]])
```
Gives the following output:
```python
[(0, 0), (0, 4), (0, 2), (0, 3), (0, 4), (0, 1), (0, 3), (0, 9), (0, 0)]
(0, 4) This
(0, 2) is
(0, 3) not
(0, 4) what
(0, 1) i
(0, 3) was
(0, 9) expecting
```
RagTokenForGeneration: Fixed parameter name for logits_processor
Author: michaelrglass
Pull request: yes

# What does this PR do?
The parameter name for the beam_search and greedy_search functions of the GenerationMixin is (now?) logits_processor not pre_processor.  This fix makes prefix_allowed_tokens_fn work (again?).

## Who can review?

 Rag: @patrickvonplaten, @lhoestq
 


Comments:
Fix fine-tuning translation scripts
Author: mbiesialska
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

In the [seq2seq README.md](https://github.com/huggingface/transformers/tree/master/examples/seq2seq#new-script) there are some errors. This PR fixes typos that cause the following problem:
```
Traceback (most recent call last):
  File "transformers/examples/seq2seq/run_seq2seq.py", line 536, in <module>
    main()
  File "transformers/examples/seq2seq/run_seq2seq.py", line 419, in main
    load_from_cache_file=not data_args.overwrite_cache,
  File ".../lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1240, in map
    update_data = does_function_return_dict(test_inputs, test_indices)
  File ".../lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1211, in does_function_return_dict
    function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
  File "transformers/examples/seq2seq/run_seq2seq.py", line 388, in preprocess_function
    inputs = [ex[source_lang] for ex in examples["translation"]]
  File "transformers/examples/seq2seq/run_seq2seq.py", line 388, in <listcomp>
    inputs = [ex[source_lang] for ex in examples["translation"]]
KeyError: 'en-XX'
```

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
<!--
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? -->


## Who can review?

@sgugger, @patil-suraj


Comments:
Add head_mask/decoder_head_mask for BART
Author: stancld
Pull request: yes

This PR implement `head_mask` and `decoder_head_mask` for PyTorch BART-based models. The full list, please, see below:
- **BART**
- **MBart**
- **Blenderbot**
- **BlenderbotSmall**
- **Marian**
- **Pegasus**

This PR is a follow up on the closed PR #9404.

**Motivation**:
According to HuggingFace's websites "There is a growing field of study concerned with investigating the inner working of large-scale transformers like BERT (that some call “BERTology”)." This PR enables to mask attention heads in encoder and decoder models exactly like for BERT. This PR thus creates an opportunity to study the importance of attention heads in encoder-decoder BERT-like model.


**Description**
- New arguments `head_mask` and`decoder_head_mask` are passed to all the BART-based models `...Model`, `...ForConditionalGeneration` and `...ForQuestionAnswering` after four arguments `input_ids, attention_mask, decoder_input_ids, decoder_attention_mask` so that a testing and whole pipeline remains smooth.

- This PR also contains updated `test_headmasking`, which currently works fine with one problem - BART-based models do not satisfy a condition:
```
self.assertNotEqual(attentions[1][..., 0, :, :].flatten().sum().item(), 0.0).

```
Fixing this problem is currently underway. 

**Reviewer:** @patrickvonplaten 

Comments:
Thanks for opening a new PR. Let me know if you need a review (It's also ok if I go into the PR and fix some things if your stuck :-) )

@patrickvonplaten I hope this PR is again ready for review. The only thing remaining to resolve is that issue in `test_headmasking` described above. Currently, I've been trying to fix this one, but I'll be grateful for sure if you can have a look at that too :)

Hey @patrickvonplaten. I would like to inform you I fixed `test_headmasking` for BART-based. The problem was that code inside
```
self.assertNotEqual(attentions[1][..., 0, :, :].flatten().sum().item(), 0.0)
```
pointed to the last layer of encoder/decoder (encoder-decoder models have only 2 layers in each module while BERT has 5 layers during testing). At the end of the day, this condition was invalid for BART-based models considering the `head_mask` to be
```
head_mask = torch.ones(
    self.model_tester.num_hidden_layers,
    self.model_tester.num_attention_heads,
    device=torch_device,
)
head_mask[0, 0] = 0
head_mask[-1, :-1] = 0
```

I hope this PR is then ready for review.
Add separated decoder_head_mask for T5 Models
Author: stancld
Pull request: yes

### Fix issue #9632 

<hr>

This PR separates `head_mask` and `decoder_head_mask` for T5 models, and thus enables to specify different head masks for an encoder and decoder.

**Description:**
- Replace a single input argument `head_mask` with a separated couple `head_mask` and `decoder_head_mask` for the T5 models: `T5Model, T5ForConditionalGeneration, TFT5Model, TFT5ForConditionalGeneration`
- Slightly change the order of input arguments to follow the convention of first 7 arguments introduced in PR #9569 for BART-based models, i.e. `"input_ids", "attention_mask", "decoder_input_ids", "decoder_attention_mask", "head_mask", "decoder_head_mask", "encoder_outputs"`
- Currently, the updated PyTorch T5 model does not pass `test_forward_signature` in `tests/test_modeling_common.py`. This problem will be diminished once PR #9569 to be merged.

Reviewer: @patrickvonplaten (the code is ready for review)

Comments:
Great, that looks nice! Let's first merge https://github.com/huggingface/transformers/pull/9569 and then rebase this PR so that it passes all tests :-) 

Thanks for fixing this!

I have one note/question: This seems to only apply to self-attention heads, not heads in the cross attention module, right? Is this intentional?

@talkhaldi Thank you very much for pointing this out. It seems you're right and this is not intentional by myself. It'll be fixed in another commit.

Hey @patrickvonplaten and @LysandreJik. I've added some `FutureWarning` into the code to handle cases when only `head_mask` is passed by a user. Also, I fixed a cross-attention issue noted by @talkhaldi.
I believe, the PR is now ready for review as all the checks have passed after the rebasing.
AllenNLP SPECTER model
Author: ckald
Pull request: no
Labels: name: wontfix

name: New model

# 🌟 New model addition

SPECTER: Document-level Representation Learning using Citation-informed Transformers

## Model description

[SPECTER](https://arxiv.org/pdf/2004.07180.pdf) by AllenAI is a new method to generate document-level embedding of scientific documents based on pretraining a Transformer language model on a powerful signal of document-level relatedness: the citation graph. It is based on SciBERT.

This model is showing us promising results for scientific paper similarity tasks. However, the repo is quite rough around the edges and inference performance could be better. According to the AllenNLP repo issues, they do not plan to adapt the models to ONNX runtime.

It would be great to have a plug'n'play implementation for Transformers. Instructions on how to do this are also welcome 🤝 

## Open source status

* [x] the model implementation is available: https://github.com/allenai/specter 
* [x] the model weights are available: see repo above 
* [x] who are the authors: AllenAI, @armancohan, @sergeyf 


Comments:
Wonderful if we can use embeddings of scientific documents in transformers.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


SPECTER is a great work. Is there any plan to have the Huggingface version of SPECTER?

This should be addressed now: https://huggingface.co/allenai/specter
How to use: https://github.com/allenai/specter
Allow RAG to output decoder cross-attentions
Author: dblakely
Pull request: yes

# What does this PR do?

This PR makes RAG output the generator model's decoder cross-attentions when `output_attentions=True`.

Motivation and context: before this PR, RAG's output objects had attributes for the generator's encoder self-attentions and decoder self-attentions, but no option for the encoder-decoder cross-attentions. So this simply allows cross-attentions to be extracted, as well as fixing a small bug where `output_attentions` wasn't being passed into the generator.

Fixes #9468


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. Yes - #9468 
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? - I don't believe any new tests are necessary. Existing tests pass.


## Who can review?
@patrickvonplaten, @lhoestq


Comments:
@lhoestq Thanks for the suggestions! All the CI checks pass now.
Have RAG return generator cross-attentions when output_attentions=True
Author: dblakely
Pull request: no

# 🚀 Have RAG return generator cross-attentions when output_attentions=True

This feature request is for the RAG code to be modified so that if `output_attentions=True`, it returns the generator's cross-attentions in addition to the attentions it already returns.

## Motivation

I'm interested in extracting the generator's attentions from a RAG generator model. Currently, `transformers` allows you to extract the generator's encoder attentions and decoder attentions, but not it's cross attentions. For example, inside `modeling_rag.py`, the return objects such as [RetrievAugLMMarginOutput](https://github.com/huggingface/transformers/blob/master/src/transformers/models/rag/modeling_rag.py#L38), have fields for these other attentions, but not the cross-attentions.

Because both T5 and BART can output cross-attentions, I think they could simply propagate up through the RAG code. Is there a reason this isn't already the case? Or could I do a PR to include the cross attentions along with the other attentions in the model output?

## Your contribution

On my own fork of `transformers`, I've already added this feature and would happily submit a PR!


Comments:
@patrickvonplaten, @lhoestq Any feedback on this? 

Feel free to open a PR indeed :) 
What do you think @patrickvonplaten ? I guess it can be part of the RetrievAugLMMarginOutput attributes.

Hey @dblakely,

It would be great if you could open a PR. Both Bart and T5 already return the `cross_attentions`, so it should be a pretty easy change by just adding 

```python
generator_cross_attentions=gen_outputs.cross_attentions,
```
here:

https://github.com/huggingface/transformers/blob/fac7cfb16a437a97584f6a14c3856b2e06bf0eaa/src/transformers/models/rag/modeling_rag.py#L657

and then adding `generator_cross_attentions` to all output classes as suggested by @lhoestq
Commit the last step on world_process_zero in WandbCallback
Author: tristandeleu
Pull request: yes

# What does this PR do?

Only commit the last step on the first process (`is_world_process_zero == True`), to avoid calling `wandb.log()` without a prior call to `wandb.init()` (the latter being only called on the first process) with DDP.

Fixes (at least partially) #9623 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@borisdayma @sgugger 


Comments:
Would it make sense to move [those 2 lines](https://github.com/huggingface/transformers/blob/0dd939bf1e01594eadf21b40e5cdb07001233cbf/src/transformers/integrations.py#L573-L574) to the init method instead of setting `self._log_model` to False?

This should be equivalent indeed, and probably easier to read

Looks great!