noamaanMulla-03/transformers-issues huggingface dataset:
Store transformers version info when saving the model
Author: JetRunner
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR stores the transformers version info in the model config. It makes debugging saved models from the model hub easier without affecting any actual function.


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
[Generation] Add generation outputs
Author: patrickvonplaten
Pull request: no

# 🚀 Feature request

We've had multiple issues asking for the possibility to output the scores/probabilities of each token during generation, see:
https://github.com/huggingface/transformers/issues/7654
https://github.com/huggingface/transformers/issues/3891
https://github.com/huggingface/transformers/issues/8656

Also we should be able to output the models attentions `hidden_states` at each generation step, *a.k.a* make use of those model outputs: 
https://github.com/huggingface/transformers/blob/c19d04623eacfbc2c452397a5eda0fde42db3fc5/src/transformers/models/bert/modeling_bert.py#L883 in generation as well.

To do so we should create a new generation output class for each "sub" generation function:

1) `GreeySearchDecoderOnlyOutput(output_ids,  logits, attentions, hidden_states)` for decoder-only models, where as `output_ids` are the current outputs of generate, `logits` will be the logit vectors at each generation step (so should be of shape `Tuple((logits_1,), ..., (logits_max_length,))`) and `attentions` and `hidden_states` should be of shape `Tuple((attentions_1,), ..., (attentions_max_length,))`. As before `attentions` and `hidden_states` will be output if a flag `output_attentions` or `output_hidden_states` iset to True and for the logits we should add a flag `output_scores`. Also we should have a `GreeySearchEncoderDecoderOutput(output_ids,  logits, encoder_attentions, decoder_attentions, encoder_hidden_states, decoder_hidden_states)` class with the respective enc and dec outputs.

2) `SampleDecoderOnlyOutput(output_ids,  probabilities, attentions, hidden_states)` -> the same outputs only that we replace the logits output with the softmax probabilities (of the same shape); same flags as in 1) and encoder-decoder class as well 

3) `BeamSearchDecoderOnlyOutput(output_ids, probs, attentions, hidden_states)` -> `probs` should be this tensor: https://github.com/huggingface/transformers/blob/c19d04623eacfbc2c452397a5eda0fde42db3fc5/src/transformers/generation_utils.py#L1235 at each step same flags as in 1) and encoder-decoder class as well

Each output class should be derived from https://github.com/huggingface/transformers/blob/c19d04623eacfbc2c452397a5eda0fde42db3fc5/src/transformers/file_utils.py#L1306 just as the model output classes are in https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_outputs.py . 

A PR should start with the "GreedySearchOutput" model classes and add this to `generation_utils.py` => then we should add the  three flags to both `generate()` and `greedy_search()`. Then `SampleOutput` and `BeamSerachOutput` should be added. The PR should also include good documentation for each of the outputs as it is the case for the current model outputs.

## Your contribution

I'm happy to help the contributor throughout the PR :-) 


Comments:
fix order of input/target of cross_entropy
Author: HHoofs
Pull request: yes
Labels: name: wontfix

https://pytorch.org/docs/stable/nn.functional.html#cross-entropy

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6642?src=pr&el=h1) Report
> Merging [#6642](https://codecov.io/gh/huggingface/transformers/pull/6642?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d0e42a7bed3de9271ae39c575d7eeb54cf985921?el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6642/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6642?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #6642   +/-   ##
=======================================
  Coverage   79.14%   79.14%           
=======================================
  Files         156      156           
  Lines       28248    28248           
=======================================
  Hits        22358    22358           
  Misses       5890     5890           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6642?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6642/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `82.45% <0.00%> (-0.26%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6642/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.41% <0.00%> (+0.25%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6642?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6642?src=pr&el=footer). Last update [d0e42a7...502f692](https://codecov.io/gh/huggingface/transformers/pull/6642?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


As far as I can see it, this snippet in the Readme is not working correctly.
[utils/get_modified_files.py] fails with a few PR checkout tools
Author: stas00
Pull request: no

I have noticed that when using [gh cli](https://github.com/cli/cli) to checkout a pr
```
git merge-base --fork-point master
```
fails, which breaks `utils/get_modified_files.py`
e.g.:

```
gh pr checkout 9423
python utils/get_modified_files.py
Traceback (most recent call last):
  File "utils/get_modified_files.py", line 27, in <module>
    fork_point_sha = subprocess.check_output("git merge-base --fork-point master".split()).decode("utf-8")
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'merge-base', '--fork-point', 'master']' returned non-zero exit status 1.
```

So `make fixup` fails to check the modified files then.

It fails if I use `git-pr` too (this tool is from the git-extras package).

It works fine if I use the native `git pr`.

This needs to be investigated.

Until this is resolved if you use those tools please use `make style` /  `make quality`, but `make fixup`  should work just fine elsewhere.


Comments:
Prophetnet optimization
Author: guillaume-be
Pull request: yes

# What does this PR do?

This PR proposes an optimization for the ProphetNet model. The current implementation calculates an attention bias mask by looping through the position to unmask. It performs a high number of assignments (`ngram` * `sequence_length`) which can be in the order of ~1000. Single tensor assignments, especially on accelerators, are inefficient.

This PR proposes a vectorized implementation which performs at most `ngram` assignments, which should be significantly lower than `ngram * sequence_length`.

A quick experiment shown at https://gist.github.com/guillaume-be/e6b862c701fac1b54765e7af7e71c641 shows that:
1. this `ngram_attention_bias` calculation is very expensive, taking close to 230ms (!) on a GPU
2. the vectorized implementation is several orders of magnitude faster (the same calculation takes less than 1ms on the same example)

## Who can review?

@patrickvonplaten maybe you would be a good candidate? I could not find anyone assigned for ProphetNet

edit: pushed some further optimization, further accelerating by ~40%

Comments:
All slow tests are passing! Very nice PR - thanks a mille @guillaume-be
[EncoderDecoder] Make sure `use_cache` is set to `True` for all Bert2Bert, Roberta2Roberta by default
Author: patrickvonplaten
Pull request: no

At the moment when one loads a Bert2Bert:

```python
model = EncoderDecoderModel.from_encoder_decoder_pretrained("bert-base-cased", "bert-base-cased")
```

does not automatically set `use_cache` to True -> so that the user "silently" has to be fine with a much slower than optimal inference speed. Also all Bert2Bert configs online don't have `use_cache` set to True. This should be changed at least for the heavily used Bert2Bert models.

I'll try to take care of that in the next couple days. Also pinging @patil-suraj for information. Thanks @Narsil for binging up the topic. 

Comments:
@patrickvonplaten 

Can we instead set `use_cache` to `True` by default in `generate`? That way we won't need to rely on `config` 

Right now, the `generate` [docstring](https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L689) says that it defaults to `True`, but it's set to `None`

https://github.com/huggingface/transformers/blob/28d74872cc049e0cbee3fafd15cbbabfe348ebd4/src/transformers/generation_utils.py#L618


Hmm, that goes a bit against the philosophy because we never "set" any variables in `generate()`. We should do it in `EncoderDecoderConfig` and in `from_encoder_decoder_pretrained`. Note that all args in `generate()` are set to `None`, but default to the respective config defaults which should be set correctly

Also `use_cache` is newly introduced in bert/roberta config and is `True` by default, so even if the model's config file online doesn't have  `use_cache` it should still be `True,` no?

Could you maybe provide an example where the above issue occurs?

@patil-suraj, you're 100% right!

I initially thought it's a problem because `EncoderDecoderConfig` does not have a `use_cache` param set to `True`, but it doesn't actually matter since `model.decoder.config.use_cache` will always be set to `True` by default which forces `use_cache` to be True in the decoder which makes it return the `past_key_values` => so all good then - thanks a lot for double-checking this :-)
Turn attentions and hidden-states into a tensor
Author: jplu
Pull request: yes

# What does this PR do?

This PR turns the `all_attentions` and `all_hidden_states` values into tensors instead of a tuple. This update is to properly allow the dict outputs in TF serving, because the value of each key cannot be something else than a TF tensor.

Here a simple piece of code to produce the issue:
```
from transformers import TFBertModel, BertConfig
import tensorflow as tf
config = BertConfig.from_pretrained("bert-base-cased", output_attentions=True)
model = TFBertModel.from_pretrained("bert-base-cased", config=config)
tf.saved_model.save(model, "my_model")
```

Gets the error:
```
ValueError: Got a dictionary containing non-Tensor value (<tf.Tensor 'StatefulPartitionedCall:0' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:1' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:2' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:3' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:4' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:5' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:6' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:7' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:8' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:9' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:10' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:11' shape=(None, 12, None, None) dtype=float32>) for key attentions in the output of the function __inference_serving_15889 used to generate a SavedModel signature. Dictionaries outputs for functions used as signatures should have one Tensor output per string key.
```

Comments:
After an offline discussion we decided to proceed differently. Then closing this PR.
FileNotFoundError when instantiating RagRetriever
Author: poccio
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 4.1.1
- Platform: Linux-5.4.0-58-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.1 (True)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help
@patrickvonplaten, @lhoestq

## Information

I am trying to use RAG but I am having issues downloading the compressed index.

## To reproduce

```python
from transformers import RagRetriever
retriever = RagRetriever.from_pretrained('facebook/rag-sequence-nq', dataset='wiki_dpr', index_name='compressed')
```
Which results in:
```python
FileNotFoundError: Couldn't find file at https://storage.googleapis.com/huggingface-nlp/cache/datasets/wiki_dpr/psgs_w100.nq.compressed/0.0.0/psgs_w100.nq.IVF4096_HNSW128_PQ128-IP-train.faiss
```


Comments:
Hi ! Thanks for reporting
Can you try again ? I fixed the missing file

Thanks a lot! I'll try right away (with my internet speed, it should take ~1h30)

Yup, I can confirm now the issue is resolved. Thanks a lot! Shall I close the issue?

Hey @poccio,

usually, always feel free to close issues that you opened. As maintainers, we don't always feel comfortable closing an issue since it's not always clear whether the author's issue is solved. So if it's solved for you, it's great if you close it :-) Thanks for reporting the issue.
Transformers fast import part 2
Author: sgugger
Pull request: yes

# What does this PR do?

This is the second test to allow a fast import of transformers by deferring the imports of dependencies when they are actually needed (almost but not quite, see below). It results in the line `import transformers` running in 239ms instead of 2.3s, so quite a nice speedup.

To do this, the main init is changed to have a bid private dictionary that maps modules names to public object names instead of directly importing those objects. A submodule or object is then only imported when explicitly requested, which means the line `import transformers` by itself doesn't import any of the dependencies.

This mechanism is incompatible with absolute imports inside the library, hence the big diff as I had to change quite a few `from transformers.xxx import yyy` to `from .xxx import yyy`.

Also, this misses the last piece to be completely efficient: the intermediate init (in models) should use the mechanism to avoid importing TensorFlow when we only request a PyTorch model. This will be done in another PR as this one is already quite big by itself.

The script that created the dummy objects needed some update because it used to parse the init. I took this opportunity to also refactor the dupe code. Obviously the templates also needed an update.

The rework of the init also makes it important to have the intermediate init of `models` be nonempty, otherwise things like
```
import transformers
auto_module = transformers.model.auto
```
will break. I don't think this is a big inconvenience (especially since the update template will fill this for the user).

Comments:
Just followed a bug back to this PR, wanted to send a message here since it seemed relevant to ping @sgugger 

The check for version in file_utils.py:
`if version.parse(sys.version) < version.parse("3.8"):` doesn't seem to be reliable for me (or on multiple machines and images I have)

Specifically: 
```
Python 3.8.5 (default, Aug  6 2020, 14:13:36) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> from packaging import version
>>> sys.version
'3.8.5 (default, Aug  6 2020, 14:13:36) \n[GCC 9.3.0]'
>>> version.parse("3.8.5")
<Version('3.8.5')>
>>> version.parse("3.8.5") < version.parse("3.8")  # expect false
False
>>> version.parse(sys.version) < version.parse("3.8")  # expect false
True
```

Instead, it seems more reliable/functional to not rely on `packaging.version` at all and instead do `sys.version_info < (3, 8)`.

I can also put in an Issue if that's a more appropriate way to raise / flag a concern. Just thought i'd ping here since i was able to trace it back to this PR from today. 

Oh, thanks for reporting! Will add this to #9474 which should be merged tomorrow.
[make fixup] a more reliable version of branching point discovery
Author: stas00
Pull request: yes

This PR replaces:
```
git merge-base --fork-point master
```
with:
```
git merge-base master HEAD
```
in `utils/get_modified_files.py` (which is used by `make fixup`)

As I reported in https://github.com/huggingface/transformers/issues/9425 the former method sometimes doesn't work when used with `gh pr checkout` or `git-pr`, rendering the relatively recently added git ` --fork-point` feature unreliable.

I have re-tested and the new way works for any of:
1. `gh pr checkout`
2. `git-pr`
3. `git pr`
4. normal local git branch

So this is what we are doing now to get only the modified files of the current branch:
```
git diff --name-only $(git merge-base master HEAD)
```

If we get complex branches that have various re-merges we will want to find not the most recent ancestor which the above gives, but the oldest ancestor - after some research found this: https://stackoverflow.com/a/4991675/9201239, which suggests:
```
diff --changed-group-format='' <(git rev-list --first-parent "${1:-master}") <(git rev-list --first-parent "${2:-HEAD}") | head -1
```
and in the simple case where there is just one common ancestor it will find it too. So let's keep this as an option if you find the current solution isn't satisfactory.

Fixes: #9425 

@LysandreJik 

Comments:
> why --fork-point doesn't work with the GitHub CLI

I wasn't able to figure out what exactly those 2 tools do differently, but yes, `--fork-point` only works when specific conditions are met in the reflog, and it fails when some entries (the sha we are after) missing from it. I suppose `gh and` `git-pr` fetch just part of the reflog?

Apparently there are multiple causes. The first one is described at https://stackoverflow.com/a/53981615/9201239 and then it links to a discussion with additional causes.

I see! Thank you, this is interesting!
[README] Add new models
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Adds LED and BlenderbotSmall to the Readme.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments: