noamaanMulla-03/transformers-issues huggingface dataset:
[Flax] Add attention weights outputs to all models
Author: patrickvonplaten
Pull request: no
Labels: name: Good First Issue

# 🚀 Feature request

At the moment we cannot return a list of attention weight outputs in Flax as we can do in PyTorch. 

In PyTorch, there is a `output_attentions` boolean in the forward call of every function, see [here](https://github.com/huggingface/transformers/blob/42fe0dc23e4a7495ebd08185f5850315a1a12dc0/src/transformers/models/bert/modeling_bert.py#L528) which when set to True collects all attention weights and returns them as a tuple. 
In PyTorch, the attention weights are returned (if `output_attentions=True`) from the self-attention layer *e.g.* here: https://github.com/huggingface/transformers/blob/42fe0dc23e4a7495ebd08185f5850315a1a12dc0/src/transformers/models/bert/modeling_bert.py#L331 and then passed with the outputs. 

Currently, this is not implemented in Flax and needs to be done. At the moment the function [`dot_product_attention`](https://github.com/google/flax/blob/6fb839c640de80f887580a533b222c6dddf04c0d/flax/linen/attention.py#L109) is used in every Flax model which makes it impossible to retrieve the attention weights, see [here](https://github.com/huggingface/transformers/blob/42fe0dc23e4a7495ebd08185f5850315a1a12dc0/src/transformers/models/bert/modeling_flax_bert.py#L244). However recently the Flax authors refactored this function into a smaller one called [`dot_product_attention_weights`](https://github.com/google/flax/blob/6fb839c640de80f887580a533b222c6dddf04c0d/flax/linen/attention.py#L37) which would allow us to correctly retrieve the attention weights if needed. To do so all `dot_product_attention` functions should be replaced by `dot_product_attention_weights`, followed by a `jnp.einsum`, see [here](https://github.com/google/flax/blob/6fb839c640de80f887580a533b222c6dddf04c0d/flax/linen/attention.py#L162) so that we can retrieve the attention weights. 
Next, the whole `output_attentions` logic should be implemented for all Flax models analog to `output_hidden_states`.


Comments:
Also, adding this feature will require us to bump up the Flax dependency to `>=0.3.4` for `flax` in https://github.com/huggingface/transformers/blob/master/setup.py

I am starting with Bert.

Hi @patrickvonplaten @patil-suraj https://github.com/huggingface/transformers/pull/11918
[Flax] Add FlaxBart model
Author: stancld
Pull request: no

# 🚀 Feature request

It would be nice to implement Flax version of BART model.

## Motivation

Narrow the gap in support between encoder transformers (BERT, RoBERTa, ...) and encoder-decoder models (BART,...).

## Your contribution

I've been working on this now so I hope I will send a PR soon.

@patrickvonplaten @sgugger 

Comments:
Wow this sounds awesome! Let me know if you need help! (I think we will need to look at the `generate()` function) together :-)

Hi @patrickvonplaten, as I indicated I've started working on `FlaxBart` which can be found on this branch https://github.com/stancld/transformers/tree/FlaxBart . So far, I've implemented `FlaxBartModel` and `FlaxBartForConditionalGeneration` with some remaining pieces to do, but it is possible to run them. 

As there is no official template for Flax encoder-decoder models, I've tried to follow Torch implementation of Bart and Flax implementation of Bert. Before diving deeper and finishing all the components, tests etc, could I, please, ask you to provide me with short feedback if this structure seems ok to you? Thanks a lot in advance! :)
(I guess I left some redundant code there but I'm gonna polish it soon)

This sounds great :-) Thanks a lot for tackling this! Could you maybe make a [WIP] PR from your branch and ping me - this would make it a bit easier to review the code

Hey @stancld, 

I looked quickly and in general the PR already looks great :-) 

A couple of things:
- we don't allow `labels` as an input argument to Flax models (and actually probably even won't do this in the future). Flax/Jax is inherently functional which means that a loss function should wrap the model forward function and not the other way around
- Weight tying is done a bit differently as in PyTorch, thus we don't need `    def get_input_embeddings(self):` for now
- `return_dict` is now also implemented in `FlaxBert` -> so this can be copied from there :-) 

=> In short the design already looks great :-) I think you can open a PR & we'll discuss everything on the PR 

@patrickvonplaten Thanks a lot for the feedback! I will create [WIP] PR later today :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

ping
[lm examples] replicate --config_overrides addition to other LM examples
Author: stas00
Pull request: no
Labels: name: Good First Issue

This PR https://github.com/huggingface/transformers/pull/11798 created for `run_clm.py` which adds a new feature `--config_overrides` needs to be replayed for other scripts under `examples/pytorch/language-modeling/`.

If you choose to work on this small project, please comment that you're working on it.

And thank you!

Comments:
I am getting started with this task. 

Thank you, @kumar-abhishek!
Model type to AutoModelForQuestionAnswering incorrect
Author: AjaxFB
Pull request: no

`model = AutoModelForQuestionAnswering.from_pretrained("dbmdz/german-gpt2")

ValueError: Unrecognized configuration class <class 'transformers.models.gpt2.configuration_gpt2.GPT2Config'> for this kind of AutoModel: AutoModelForQuestionAnswering.
Model type should be one of BigBirdConfig, ConvBertConfig, LEDConfig, DistilBertConfig, AlbertConfig, CamembertConfig, BartConfig, MBartConfig, 
LongformerConfig, XLMRobertaConfig, RobertaConfig, SqueezeBertConfig, BertConfig, XLNetConfig, FlaubertConfig, MobileBertConfig, XLMConfig, 
ElectraConfig, ReformerConfig, FunnelConfig, LxmertConfig, MPNetConfig, DebertaConfig, DebertaV2Config, IBertConfig.`

How i can convert Model Type? If its not possible, i need suggestions to solve this problem ;(

Comments:
`AutoModelForQuestionAnswering` expects a model with a span classification head on top, GPT2 is an auto-regressive language model and does not have `ForQuestionAnswering` class defined for it. So it can't be used with `AutoModelForQuestionAnswering`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[lm examples] Replicate --config_overrides addition to other LM examples
Author: kumar-abhishek
Pull request: yes

# What does this PR do?
This PR replays the new feature `--config_overrides` for other scripts under `examples/pytorch/language-modeling/` which was added by https://github.com/huggingface/transformers/pull/11798/



<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->


Fixes: https://github.com/huggingface/transformers/issues/11875


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@stas00 

Comments:
I don't think this change applies to `run_clm_no_trainer.py` and `run_mlm_no_trainer.py` since the argument `model_name_or_path` is a required argument and we can't have both arguments `model_name_or_path` and `config_overrides` at the same time.
[style] consistent nn. and nn.functional
Author: stas00
Pull request: yes

As discussed in https://github.com/huggingface/transformers/issues/11600 this PR normalizes to `nn.functional.foo()` replacing `F.` and `torch.nn.` with `nn.`.

This is all automated by:
```
# deal with torch.nn
perl -pi -e 's|^import torch\n|from torch import nn\nimport torch\n|' `grep -Ilr torch.nn src`
find src -type f -exec perl -X -pi -e 's{(?<!(from |import |[`#/]))torch\.nn\.}{nn.}g' {} \;
find src -type f -exec perl -pi -e 's|import torch\.nn as nn|from torch import nn|g' {} \;

# deal with F
find src -type f -exec perl -pi -e 's|from torch.nn import functional as F|from torch import nn|g' {} \;
find src -type f -exec perl -pi -e 's|import torch.nn.functional as F|from torch import nn|g' {} \;
find src -type f -exec perl -pi -e 's|(?<!\w)F\.|nn.functional.|g' {} \;

git checkout src/transformers/data/data_collator.py
perl -pi -e 's|import torch||' src/transformers/models/prophetnet/convert_prophetnet_original_pytorch_checkpoint_to_pytorch.py
make fixup
```

This is just `src/transformers` for now. If happy I can do the same for `templates`, `tests` and `examples` next

To kind reviewers: This is a massive auto-rewrite, if you notice some missed patterns please just point one instance to me, so I will adjust regex to catch them all. (since we need to do at least `tests`/`examples` too)

@sgugger, @LysandreJik, @patrickvonplaten


Comments:
[style] consistent nn. and nn.functional: part2: templates
Author: stas00
Pull request: yes

Continuing https://github.com/huggingface/transformers/pull/12124 this PR takes care of `templates` - had to do some manual tweaking over automated rewrite here since `make fixup` can't process templates.

```
# deal with torch.nn
perl -pi -e 's|^import torch\n|from torch import nn\nimport torch\n|' `grep -Ilr torch.nn templates`
find templates -type f -exec perl -X -pi -e 's{(?<!(from |import |[`#/]))torch\.nn\.}{nn.}g' {} \;
find templates -type f -exec perl -pi -e 's|import torch\.nn as nn|from torch import nn|g' {} \;

# deal with F
find templates -type f -exec perl -pi -e 's|from torch.nn import functional as F|from torch import nn|g' {} \;
find templates -type f -exec perl -pi -e 's|import torch.nn.functional as F|from torch import nn|g' {} \;
find templates -type f -exec perl -pi -e 's|(?<!\w)F\.|nn.functional.|g' {} \;
make fixup
```
and some manual corrections to remove duplicated imports.

@sgugger 

Comments:
TFWav2Vec2Model
Author: will-rice
Pull request: no
Labels: name: New model

# 🌟 New model addition
I want a Tensorflow version of Wav2Vec2 and it's something I'm willing to work on. I just wanted to make sure no one else was working on it. I looked at the issues, PR's, and source code. The only mention of TFWav2Vec2Model is in the current docs so just wanted to double-check no one was working on this before I dive in. 

> vocab_size (int, optional, defaults to 32) – Vocabulary size of the Wav2Vec2 model. Defines the number of different tokens that can be represented by the inputs_ids passed when calling Wav2Vec2Model or TFWav2Vec2Model. 


## Model description
Tensorflow port of Wav2Vec2 
<!-- Important information -->

## Open source status
Tensorflow version of existing Transformers Model
* [x] the model implementation is available: (give details)
* [x] the model weights are available: (give details)
* [ ] who are the authors: (mention them, if possible by @gh-username)


Comments:
Sure, there's a [general conversion script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_pytorch_checkpoint_to_tf2.py) that allows you to convert the weights from a PyTorch model to its TF counterpart. You only need to make sure that all the attributes of the TF model correspond to the names of the PyTorch model.

cc @patrickvonplaten  

@will-rice, adding TFWav2Vec2Model would be a great addition indeed -> I'm more than happy to help you on the PR
[style] consistent nn. and nn.functional: part 4 `examples`
Author: stas00
Pull request: yes

This concludes the work on https://github.com/huggingface/transformers/issues/11600 with normalizing `examples` using fully automated:

```
# deal with torch.nn
perl -pi -e 's|^(\s*)import torch\n|$1from torch import nn\n$1import torch\n|' `grep -Ilr torch.nn examples`
find examples -type f -exec perl -X -pi -e 's{(?<!(from |import |[`#/]))torch\.nn\.}{nn.}g' {} \;
find examples -type f -exec perl -pi -e 's|import torch\.nn as nn|from torch import nn|g' {} \;

# deal with F
find examples -type f -exec perl -pi -e 's|from torch.nn import functional as F|from torch import nn|g' {} \;
find examples -type f -exec perl -pi -e 's|import torch.nn.functional as F|from torch import nn|g' {} \;
find examples -type f -exec perl -pi -e 's|(?<!\w)F\.|nn.functional.|g' {} \;
perl -pi -e 's|import torch||' examples/research_projects/pplm/pplm_classification_head.py

# leave legacy unmodified as we can't test is easily
git checkout examples/legacy

make fixup
```

@sgugger 

Comments:
> You have changes in two png files here too, which is weird. 

Whoah! Found some secret stenography embeddings! :)

Probably got triggered by `s|(?<!\w)F\.|nn.functional.|g`

Thank you for noticing, @sgugger  - will fix it up!

> Not sure if we really need to apply this to the research projects which are not actively maintained.

at least - `wav2vec2` is. Do you want me to reset all but `wav2vec2` under research?


No, it's easier to go for all of them in that case.
[style] consistent nn. and nn.functional: part 3 `tests`
Author: stas00
Pull request: yes

Continuing https://github.com/huggingface/transformers/pull/12124 this PR takes care of `tests` - a slight variation of the automated code:

```
# deal with torch.nn
perl -pi -e 's|^(\s*)import torch\n|$1from torch import nn\n$1import torch\n|' `grep -Ilr torch.nn tests`
find tests -type f -exec perl -X -pi -e 's{(?<!(from |import |[`#/]))torch\.nn\.}{nn.}g' {} \;
find tests -type f -exec perl -pi -e 's|import torch\.nn as nn|from torch import nn|g' {} \;

# deal with F
find tests -type f -exec perl -pi -e 's|from torch.nn import functional as F|from torch import nn|g' {} \;
find tests -type f -exec perl -pi -e 's|import torch.nn.functional as F|from torch import nn|g' {} \;
find tests -type f -exec perl -pi -e 's|(?<!\w)F\.|nn.functional.|g' {} \;
make fixup
```

One concern is for slow tests that would be missed by CI, so let's be on the lookout for the nightly slow run after this PR is merged.

@sgugger 

Comments:
> It looks like you have changes in two test fixtures, is that intended?

Oh, I was studying `git diff` and missed the binary change - thank you for noticing it, @sgugger - fixed.
[consistent use] `F` vs. `nn.functional`
Author: stas00
Pull request: no
Labels: name: WIP

We use 3 different ways of doing the same:
1. `F.foo()`
2. `nn.functional.foo()`
3. `torch.nn.functional.foo()`

and these could also be imported:

4. `from torch.nn.functional import foo; foo()`

Asking others it appears that `F` is not quite liked, so it's 2, 3 or 4.

2 and 3 often lead to longer lines which autoformatter wraps, leading to 3 lines of code instead of 1 and which gives less readable code.

So it seems that option 4 might be the best outcome.

For 2, the global update would be easy:
```
find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's|from torch.nn import functional as F||'  {} \;
find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's|import torch.nn.functional as F||'  {} \;
find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's| F\.| nn.functional.|g'  {} \;
make fixup
```

For 4, it will take much more work, but can be semi-automated.

@LysandreJik, @sgugger, @patrickvonplaten 


Comments:
I personally vote for 4, and if there is any risk of ambiguity, doing
```
from torch.nn.functional import foo as torch_foo
```


2 or 4 for me, but I don't have strong feeling about this issue

2 looks good to me but no strong feelings either

I further analyzed the codebase and it looks like we use `nn.` almost exclusively, for all other cases outside of `nn.functional`, with an occasional `torch.nn.` 

So I'd say 2 is probably the most consistent way, unless we start explicitly importing `Parameter`, `Embedding`, `ModuleList` from `nn`.

I know the additional "functional" string pushes some single lines into autoformatter's 3 lines mode. But for consistency I think sticking to just a single:
```
from torch import nn
```
is a goodness.

@sgugger,  will option 2 work for you?

If we stick to 2, then I will also normalize `s|torch.nn.foo|nn.foo|` calls unless you object. 

That works for me.

Status:

merged:
- [x] src https://github.com/huggingface/transformers/pull/12124
- [x] examples https://github.com/huggingface/transformers/pull/12156
- [x] templates https://github.com/huggingface/transformers/pull/12153
- [x] tests https://github.com/huggingface/transformers/pull/12155
- [x] docs https://github.com/huggingface/transformers/pull/12161

just need to merge docs, but all is done here.
consistent nn. and nn.functional: part 5 docs
Author: stas00
Pull request: yes

Continuing https://github.com/huggingface/transformers/pull/12124 this PR takes care of `docs` - had to do a bit of extra filtering to not break the images:

```
# deal with torch.nn
perl -pi -e 's|^(\s*)import torch\n|$1from torch import nn\n$1import torch\n|' `grep -Ilr torch.nn docs`
find docs -type f -regextype posix-egrep -regex '.*(md|rst)$' -exec perl -X -pi -e 's{(?<!(from |import |[`#/]))torch\.nn\.}{nn.}g' {} \;
find docs -type f -regextype posix-egrep -regex '.*(md|rst)$' -exec perl -pi -e 's|import torch\.nn as nn|from torch import nn|g' {} \;

# deal with F
find docs -type f -regextype posix-egrep -regex '.*(md|rst)$' -exec perl -pi -e 's|from torch.nn import functional as F|from torch import nn|g' {} \;
find docs -type f -regextype posix-egrep -regex '.*(md|rst)$' -exec perl -pi -e 's|import torch.nn.functional as F|from torch import nn|g' {} \;
find docs -type f -regextype posix-egrep -regex '.*(md|rst)$' -exec perl -pi -e 's|(?<!\w)F\.|nn.functional.|g' {} \;


make fixup
```
and one manual tweak. docs are hard to automate rewrites and there is no validation, so had to carefully check each diff.


@sgugger 

Comments:
Documentation for tiny-gpt2 in transformers/examples/pytorch
Author: tginart
Pull request: no

# 🚀 Documentation request

The **tiny-gpt2** transformer model is great for fast prototyping, but it seems sparsely documented on the Huggingface hub: https://huggingface.co/sshleifer/tiny-gpt2

## Motivation

It would be helpful if users knew basic info about how tiny-gpt2 was trained. Is it the same corpus as the standard gpt2? Was it distilled from a larger model or trained from scratch? Etc.

## Your contribution

As I did not train tiny-gpt2, I don't know any info about it.

Comments:
It is randomly initialized and trained for 2 steps. So basically it can only be used for prototyping.
Ouput Includes Input
Author: kurbster
Pull request: no

Whenever I am generating text the input is included in the output. When the input is close to the maximum length the model barely produces any useful output.
# Information
When using transformers.pipeline or transformers.from_pretrianed, the model is only generating the input, when the input is long. For example,
`generator = transformers.pipeline('text-generation', model='gpt2')`
`prompt = "really long text that is 1023 tokens ..."`
`output = generator(prompt, mex_length=1024, do_sample=True, temperature=0.9)`
output in this case would be equal to the input prompt.
## To reproduce
[Here is a Collab notebook](https://colab.research.google.com/drive/1JzwSmFGrWY1bU6f-t-mgMug88NsRVllp#scrollTo=OgOhZxQJNseL) with simple examples of the problem. I am looking to generate output from input ~1300 tokens and running into this issue consistently. Is there a way around this?



Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

Ok, I posted my question [here](https://discuss.huggingface.co/t/output-includes-input/6831).

Thank you!
Use a released version of optax rather than installing from Git.
Author: avital
Pull request: yes

(We were using a new API that hadn't been released until a few weeks
ago)

# What does this PR do?

Update the version of Optax we depend on in Flax examples' requirement.py to the latest released version.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? <-- I installed dependencies via requirements.txt and then ran run_flax_glue.py.




Comments:
Import `SPIECE_UNDERLINE` from `file_utils` instead of WET definition
Author: PhilipMay
Pull request: no
Labels: name: WIP

Many places define `SPIECE_UNDERLINE` in the code like this: `SPIECE_UNDERLINE = "▁"`

Instead it should me imported: `from transformers.file_utils import SPIECE_UNDERLINE`.

I can provide a PR...

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I am still planning to provide a PR later.