noamaanMulla-03/transformers-issues huggingface dataset:
[Flax] remove jax.ops.index
Author: patil-suraj
Pull request: yes

# What does this PR do?

Remove `jax.ops.index` as this method is removed in latest jax release `v0.3.2`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

There seem to be some failing tests still :-) 

The failing tests are not related to this PR.

Hi, I am not very into the detail here, but curious about that if this implies from this version, `transformers` will only work with the new version of `jax`? 

> Hi, I am not very into the detail here, but curious about that if this implies from this version, `transformers` will only work with the new version of `jax`?

No, the `.at` syntax is also supported in older jax versions.

Feel free to merge if you want @patil-suraj
Add compatibility for Postponed Evaluation of Annotations (PEP 563)
Author: function2-llx
Pull request: no
Labels: name: Good First Issue

Hello,

The code says that it will add compatibility for Postponed Evaluation of Annotations ([PEP 563](https://www.python.org/dev/peps/pep-0563/)) when Python 3.9 is released (which already happened on 2020.10.5). Is there any plan to complete this?

https://github.com/huggingface/transformers/blob/2c2a31ffbcfe03339b1721348781aac4fc05bc5e/src/transformers/hf_argparser.py#L85-L90

Comments:
Hey! We don't have to do the bandwidth to do it right now, but we'd welcome contributions! Let me tag this as a first good issue, and let me know if you're interested in taking a stab at it!

I'm glad to help with that, maybe it'll take some time. I never contribute here, I'll try to follow the CONTRIBUTING.md, post progress here and submit PR later, any discussion telling me if I'm doing right would be great. 

According to [discussion here](https://bugs.python.org/issue39442) and solution provided by [Pydantic](https://pydantic-docs.helpmanual.io/usage/postponed_annotations/), we may just call [typing.get_type_hints](https://docs.python.org/3.9/library/typing.html#typing.get_type_hints) on some dataclass to get type of a field instead of relying on `field.type`.

Also, `typing` module is still under development, thus changes notably across different versions of Python. Since Python 3.6 reached its end-of-life last year (https://endoflife.date/python), dropping support for Python 3.6 would be reasonable and make this implementation much easier as well. There seems to be no plan on this (see also #15720).

I understand; as mentioned in the thread you linked, we're unlikely to drop support for Python 3.6 just yet, but we'll definitely keep in mind that this particular issue would be solved in a simpler manner were we to drop support for it.

I'm glad to help with that, maybe it'll take some time. Like [sparkanime.com](https://sparkanime.com/)

@rahimt420 Hello, thanks for the offer! However, I've already submitted #15795 to resolve this issue. It will be great if you could leave any suggestions there!
Support PEP 563 for HfArgumentParser
Author: function2-llx
Pull request: yes

# What does this PR do?

Fixes #15739, support for [PEP 563](https://www.python.org/dev/peps/pep-0563/) is added, making it possible to write `from __future__ import annotations` in the file of dataclasses for `HfArgumentParser`. But this will not be available for Python with a version lower than 3.7, thus adding `from __future__ import annotations` in the test is currently infeasible because `__future__` imports should always occur at the beginning of the file. Instead, I write a test case with string literal annotations.

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes?
- [x] Did you write any new necessary tests?


## Who can review?
@LysandreJik 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15795). All of your documentation changes will be reflected on that endpoint.

@LysandreJik Hello, would be great if you could help or ask someone to give any feedback on this, thanks!
Seems #15951 updated the parser as well (just resolved the conflict), I've done the same work, and added a comment at the corresponding changing line.

Looking forward to any feedback.

Thanks for the ping! @sgugger could you give this a quick look? It's similar to the work merged in https://github.com/huggingface/transformers/pull/15951

It looks like you did a rebase on your PR that GitHub did not appreciate (the diff shows 292 files). Could you close this PR and open a fresh one?

@sgugger Github encountered some internal error a little while ago, it seems to be fine now.

_The documentation is not available anymore as the PR was closed or merged._

@sgugger Thanks so much for the feedback! I've adopted that. In addition, I've replaced `... != type(None)` with `isinstance` to pass the quality check.

Cool, thanks again for your contribution!
add unpack_inputs decorator for marian
Author: johko
Pull request: yes

# What does this PR do?

Add the unpack_inputs decorator to marian model and remove unneeded code

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@gante 


## Tests
The tests via `RUN_SLOW=1 py.test -vv tests/marian/test_modeling_tf_marian.py` ran without problems on the current state



Comments:
_The documentation is not available anymore as the PR was closed or merged._

CI is not fully green, but I will merge -- the issue is separated from this PR, and is being tracked internally
[FlaxSpeechEncoderDecoderModel] Skip from_encoder_decoder_pretrained
Author: patil-suraj
Pull request: yes

# What does this PR do?

Skip `test_encoder_decoder_model_from_encoder_decoder_pretrained` until this issue is: https://github.com/google/jax/issues/9941

cc @sanchit-gandhi @patrickvonplaten 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16236). All of your documentation changes will be reflected on that endpoint.
Question about the why num_beams is multiplied by 2 in Beamsearch
Author: cloudygoose
Pull request: no

Hi, I have a simple question, in the code for beam_search, here:

https://github.com/huggingface/transformers/blob/ef9c3ca348cb11c4ba951f8bdab068ecd8b847ce/src/transformers/generation_utils.py#L2137

Why the num_beams is multiplied by 2 ? Thanks!

Comments:
Ping @patrickvonplaten 

Good question @cloudygoose!

The reason is simply due to the fact that a beam might finish in an EOS in which case the beam search would not be able to continue anymore. 

E.g. think about the following scenario where we have num_beams=3 


1. step

1. ["hello"]
2. ["hey"]
3. ["hi"]

The best three beams from this are;

1. ["hello", "my"]
2. ["hello", "are"]
3. ["hi", "there"]

Now in the next step the following three beams are the best three beams

1. ["hello", "my", "dear"]
2. ["hello", "are", "there"]
3. ["hi", "there", "EOS"]

No we have a problem because we can't continue with EOS (end of sentence). So what we do now is to put `["hi", "there", "EOS"]` in the finished beams and take the 4th best beam and continue from there:

1. ["hello", "my", "dear"]
2. ["hello", "are", "there"]
4. ["hello", "are", "any"] -> becomes the 3rd best beam

So in the worst case we have all three best beams ending in EOS. In this case we might have to take the 4th, 5th, 6th beams to continue beam search which is why we do `2 * num_beams`


thanks for the reply, could you explain more about where the beam with EOS is saved? I did not find the place where "finished beams" is saved in the code...
[Deepspeed] non-HF Trainer doc update
Author: stas00
Pull request: yes

Adding some clarifications that `HfDeepSpeedConfig` is not needed unless Deepspeed ZeRO-3 is used w/o HF Trainer:
based on this communication https://github.com/huggingface/transformers/issues/15399#issuecomment-1070364836

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
A question about the cross-attention layer shape in encoder-decoder model
Author: HongkuanZhang
Pull request: no

I am trying to intialize a bert2bert model with bert-base-uncased as encoder and bert-large-uncased as decoder with the following codes:
```
from transformers import EncoderDecoderModel, BertTokenizer
import torch

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
model = EncoderDecoderModel.from_encoder_decoder_pretrained(
    "bert-base-uncased", "bert-large-uncased") 
```
When I print the parameter shape of random intialized cross-attention layer, the results are:
```
(attention): BertAttention(
  (self): BertSelfAttention(
    (query): Linear(in_features=1024, out_features=1024, bias=True)
    (key): Linear(in_features=1024, out_features=1024, bias=True)
    (value): Linear(in_features=1024, out_features=1024, bias=True)
    (dropout): Dropout(p=0.1, inplace=False)
  )
  (output): BertSelfOutput(
    (dense): Linear(in_features=1024, out_features=1024, bias=True)
    (LayerNorm): LayerNorm((1024,), eps=1e-12, elementwise_affine=True)
    (dropout): Dropout(p=0.1, inplace=False)
  )
)
(crossattention): BertAttention(
  (self): BertSelfAttention(
    (query): Linear(in_features=1024, out_features=1024, bias=True)
    (key): Linear(in_features=1024, out_features=1024, bias=True)
    (value): Linear(in_features=1024, out_features=1024, bias=True)
    (dropout): Dropout(p=0.1, inplace=False)
  )
  (output): BertSelfOutput(
    (dense): Linear(in_features=1024, out_features=1024, bias=True)
    (LayerNorm): LayerNorm((1024,), eps=1e-12, elementwise_affine=True)
    (dropout): Dropout(p=0.1, inplace=False)
  )
)
```
For the cross attention part, I think the shape for query and key should be (768,1024) instead of (1024,1024), since it converts the bert-base output dim (768) to bert-large dim (1024). Who can tell me where I made a mistake and give me a explanation? Many thanks!

Comments:
How to pass args of generate() method via batch_decode
Author: rafikg
Pull request: no

I want to specify the generate params in the line below:

```
generate(tokenized_input.input_ids,
                                     max_length=64,
                                     no_repeat_ngram_size=2,
                                     early_stopping=True,
                                     num_beams=4,
                                     num_return_sequences=3)
```
https://github.com/huggingface/transformers/blob/93d3fd86459eb27bc584da29a3d542817a395bca/examples/pytorch/summarization/run_summarization.py#L662

I specified the args as following. I did not get any error, but it is still getting one sequence instead of 3 as expected.
```
batch_decode(
          predict_results.predictions,
          skip_special_tokens=True,
          early_stopping=True,
          clean_up_tokenization_spaces=False,
          no_repeat_ngram_size=2,
          num_return_sequences=3
)
```



Comments:
Currently the seq2seq trainer doesn't support `num_return_sequence` argument for generation. 

You might need to tweak it here:
https://github.com/huggingface/transformers/blob/v4.17.0/src/transformers/trainer_seq2seq.py#L158-L162

Thanks @qqaatw. Because batch_decode cotains **kwargs, I thought maybe I can pass them through it but it seems that is not implemented. So, in this case `Generate()` will use default values and all metrics will calculated based on that. `no_repeat_ngram_size` is really important to avoid generating duplicated tokens. 

`batch_decode` isn't responsible for generation by design, it only converts ids to tokens and does some normalizations. So passing generation-related arguments for it won't work.
Add unpack_inputs decorator for ctrl
Author: johko
Pull request: yes

# What does this PR do?

Add the `unpack_inputs` decorator for `ctrl`. It also replaces the `past` parameters in models in and output by `past_key_values` as there was an irregularity in the naming that caused an error with the new input processing.

<!-- Remove if not applicable -->

Fixes # (issue)
#16051 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@gante


## Tests
I ran `RUN_SLOW=1 py.test -vv tests/ctrl/test_modeling_tf_ctrl.py` but it only came to around 69% and failed the pre-trained model test, because it is too big for my local machine to test it.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Going to run the slow tests locally, to confirm they pass

Can confirm that they pass, merging 👍
[Constrained Beam Search] Adding Notebook Example & Minor Typo Fix
Author: cwkeam
Pull request: yes

# [CBS] Adding Notebook Example & Minor Typo Fix

Very lightweight PR for:
- a fix to a somewhat embarrassing copy-and-paste error I made in my initial PR
- Thought it'd be nice if the notebook on `huggingface/blog` could be added to the list of Pytorch notebooks as others have done (both the Colab and SageMaker link imports from the `huggingface/blog` `master` branch with no issue.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten 




Comments:
_The documentation is not available anymore as the PR was closed or merged._

The current test fail is unrelated to the PR (cbs is separated from `sample_generate`):

```
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/pegasus/test_modeling_pegasus.py::PegasusStandaloneDecoderModelTest::test_sample_generate
=== 1 failed, 5253 passed, 4576 skipped, 4839 warnings in 498.21s (0:08:18) ====
```
ViT-mae missing parameter at comments of `copied  from`
Author: johnnv1
Pull request: no

### Who can help
@NielsRogge

## Information
As describe in #16151

When i was type annotating the ViT model, i get some problem when running `make fix-copies` for vit-mae. When i annotated the cofig parameter at ViT, and run  `make fix-copies` the script fix at deit model (update the type annotation to Deit config), but at vit-mae he just copied the type annotation of ViT config.

This seems to be because don't have with ViT->VitMAE at the end of the “Copied from …” comment in vit_mae model;

But I don't know if this annotation is really missing, or it was supposed to be this way


## To reproduce

1. Add type annotation at ViTSelfAttention src/transformers/models/vit/modeling_vit.py
2. run `git diff`
```diff
--- a/src/transformers/models/vit/modeling_vit.py
+++ b/src/transformers/models/vit/modeling_vit.py
@@ -192,7 +192,7 @@ class PatchEmbeddings(nn.Module):
 
 
 class ViTSelfAttention(nn.Module):
-    def __init__(self, config) -> None:
+    def __init__(self, config: ViTConfig) -> None:
         super().__init__()

```

3. Run `make fix-copies`
4. Run `git diff src/transformers/models/vit_mae/`
```diff
@@ -318,7 +318,7 @@ class PatchEmbeddings(nn.Module):
 
 # Copied from transformers.models.vit.modeling_vit.ViTSelfAttention
 class ViTMAESelfAttention(nn.Module):
-    def __init__(self, config) -> None:
+    def __init__(self, config: ViTConfig) -> None:
```

## Expected behavior
I was expecting 

```diff
@@ -318,7 +318,7 @@ class PatchEmbeddings(nn.Module):
 
 # Copied from transformers.models.vit.modeling_vit.ViTSelfAttention
 class ViTMAESelfAttention(nn.Module):
-    def __init__(self, config) -> None:
+    def __init__(self, config: ViTMAEConfig) -> None:
```

This can be fixed with 

```diff
@@ -316,9 +316,9 @@ class PatchEmbeddings(nn.Module):
         return x
 

-# Copied from transformers.models.vit.modeling_vit.ViTSelfAttention
+# Copied from transformers.models.vit.modeling_vit.ViTSelfAttention ViT->VitMAE
```

But I don't know if this annotation is really missing, or it was supposed to be this way


Comments:
Hey,

Yeah you can add the `with ViT->ViTMAE` if you need to.
[WIP] add `has_attentions` as done in PyTorch side
Author: ydshieh
Pull request: yes

# What does this PR do?

My original intention is to clean up `TFModelTesterMixin.test_pt_tf_model_equivalence`. This is done in this first commit.
Would like to hear from @sgugger to see if it is better to apply the same change found in #15909 here.

 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

So I will merge as it is, and work in a follow up PR to use `has_attentions` to other (TF) test methods like `test_model_outputs_equivalence` etc.
Fine-tune blenderbot 2.0 to retrieve from my own documents instead internet
Author: BayanIssa
Pull request: no

i have database contain of many documents , and try to build chatbot which reply for users from those documents in my database .
can i use this [way(ONLY GOLD DOCS)](https://github.com/facebookresearch/ParlAI/tree/main/projects/blenderbot2/agents)?

Comments:
Hi @BayanIssa ! It would be nice if you could use the [forum](https://discuss.huggingface.co/) for such general questions. We use issues to report bugs and for feature requests.

Also blenderbot 2.0 will soon be included in Transformers :) #12807

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to implement generate function for seperate encoder decoder T5 model?
Author: VikasOjha666
Pull request: no

I was working on optimizing the T5 model. The version of the transformer I am using is  4.8. For optimization, I separated the model into encoder and decoder with LM(Language modeling) head. Earlier for generation, I just passed input_ids, attention_mask,max_length, and num_beams. But now since the model is separated into encoder and decoder I can't use generate function. But I am able to generate logits using the below code:

encoder_last_hidden_state = t5_trt_encoder(input_ids=input_ids)
outputs = t5_trt_decoder(input_ids, encoder_last_hidden_state)

But how can I use these logits to generate sequences?. I mean I am unclear how can I use the encoder and decoder to write a class which supports generate function?

----

note: edited by @stas00 to remove a dozen of tags 

Comments:
Hi! Thanks for the issue. But please don't tag every maintainer on an issue. In the issue template you can find who to tag depending on the model or feature. 
Also for such general questions please use the [forum ](https://discuss.huggingface.co/). We use issues for bug reports and feature requests. Thank you !


Hi @patil-suraj. Thanks for letting me know about the community. I was trying to optimize the T5 based QA-QG model from your repository using https://github.com/NVIDIA/TensorRT/tree/main/demo/HuggingFace repo. Here they have separated the model into encoder and decoder with LM(Language model) for optimization. But when I am doing inference in the following way:
max_length = 64

decoder_input_ids = torch.full(
    (1, 1), tokenizer.convert_tokens_to_ids(tokenizer.pad_token), dtype=torch.int32
)
encoder_last_hidden_state = t5_trt_encoder(input_ids=input_ids)

outputs = t5_trt_decoder.greedy_search(
            input_ids=decoder_input_ids,
            encoder_hidden_states=encoder_last_hidden_state,
            stopping_criteria = StoppingCriteriaList([MaxLengthCriteria(max_length)])
        )
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

I am getting an empty result. I tried with an unoptimized torch model too but the results are the same. Hence I thought maybe that can be resolved if somehow I am able to use logits from these in generate function. Hence I asked the question.
The same prediction method works for translation case.

The tokenizer is the same as Valhalla/t5-small-qa-qg-hl

Please help.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
train_file and validation_file in jsonl format with examples/pytorch/language-modeling/run_clm.py  
Author: andrescodas
Pull request: no

I'm using `./examples/pytorch/language-modeling/run_clm.py` providing the arguments _--train_file_ and _--validation_file_ with the **json** extension, however the underlying file is really a **jsonl** file.  This is confusing as I need to rename my jsonl files.

Since run_clm.py is compatible with jsonl files, is it possible to include the *jsonl* extension [here](https://github.com/huggingface/transformers/blob/eed3186b79d7ef9187113849f9ee7882bdd359fd/examples/pytorch/language-modeling/run_clm.py#L186) and [here](https://github.com/huggingface/transformers/blob/eed3186b79d7ef9187113849f9ee7882bdd359fd/examples/pytorch/language-modeling/run_clm.py#L189)?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.