noamaanMulla-03/transformers-issues huggingface dataset:
type annotations for rembert w/ copies from bert (torch)
Author: jacobdineen
Pull request: yes

Added type annotations to Rembert per the guidelines. Modified Bert where copied from transformers.models.bert.modeling_bert

 #16059

@Rocketknight1 



Comments:
Note make fixup unable to run in my environment w/ the error message:

```
/usr/bin/sh: -c: line 2: unexpected EOF while looking for matching `"'
/usr/bin/sh: -c: line 3: syntax error: unexpected end of file
make: *** [Makefile:10: modified_only_fixup] Error 2
```


_The documentation is not available anymore as the PR was closed or merged._

ran make fix-copies. Many copies stemming from Bert (torch), some of which don't have typing imports already. Added those where applicable.
Add the XTREME-S fine-tuning example
Author: anton-l
Pull request: yes

# What does this PR do?

This adds and example script and benchmark results for fine-tuning speech models on the [XTREME-S](https://huggingface.co/datasets/google/xtreme_s) becnhmark tasks, namely Speech Recognition and Speech Classification.

The results so far:
| Task                  | Dataset   | Result                | Fine-tuned model & logs                                            | Training time | GPUs   |
|-----------------------|-----------|-----------------------|--------------------------------------------------------------------|---------------|--------|
| Speech Recognition    | MLS       | 30.33 WER             | [here](https://huggingface.co/anton-l/xtreme_s_xlsr_300m_mls/)     | 18:47:25      | 8xV100 |
| Speech Classification | Minds-14  | 94.74 F1 / 94.70 Acc. | [here](https://huggingface.co/anton-l/xtreme_s_xlsr_300m_minds14/) | 04:46:40      | 2xA100 |


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Looks very nice! I'm not 100% sure under which folder to put the script, but I tend to `audio-classification` as `run_glue.py` is under `text-classification`. 

This xtreme-s is a new speech benchmark that has both speech recognition, speech translation and speech classification. Think we have the following options:

a) Put it under something like `speech-representation` / `speech-embeddings` because the benchmark is supposed to evaluate exactly this
b) Put it under `audio-classification` similar to `run_glue.py` under text classification
c) ...other ideas? 

@sgugger - any preference? Here a short summary of the benchmark: https://huggingface.co/datasets/patrickvonplaten/xtreme-s (see diagram)
clearer model variable naming: funnel
Author: saglamutku
Pull request: yes

# What does this PR do?
clearer model variable naming: funnel

<img width="1512" alt="Ekran Resmi 2022-03-15 22 59 16" src="https://user-images.githubusercontent.com/58150504/158461148-2dc37be1-e491-49ac-9d0d-a47ee5f2d8b4.png">

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [*] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
 Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@gante 


Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.



Comments:
_The documentation is not available anymore as the PR was closed or merged._
clearer model variable naming: blenderbot
Author: saglamutku
Pull request: yes

# What does this PR do?
clearer model variable naming: blenderbot

<img width="1512" alt="Ekran Resmi 2022-03-16 15 12 18" src="https://user-images.githubusercontent.com/58150504/158587347-5934dc62-b07e-4fa1-b49a-5f3be4b0f250.png">

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [* ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@gante 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix-clip-configs
Author: patil-suraj
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
unpack_input decorator for tf_convnext
Author: johko
Pull request: yes

# What does this PR do?

Add the unpack_input decorator for convnext (see #16051) 

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Rocketknight1

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Sure, no problem @gante . The tests ran without a problem.
clearer model variable naming: blenderbot_small
Author: saglamutku
Pull request: yes

# What does this PR do?
clearer model variable naming: blenderbot_small
<img width="1512" alt="Ekran Resmi 2022-03-16 16 27 40" src="https://user-images.githubusercontent.com/58150504/158600522-808954f1-9519-4a56-9a07-c3649faac819.png">

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ *] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?
## Who can review?
@gante 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
How to use pipeline with Pytorch framework on Windows?
Author: dimka11
Pull request: no

I used framework parameter with "pt" in pipeline constructor, but after run I get: 

2022-02-05 13:48:25.332617: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll

Why doesn't  transformers detect Pytorch installation?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fix loading CLIPVisionConfig and CLIPTextConfig
Author: patil-suraj
Pull request: yes

# What does this PR do?

As described in #16185, there's a bug when loading `CLIPVisionModel` from a `CLIPModel` checkpoint. This is because the 
`CLIPVisionConfig` is not loaded correctly. And all the arguments use default values which correspond to the `openai/clip-vit-base-patch32` model, which results in shape mismatch when loading vision/text model from other CLIP checkpoints.

This PR override the `from_pretrained` methods for `CLIPVisionConfig` and `CLIPTextConfig` to allow loading from `CLIPConfig`.


Fixes #16185

Comments:
_The documentation is not available anymore as the PR was closed or merged._
JAX 0.2.22: Replace all deprecated `jax.ops` operations with jnp's `at`
Author: sanchit-gandhi
Pull request: no

The `jax.ops.index_update` function used for in-place `jax.ndarray` operations was deprecated in [JAX 0.2.22](https://jax.readthedocs.io/en/latest/changelog.html?highlight=0.2.22#jax-0-2-22-oct-12-2021). It is advised that any in-place array modifications be made with [`jax.numpy.ndarray.at`](https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.ndarray.at.html). We should update the following scripts to conform to this new approach:

- [x] https://github.com/huggingface/transformers/blob/baab5e7cdf04c5b2cd209de4e9af6cb2c51a30d2/src/transformers/generation_flax_logits_process.py#L145
- [x] https://github.com/huggingface/transformers/blob/d3ae2bd3cf9fc1c3c9c9279a8bae740d1fd74f34/tests/generation/test_generation_flax_utils.py#L222
- [x] https://github.com/huggingface/transformers/blob/2596f95e8499bf350b18e1fa0492d38b6f8148fa/src/transformers/models/marian/modeling_flax_marian.py#L895
- [x] https://github.com/huggingface/transformers/blob/60b81dfa6faae3aa90c34a7df9304036f513d055/src/transformers/models/wav2vec2/modeling_flax_wav2vec2.py#L960
- [x] https://github.com/huggingface/transformers/blob/d25e25ee2b63ebfcd099deb689a5a7272574a10f/src/transformers/models/xglm/modeling_flax_xglm.py#L134
- [x] https://github.com/huggingface/transformers/blob/7732d0fe7a759c9844215920e9f1c5540eafb1a6/src/transformers/models/big_bird/modeling_flax_big_bird.py#L2127
- [x] https://github.com/huggingface/transformers/blob/2596f95e8499bf350b18e1fa0492d38b6f8148fa/src/transformers/models/blenderbot/modeling_flax_blenderbot.py#L886
- [x] https://github.com/huggingface/transformers/blob/2596f95e8499bf350b18e1fa0492d38b6f8148fa/src/transformers/models/blenderbot_small/modeling_flax_blenderbot_small.py#L898
- [x] https://github.com/huggingface/transformers/blob/2596f95e8499bf350b18e1fa0492d38b6f8148fa/src/transformers/models/bart/modeling_flax_bart.py#L925
- [x] https://github.com/huggingface/transformers/blob/2596f95e8499bf350b18e1fa0492d38b6f8148fa/src/transformers/models/mbart/modeling_flax_mbart.py#L949

Comments:
Adding type hints for Distilbert
Author: johnryan465
Pull request: yes

# What does this PR do?

Added type hints for DistillBert


Fixes # (https://github.com/huggingface/transformers/issues/16059)

There are some slight issues re the type signatures, eg the example below where the code will fail if the head_mask paramerter is actually none, however I didn't remove the default paramerter as that would require a change in parameter order etc.

(https://github.com/johnryan465/transformers/blob/a11858c3dd7e28dec615c2d00112267ed231c657/src/transformers/models/distilbert/modeling_distilbert.py#L318) 

@Rocketknight1 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@johnryan465 I reviewed things and you're right - `Tuple[torch.Tensor, ...]` is correct! Let me know when you're happy with the rest and I'll do another review and hopefully merge.

@Rocketknight1  I have gone through your comments and I hopefully addressed them. I'm happy enough with the code now I think. Have a good St. Patrick's Day weekend ☘️!

It looks perfect now, thank you! :shamrock:
rembert type annotations and dependencies
Author: jacobdineen
Pull request: yes

Added type annotations to Rembert per the guidelines. Modified Bert where copied from transformers.models.bert.modeling_bert

#16059 

@Rocketknight1 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@Rocketknight1 Any idea on why this is failing?
Only made modifications to Bert and Rembert, with copies to the other seven models based on make fix-copies.
Model template seems to fail due to a bad import of Optional, from what I can tell.

@jacobdineen Thanks for this! I would guess that `make fix-copies` has copied code to the other models, but one or more of the files it copied to hasn't imported `Optional`, and so the code crashes there. If you add the missing imports to those files it should work!

Also, some of the return types here seem to be missing bits - they're just `Union[Tuple]` but usually there's another return type as well in the union.
Swin support for any input size
Author: FrancescoSaverioZuppichini
Pull request: yes

# What does this PR do?
This PR adds padding to Swin allowing to support any (if divisible by `32`) input size.

Example:

```python
from transformers import SwinConfig, SwinModel
import torch

model = SwinModel(SwinConfig(image_size=384))
x = torch.randn((1, 3, 1024, 640))
out = model(x)
```

Moreover, it adds a new field to the outputs, `hidden_states_spatial_dimensions`, containing the spatial dimension of all the stages' inputs

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15986). All of your documentation changes will be reflected on that endpoint.

Swin now returns a list of reshaped `hidden_states` (B, C, H, W). However, due to the `view` operation, the viewed tensor won't have `.grad` in it, so `test_retain_grad_hidden_states_attentions` fails. Not sure how to proceed

```python
from transformers import SwinConfig, SwinModel
import torch

model = SwinModel(SwinConfig(image_size=384))
x = torch.randn((1, 3, 1024, 640))
out = model(x, output_hidden_states=True)
[print(e.shape) for e in out.hidden_states]

torch.Size([1, 96, 256, 160])
torch.Size([1, 192, 128, 80])
torch.Size([1, 384, 64, 40])
torch.Size([1, 768, 32, 20])
torch.Size([1, 768, 32, 20])
```

Maybe I am missing something, kindly pinging @sgugger 

Following @NielsRogge suggestion, we now return the reshaped hidden sized inside `reshape_hidden_sizes` in all four (`Encoder/Model/MaskedImage/ImageClassifier`) outputs

Thanks to all the reviewers. I've resolved all the conversation and renamed some layers to match our convention of `Stage` and `Layer`
Add/type annotations/model vision
Author: johnnv1
Pull request: yes

I added type annotations for  Beit, ViT and Deit models at PyTorch as described in #16059 

@Rocketknight1 

fix the PR #16092 


Also, the issue with `make fix-copies` for ViT_mae, maybe I should open an issue?

For this PR, I just remove the type annotation of config parameter that cause the problem

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This looks really good! Good job handling the more intricate `Tuple` annotations as well as forward references well - this is a lot bigger and more challenging than most of the other type hint PRs, but it looks like you did it well. 

I have one final small issue - types should only be `Optional` when `None` is an allowable option. It looked like there were some cases when `Optional[bool]` was used for boolean arguments to internal methods which have to be either `True` or `False` - these should just be `bool`. Can you double-check that? Of course, when `None` is allowable or the default option, then `Optional[bool]` is correct. I'm not 100% certain on this one (maybe `None` really is allowable in all of those cases!), so if you think all your annotations were correct, let me know.

Hi, i have kept `Optional[bool]` just to keep everything equals. But I will take a look, if it can be changed to "bool", I will update it and let you know.

@Rocketknight1 i have changed to just `bool`, both cases work, but just `bool` seems more correct and clean for me.

When has `None` as default, the code will be `a if x is not None else b`, and with `False`/`True` will be just `a if x else b`




Sounds good, reviewing now!

@Rocketknight1 about the `make fix-copies` isn't working for `ViT_mae` (config parameters), this is because don't have `with ViT->VitMAE` at the end of the “Copied from …” comment;

But i don't know if this annotation is really missing, or it was supposed to be this way
Fix saving FlaubertTokenizer configs
Author: vmaryasin
Pull request: yes

All specific tokenizer config properties must be passed to its base
class (XLMTokenizer) in order to be saved. This was not the case for
do_lowercase config. Thus it was not saved by save_pretrained() method
and saving and reloading the tokenizer changed its behaviour.

This commit fixes it.

# What does this PR do?

Fixes # 14489

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@n1t0, @LysandreJik

## Comment:

As I have said in the issue, a similar problem may happen to other models. For me, it is not obvious, that one should pass a model-specific config to the general base class constructor for the config to be saved later.

I feel like some comment should be left somewhere for future developers, but I cannot come up with a good place for it. On the other hand, maybe it is obvious from all the base class' descriptions, and I'm just not experienced enough.


Comments:
Fix reproducibility in Training for PyTorch 1.11
Author: sgugger
Pull request: yes

# What does this PR do?

It used to be that PyTorch called just one random operation in the `RandomSampler` code, so to ensure reproducibility, we only had to get the first batch of a dataloader to have its generator RNG be in the same state as if we had done the whole epoch.

Well not anymore... Introducing PyTorch 1.11, where the `RandomSampler` now does two random operations, one being at the very end. So we need to iterate through the whole sampler to guarantee reproducibility. This PR deals with that.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix Type Hint of Nan/Inf Logging Filter Arg
Author: Sophylax
Pull request: yes

# What does this PR do?

Nan/Inf Logging Filter Training Argument is type hinted as `str` even though it looks like it is a `bool`. Documentation marks it as a `bool`, the default value is a `bool`, and all the usages that I could see are for a `bool`.

We are doing some work involving dependency injection and rely on type hints. This mismatch is throwing some of our systems off, which is how I found out about it.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16227). All of your documentation changes will be reflected on that endpoint.