noamaanMulla-03/transformers-issues huggingface dataset:
Issue in wav2vec2ForPretraining
Author: annihi1ation
Pull request: no
Labels: name: bug

### System Info

```shell
In the example mentioned in the doc, when I print the loss, it is "None"
```


### Who can help?

@patrickvonplaten

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

import torch
from transformers import AutoFeatureExtractor, Wav2Vec2ForPreTraining
from transformers.models.wav2vec2.modeling_wav2vec2 import _compute_mask_indices
from datasets import load_dataset

feature_extractor = AutoFeatureExtractor.from_pretrained("facebook/wav2vec2-base")
model = Wav2Vec2ForPreTraining.from_pretrained("facebook/wav2vec2-base")

ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
input_values = feature_extractor(ds[0]["audio"]["array"], return_tensors="pt").input_values  # Batch size 1

# compute masked indices
batch_size, raw_sequence_length = input_values.shape
sequence_length = model._get_feat_extract_output_lengths(raw_sequence_length)
mask_time_indices = _compute_mask_indices((batch_size, sequence_length), mask_prob=0.2, mask_length=2)
mask_time_indices = torch.tensor(mask_time_indices, device=input_values.device, dtype=torch.long)

with torch.no_grad():
    outputs = model(input_values, mask_time_indices=mask_time_indices)

# compute cosine similarity between predicted (=projected_states) and target (=projected_quantized_states)
cosine_sim = torch.cosine_similarity(outputs.projected_states, outputs.projected_quantized_states, dim=-1)

# show that cosine similarity is much higher than random
cosine_sim[mask_time_indices.to(torch.bool)].mean() > 0.5

# for contrastive loss training model should be put into train mode
model = model.train()
loss = model(input_values, mask_time_indices=mask_time_indices).loss

### Expected behavior

```shell
Modify the example doc
```


Comments:
Hey @annihi1ation,

What `transfromres` version are you using?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix ROUGE add example check and update README
Author: sgugger
Pull request: yes

# What does this PR do?

This PR contains a few fixes for the examples all linked to #18381 
- it adds the version check of Transformers to the `no_trainer` examples, so that the user is not surprised when the example script that uses main fails if they use a different version
- it expands the table of examples for each version to get to the current version
- it fixes the ROUGE metric return type since that metric was broken in https://github.com/huggingface/evaluate/pull/158

Fixes #18381

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Failures are flaky, so merging.
Split model list on modality
Author: stevhliu
Pull request: yes

This PR splits the model API list on modality because it was becoming difficult for users to discover what vision or audio models are supported because they just disappear in the long list. I set these sections to not expand by default since users are more likely interested in going straight to a modality and then expanding the list of supported models instead of scrolling past all the text models, for example.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This requires more work than just splitting the navigation bar (as seen with the failing quality test). There are scripts that format this model ToC/check all models are documented which then need to be adapted.

Likewise splitting the big tables of models in the main README will require to rewrite a lot of the scripts that copy it to other READMEs and the index.

I have made the necessary changes to the quality script that is failing but:
- I don't have permission to push on this branch, so can't push them here
- Your fork does not appear in the GitHub website for some reason, so I can't make a pull request either

You can find the changes in in [this branch](https://github.com/huggingface/transformers/tree/split-model), just cherry-pick the commit.

Giving authorization to the maintainers to push on your branch when opening a pull request would make the whole process way smoother. Or just use the main fork (huggingface/transformers) instead of your personal fork :-)

Edit: Managed to open a PR by hacking some urls in GitHub (thank you for showing me @LysandreJik !). It's [here](https://github.com/stevhliu/transformers/pull/1). The changes in the toc are just rewrites from the Python yaml library.
Layoutlmv2 tesseractconfig
Author: kelvinAI
Pull request: yes

# What does this PR do?

Giving user option to set config parameter used by Tesseract when performing feature extraction. Eg. to change psm levels while performing transcription by passing in '--psm 10' to config parameter while invoking image_to_data

It is shown that changing the psm values greatly influences the end result of LayoutLMV2/XLM/V3, and the specific psm value is different depending on the document formatting. Refer : [PSM](https://github.com/tesseract-ocr/tesseract/issues/434)

```python
pytesseract.image_to_data(image, lang=lang, output_type="dict", config="--psm 10") 
``` 

Users can now set the tesseract config parameter during Processor initialization, like so:
```python
# LayoutLMV2
processor = LayoutLMv2Processor.from_pretrained("microsoft/layoutlmv2-base-uncased", ocr_lang="eng", tesseract_config="--psm 5")

# LayoutLMV3
processor = AutoProcessor.from_pretrained("microsoft/layoutlmv3-base", ocr_lang="eng", tesseract_config="--psm 5")
```


## Before submitting
- [❌] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [✔️] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [❌] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [✔️] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [❌] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@NielsRogge kindly review. Thanks!

> LGTM! Could you add it to LayoutLMv3's feature extractor as well?

Sounds good! I'll ping you when it's done. Do I need to do anything to merge this to huggingface:main? You'll trigger the merge is that right?

>You'll trigger the merge is that right?

Yes, indeed.

Hi @kelvinAI, could you add it to LayoutLMv3 as well in this PR?

Thanks!

> Hi @kelvinAI, could you add it to LayoutLMv3 as well in this PR?
> 
> Thanks!

Done! @NielsRogge 

@NielsRogge pls review.
Thanks!

Hi @kelvinAI, could you apply the suggestions such that I can merge your PR?

Thanks! 

@NielsRogge done! :)
[WIP]-Add Fast Pitch 1.1
Author: ArEnSc
Pull request: yes

# What does this PR do?

Adds FastPitch 1.1# (issue)[https://github.com/huggingface/transformers/issues/16349]

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?



Comments:
Thanks for the PR @ArEnSc!

Let us know if you need assistance at any point!

> Thanks for the PR @ArEnSc!
> 
> Let us know if you need assistance at any point!

Will do! have been lagging on this due to family and my day job =)

update: going to continue to work on this soon

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Still working on this, just doing some reading working on burn out.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

closing for now while I deal with covid =(
Fix OPT doc tests
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes the doctest for OPT by adding slight modifications. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @ArthurZucker Do you know why the expected value has extra ``` ? `OPTForCausalLM` has the same expected value, but it doesn't have test failure. Seems strange to me 😕 

```
 1003         >>> # Generate
 1004         >>> generate_ids = model.generate(inputs.input_ids, max_length=30)
 1005         >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
 Expected:
     "Hey, are you consciours? Can you talk to me?\nI'm not consciours, but I can talk to you."
     ```
 Got:
     "Hey, are you consciours? Can you talk to me?\nI'm not consciours, but I can talk to you."
```

Actually it seems that the automatic doc préparation that adds a line does not affect the TFOPTForCausalLM forward doc. Thus I removed the manually created example and defeated to using **add_code_sample** which worked in that case. 
Now why does the add extra line does not work is another mystery and I could not setup a debugging instance to check that  

Thank you. I will check this on Monday.

Regarding the mystery , it is due to this line

> ``decoder_input_ids```

https://github.com/huggingface/transformers/blob/b2e4b091f08f1aaf21855d588c6c8d284baba9eb/src/transformers/models/opt/modeling_tf_opt.py#L601

~~We can fix this in this PR too :-) It should be simply~~

> \`decoder_input_ids\`

I forgot to change the name in the `add_docstring` will fix that
Load sharded pt model to flax (from the hub )
Author: ArthurZucker
Pull request: yes
Labels: name: Core: Modeling

name: Flax

# What does this PR do?

Implements the automatic conversion from `pt` to `flax` if the model is sharded but does not already have flax weights. This could be useful for automatic conversion 

## Who can review?

A few tests still need to be implemented, for now works well with `hf-internal-testing/tiny-random-bert-sharded`



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Follows  #17537
Fix uninitialized parameter in conformer relative attention.
Author: PiotrDabkowski
Pull request: yes

`torch.Tensor` creates an unitialized tensor (as via `torch.empty`), this leads to undeterministic behavior, poor initialization, and nans if you have unlucky init. The paper does not specify the initialization for bias terms, so I guess zero seems like a good choice - no bias initially. `torch.Tensor` is usually populated with zeros, so this fix will be close to the intended behavior:

```
>>> torch.Tensor(100, 100).sum()
tensor(0.)
>>> torch.Tensor(100, 100).sum()
tensor(nan)
>>> torch.Tensor(100, 100).sum()
tensor(0.)
```


## Who can review?

@patrickvonplaten



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @PiotrDabkowski, won't these be overridden by the weights when loaded with `from_pretrained`?

Edit: ah, understood - you'd like to have a better initialization at zeros instead of random.

Will let @patrickvonplaten or @sanchit-gandhi comment, we try to stay as close as possible to the original implementation so this will likely be the deciding factor.

Hey @PiotrDabkowski! That's a great catch! It looks like the initialisation of the (u, v) bias terms in the relative positional encoding transformation were copied one-to-one from the original modelling code in the fairseq repo:
https://github.com/facebookresearch/fairseq/blob/4fe8583396191c22011350248119db98ec1b5cb8/fairseq/modules/espnet_multihead_attention.py#L127-L128

Indeed, using `torch.Tensor` leads to undesirable behaviour for the initialisation of tensors, and `torch.zeros` is an apt replacement. I've opened an issue on fairseq: https://github.com/facebookresearch/fairseq/issues/4622 and PR with the corresponding fix: https://github.com/facebookresearch/fairseq/pull/4623

If it's good with you, we'll wait for the original authors to respond and then hopefully update the two in tandem!
Rewrite push_to_hub to use upload_files
Author: sgugger
Pull request: yes

# What does this PR do?

As asked in #18299 this PR rewrites the `PushToHubMixin` completely to use `upload_file` behind the scenes. This is breaking since there is no more git repository involved, so while the user might have been left with a proper repository in the past, this is not the case anymore. The trade-off is that there is no need to clone anything now, so the upload should be faster.

Another breaking change is to default `use_temp_files` to None now, which then defaults to `True` if there is a local folder of the same name, `False` otherwise. This felt way more natural with this change, but we can discuss and revisit if needed. For the rest, normally backward compatibility is ensured with proper deprecation warnings when needed.

Using `push_to_hub=True` in `save_pretrained` is adapted to this rewrite but with 0 breaking change normally.

You can see in the tests how this simplifies a lot of things at the end of the day, so I'm quite happy with the final API.

Oh and one last change I made is to move the code specific to TensorFlow models in the TF modeling utils file.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Added it documentation for fast_tokenizers #17459
Author: andreafailla
Pull request: yes

# What does this PR do?


Addresses #17459 with italian translation of fast_tokenizers.mdx


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17622). All of your documentation changes will be reflected on that endpoint.

Hi @andreafailla, I'm just taking a look at your PR. Can I ask you to change a little of thing? In the toctree please remove the quotation marks here "Usare i tokenizzatori di 🤗 Tokenizers". Everything else looks perfect to me! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
MaskFormer documentation - `is_thing_map`
Author: morrisalp
Pull request: no

The MaskFormer documentation states: "Both tasks can be solved using [MaskFormerForInstanceSegmentation](https://huggingface.co/docs/transformers/v4.20.1/en/model_doc/maskformer#transformers.MaskFormerForInstanceSegmentation) output, the latter needs an additional is_thing_map to know which instances must be merged together.."

However, `is_thing_map` does not appear in the source code, and it looks like this was replaced with `label_ids_to_fuse`.

Comments:
cc @NielsRogge 

@LysandreJik @NielsRogge I updated the MaskFormer docs to reflect the current code and added a [PR](https://github.com/huggingface/transformers/pull/18423)



I merged the PR, closing this issue!
Documentation 404 error
Author: gwc4github
Pull request: no

For this page: https://huggingface.co/transformers/examples.html
several (all?) of the links in the "The Big Table of Tasks" section are getting a 404 error.
Ex: https://github.com/huggingface/transformers/tree/master/examples/token-classification


Comments:
thanks alot 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

[https://huggingface.co/transformers/examples.html](https://huggingface.co/transformers/examples.html) now points towards [https://huggingface.co/docs/transformers/main/en/examples](https://huggingface.co/docs/transformers/main/en/examples), which is a 404.
fixing error when using sharded ddp
Author: pacman100
Pull request: yes

# What does this PR do?

Fixes #18410 
1. conditional logic fixed

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update pinned hhub version
Author: osanseviero
Pull request: yes

# What does this PR do?

This PR updates the `huggingface_hub` pinned version. https://github.com/huggingface/transformers/pull/18366 uses new versions from the library which are not supported in previous versions, so we need to upgrade here.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I still need to update https://github.com/huggingface/transformers/blob/main/src/transformers/dependency_versions_table.py, will do once back in laptop and let you know

Thank you :hugs:
Fix failing tests for XLA generation in TF
Author: dsuess
Pull request: yes

# What does this PR do?

This PR fixes the failing tests for text generation in XLA mentioned in #17935. The issue was that the failing models had additional settings limiting the sequence length, which weren’t updated when creating the config-objects for the tests.

Fixes # 17935


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@gante @LysandreJik 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @dsuess! Thank you so much for looking into this -- I can confirm that the tests are passing after these changes. 

This implies two things:
1. The models are indeed XLA-ready, so I will tick their boxes in the original issue #17935 
2. The tests were failing due to poor model parametrization at test time -- I will open a follow-up PR to fix them

cc @sgugger (with these changes the XLA tests will run instead of being skipped, but the root cause still needs to be addressed to avoid this ad hoc logic, see my comment above)
Add zero-shot obj detection notebook to docs
Author: alaradirik
Pull request: yes

Adds OWL-ViT demo notebook links to the official notebooks docs.

I'm currently working on adding TF support for this model and we'll be promoting it soon.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Thanks for adding! Although I think we need to split up that long list of notebooks by modality/task.

I agree! I will add another PR to organize the notebooks page.
[BLOOM] Deprecate `position_ids`
Author: thomasw21
Pull request: yes

# What does this PR do?

Deprecate `position_ids` as they never made sense for BLOOM.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

With @NouamaneTazi we discovered that this actually breaks scripting for torch 1.11 (and seems to work fine in torch 1.12) . In particular, the `**` operator breaks it,  should we revert back to set `position_ids=None` in the signature of the function? cc @sgugger 

> your solution works against everything :-)

I guess it didn't ...

Does this mean `torchscript` can't work with `**kwargs`?

No it didn't work with kwargs, that's why it doesn't work for some specific models like XLNet. Since this seems fixed in PyTorch 1.12 and torchscripting is not the mainstream use, I would leave it as is (so users need to use PyTorch >= 1.12 for scripting).