noamaanMulla-03/transformers-issues huggingface dataset:
Add support to MPNetForCausalLM
Author: jwengr
Pull request: no

### Feature request

Add MPNetForCausalLM for various usage. 


### Motivation

The pre-trained mpnet model provides the best performance in [sentence embedding](https://www.sbert.net/docs/pretrained_models.html)
And it can aim for additional performance improvements through using [TSDAE (Tranformer-based Denoising AutoEncoder)](https://www.sbert.net/examples/unsupervised_learning/README.html#tsdae).
For using TSDAE, model need decoder part implementation(MPNetForCausalLM). but current MPNet model does not support MPNetForCausalLM

With support to MPNetForCausalLM, all the decoding functions will be possible for mpnet models, e.g. the TSDAE learning, seq2seq tasks, etc.

simillar issue : [Add support to DistilBertLMHeadModel](https://github.com/huggingface/transformers/issues/14737)

### Your contribution

I have been working on the details by forking the recent master branch , and if there is no problem, I would like to proceed.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[TensorFlow] Adding LeViT
Author: ariG23498
Pull request: yes

# Adding TensorFlow version of LeViT

This PR adds the TensorFlow version of [LeViT](https://arxiv.org/abs/2104.01136).

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. Issue linked: #19123 
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19152). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Still working on it.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Still working.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Working!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Strange output from summarization models 
Author: Zahz1
Pull request: no

I am trying to get some models working for summarizing news articles, but for some reason I keep getting this strange out

Output:
"In our series of letters from African journalists, film-maker and columnist Farai Sevenzo looks at ... [subject of input article]"

This has happened on multiple models (Pegasus, Bart, and Roberta) and multiple different inputs. The output is either the correct summary for the article or this incorrect output listed above. Does anyone have any idea how to fix this problem?

code:

from transformers import PegasusTokenizer, TFPegasusModel, PegasusModel, TFPegasusForConditionalGeneration
import tensorflow as tf

src_text = """ article text put here """

tokenizer = PegasusTokenizer.from_pretrained('google/pegasus-xsum')
model = TFPegasusForConditionalGeneration.from_pretrained('google/pegasus-xsum')

inputs = tokenizer(src_text, truncation=True, padding='longest', return_tensors="tf", )
translated = model.generate(**inputs)
tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)
print(tgt_text)



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I see this happening now with pegasus models. Similar to the @Zahz1 I get the following:

"In our series of letters from African journalists, filmmaker and columnist Ahmed Rashid looks at some of the issues facing the continent."

The text that is being summarized has nothing to do with the above generated summarization.
[WIP] Support BLIP and GIT in image-to-text and VQA pipelines
Author: atturaioe
Pull request: yes

# What does this PR do?
Support BLIP and GIT models in image-to-text and VQA pipelines.

Fixes #21110 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@NielsRogge 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21227). All of your documentation changes will be reflected on that endpoint.

Hi @NielsRogge, should I remove the return of the topk scores in the VQA pipeline that used ViltForQuestionAnswering only?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Can not init BertTokenizerFast
Author: wqh17101
Pull request: no

### System Info

Linux
python3.7
tokenizers                         0.12.1
transformers                       4.26.1
@ArthurZucker

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
>>> from transformers import BertTokenizerFast
>>> BertTokenizerFast()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/anaconda3//python3.7/site-packages/transformers/utils/dummy_tokenizers_objects.py", line 31, in __init__
    requires_backends(self, ["tokenizers"])
  File "/home/anaconda3/python3.7/site-packages/transformers/utils/import_utils.py", line 935, in requires_backends
    raise ImportError("".join(failed))
ImportError: 
BertTokenizerFast requires the 珞 Tokenizers library but it was not found in your environment. You can install it with:

pip install tokenizers

In a notebook or a colab, you can install it by executing a cell with

!pip install tokenizers

```


### Expected behavior

work normally.

Comments:
Not sure I understand, to use `TokenizerFast` you need the tokenizers llibrary. 
Either use:
```python
>>> from transformers import BertTokenizer
>>> BertTokenizer 
```
Or run `pip install tokenizers`. 
This is not an issue but expected.

What is the tokenizers llibrary? and how to install them? @ArthurZucker 
I have run `pip install tokenizers` to install tokenizers

The `tokenizers` library is available [here](https://github.com/huggingface/tokenizers), it implements the backend of fast tokenizers in rust. If it is installed you should be able to import without any issues! Make sure it was installed in the environment you are using. 

@ArthurZucker 
```
[root@localhost home]# pip list |grep tokenizers
tokenizers                         0.13.2
```
```
[root@localhost home]# python -c "import tokenizers"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'tokenizers'
```
Why @ArthurZucker 

Sorry , it looks like the problem of my conda env
Add AutoModelForZeroShotImageClassification
Author: alaradirik
Pull request: yes

# What does this PR do?
Adds `AutoModelForZeroShotImageClassification` and `TFAutoModelForZeroShotImageClassification` to transformers. 

CC @MKhalusova will be adding a task guide in a separate PR


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?




Comments:
_The documentation is not available anymore as the PR was closed or merged._

The CI tests are failing due to unrelated issues, I will rebase the PR to the main branch once they are fixed.
Add a vit-based ocr model to hugging face
Author: wdp-007
Pull request: no
Labels: name: New model

### Model description

We want to add MGPSTR model(ECCV 2022)  to hugging face.
MGPSTR is a ViT(Vision Transformer)-based pure vision model for STR, which shows its superiority in recognition accuracy. It has a Multi-Granularity Prediction (MGP) strategy to inject information from the language modality. MGPSTR algorithm achieves state-of-the-art performance.
We followed the guidance of https://github.com/huggingface/transformers/tree/main/templates/adding_a_new_model, but encountered some problems, such as being unable to find a suitable huggingface-hub version when installing the environment.
```
ERROR: Could not find a version that satisfies the requirement huggingface-hub<1.0,>=0.8.1 (from transformers[dev]) (from versions: 0.0.1, 0.0.2, 0.0.3rc1, 0.0.3rc2, 0.0.5, 0.0.6, 0.0.7, 0.0.8, 0.0.9, 0.0.10, 0.0.11, 0.0.12, 0.0.13, 0.0.14, 0.0.15, 0.0.16, 0.0.17, 0.0.18, 0.0.19, 0.1.0, 0.1.1, 0.1.2, 0.2.0, 0.2.1, 0.4.0)
ERROR: No matching distribution found for huggingface-hub<1.0,>=0.8.1
```
Can I get some help or guidance?

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The paper will be published soon.

Comments:
Sure, do you have an email address? We can set up a slack channel for easier communication 

Yes, my email address is wdp0072012@gmail.com.
Do I need to register for slack in advance?

You should have received an invite by email :)
[i18n-<languageCode>] Translating docs to <languageName>
Author: dhckdduf
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @ArthurZucker, @sgugger for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.mdx) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.mdx) (waiting for initial PR to go through)
- [ ] [installation.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.mdx).

## Tutorial section
- [ ] [pipeline_tutorial.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.mdx)
- [ ]  [autoclass_tutorial.mdx](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.mdx)
- [ ]  [preprocessing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.mdx)
- [ ]  [training.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.mdx)
- [ ]  [accelerate.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.mdx)
- [ ]  [model_sharing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.mdx)
- [ ]  [multilingual.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.mdx)

<!--
Keep on adding more as you go 🔥
-->

Comments:
Add pr_checks.mdx Italian translation (#17459)
Author: alexcalabrese
Pull request: yes

## What does this PR do?

Italian translation of doc related to the checks on a PR of :hugs: Transformers.
* updated _toctree.yml
* added pr_checks.mdx

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
[documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

See issue: [#17459](https://github.com/huggingface/transformers/issues/17459)

@omarespejel
@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@omarespejel @sgugger for me is ok.
Thanks @alexcalabrese
Fix gradient checkpointing bug in xlm_roberta_xl
Author: KMFODA
Pull request: yes

This PR fixes a bug that a user can encounter while using generate and models that use gradient_checkpointing.

Fixes Issue https://github.com/huggingface/transformers/issues/21737

cc @younesbelkada or @gante

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Added big_models.mdx italian translation #17600 
Author: nickprock
Pull request: yes

## What does this PR do?

Italian translation of doc related to the preprocessing of :hugs: Transformers.
* updated _toctree.yml
* added big_models.mdx

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

See issue: [#17459](https://github.com/huggingface/transformers/issues/17459)

@omarespejel 
@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
torchmetrics support
Author: izapolsk
Pull request: no

### Feature request

In eval loop the HF transformers collects and keeps in memory predictions and targets up to the end of eval and then computes metrics passed via callback.
When transformers is used to fine-tune model with big dataset and many labels (in my case up to 220k labels), predictions and targets take huge amount of RAM. It's especially noticable with DDP and many GPUs per node.
It would be great to add [torchmetrics](https://torchmetrics.readthedocs.io/en/stable/index.html) support where metrics can be computed for every eval step and then averaged at the end

With torchmetrics it's also possible to compute metrics on GPU in distibuted way. What makes eval significantly faster. 

### Motivation
With current metrics callback I can't run eval with > 120k labels.
I tried to sparsify and/or cut data. However, with growing number of labels and etc it anyway became inefficient.  
Eventually I had to update trainer to compute torchmetrics for every eval step on GPU what fully solved this issue in my case.

So, it would be great to have this feature out-of-the-box.

### Your contribution
I can create PR with appropriate changes if you also view this feature as useful.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@sgugger, could you please share your opinion on above feature request ? ^^

At this stage we're not planning to add this no. You can always subclass the Trainer to put your own evaluation loop in it however.

> At this stage we're not planning to add this no. You can always subclass the Trainer to put your own evaluation loop in it however.

this is exactly what I did. I just thought that it could be useful for others because I saw some other reports in GH about OOM in eval.

Note that you can use `eval_accumulation_steps` to avoid taking too much space on the GPU. There is no need to use torchmetrics for that.
[`Blip2`] skip accelerate test
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR skips a test that is currently failing: https://github.com/huggingface/transformers/actions/runs/4395315343/jobs/7697061404

The tiny BLIP2 models uses T5 as a text decoder, that is itself having a parameter named `shared` that is tied with 3 other parameters of the model: `encoder.embed_tokens`, `decoder.embed_tokens`, `lm_head`.

In some very specific usecases (small model, + small `max_memory`), the test hits some corner cases as `accelerate` does not support handling multiple tied weights yet: https://github.com/huggingface/accelerate/blob/37831808444e089a182f66713935d27c39a0cf2c/src/accelerate/utils/modeling.py#L232 & https://github.com/huggingface/accelerate/blob/37831808444e089a182f66713935d27c39a0cf2c/src/accelerate/utils/modeling.py#L566

Not that a similar test is also currently being skipped for T5: https://github.com/huggingface/transformers/blob/102b5ff4a813eea848bb82ff2f451e0f6b17b30c/tests/models/t5/test_modeling_t5.py#L689

As this usecase is very corner case and less likely to happen (most of BLIP2 models are large) and fixing this would require a lot of work on `accelerate`, let's skip this test as we did it for T5

cc @sgugger @ydshieh

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I am trained by @sgugger 's words and my prediction of the response to this PR is

Yes
```
the test doesn't make sense for tiny models and triggers some undefined behaviors.
Probably better to just skip it at this stage.
```
and
```
We should have slow integration tests instead (on a regular-size model)
```
(but I am not sure if it's required to do this in the same PR, or just something we should change for all such tests in another PR)
Fix gradient checkpointing bug in LongT5
Author: KMFODA
Pull request: yes

This PR fixes a bug that a user can encounter while using generate and models that use gradient_checkpointing.

Fixes Issue https://github.com/huggingface/transformers/issues/21737

cc @younesbelkada or @gante

Comments:
No problem! Yeah I added it because I thought it was incorrectly omitted from LongT5Stack. Should it be removed?

_The documentation is not available anymore as the PR was closed or merged._

I think your changes are fine and it was something we forgot to add on our side at the first place !