noamaanMulla-03/transformers-issues huggingface dataset:
[`Blip2`] Add int8 support for `blip2-flan-t5-xxl` 
Author: younesbelkada
Pull request: yes

# What does this PR do?

Similarly as https://github.com/huggingface/transformers/pull/20683 - currently the largest blip2 + flan-t5-xxl fails to run in int8 since we forgot to add the patch presented in the aforementioned PR

This PR adds the `_keep_in_fp32_modules` support for `Blip2` so that users can run this model in fp16 and int8 with no performance degradation as the original model trained in bf16

cc @sgugger @NielsRogge 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Query related to finetuning bert models for QA
Author: gokul427
Pull request: no

When we run run_qa.py on bert-base using squad data and once again fine tune it on custom data, will it retrain all the layers? or will it train only the last layer(head) freezing the other layers ? please explain the kind of finetuning done with run_qa.py

Comments:
Please use the [forums](https://discuss.huggingface.co/) for such questions as we keep issues for bugs and feature requests only.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Command needed to finetune bert on squad using TPU
Author: gokul427
Pull request: no

Please provide the command to execute run_qa.py on TPU to finetune bert models.

Comments:
Please use the [forums](https://discuss.huggingface.co/) for such questions as we keep issues for bugs and feature requests only.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Domain-specific word similarity from documents question-answer
Author: VikasRathod314
Pull request: no

I am trying to create a chat boat-like application (inspired by chat GPT). The boat or you can say an application should be able to answer questions of a software/products respective Help document. 

I have tried to finetune tilbert_base_uncased model from hugging face on less than 100 annotated Question-answer in the form of squad format. but my model is not performing well. the F1 score is about 0.3. Can anyone suggest important approaches or docs related to Question answering-based QA implementation who worked on the same problem? 

Comments:
Please use the [forums](https://discuss.huggingface.co/) to ask such questions as we keep issues for bugs and feature requests only.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Project_Test01
Author: RutujaTalekar
Pull request: no
Labels: name: New model

### Model description

This model takes parameters - training data(optional) and rubrics(list of strings) to train. It gives us zero shot results for scoring logical reasoning answers provided by the students.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://arxiv.org/pdf/2301.08771.pdf

Comments:
Generate: TF supports multiple eos tokens
Author: gante
Pull request: yes

# What does this PR do?

TF generation test addition PR 4 (out of 4) 👉 All PT generation integration tests for features that also exist in TF are now framework-agnostic 🎉 

To complete this last step, I've added support to multiple EOS tokens in TF. 

Because this touches the core of the TF generation code, slow tests were run for:
- [x] TF GPT2
- [x] TF T5

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The two failing tests are known failures -- merging
added docs to time series transformer's generate function
Author: kashif
Pull request: yes

# What does this PR do?

Added docs to the time series transformer's generate function.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20524). All of your documentation changes will be reflected on that endpoint.
BertTokenizer not release gpu memory after del 
Author: jaewoo-so
Pull request: no

### System Info

transformers                       4.20.1
tensorflow                         2.9.1

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
from transformers import BertTokenizer, TFBertModel
import torch
max_seq_len = 2028
tokenizer = BertTokenizer.from_pretrained('klue/bert-base', truncation=True, max_seq_len=max_seq_len)
del tokenizer
torch.cuda.empty_cache()
```



### Expected behavior

In ubuntu20.04 with RTX3090, after del tokenizer, gpu memory is not released. 
how can i release gpu memory?

Comments:
This code doesn't use any GPU memory as tokenizers don't even import `torch`.

@sgugger In my case, after run this code, gpu memory is fully occupied.  Is your pytorch the gpu version correct?

This is because you import `TFBertModel` (I didn't catch it in your first code sample). This imports `tensorflow` which then takes all the GPU memory.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix CLIP pooling for textual inversion so that eos tokens are taken
Author: isamu-isozaki
Pull request: no

### Feature request

For textual inversion in diffusers, we are adding tokens that have a higher token id than the eos token. So when we get clip embeddings for textual inv tokens, we need to change the pooling so it gets the eos token and not the arg max token.



### Motivation

This is an issue that should be fixed as the clip embeddings won't work once we add more tokens to the tokenizer.

### Your contribution

I can make a pr for this. This is not an issue in the original implementation of the clip since they use pre-existing tokens in the embedding which has its own pro-cons.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Remove trailing 'extractive' word from en documentation
Author: tpaviot
Pull request: yes

# What does this PR do?

Removes an extra word from the documentation.

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Documentation: @sgugger, @stevhliu and @MKhalusova



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: Fix flaky indexing error in `test_constrained_beam_search_generate_dict_output`
Author: gante
Pull request: yes

# What does this PR do?

Fixes a random indexing error in `test_constrained_beam_search_generate_dict_output` -- For models without integer inputs (like whisper), we were requesting to force a random token between 3 and 99 (inclusive), but the vocab size was also 99. 

This means that when the test attempted to force a `99`, the test would fail with an indexing error.

Comments:
> For models without integer inputs (like whisper), we were requesting to force a random token between 3 and 99

Sorry, I lack of a bit the context. `models without integer inputs`, but in this failed test, the `input_ids` is still passed to the model, and the model use it (and get index error)? I am a bit confused by reading it literally. 

_The documentation is not available anymore as the PR was closed or merged._

@ydshieh `input_ids` is a poor variable name in this test (and in many others). It holds a dummy input with the format expected by the model -- for `Whisper`, it is a tensor of type fp32. See below what the debugger prints for this test on `Whisper`

![Screenshot 2023-02-10 at 12 33 14](https://user-images.githubusercontent.com/12240844/218093435-8fb7a755-3a91-4ef4-a943-693a7d1387a1.png)

The touched variables (`min_id` and `max_id`) are used to force a random token (selected between those ids) at generate time. These were the indices triggering the indexing error. Since `generate` always outputs tokens, regardless of the model, the if/else I removed didn't make much sense :)
[`bnb`] Let's make the daily CI green 🍏 
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR fixes a test that is currently failing on the `main` branch.

Link to failing test: https://github.com/huggingface/transformers/actions/runs/4154270129/jobs/7186572507

Since the introduction of https://github.com/huggingface/transformers/pull/21524 - when loading `bigscience/bloom-1b7` it  detects `fp32` weights when using `torch_dtype="auto"`. Therefore the expected relative difference of the memory footprint is different than the expected one. 

This PR fixes the test by forcing `torch_dtype=torch.float16` when loading the fp16 model

cc @ydshieh @sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yeah I am unsure too, `auto` shold give `float16` according to the weights size: https://huggingface.co/bigscience/bloom-1b7/tree/main (1b7 parameters = 3.4GB in fp16)
Add missing arguemtn to run_clip.py
Author: WarrenGreen
Pull request: yes

# Missing test_file argument in run_clip.py

## Bug Experience
- Including the test_file parameter during runtime causes HfArguments to throw error for extra parameters. 
- Not including the test_file parameter during runtime causes NPE [here](https://github.com/huggingface/transformers/blob/main/examples/pytorch/contrastive-image-text/run_clip.py#L299)

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix Blip-2 CI
Author: ydshieh
Pull request: yes

# What does this PR do?

Avoid GPU OOM by using FP16.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

since we have put `_keep_in_fp32_modules` [in a previous PR](https://github.com/huggingface/transformers/pull/21574), I think it should work!

@sgugger I never merge my PR without running on a CI runner-like machines: ran it 3 times and all pass.

Yes I think we can be confident to say that @sgugger
Documentation code sample fixes
Author: MKhalusova
Pull request: yes

Several code examples in the docs will fail if used as is. In many cases, it's a missing dependency, other times, this is due to naming inconsistency, one may be related to a change in API, and a few are due to the execution order (i.e., things used before defined in a tutorial). 
This maintenance PR fixes these issues so that the code samples in the docs work as expected and do not cause unnecessary frustration. 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Automated compatible models list for task guides
Author: MKhalusova
Pull request: yes

# What does this PR do?

This PR adds a script that aggregates model architectures compatible with a task illustrated in a task guide and adds a list of links to them in a <Tip> in the guide. This serves several purposes: 
1. Reinforces the idea that task guides are applicable to more than one model architecture. 
2. Improves discoverability of models. 
3. Serves as the first step to improve navigation between task guides and model docs. 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add: document question answering task guide
Author: MKhalusova
Pull request: yes

This PR adds a task guide on document question answering. 

The example is largely based on @NielsRogge 's [notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LayoutLMv2/DocVQA/Fine_tuning_LayoutLMv2ForQuestionAnswering_on_DocVQA.ipynb) with some inspuration from [this blog post](https://www.philschmid.de/fine-tuning-layoutlm#3-fine-tune-and-evaluate-layoutlm)

A Colab notebook to test the code examples is [here](https://colab.research.google.com/drive/1vIc-YoGbWdNOBGSb6mpkw9myJ4nr2vcA?usp=sharing).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@MKhalusova is it okay if I reviewed the latest changes on Monday? I am currently on travel. 

> @MKhalusova is it okay if I reviewed the latest changes on Monday? I am currently on travel.

Absolutely!
[i18n-fr] Translate quicktour page to French
Author: NoB0
Pull request: yes

# What does this PR do?

Translated the `quicktour.mdx` file of the documentation to French. 

Part of #21456 

Thank you in advance for your review.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger, could you review this PR?

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update setup.py
Author: stas00
Pull request: yes

I was tweaking m4's `setup.py` with some goodies from here and saw a few minor improvements that could be made here:

- clarify the license and description
- add `jax` to the keywords
- add recent python versions

If anything doesn't fit please let me know and I will remove it.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: correct default model input creation for decoder-only models
Author: gante
Pull request: yes

# What does this PR do?

Fixes #21578 and addresses concerns in #21575 

## Context
Support for `.generate()` from `inputs_embeds` with selected decoder-only models was added recently (#21405). This feature enables a `.generate(inputs_embeds=inputs_embeds)` call, i.e. without a `input_ids`. 

This specific call strategy, under the hood, implies that `.generate()` is in charge of creating a) an `input_ids` for later use (to concatenate the generated tokens) and b) the corresponding `attention_mask`. This automated creation was not working properly for `batch_size>1`.

## Changes
The changes in the PR respect the following desiderata (which required moving a few things around):
1. The `attention_mask` can be automatically inferred (with all ones) regardless of the shape of `inputs_embeds`;
2. When `inputs_embeds` is passed and `input_ids` is not, the automated `input_ids` has a sequence length of 1. This is particularly relevant for BLIP, as we don't want `input_ids` to start with the embeddings' sequence length.

This PR also adds/enhances tests, to ensure we don't regress on this capability.

⚠️ if approved, I will make the corresponding TF changes before merging.

Comments:
cc @dimitry12 this fixes the error you reported in #21575 (GPT2 was throwing the same error as GPTJ, and gets fixed here)

cc @NielsRogge please don't forget to add batched tests on models with generation capabilities, generate+batching is surprisingly tricky🙏 

_The documentation is not available anymore as the PR was closed or merged._

But it does seem like it breaks a lot of tests 😅 

(Merging -- the failing test is a known failing test)

@dimitry12 lmk if you see the error when using GPT-J :)

> @dimitry12 lmk if you see the error when using GPT-J :)

@gante GPT-J generation without dummy `input_ids` using only `inputs_embeds` works without errors now. Thank you!
Fix env. variable type issue in testing
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix env. variable type issue in testing.

If `PYTEST_TIMEOUT` is set by `export PYTEST_TIMEOUT=...` or `PYTEST_TIMEOUT=xxx python3 -m pytest ...`, we actually get a `string` instead of `int`, and the test fails.

On our CI (within docker), we don't have this issue, probably due to the way of docker dealing with env. variable.
Let's try to avoid unexpected error though.



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add in big model inference to issue template
Author: muellerzr
Pull request: yes

# What does this PR do?

Adds in @sgugger and myself as @'s on the big model inference as part of the issue template


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._