noamaanMulla-03/transformers-issues huggingface dataset:
[CodeGen] RuntimeError: where expected condition to be a boolean tensor, but got a tensor with dtype Half
Author: jrdzha
Pull request: no

### System Info

transformers==4.21.2
torch==1.10.2+cu113

HW: 12 cpu, 90gb ram, 1xA100-80g

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

**The following works with CodeGen-350M-mono / CodeGen-6B-mono**

```
import torch
from transformers import AutoTokenizer, CodeGenForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-350M-mono")
model = CodeGenForCausalLM.from_pretrained("Salesforce/codegen-350M-mono", device_map="auto")
```

**But throws the RuntimeError for CodeGen-16B-mono**

```
import torch
from transformers import AutoTokenizer, CodeGenForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-16B-mono")
model = CodeGenForCausalLM.from_pretrained("Salesforce/codegen-16B-mono", device_map="auto")
```

### Expected behavior

Expect the 16B model to work as well.

Comments:
cc @younesbelkada 

Another way to reproduce is to use https://github.com/huggingface/transformers-bloom-inference

and run:

```
make codegen-mono
```

Hi @jrdzha 
Thanks so much for your issue! Unfortunately I did not managed to reproduce your issue. How are you getting this error? Could you provide me a full example script please?
Here is the snippet I used: 
```
import torch
from transformers import AutoTokenizer, CodeGenForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-16B-mono")
model = CodeGenForCausalLM.from_pretrained("Salesforce/codegen-16B-mono", device_map="auto")

text = "def main():"
inputs = tokenizer(text, return_tensors="pt")

model.generate(**inputs)
```
Also can you make sure to use the latest version of `accelerate` ? `pip install --upgrade accelerate` + is there anything blocking on your side not to use the latest version of `transformers`? `pip install --upgrade transformers` 

Seems also to be a duplicate of https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram/issues/4 I also ran the experiment with `torch==1.10` and not getting any error ! 

It seems this issue is inconsistent. I did have it work once, but without changing any code... I'll spend some more time trying to reproduce this more reliably...

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
can't from transformers import TFBertModel 
Author: minig0lem
Pull request: no

### System Info

- `transformers` version: 4.24.0
- Platform: Windows-10-10.0.19044-SP0
- Python version: 3.10.8
- Huggingface_hub version: 0.11.0
- PyTorch version (GPU?): 1.13.0+cpu (False)
- Tensorflow version (GPU?): 2.11.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@LysandreJik @Rocket

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

![image](https://user-images.githubusercontent.com/92259521/203403557-481c7ac6-fbd7-4a75-8cbb-088f25a8f43e.png)

### Expected behavior

from transformers import BertTokenizer works well.

but from transformers import TFBertModel doesn't work .

It runs like picture above.

How do I resolve this error(ModuleNotFoundError: No module named 'keras.saving.hdf5_format')?

Comments:
This is due to the latest release of TensorFlow which broke many things. You need to either install `transformers` from source (this is fixed on the main branch) or downgrade `Tensorflow` to 2.10. :-)

This also broke the `evaluate` CI :) Will fix to `<=2.10` for now.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Can we add optional kwargs to various model in addition to their required fixed inputs? 
Author: LTEnjoy
Pull request: no

### Feature request

For example in class BertForMaskedLM, we have to input needed arguments as follows:
    def forward(
            self,
            input_ids=None,
            attention_mask=None,
            token_type_ids=None,
            position_ids=None,
            head_mask=None,
            inputs_embeds=None,
            encoder_hidden_states=None,
            encoder_attention_mask=None,
            labels=None,
            output_attentions=None,
            output_hidden_states=None,
            return_dict=None,
        ):

I wonder if it's possible to add optional **\*\*kwargs** so that we could customize model layer easier. Like this:
  def forward(
            self,
            input_ids=None,
            attention_mask=None,
            token_type_ids=None,
            position_ids=None,
            head_mask=None,
            inputs_embeds=None,
            encoder_hidden_states=None,
            encoder_attention_mask=None,
            labels=None,
            output_attentions=None,
            output_hidden_states=None,
            return_dict=None,
            **kwargs,
        ):

### Motivation

When I want to add an additional operation to a sub module in a big whole model, I need to use extra data as input. But to implement this, I have to add the extra inputs to every module so they can be finally transferred to the correct module. I don't know whether there are more efficient ways to solve the problem. If so, I'd be appriciated for you to point it out.

### Your contribution

Maybe None

Comments:
We can't add blank kwargs like this, as a user who makes a typo in their inputs will then not get any error and not realize they did something wrong.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Yeah, now I see the hidden trouble if we add blank kwargs to a model. But what about adding some arguments that are specified by users?

For example, we add an argument "custom_arg=None" to __init__(```, custom_arg=None), and then the model will add the argument to its forward function like "self.forward = partial(self.forward, custom_arg=None)", and finally this argument is iteratively passed to all the submodel.

Maybe in this way we could keep the input safety for unaware mistakes but increase a model's flexibility?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add FlexiBERT
Author: shikhartuli
Pull request: yes

# What does this PR do?

Implements the FlexiBERT suite of 3.32 billion transformer architectures (and also FlexiBERT 2.0 design space with 1.7 $\times$ 10<sup>88</sup> architectures). The design space supports *flexible* and *heterogeneous* transformer models with diverse attention types. The [paper](https://arxiv.org/abs/2205.11656) has been accepted for publication at the Journal of Artificial Intelligence Research. 

Fixes #20362

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@LysandreJik @sgugger 

## Tasks Completed

From those provided in the [add new model](https://huggingface.co/docs/transformers/add_new_model) contribute section

- [x]  (Optional) Understood the model’s theoretical aspects
- [x]  Prepared 🤗 Transformers dev environment
- [x]  Set up debugging environment of the original repository
- [x]  Created script that successfully runs the forward() pass using the original repository and checkpoint (Available in Demo Colab)
- [x]  Successfully added the model skeleton to 🤗 Transformers
- [x]  Successfully converted original checkpoint to 🤗 Transformers checkpoint
- [x]  Successfully ran forward() pass in 🤗 Transformers that gives identical output to original checkpoint
- [x]  Finished model tests in 🤗 Transformers
- [x]  Successfully added tokenizer in 🤗 Transformers
- [x]  Run end-to-end integration tests
- [x]  Finished docs
- [x]  Uploaded model weights to the Hub (at this [link](https://huggingface.co/shikhartuli/flexibert-mini), shall add more models soon)
- [x]  Submitted the pull request
- [ ]  (Optional) Added a demo notebook 

Thanks so much!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

The PR was closed by bot. Please reopen the PR and let me know if anything needs to be done to merge the PR.
Fix CIs for PyTorch 1.13
Author: ydshieh
Pull request: yes

# What does this PR do?

Before we can update CI to use PyTorch 1.13, there are a few things to fix.
This PR avoids `test_model_parallelism` to fail with some models, where `PyTorch 1.13` requires more strictly the indices to be on the same device of the indexed tensor.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hello and have a good time
Can anyone here help me with the "multi_label_classification" and "single _label_classification" functions?
I am writing a code for dynamic quantization, and I have encountered problems with these functions and their writing.

Hi @fkoeini 

First of all, your question is  unrelated to this PR, and here (this PR page) is not the correct  place for your question above.

Also, from you description, [Hugging Face Forums](https://discuss.huggingface.co/)](https://discuss.huggingface.co/) is the place to post the question. On GitHub, it's for bugs or feature requests :-) Thank you!
Generate: TF XLA beam sample
Author: gante
Pull request: yes

# What does this PR do?

## Context
This is a 2-in-1 PR. While working on adding `generation_config` to TF's `generate`, I noticed that I would have twice the work. This is because on `main` we have `generate()` and `_generate()`, where the former is the legacy version that calls the latter except for beam sample (which is not XLA compatible before this PR). As such, this PR completes the transition to XLA and removes most legacy code, simplifying the transition to the generation config.

## Changes
1. Replaces `generate()` by `_generate()`, which was the original goal of the XLA refactor (and will make my life easier);
2. Deletes many private functions that are no longer reached;
3. Updates RAG accordingly (from the old beam search to the XLA-compatible beam search), slow tests are passing;
4. ⚠️ Adds beam sample on the existing `beam_search` function. Unlike PT implementation, this is NOT a stand-alone function. This was a deliberate decision to decrease maintenance costs, as I don't think it would be wise to add ~500 lines of code for a functionality that is infrequently used, and can be solved with a few extra lines.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[i18n-<languageCode>] Translating docs to <languageName>
Author: 7565A
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @ArthurZucker, @sgugger for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.mdx) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.mdx) (waiting for initial PR to go through)
- [ ] [installation.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.mdx).

## Tutorial section
- [ ] [pipeline_tutorial.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.mdx)
- [ ]  [autoclass_tutorial.mdx](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.mdx)
- [ ]  [preprocessing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.mdx)
- [ ]  [training.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.mdx)
- [ ]  [accelerate.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.mdx)
- [ ]  [model_sharing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.mdx)
- [ ]  [multilingual.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.mdx)

<!--
Keep on adding more as you go 🔥
-->

Comments:
Flax implementation of BART contain NaN in hidden_states 
Author: amankhandelia
Pull request: no

### System Info

While implementing Donut model #19831, one of my test was failing `test_from_pretrained_save_pretrained`, while debugging the testcase failure, It is happening because the test is trying to compare nan with nan which causes the failure, on tracing the root cause, it came down to this line [`jnp.full(attention_mask.shape, float("-inf")).astype(self.dtype)`](https://github.com/huggingface/transformers/blob/fdffee8a601d0408c7e0f57fbb56217f8b57e62a/src/transformers/models/mbart/modeling_flax_mbart.py#L386), `float("-inf")` is causing the `dot_product_attention_weights` to return NaN instead of 0, which is getting cascaded downstream. Since this code is copied from BART, and that code has been copied to several different models (OPT, PEGASUS, BLENDERBOT etc), I am raising this issue against BART

IMHO, we should replace `float("-inf")` with `-1e10` as is the case for multiple different model, RoBerta, if the maintainers agree with this understanding and solution, I can raise a quick PR to fix the same, or otherwise please suggest a solution

@patil-suraj @patrickvonplaten 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run testcase from my branch in the above mentioned PR

### Expected behavior

Hidden States should contain 0 instead of instead of NaN

Comments:
@ArthurZucker and also @sanchit-gandhi since you know Flax Bart quite well 

Hey @amankhandelia - the test of interest passes for the current BART and BART-derived models, so I wonder whether the issue is with the BART model or rather the mBART one? In general, I'm happy with the notion of changing the mask value from `-inf` to a large non-negative number, I just want to determine whether the issue lies with BART or FlaxMBartForCausalLM!

I've noticed in your PR that you're adding FlaxMBartForCausalLM as well as the Flax DONUT model in the same PR (https://github.com/huggingface/transformers/pull/19831). Perhaps you could first add FlaxMBartForCausalLM in a smaller separate PR? We could then run through the failing test together and try to assert whether it's an issue with FlaxMBartForCausalLM or Flax BART and fix any other issues that crop up 🤗

Hey @sanchit-gandhi, thanks for the feedback, 
Makes sense, will raise a separate PR for the FlaxMBartForCausalLM and check the same.

Hey @amankhandelia, thanks for understanding! Feel free to tag me on the PR as soon as it's ready and I'll try to get you a review ASAP!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Use memory efficient attention in CLIP
Author: changjonathanc
Pull request: yes

# What does this PR do?

This PR adds support for clip model to use memory efficient attention, similar to https://github.com/huggingface/diffusers/pull/532

~This is critical when you want to use stable diffusion with large batch size. Because when unet is using memory efficient attention, the bottleneck becomes the initial text encoding step.~
edit: I just tested to see what's the batch size that it becomes the bottleneck, but it seems that's not the bottleneck now. I'm not sure now. 🤔 

Anyways. With this improvement, I'm able to run stable diffusion with batch size 128 on RTX 3090, in here [cccntu/accelerated-stable-diffusion](https://github.com/cccntu/accelerated-stable-diffusion)
@patrickvonplaten


Comments:
Is https://github.com/huggingface/diffusers/pull/532 for CLIP in Stable Diffusion? Is it necessary to port the implementation to `transformers` **CLIP** here to make running SD more efficient ...? @patrickvonplaten is the best one to make the decision.

Looks like this would be a great use of [custom modeling code](https://huggingface.co/docs/transformers/custom_models#writing-a-custom-model) instead of trying to change the model code.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Note that in `transformers` we don't want to necessarily support all the optimizations natively in the core library - also because the fundamental design is different (modeling code is copied rather than abstracted like in `diffusers`). 

Maybe also cc @michaelbenayoun here 

Hi @cccntu,
I agree with @patrickvonplaten here. Also, note that we have a library for optimizations called [Optimum](https://github.com/huggingface/optimum). Altough we do not support custom optimization for each modeling, I think that you might be interested in contributing optimizations such as the ones on your PR there!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix T5 docstring
Author: IvanLauLinTiong
Pull request: yes

# What does this PR do?
Fix  docstring for  `T5Stack` 's  `deparallelize` method : 
`PARALLELIZE_DOCSTRING` -> `DEPARALLELIZE_DOCSTRING`

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger and @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._