noamaanMulla-03/transformers-issues huggingface dataset:
pytest jax,jaxlib,flax versions incompatibility
Author: darxradi3nt
Pull request: no

### System Info

When running pytest (```pytest --collect-only -q``` is enough), it fails due to
```
AttributeError: module 'jax.tree_util' has no attribute 'register_pytree_with_keys_class'
```

From setup.py deps:
```
    "jax>=0.2.8,!=0.3.2,<=0.3.6",
    "jaxlib>=0.1.65,<=0.3.6",**
```

## Fix:
pip install jax==0.3.25 jaxlib==0.3.25 flax==0.6.2

### Who can help?

@sanchit-gandhi

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

pytest --collect-only ./tests/models/<some_model>

### Expected behavior

"tests collected" without any error

Comments:
Could you try:
```
pip install -e ".[flax]"
```
To get the correct version of all three packages?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

More detailed instructions for installing Flax & Transformers can be found here: https://github.com/huggingface/transformers/tree/main/examples/research_projects/jax-projects#how-to-install-relevant-libraries

Leaving this one closed for now. Feel free to open if the above guide doesn't solve your issue and I can take another look
save full_osd in fsdp mode
Author: amartino1
Pull request: yes

Fixes a typo in which the variable full_osd is referenced before definition if run in fsdp mode. The fix allows model files to be saved when running in fsdp.

Links to issue: https://github.com/huggingface/transformers/issues/24057

Fixes # 24057


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24328). All of your documentation changes will be reflected on that endpoint.

Hello, PR #24446 addresses this issue. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
🌐 [i18n-KO] Translated `tf_xla.md` to Korean
Author: 54data
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `tf_xla.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
<!-- May you please review this PR? @member1 @member2 ... -->
@kihoon71, @0525hhgus, @54data, @Sunmin0520, @seank021, @augustinLib

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
May you please review this PR? @sgugger, @ArthurZucker, @eunseojo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

리뷰 사항 없습니다!

꼼꼼한 번역 잘 보았습니다! 수고 많으셨습니다!
Generate: `ImageToTextPipeline` now supports generation kwargs
Author: gante
Pull request: yes

# What does this PR do?

As the title indicates -- previously the generation kwargs had to be passed as a separate dictionary, which was inconsistent with the other pipelines. 

This PR borrows code from the other pipelines, to allow things like `pipe(input, min_new_tokens=10)` (as opposed to `pipe(input, generate_kwargs={"min_new_tokens":10})`). It also updates the docs, which were slightly outdated.

Fixes #24836 

Comments:
 > This PR borrows code from the other pipelines, to allow things like `pipe(input, min_new_tokens=10)` (as opposed to `pipe(input, generate_kwargs={"min_new_tokens":10})`). It also updates the docs, which were slightly outdated.

Actually, other pipelines that accept anything, are the ones that have been done so because of legacy reasons.
Accepting anything causes headaches because `generate` can arbitrarily add new kwargs, some of which could clash with existing pipeline kwargs.
The first one that comes to mind is `max_length` which already clashed with tokenizer `max_length`.
I really think this is a bad idea, and that whitelisting is better.

Ah ah, glad I asked 😅 

_The documentation is not available anymore as the PR was closed or merged._

@Narsil @sgugger I see, arg clashing is absolutely undesirable!

Explicit whitelisting (like `max_new_tokens` in `ImageToTextPipeline`) also seems excessive -- different modalities/tasks benefit from controlling specific parameters, so we would have to pick between additional maintenance burden + large docs and usage capabilities.

How about we move all generative pipelines to accept a `generation_config` (with the proper deprecation cycles and doc changes)? It would be an explicitly whitelisted argument from the pipeline perspective while enabling the vast majority of generation modes.

[Regardless of the specific decision here, I believe we would benefit from making all pipelines consistent with each other.]

That works for me!

`generate_config` seems even better than `generate_kwargs` !

Awesome, I'll close this PR and open a new one reflecting the decision here 👍
[`TF`]  Also apply patch to support left padding
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes red test on main GPTj test equivalence. Follows #24979

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25085). All of your documentation changes will be reflected on that endpoint.
[ `PreTrainedTokenizerFast`] Keep properties from fast tokenizer
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Start a potential fix to #24179, ended up being releated to #24441, calling modifies the values of the underlying tokenizer, but never changes anything on the surface so will probably add some kind of warning in the documentation. 

TLDR; adds the possibility  of initialize a `PreTrainedTokenizerFast` from a `tokenizer.Tokenizer`, keeping the `padding` and `truncation` informations. 



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yes, currently everything is ignored
[`IDEFICS`] Fix idefics ci
Author: younesbelkada
Pull request: yes

# What does this PR do?

Attempt to fix IDEFICS failing CI as discussed offline @stas00 - more info coming soon

original PR: https://github.com/huggingface/transformers/pull/24796


Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25056). All of your documentation changes will be reflected on that endpoint.
Allow generic composite models to pass more kwargs
Author: ydshieh
Pull request: yes

# What does this PR do?

Generic composite models: `(Text|Vision|Speech)EncoderDecoder`: their `forward` don't have the full info. in their 2 components.

Fix #24919

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Looking at the errors, we probably need to check whether the model has `self.encoder` or `self.model.encoder` 

> Looking at the errors, we probably need to check whether the model has `self.encoder` or `self.model.encoder`

I can do this, but the `model` in `self.model` is not something from convention I guess: it's more a implementation detail of each (concrete) encoder decoder model class (like `bart`)

@gante 

Let me know if the latest change LGTY 🤗 . Thanks!
BLOOM Flax
Author: younesbelkada
Pull request: yes

# What does this PR do?

An attempt of adding Flax implementation of BLOOM - original PR from @haileyschoelkopf #17761 

## TODOs:

- [x] alibi shifting for batched generation
- [ ] change mask fill value cc @patil-suraj 
- [x] optimize code (alibi creation + mask creation)


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18022). All of your documentation changes will be reflected on that endpoint.

Is this ready to merge @younesbelkada ? @sanchit-gandhi @patil-suraj could you take a final look?

Yeah ready to merge for me, for partitioning compatibility I think that we can address another PR for that

Also we need to add the credits to @haileyschoelkopf for starting the contribution

Modelling file and tests LGTM. Am in agreement that the the named axes should come as a follow-up PR - better to modify a script that's already in-place and working rather than merging a complete overhaul of how we structure our code in one go.

Good to merge for me once @patil-suraj and @sanchit-gandhi give the green light here :-) 

Thanks all for your comments! 
I had to fix a small test since the main modeling code has changed a bit in #18344 !
I think this PR is ready to be merged 💪 cc @patil-suraj 

@patil-suraj @sanchit-gandhi - should we merge this one? 

Good for me! Awaiting the green light from @patil-suraj who requested a look over the modelling code prior to merging. Regarding `setup`, I previously asked in the G Chat with the JAX/Flax guys how to make this work with `scan`. Think it went under the radar, so will bring this issue back up!

Hi @sanchit-gandhi @patil-suraj @patrickvonplaten 
I think that the PR is ready for a final round of review! 💪  I proposed a potential solution to re-write the model definition with `setup` but I am unsure how this fits the HF+Jax approach regarding `scan` feature

@sanchit-gandhi @patil-suraj could you check here? I won't have time do review the PR this week I think

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Should we merge this one ? cc @patrickvonplaten @patil-suraj @sanchit-gandhi 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18022). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Both https://github.com/huggingface/transformers/pull/17761 and this PR show a lot of work. Why isn't it merged? 

Adding this to my TODOs
Hotfix for failing `MusicgenForConditionalGeneration` tests
Author: ydshieh
Pull request: yes

# What does this PR do?

A exceptional case (for `MusicgenForConditionalGeneration`) is not detected by the CI triggered in #24927.

Test is currently failing on `main`, see

https://app.circleci.com/pipelines/github/huggingface/transformers/68971/workflows/503866ea-e425-4817-9849-52c6e7fd2865/jobs/863722

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Added support for dtype in .to() method. 
Author: amannagarkar
Pull request: yes

Issue #24068. The updated  method now accepts both  "device" and "dtype"  as keyword arguments. When "dtype" is provided, the tensors within the  object will be cast to the specified data type.

# What does this PR do?

This PR adds support for the `dtype` parameter in the `.to()` method of the `BatchEncoding` class. Previously, only the `device` parameter was supported. With this enhancement, users can now specify the desired data type for tensor casting and allocation.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. 

@amyeroberts @NielsRogge 

Comments:
@amyeroberts hey, yes will do!

Thanks for your PR but what is the purpose of this? The tokenization results are all integers and changing their dtype will make then unusable by the model.

@sgugger I was using `BatchEncoding` in a Processor class (`InstructBlipProcessor`) and noticed that it didn't support `dtype` as `BatchFeature` does. However it's a bit unclear to me whether I need to use `BatchFeature` or `BatchEncoding` for multi-modal processors

Probably batch feature if you have float values. As the name indicates, `BatchEncoding` is for encoded values (so ints).

@amyeroberts passed all cases. Kindly check!

> Thanks for your PR but what is the purpose of this? The tokenization results are all integers and changing their dtype will make then unusable by the model.

You did not answer that question though.

@amannagarkar thanks for your PR, but given the comment by @sgugger it probably makes sense to close this PR, and instead update multimodal processors in the library that return a`BatchEncoding` instead of a `BatchFeature`.

This is because `BatchEncoding` is only used by text-only tokenizers, for which the `dtype` isn't relevant, since they always return LongTensors.

@sgugger sorry for not responding, I thought Niels answered your question. I will be more careful in the future!
@NielsRogge okay, noted. I will take a look into it!
fix tied_params for meta tensor
Author: SunMarc
Pull request: yes

# What does this PR do?
This PR fix the retrieval of `tied_params` when we load a model with accelerate. The issue was that we were not able to retrieve the tied parameters by using the `id` function to check equality between meta `tensor`. Instead, we use `find_tied_parameters` from accelerate library. 

Solves [#1772](https://github.com/huggingface/accelerate/issues/1772)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
🌐 [i18n-KO] Translated `perf_hardware.md` to Korean
Author: augustinLib
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `perf_hardware.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)
@0525hhgus, @Sunmin0520, @54data, @seank021, @kihoon71

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
<!-- May you please review this PR? @member1 @member2 ... -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
May you please review this PR? @sgugger, @ArthurZucker, @eunseojo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

위에서 @0525hhgus 님의 댓글을 제외하고는 별도의 리뷰사항 없습니다 :)  

> Looks good! Could you update the ## GPU [[gpu]] in the english version as well? Seems like it is not rendering properly, let's kill two birds with one stone!
> 
> <img alt="image" width="959" src="https://user-images.githubusercontent.com/48595927/255879037-d22d9839-7263-47d6-988b-408ed3bec0aa.png">

@ArthurZucker 
I updated that issue! and also i could confirm [here](https://huggingface.co/docs/transformers/main/en/perf_hardware) that it worked successfully. 
This part didn't work well in previous versions, but it works well in the current `main`. It was a problem that appeared because the newline was omitted in the markdown file

Thank you for your hard work and dedication, it is greatly appreciated.
[`T5`] Adding model_parallel = False to `T5ForQuestionAnswering` and `MT5ForQuestionAnswering`
Author: sjrl
Pull request: yes

# What does this PR do?

<!-- Remove if not applicable -->

Fixes https://github.com/huggingface/transformers/issues/24682 by adding `self.model_parallel = False`


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker and @younesbelkada


Comments:
Is there a good way to add a test for this? I wasn't sure where a test like this would be added. 

> Is there a good way to add a test for this? I wasn't sure where a test like this would be added.

No need to add a test for this. We have `test_model_parallelization` which tests model parallelization (the opposite).
As we are dealing with some  deprecated thing, it doesn't worth much time on it.



Thanks! Would be nice if you can check this change works (i.e. fix the issue you opened in #24682) 🙏 

_The documentation is not available anymore as the PR was closed or merged._

> Thanks! Would be nice if you can check this change works

Definitely! I just ran the code locally and it works. 

Yes. It seems a bit strange indeed. So far all `is_parallelizable` is set at the `XXXPreTrainedModel` level.

I think it's fine as the only usage of `is_parallelizable` is here

```python
        if hasattr(model, "is_parallelizable") and model.is_parallelizable and model.model_parallel:
            self.is_model_parallel = True
        else:
            self.is_model_parallel = False
```
There shouldn't bee too much confusion between `is_parallelizable = True` and `always model_parallel = False` for just this single (and new) model class.
Replace BertLayerNorm with LayerNorm
Author: eldarkurtic
Pull request: yes

Running Movement pruning experiments with the newest HuggingFace would crash due to non-existing BertLayerNorm.

Comments:
@VictorSanh @eldarkurtic 

Hello guys!

Thanks for your effort for such a great repo!

I'm wondering why the implementation of LayerNorm in BERT seems not to be identical to what's commonly discussed version of `LayerNorm`.
I've read several tutorials about `LayerNorm`, and according to their explanation, the implementation seems to be like following:
```
input_x = torch.rand(batch_size, sequence_length, hidden_size)
layer_norm = torch.nn.LayerNorm([sequence_length, hidden_size])
output = layer_norm(input_x)
```
In above implementation, the normalization is calculated through the dim of `sequence_length` as well as `hidden_size`, but the implementation in this repo only normalize the tensors in the last dim, which I think is somehow identical to `InstanceNorm`?
Need documentation for understanding the difference between `7B` and `7Bf`
Author: scottfleming
Pull request: no

Unclear what the tradeoffs are between using eg `7B` and `7Bf` as an argument for `model_size` in the `convert_llama_weights_to_hf.py` script:
https://github.com/huggingface/transformers/blob/f9cc333805c47665c6afee8b5867931e54abe0c6/src/transformers/models/llama/convert_llama_weights_to_hf.py#L65

Comments:
cc @ArthurZucker 

Sure, we can add a bit of documentation, mentioning that this is specific to llama2 official release 😉
documentation for llama2 models
Author: shauray8
Pull request: yes

# What does this PR do?
Adds some documentation for 'f' models.

Fixes #25090

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@ArthurZucker 

** I'm not sure if this is even the right way to include the documentation, but let me know if there's something more to add.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

changes incorporated!
Fix `PvtModelIntegrationTest::test_inference_fp16`
Author: ydshieh
Pull request: yes

# What does this PR do?

Failing

```bash
FAILED tests/models/pvt/test_modeling_pvt.py::PvtModelIntegrationTest::test_inference_fp16 - ValueError: PvtForImageClassification does not support `device_map='auto'`. To implement support, the modelclass needs to implement the `_no_split_modules` attribute.
```

A fix for this test will fix the other 2 shown in the report (due to bad GPU state or something else)

Comments:
_The documentation is not available anymore as the PR was closed or merged._