noamaanMulla-03/transformers-issues huggingface dataset:
Fix max length for BLIP generation
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

In the previous PR #28994, I did not make changes in multimodal models, which have custom `generate` and pass in `input_embeds`. This PR fixes it. Currently the only model in `transformers`  failing in generation from embeds is BLIP2 and its instruct version. I checked other multimodal generation models, everything is fine there.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@gante 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29296). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

it also needs a `make fixup` run :)

@ArthurZucker the failing test is unrelated (safetensors usage test, timeout) and reruning tests doesn't make it green -- would you be able to merge this PR? 🤗
when running rag example, errors in the 'generating train split' step (wiki_dpr.py)
Author: kiehls90
Pull request: no

### System Info

I'm trying to run a rag example, and the dataset is wiki_dpr.
wiki_dpr download and extracting have been completed successfully.
However, at the generating train split stage, an error from wiki_dpr.py keeps popping up.

Especially in "_generate_examples" :

1. The following error occurs in the line **id, text, title = line.strip().split("\t")**
ValueError: not enough values ​​to unpack (expected 3, got 2)
-> This part handles exceptions so that even if an error occurs, it passes.

2. **ID mismatch between lines {id} and vector {vec_id}**
This error seems to occur at the line " assert int(id) == int(vec_id),".
After I handled the exception in the split error, generating train split progressed to 80%, but an id mismatch error occurred at about the 16200000th vector id.

Debugging is even more difficult because it takes a long time to download and split wiki_dpr. I need help. thank you in advance!!
version: python 3.8, and others are referenced from requirements.txt.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. running rag example: python examples/research_projects/rag/finetune_rag.py \
    --data_dir $DATA_DIR \
    --output_dir $OUTPUT_DIR \
    --model_name_or_path $MODEL_NAME_OR_PATH \
    --model_type rag_sequence \
    --fp16 \
    --gpus 8
2. after downloading and extracting wiki_dpr, then error occurs in "generating train split"

### Expected behavior

.

Comments:
Hi @kiehls90, thanks for raising an issue! 

The scripts under `research_examples` aren't actively maintained. If a dataset is very large, you can try [streaming it](https://huggingface.co/docs/datasets/stream) instead of downloading the whole thing. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Use the old style of cache-management when using DS-Inference
Author: RezaYazdaniAminabadi
Pull request: yes

This PR intends to revert back the cache-management to the old style when optimizing HF models with DeepSpeed-Inference. 
I just added some changes to make it work with Llama.
I will add a test to show the usage of this and why this is needed for the DS-Inference to work.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[`UdopTokenizer`] Fix post merge imports
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Some of the slow tokenizer files were imported in the fast. This means `sentencepiece` dep is forced.
cc @NielsRogge 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29451). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Revert-commit 0d52f9f582efb82a12e8d9162b43a01b1aa0200f
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Revert wrong commit to main https://github.com/huggingface/transformers/commit/0d52f9f582efb82a12e8d9162b43a01b1aa0200f 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29455). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`Udop imports`] Processor tests were not run.
Author: ArthurZucker
Pull request: yes

# What does this PR do?
This is the last bit of non direct import from transformers. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29456). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Generate: inner decoding methods are no longer public
Author: gante
Pull request: yes

# What does this PR do?

This PR sets the inner decoding methods (such as `greedy_search`) as private methods, with a deprecation warning.

This will enable us to complete the following projects with fewer speed bumps:
1. `torch.compile`: as seen in the [PoC PR](https://github.com/huggingface/transformers/pull/29374), a few interface pieces should be touched. E.g. Some special tokens can be a `list` (e.g. `eos_token_id`), which is not compile-friendly, and thus `torch.tensor` should be used instead;
2. Separate prefill function. This will significantly change the interface to the inner decoding methods;
3. More flexible, composable `generate`. Having multiple interfaces to maintain will constrain our ability to refactor the code.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29437). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add Mamba model and Architecture.
Author: Tanmaypatil123
Pull request: no
Labels: name: New model

### Model description

Mamba is a new state space model architecture showing promising performance on information-dense data such as language modeling, where previous subquadratic models fall short of Transformers. It is based on the line of progress on [structured state space models](https://github.com/state-spaces/s4), with an efficient hardware-aware design and implementation in the spirit of [FlashAttention](https://github.com/Dao-AILab/flash-attention).

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Github Repo : [https://github.com/state-spaces/mamba](https://github.com/state-spaces/mamba) (licensed under Apache 2.0)

model checkpoints ; [https://huggingface.co/state-spaces/mamba-2.8b-slimpj](https://huggingface.co/state-spaces/mamba-2.8b-slimpj)
Author :  @tridao

Comments:
Hi @Tanmaypatil123, thanks for opening this new model request! 

There's an open PR to add Mamba here #28094 and an existing issue here #28086 

This was completed by #28094
PyTest import error
Author: loadams
Pull request: no

### System Info

Current head of transformers shows this issue, when importing functions from pytest, the `import_path` function is not found.  Sample error from DeepSpeed's unit tests [here](https://github.com/microsoft/DeepSpeed/actions/runs/7977730884/job/21781270161?pr=5164#step:7:391).

```
______________ ERROR collecting tests/deepspeed/test_deepspeed.py ______________
ImportError while importing test module '/tmp/actions-runner/_work/DeepSpeed/DeepSpeed/accelerate/tests/deepspeed/test_deepspeed.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../unit-test-venv/lib/python3.8/site-packages/_pytest/python.py:538: in importtestmodule
    mod = import_path(path, mode=importmode, root=config.rootpath)
../unit-test-venv/lib/python3.8/site-packages/_pytest/pathlib.py:566: in import_path
    importlib.import_module(module_name)
/opt/conda/envs/ptca/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1014: in _gcd_import
    ???
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:671: in _load_unlocked
    ???
../unit-test-venv/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:178: in exec_module
    exec(co, module.__dict__)
tests/deepspeed/test_deepspeed.py:26: in <module>
    from transformers.testing_utils import mockenv_context
../unit-test-venv/lib/python3.8/site-packages/transformers/testing_utils.py:129: in <module>
    from _pytest.doctest import (
E   ImportError: cannot import name 'import_path' from '_pytest.doctest' (/tmp/actions-runner/_work/DeepSpeed/DeepSpeed/unit-test-venv/lib/python3.8/site-packages/_pytest/doctest.py)
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 4.71s ===============================
```

### Who can help?

@pacman100 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

With pytest 8.0.1:
1. `from _pytest.doctest import import_path`
2. observe error.

### Expected behavior

No errors.

Comments:
Please run `pip3 install -e .[testing]`.

Hi @DonggeunYu - I believe the linked PR alone should be all we need here, the latest version of pytest (>= 8.0.1) have finally deprecated including import_path from _pytest.doctest to _pytest.pathlib, so this import path needs to be updated.
Solve: Inconsistent decoding with additional special tokens between slow and fast tokenizers.
Author: hisushanta
Pull request: yes

With this pull request, I have endeavored to remedy a minor decoding issue. If any problems remain with my proposed solution, I welcome your thoughtful feedback and suggestions for improvement.

Fixes #28287 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@ArthurZucker   @amyeroberts 



Comments:
Would you be able to check my solution?

I don't understand why this error occurs, but my code does the same work to define the issue section.

@hi-sushanta To figure out the errors, you'll need to look at the CI runs and debug locally so that the currently failing tests pass. For the quality checks, you'll need to fun `make fix-copies` and `make fixup`

But this command returns to the original codebase. that does not make sense because my code works exactly like the issues section mentioned.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Update pytest `import_path` location 
Author: loadams
Pull request: yes

# What does this PR do?

Fixes location of `import_path` from pytest from `_pytest.doctest` to `_pytest.pathlib` when using PyTest 8.0.1+ since it is finally deprecated from being in `_pytest.doctest`.  It is provided in `_pytest.pathlib` from at least 7.2.0+ so we do not need to modify the supported pytest range in `setup.py`

Tested [here in DeepSpeed](https://github.com/microsoft/DeepSpeed/pull/5164) and tests appear to be passing.

Fixes: #29155 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@muellerzr and @pacman100


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29154). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks @amyeroberts!  @ydshieh - could you review when you have time, I'd like to update DeepSpeed to the latest here when possible.

Hi @ydshieh, curious if you'd have some time to review this PR?

Hi @amyeroberts @pacman100 - gentle ping on being able to merge this PR?

I'm going to merge, as I'm pretty sure it's OK - @ydshieh for reference 

Thanks @amyeroberts!
[i18n-zh] Translate add_new_pipeline.md into Chinese
Author: windsonsea
Pull request: yes

Add zh text to this page:

```
docs/source/zh/add_new_pipeline.md
```
cc @stevhliu @Fan-Lin @jiaqiw09

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29432). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for your review. All suggestions are applied.

Hello @stevhliu, I am interested in becoming a member of Hugging Face. Could you please provide information on how to apply and where I can raise any issues or submit a pull request? Thank you.

Thanks for your interest! Feel free to check out the job postings [here](https://apply.workable.com/huggingface/?lng=en) and apply 🤗
🌐 [i18n-KO] Translated generation_strategies.md to Korean
Author: AI4Harmony
Pull request: yes

# What does this PR do?

Translated the generation_strategies.md file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179


## Before submitting
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. [[lowercased-header]])
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)


## Who can review?

As followed the guide, I would like to give a review request to @ArthurZucker, @sgugger, @eunseojo.
and also send the review request to Team PseudoLab just in case. @0525hhgus, @kihoon71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo.

## Before submitting

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x]  Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ]  Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x]  Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ]  Did you write any new necessary tests?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29086). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Send the reminder  to Team PseudoLab @0525hhgus, @kihoon71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo. Can you help me contribute to this repo?

And thank you for the quick response! @stevhliu 

Hi! @AI4Harmony 👋 I'm a member of Team PseudoLab.
I'll review your documentation soon, sorry for the wait.
[`Docs` / `Awq`] Add docs on exllamav2 + AWQ
Author: younesbelkada
Pull request: yes

# What does this PR do?

cc @IlyasMoutawwakil @ArthurZucker 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29474). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

lgtm thanks @younesbelkada
[`docs`] Add starcoder2 docs
Author: younesbelkada
Pull request: yes

# What does this PR do?

cc @loubnabnl @lvwerra @ArthurZucker 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29454). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Pydantic V2 support
Author: FanaticPythoner
Pull request: no

### Feature request

Please migrate to the latest Pydantic version.

### Motivation

The current library appears to be incompatible with Pydantic Version 2. I find that being able to utilize new features, such as the `model_dump` function, would be highly beneficial. This function is particularly useful for ensuring robust data validation in a scenario like mine, where JSON serialization of a Pydantic model is required for an HTTP request body.

### Your contribution

I am not able to perform the migration myself due to lack of time. Let me know if you guys need screenshots/tracebacks of errors when attempting to use more recent versions of Pydantic.

Comments:
Hi @FanaticPythoner, thanks for opening this feature request! 

Pydantic v2 is supported on the development branch c.f. #28728 #27933 

To use, install from source: `pip install git+https://github.com/huggingface/transformers`


Great thank you. Is there an ETA for the next release?

The schedule for releases is around once a month. The latest release was last week, so probably around 4 weeks or so. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Handling offload when calling AutoModelForCausalLM.from_pretrained()
Author: YourSaDady
Pull request: no

### System Info

Transformers version: 4.33.3
Python version:  3.9.18
Platform: Win11 WSL

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. `from transformers import AutoTokenizer,AutoModelForCausalLM`
2. `model = AutoModelForCausalLM.from_pretrained(args.model,device_map='auto')`
3. Error occurs: 
<img width="866" alt="error2" src="https://github.com/huggingface/transformers/assets/99607923/86201897-10a7-41b2-a1f8-3c961b198337">


### Expected behavior

Expected: model should be loaded without any errors.
I also tried to specify the offload folder by changing the line 67 to `model = AutoModelForCausalLM.from_pretrained(args.model,device_map='auto', offload_folder='offload')', but the program is killed without any error or tracebacks shown.

Comments:
Hi @YourSaDady, thanks for raising this issue! 

If the program is killed when using `offload_folder` it would indicate that you're hitting OOM. If you look at your GPU memory utilization - `watch -n1 nvidia-smi` - and your CPU utilization e.g. with`top` are you able to see these topping out when loading the model?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Generate: add tests for caches with `pad_to_multiple_of` 
Author: gante
Pull request: yes

# What does this PR do?

Adds tests for padding with `pad_to_multiple_of` (i.e. extra left-padding), on the different cache types. 

These tests were pulled from #29221, and should help us retain the correctness as we expand the capabilities of `generate`/`Cache` in other PRs (#29180, #29374).

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29462). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Large World Model and Ring Attention
Author: kabachuha
Pull request: no
Labels: name: New model

### Model description

Ring Attention is a new kind of attention that can expand the context size reliably (tested with precise needle search) up to several _million_ tokens. See https://arxiv.org/abs/2310.01889.

One of the models trained with RingAttention is multimodal Large World Model https://largeworldmodel.github.io/

I believe this attention replacement can be crucial for future LLMs (and not only), and should be included with the Transformers library

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

LWM checkpoints https://huggingface.co/LargeWorldModel
Ring Attention in PyTorch by Lucidrains https://github.com/lucidrains/ring-attention-pytorch
LWM in Jax https://github.com/LargeWorldModel/LWM

Comments:
Generate: get generation mode from the generation config instance 🧼
Author: gante
Pull request: yes

# What does this PR do?

 This PR moves `get_generation_mode` from a stand-alone function requiring a `generation_config` to a `generation_config` method. Also adds some missing documentation to public functions 🤗 

🧼 This PR adds no new code, except for a new test.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29441). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> This way printing the generation config always tells you which mode you will be using

That's a cool idea!

@ArthurZucker Actually I'm going to leave it as a todo -- because of assisted generation, whose trigger is not an attribute of the config, we can't store it cleanly as a property. Going to focus on more relevant feature for now :D