noamaanMulla-03/transformers-issues huggingface dataset:
Data prefetching does not occur for iterable datasets
Author: Lucaweihs
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.46.1
- Platform: macOS-15.1-arm64-arm-64bit
- Python version: 3.11.10
- Huggingface_hub version: 0.26.2
- Safetensors version: 0.4.5
- Accelerate version: 1.0.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No

### Who can help?

@muellerzr @SunMarc 

### Reproduction

PR #28498 was meant to allow for specifying the Pytorch dataloader's `prefetch_factor` argument via the huggingface `dataloader_prefetch_factor` training argument. As we can see [on this line](https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L989), this feature was added inside of a 
```python
if not isinstance(train_dataset, torch.utils.data.IterableDataset):
```
statement which results in prefetching never occurring for `IterableDataset`s (which seems like a mistake). There are a two other lines where the same error happens for [test](https://github.com/huggingface/transformers/blob/4e90b99ed916300b80bac9db793f2a96b2a87122/src/transformers/trainer.py#L1126) and [eval](https://github.com/huggingface/transformers/blob/4e90b99ed916300b80bac9db793f2a96b2a87122/src/transformers/trainer.py#L1084) dataloaders. Unless I'm missing something, I believe these lines can be moved out of the if condition and into other logic. 

### Expected behavior

Prefetching should work with iterable datasets.

Comments:
Thanks for the nice write-up. Would you be open to create a PR to fix this ? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add TextNet
Author: jadechoghari
Pull request: yes
Labels: name: New model

name: Vision

name: run-slow

# What does this PR do?

This PR adds TextNet which will be used to add Fast.

It builds up on the work of this PR: https://github.com/huggingface/transformers/pull/27425 (which was approved but not merged) and make it up to date with the library changes.

TODO:
- [x] Update the model's README file
- [x] Check why some tests are failing
- [x] Fix processing class errors
- [x] Fix modeling file errors


HF Model Cards:
TextNet-B: https://huggingface.co/jadechoghari/textnet-base
TextNet-S: https://huggingface.co/jadechoghari/textnet-small
TextNet-T: https://huggingface.co/jadechoghari/textnet-tiny

Notebook to replicate the author's logits:
https://colab.research.google.com/drive/1YsraOg-GHFh7PlvuIC9iJeBZquVWdz-r?usp=sharing




Comments:
cc @NielsRogge since you took the last PR - can you take a look at this one?

EDIT: I see you in the commits already, never mind!

cc @qubvel 🚀

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34979). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

🥳 

wohoo 🥳
VLMs: major clean up 🧼 
Author: zucchini-nlp
Pull request: yes
Labels: name: run-slow

# What does this PR do?

We have updated all the configs for VLMs on the hub so this PR removes legacy path for models, as it has been there for already 3 releases from v4.44. Also it fixes some stuff that broke on the way, like generating from only text input in LLaVA models

For Video-LLaVA the hub configs cannot be updated as the hub owner has been silent for several mmonths already. And since there is only one model with such architecture, we can hardcode the default values for `patch_num` and also remove the legacy path


fixes https://github.com/huggingface/transformers/issues/34824, fixes https://github.com/huggingface/transformers/issues/35169 and fixes https://github.com/huggingface/transformers/issues/35450, fixes https://github.com/huggingface/transformers/issues/35424

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34502). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

We can remove it after updating the files on the hub and that mean we also need to change warning to error so users have chance to see what is the reason for failure.

I think the earliest we can remove is next release, because the blocking PR will prob be merged next week. After that I will take time to update all hub configs. Maybe then we'll wait for the blocking PR and remove all deprecation warnings? 

Sounds good, let's wait a bit! 

@ArthurZucker i think this can be review now :)

Any updates on this PR? Looking forward to getting llava-hf/llava-v1.6-mistral-7b-hf running for inference without images.
Related to [this issue on HF](https://huggingface.co/llava-hf/llava-v1.6-mistral-7b-hf/discussions/36).

O o 👁️ 👁️ 

Triggered slow tests on some models, will merge when those pass (or at least it doesn't add up more failing tests...)

This is amazing! Let me know when it's merged so I can rebase Pixtral onto it

Oke, so BLIP models apparently have one extra model class which was not modified for some reason, and their official checkpoints on the hub are also not updated. Therefore, I am not adding BLIP to this PR, it will only remove legacy from LLaVAs

Slow tests are passing on my end if compared to the `main` branch, there are tests failing on `main` due to tiny logit inconsistencies. I believe it might also be my setup/hardware, we are trying to match the runner's outputs usually

I will merge this in a one hour, just last time slow test runs and done
support chat generator as input of TextGenerationPipeline
Author: lhoestq
Pull request: yes

Pipelines generally accept a generator input like `pipe(input for input in inputs)`. In this case the output is an iterator that yields results one by one as they arrive from the pipeline.

TextGenerationPipeline does accept a generator of strings `pipe(text for text in texts)`. Hoewever it doesn't accept a generator of chats `pipe(chat for chat in chats)` at the moment. This PR fixes this issue and adds support for `pipe(chat for chat in chats)`.

In particular this PR enables this use case with a nice tqdm bar :)

```python
>>> import pandas as pd
>>> from tqdm import tqdm
>>> df = pd.DataFrame({"text": ["Nvidia announces new RTX graphics cards"] * 10})
>>> prompt = "What is the main topic of this text ? REPLY IN LESS THAN 3 WORDS. Text: '{}'"
>>> df["main_topic"] = [
...     y[0]["generated_text"][1]["content"]
...     for y in pipe(([{"role": "user", "content": prompt.format(x)}] for x in tqdm(df["text"])), do_sample=False)
... ]
100%|███████████████████████████████████████████████████████████████████| 10/10 [00:05<00:00,  1.96it/s]
>>> df.head()
                                      text                    main_topic
0  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
1  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
2  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
3  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
4  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
```

currently it raises

```
File ~/hf/transformers/src/transformers/pipelines/text_generation.py:310, in TextGenerationPipeline.preprocess(self, prompt_text, prefix, handle_long_generation, add_special_tokens, truncation, padding, max_length, continue_final_message, **generate_kwargs)
    301     inputs = self.tokenizer.apply_chat_template(
    302         prompt_text.messages,
    303         add_generation_prompt=not continue_final_message,
   (...)
    307         **tokenizer_kwargs,
    308     )
    309 else:
--> 310     inputs = self.tokenizer(prefix + prompt_text, return_tensors=self.framework, **tokenizer_kwargs)
    312 inputs["prompt_text"] = prompt_text
    314 if handle_long_generation == "hole":

TypeError: can only concatenate str (not "list") to str
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35551). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix inference error in LlavaNextForConditionalGeneration with text-only input
Author: CatcherInThePy
Pull request: yes

# What does this PR do?
This simple fix addresses an issue where running inference with text-only input (a.k.a. without any image data ) causes an error on line 874, as `image_features` is `None`. The update restructures the checks for `image_features` and `legacy_processing` to properly handle cases where no image data is present, while preserving the integrity of the remaining processing steps. This is a suggested fix to the reported issue #35421


Fixes #35421


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
  - It is discussed via a Github issue: #35421
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
Vision models: @amyeroberts, @qubvel


Comments:
cc @zucchini-nlp 

Hello @zucchini-nlp,

That's fantastic news! Perfect timing as well ^-^ Thank you so much for the heads-up. I'm looking forward to more opportunities to contribute in the future. :) I am closing the PR.

Best.
[Docs] links to `logits-processor-zoo`
Author: gante
Pull request: yes

# What does this PR do?

Adds links to [`logits-processor-zoo`](https://github.com/NVIDIA/logits-processor-zoo) on relevant parts of our `generate` docs. 

`logits-processor-zoo` is an external library containing custom `LogitsProcessors`, and part of a recent [HF blog post](https://huggingface.co/blog/logits-processor-zoo).

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35552). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add support for modular with fast image processors
Author: yonigozlan
Pull request: yes

# What does this PR do?

Add support for using modular with fast image processors

Also add a fix for using modular on files other than modeling, which seem to have been broken by a recent change

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->


## Who can review?

@Cyrilvallez 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35379). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update missing model error message
Author: Rocketknight1
Pull request: yes

This PR updates the missing model error message to suggest installing from `main` for very new models. This can be important because a lot of users want to download new models, but often don't understand why it won't work!

Fixes #35362 

Comments:
cc @arthurzucker for review

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35370). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

gentle post-Christmas ping @arthurzucker! Lots of users got confused by this with the ModernBERT launch, so I want to make sure it doesn't happen again
ModernBERT does not have `inputs_embeds` input
Author: tbennun
Pull request: no
Labels: name: Feature request

### Feature request

Implement the `inputs_embeds` input field in ModernBERT and its variants.

### Motivation

As with other models (BERT, LLaMA, etc.), Transformers provides an option to feed the input embeddings directly, which is important for research on different input representations and embedding methods. This is also important for parity with BERT.

### Your contribution

I wrote some preliminary code that adds the input field. I also found a potential duplication of `_unpad_modernbert_input`. It appears both in the internal model and in the Masked LM version, which I think could be cleaned (@warner-benjamin thoughts?): https://github.com/huggingface/transformers/blob/main/src/transformers/models/modernbert/modeling_modernbert.py#L1055




Comments:
Pull Request #35373 is ongoing.

Sorry, for some reason GitHub search did not show that PR when looking up “ModernBERT”. Thanks for the pointer!
Fix Qwen2VL processor to handle odd number of frames
Author: jla524
Pull request: yes

# What does this PR do?

Fixes #35412 (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@ArthurZucker, @amyeroberts, @qubvel

Comments:
> Hey @jla524 !
> 
> Thanks for fixing! I think we should not hardcode `2` here as the temporal patch size, as the patch size is configurable when initializing the image processor. Can you generalize the logic for all temporal patch sizes?

Thanks for reviewing! Yes, it should use the actual value of the temporal patch size. I have fixed it and added a test.

( Checked that `    "tests/models/qwen2_vl/test_image_processing_qwen2_vl.py::Qwen2VLImageProcessingTest::test_video_inputs": {
        "tests_processors": "passed"
    },` passed!
iframe
Author: Hamed744
Pull request: no
Labels: name: bug

### System Info

Test

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

<iframe
    src="https://hkchengrex-mmaudio.hf.space"
    frameborder="0"
    width="850"
    height="450"
></iframe>

### Expected behavior

Hello, what is the reason for the issue of the application not functioning in an iframe for some of the spaces? For example:


<iframe
    src="https://hkchengrex-mmaudio.hf.space"
    frameborder="0"
    width="850"
    height="450"
></iframe>
When we use this in a webpage, the application does not work. Is there a solution to run it inside the iframe? Since I can't use Web components because the Gradio library is not available in my region. Is there a way to run that application using an iframe?

Comments:
This should probably be asked on the [Gradio repo](https://github.com/gradio-app/gradio) instead!
add code owners
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Adds code owner to make sure reviewers are pinged accordingly!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35528). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Being lazy to lessen my workload, could we automatically generate the mapping of models to owners, or manually generate it once and then add a CI test to ensure that new model additions have a matching CODEOWNERS line? This would mean that dispatch would happen automatically, instead of me manually doing it every time

If you want, I can write a script to guess which category each model is, and then we can fix up that list afterwards

For new model the issue is that we need to know if it's a vision model, language model or audio model etc. I can do it if you prefer 😉 

Yes, I know! But we could just add a CI test that raises an error if the model isn't in CODEOWNERS, and then the PR author knows they need to add a line to assign it to the right person. I can add some comments in the file to tell them who to assign it to (or you can do it!) WDYT?
ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation 
Author: amyeroberts
Pull request: no
Labels: name: New model

name: Vision

### Model description

ViTPose is used in 2D human pose estimation, a subset of the keypoint detection task #24044

It provides a simple baseline for vision transformer-based human pose estimation. It utilises a pretrained vision transformer backbone to extract features and a simple decoder head to process the extracted features. Despite no elaborate designs in the model, ViTPose obtained state-of-the-art (SOTA) performance of 80.9 AP on the MS COCO Keypoint test-dev set.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Code and weights: https://github.com/ViTAE-Transformer/ViTPose
Paper: https://arxiv.org/abs/2204.12484

@Annbless

Comments:
Glad you get something different to work on 🚀 👀 🎉 

Hi, @amyeroberts, I don't know if you are working on this but if not I would be more than happy to take it up.

Oh, this is the issue page, not the PR page!

@shauray8 You're very welcome to take this up! :) 

This model presents a new task for the library, so there might be some iterations and discussions on what the inputs and outputs should look like. The model translation should be fairly straightforward though, so I'd suggest starting with a PR that implements that and then on the PR we can figure out what works best.

Very excited to see this implemented! And I'm happy to contribute if needed. I'll play with #30530 and see if I find issues.

Update: The branch works really well! And the code is very clean. I love this! Can't wait to see it release. 

I made a [Space](https://huggingface.co/spaces/Dref360/vit_pose_playground) here to play with it. Even on CPU it's pretty fast.

I think we can close this as #30530 has been merged! 🥳 🥳 

Closing since this model has now been added in #30530
[docs] Remove Hiera from AUDIO MODELS in docs
Author: ADAning
Pull request: yes

Hiera is a visual model and should not appear in audio model...

<img width="847" alt="image" src="https://github.com/user-attachments/assets/d1e90f8f-5aee-42bf-8c86-a92e0b25c3a3" />

We should remove Hiera from  [_toctree.yml](https://github.com/huggingface/transformers/blob/12ba96aa3cb3e4ed2a3ffb77b59f53f8ce9ac1fa/docs/source/en/_toctree.yml#L757C26-L757C31).

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).



Comments:
Hi @ADAning, thanks for the fix! Can you:

1) Check that Hiera doesn't appear in any other incorrect categories
2) Make sure it's added to the correct category instead?

> Hi @ADAning, thanks for the fix! Can you:
> 
> 1. Check that Hiera doesn't appear in any other incorrect categories
> 2. Make sure it's added to the correct category instead?

1. Hiera is only incorrectly present in the Audio model, and seems to be incorrectly present since it was added to the transformers, see [there](https://github.com/huggingface/transformers/blame/b05df6611e6e3e6834acca2b50baeb7cdd5fbe3c/docs/source/en/_toctree.yml#L760).
2. Hiera is currently in the vision model, which is its correct category, see [there](https://github.com/huggingface/transformers/blob/b05df6611e6e3e6834acca2b50baeb7cdd5fbe3c/docs/source/en/_toctree.yml#L675C16-L675C21).