noamaanMulla-03/transformers-issues huggingface dataset:
[Modular] skip modular checks based on diff
Author: gante
Pull request: yes

# What does this PR do?

✨ Faster modular checks on most PRs ✨ 

It takes advantage of having the modular checks being done in topological order. The loop over files in topological order was modified to:
1. if a given model and its dependencies are not present in the diff, it's guaranteed to not have any issues -> skips checks. This check is conservative: any diff in any file related to the model disables the skip.
2. otherwise, run the modular checks. 
3. if the model was overwritten, update the list of models in the diff to handle dependency chains
4. (next iteration: check the next model in the topological order)

In branches with zero model changes, it shaves off ~1min [on my machine]. With one modular leaf model change, it shaves off ~55 secs. 🤗 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36130). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Keyword arguments {'add_special_tokens': False} not recognized.
Author: AAnminy
Pull request: no

I am a newbie, and I apologize for the stupid questions below.
When I run trainer.py for step, inputs in enumerate(epoch_iterator):, my terminal will output many, many lines, up to more than 1,000 lines of "Keyword arguments {'add_special_tokens': False} not recognized." I am using transformers=4.38.0, pytorch=2.1.2+cu11.8, what is the reason? How can I solve it?

![Image](https://github.com/user-attachments/assets/c1959362-27dd-4737-b806-b24e8ce8f5b6)

Comments:
Hi @AAnminy it's not related to `trainer` but to tokenization. You're sending a wrong keyword argument to your tokenizer method somwhere in your data preparation/collation, see here  https://github.com/huggingface/transformers/blob/1779f5180edfb048dad1ba406fe6cca942b47683/src/transformers/tokenization_utils.py#L643 where the warning you see comes from. 
I suggest testing your data collation/preprocess before launching the training. Closing this issue for now! 

 Thanks molbap's guidance. I carefully checked each version of transformers and found that upgrading the version to 4.38.2 and above can solve this problem! !
fix torch.export on torch=2.5.1 for llama
Author: ziereis
Pull request: yes

This pr fixes torch.export for llama models and torch=2.5.1

In transformers 4.46.3 the loop was:

```
for decoder_layer in self.layers:
```

which got changed in 4.48 to:

```
for decoder_layer in self.layers[: self.config.num_hidden_layers]:
```

which works with torch=2.6 but breaks using torch 2.5.1

error on torch=2.5.1:

```
  File "/home/ziereis/forks/transformers/src/transformers/models/llama/modeling_llama.py", line 577, in forward
    for decoder_layer in self.layers[: self.config.num_hidden_layers]:
                         ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ziereis/conda/envs/roofdev/lib/python3.11/site-packages/torch/nn/modules/container.py", line 332, in __getitem__
    return self.__class__(list(self._modules.values())[idx])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: _ModuleStackTracer.__init__.<locals>.AttrProxy.__init__() missing 1 required positional argument: 'path'
```


Comments:
Feels like a core maintainer thing - cc @arthurzucker @Cyrilvallez ?

i already referred to it here https://github.com/huggingface/transformers/issues/36131 but after making the change in the modeling_llama.py file and  running the make commands as stated in the CONTRIBUTING.md it does completely unrelated changes.

Hi @ziereis, I just tested by making the change to `modeling_llama.py` in my own repo and then running `make fix-copies`, and it only copied the change in those two lines to other files. The unrelated changes I saw in this PR did not occur.

I'm not sure how these weird changes happened - possibly you rebased the repo while `make fix-copies` was running? Either way, I would suggest:

1) Make sure your `main` branch has pulled the latest commits
2) Reset your PR branch to the latest `main` commit (or just close your current PR branch and open a new one)
3) Make the change in `modeling_llama.py`
4) Run `make fix-copies` again

That should resolve the issue, and then the PR should be ready for review! 

@Rocketknight1 thank you for the reply. I think it was some i issue in my env. i created a new ones and it worked as expected. Sorry for the trouble! 

actually cant even reproduce the problem with it not tracing in the new env so this can be scrapped all together. Sorry!

No probs, and thanks for updating us!
Fix : fix doc fp8
Author: MekkCyber
Pull request: yes

# What does this PR do?
Simple fix for the fp8 doc, changing the config class 

## Who can review ?
@SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36173). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[generate] revert change in Aria: the maximum cache length must match `max_length`
Author: gante
Pull request: yes

# What does this PR do?

Reverts `generate` changes from https://github.com/huggingface/transformers/pull/34157. 

In decoder-only models, the `max_length` flag in `generate` is related to the model's output sequence length. With the added changes, it was no longer respected when `inputs_embeds` was passed without `input_ids`, i.e. it resulted in model output length != max length.

[At a first glance, I can't see why Aria would need it -- the fasts tests are passing, which means the API for Aria is also consistent without the change. If Aria needs this line of code to work well on all cases, we'll need to find another solution.]

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36120). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add phi3 Vision
Author: ArthurZucker
Pull request: no
Labels: name: New model

name: Good Difficult Issue

### Model description

Model is here: https://huggingface.co/microsoft/Phi-3-vision-128k-instruct with code, weights and paper! 🚀 

### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
We have a contributor PR open for the 3.5-vision models already! https://github.com/huggingface/transformers/pull/36036
fix training issues
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #35990! Paligemma did not have `past_key_values` 

Comments:
cc @muellerzr and @SunMarc WDYT about just by default setting this for all PreTrainedModels? (If we forget about it, people are not bothered

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36158). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yeah sounds good to me @ArthurZucker. Maybe we can put it here since we only use it in trainer. This way, we don't add it in the config of models that potentially dont need it.

```python
        if ignore_keys is None:
            if hasattr(self.model, "config"):
                ignore_keys = getattr(self.model.config, "keys_to_ignore_at_inference", ["past_key_values"])
```
CI: avoid human error, automatically infer generative models
Author: gante
Pull request: yes

# What does this PR do?

This PR:
1. Replaces manual definition of generative models to test with `generate` with an automatic definition -- if `model_class.can_generate()`, then it runs tests from `GenerationTesterMixin`. No more human mistakes, which happened frequently in the past months 🐛 🔫 
2. Now that we run generation tests for all models that can generate, there are a few (old) bad apples. Explicitly skip them i.e. overwrite the automated `all_generative_model_classes` and explain why certain classes are being skipped. (bad apples detected with `py.test tests/models/ -k test_greedy_generate -vv`) 💔 

_____________________________________________
In a follow-up PR:
1. ~We need to manually define the model's main input name (e.g., `pixel_values`) in the model tester. Make it use `model.main_input_name` instead, to avoid human error~ Done ✅ 
2. Despite the changes in this PR, `generate` tests will only run if `GenerationTesterMixin` is inherited. We can easily forget to add the mixin, resulting in a false positive. Add an automated check: if any of the model classes can generate, then `GenerationTesterMixin` must be inherited in the tester

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33212). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Super cool! If VLMs start failing on generation tests, that's okay. I have a local draft for that, which needs to be reviewed after a few other PRs are merged

Many side-fixes later, ready for review 🙌 


@ydshieh all comments addressed 🤗 

> Thank you again.
> 
> Just to make sure the change in `tests/test_modeling_tf_common.py` is what you expect:
> 
> * removing `all_generative_model_classes = ()` OK
> * not adding `def all_generative_model_classes(self):` : if this is intended, OK, for you to check

ah , I see you put it back :-), all good then

@gante you can ping me to merge if the CI persists to be red but irrelevant to your PR :-)

@ydshieh yes, please merge 🙏 

(was trying to be autonomous :D)
Fix the key name for _load_rng_state under torch.cuda
Author: wizyoung
Pull request: yes

# What does this PR do?

Under the torch.cuda environment, train.py uses cuda as the key to save rng_state:

https://github.com/huggingface/transformers/blob/5489fea557b10fc8f21baab6e96d3b6b3b42a6c5/src/transformers/trainer.py#L3170-L3175

However, it uses "GPU" key to load rng_state when resuming. This breaks the resuming scheme. This pr fixes the name here. I think using "CUDA" is better than "GPU" as it keeps consistency with torch.

https://github.com/huggingface/transformers/blob/5489fea557b10fc8f21baab6e96d3b6b3b42a6c5/src/transformers/trainer.py#L3069-L3070


## Who can review?
@muellerzr @SunMarc



Comments:
@wizyoung : can you, please, clarify how to reproduce the failure you observe? do you have simplified reproducer or can you point to the failing test on HF side?

@dvrogozh You can do a simple train with save steps like 100 steps, and then resume from the checkpoint. There will be warnings like saying key error 'gpu'. The warning does not breaks the training but the cuda rng states can not be properly loaded.
Let me explain why this is a obvious bug:
(1) In `_save_rng_state` during training, the rng dict uses the key 'cuda'
![image](https://github.com/user-attachments/assets/4568b301-4265-4d6f-b272-d011969df116)
(2) When resuming, `_load_rng_state` passes key 'GPU' to `set_rng_state_for_device` to load rng states.
![image](https://github.com/user-attachments/assets/9be49f69-6ac3-4e4c-a951-9033cffd9012)
let's check `set_rng_state_for_device`, the lowercase key 'gpu' is not in the rng dict, then we will see logged errors at the end.
![image](https://github.com/user-attachments/assets/238e7f80-dccf-4156-a35a-5250c7bdad8b)

Actually, the name was changed from 'cuda' to 'gpu' in recent commits.


The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36138). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Follow up to SpQR integration
Author: MekkCyber
Pull request: yes

# What does this PR do?

Follow up PR to SpQR integration to specify `requires_calibration = True` because models are needed to be quantized

## Who can review ?

@SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36176). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix a mistake in #36175
Author: ydshieh
Pull request: yes

# What does this PR do?

I forgot to remove a line that is used to benchmark ... sorry

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36179). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix AutoProcessor import order issue with custom classes
Author: Rocketknight1
Pull request: yes

Fixes #34307

When initializing via AutoImageProcessor before AutoProcessor is imported, an AttributeError could occur if the model used custom processor classes. This happened because the code tried to import the processor class from the root transformers module, which failed for custom classes.

This PR modifies the `_get_arguments_from_pretrained` method to look for custom classes in the config object if they are not found in the root module. This allows models to define their own processor classes without having to modify the transformers module itself.

Added a test case to verify the fix works as expected.

Comments:
This is another test of using OpenHands to resolve issues, this time on a more complex one where the solution wasn't clearly laid out in the issue. The PR and comment above were written by the bot. I haven't reviewed it yet!
Fix make_batched_videos and add tests
Author: yonigozlan
Pull request: yes

# What does this PR do?

Fix an issue with the `make_batched_videos` function in image_utils, and add/modify tests to properly test it.

<!--
## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36143). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Will come back to it soon, but very much similar to @orrzohar 's PR for SmolVLM2 I guess. @orrzohar do you think this is how you process videos?


> Looks good to me! Currently we have a more involved loading for SmolVLM in https://github.com/huggingface/transformers/pull/36126#pullrequestreview-2611207275, so I think we need a general function to support both models
> 
> Feel free to jump in the above comment thread with ideas/feedback. We definitely would need to ship video loading as a separate PR

Yes having a `sample_indices_fn` seems like a much nicer solution! Instead of adding kwargs for every possible frame sampling methods 😅. I'll change this PR to just fix the `make_batched_videos` function and adding the test. 

Yep, a general solution would save us a lot of maintainment energy, hopefully
Update Qwen2.5-VL no split modules
Author: hiyouga
Pull request: yes

# What does this PR do?

Fix the `no_split_modules` attribute of Qwen2.5-VL models to avoid error in FSDP training.

Although the `Qwen2_5_VLDecoderLayer` class does not exist in the modular file, it will appear at the modeling file that is actually used.

https://github.com/huggingface/transformers/blob/0ed83df540c2f4964eede7b94ab85554308a5544/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1050-L1059


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker 


Comments:
Sorry just saw it! I think it was fixed on main! Thanks for the PR 🤗 

Yeah it has been fixed! thank you for review
Can't access Huggingface page and cant download models from laptop. Can it be IP ban?
Author: Rajmehta123
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.3
- Platform: Darwin-20.6.0-x86_64-i386-64bit
- Python version: 3.7.9
- PyTorch version (GPU?): 1.9.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
@LysandreJik


### Error
ERR_NAME_NOT_RESOLVED 
![image](https://user-images.githubusercontent.com/22636443/139662198-9a8c1b4e-a920-435e-bf6b-11b755299c6c.png)

Dev tools response when loading the page
![image](https://user-images.githubusercontent.com/22636443/139662006-2b4b114e-6755-4693-b947-b321925852d2.png)

Cannot download pre-trained models from python
`ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.`


When I access the page from another device connected to the same network, it works. I could access the site few days back.

Comments:
Hello! We don't IP ban. Could it be linked to your OS's DNS cache? cc @Pierrci 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

 even i m facing the same issue , is there any way to resolve it ?

Okay so i changed my wifi and then it started working , for some reason i cant access it from a certain network 


I am experiencing the same issue  as of 14 hours ago

```
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>403 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Request blocked.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: Ia2BsSzLNtQxWlurd1LzsXCV53YfR2xzFOJG24dU-zEOhxAtcnNARA==
</PRE>
<ADDRESS>
</ADDRESS>
</BODY></HTML>
```

Using the same token from another (external) IP works just fine
run_audio_classification.py freeze_base_model in w2v-bert
Author: HERIUN
Pull request: yes

# What does this PR do?
In audio classification using wav2vec. it needs freeze_base_model in w2v-bert


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ylacombe

Comments:
cc @eustlb
Chat template docs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Same as https://github.com/huggingface/transformers/pull/35657/ with some additions to Multimodal docs. We recently added batch support and some new kwargs for video sampling

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36163). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
CI: fix `test-save-trainer`
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, fixes the test. The default directory is indeed called `trainer_output`, so the test is not correctly checking it

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36191). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Prepare processors for VideoLLMs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As discussed internally, these changes will help SmolVLM2 to be consistent with how processors usually work for other VLMs and lay ground for other video-LLMs in the future. The current API for chat templates should be more flexible now to handle special models

TODO:
- [x]  allow video decoders to accept a callable
- [X] Tests to cover new cases (more tests should be added with SmolVLM's processor)


At the end we should have the below working before release. We might do a short cut by overriding `apply_chat_template` if some things fail on the way, and then refactor out after release

```python
inputs = processor.apply_chat_template(history, num_frames=8, sampling_fps=1.0, skip_secs=1.0, return_tensors='pt', **kwargs)
model.generate(**inputs)
```
 
cc @molbap @yonigozlan 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36149). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.