noamaanMulla-03/transformers-issues huggingface dataset:
AttributeError: 'Config' object has no attribute '_get_non_default_generation_parameters'
Author: XYZ-916
Pull request: no
Labels: name: bug

### System Info

transformers version: 4.47.1
I finetuned the rmbg-2.0 with my own dataset. When I save the model using model.save_pretrained, I got the error as follows:
 File "/usr/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2815, in save_pretrained
    misplaced_generation_parameters = model_to_save.config._get_non_default_generation_parameters()
AttributeError: 'Config' object has no attribute '_get_non_default_generation_parameters'

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. finetune rmbg-2;
2. save model.

### Expected behavior

save the model sucessfully

Comments:
Hi @xyz-916, can you create some simple reproducer code we can run to diagnose this? For example, just load and save the model or even just the config and check if the bug still occurs

Hi @Rocketknight1, I encountered the same issue when trying to save a pretrained model.

Here is an example to reproduce the problem:

- transformers==4.48.1
- torch==2.2.0+cu121
- CUDA 12.2

```
from transformers import AutoModelForImageSegmentation

model = AutoModelForImageSegmentation.from_pretrained(
    'ZhengPeng7/BiRefNet',
    trust_remote_code=True,
)

// This line raised AttributeError: 'Config' object has no attribute '_get_non_default_generation_parameters'
model.save_pretrained("./BiRefNet/")
```

Hi @cycycc, the problem with `BiRefNet` is caused by them using a config class that does not inherit from `PretrainedConfig`. Even though they have a [BiRefNet_config.py](https://huggingface.co/ZhengPeng7/BiRefNet/blob/main/BiRefNet_config.py) file, the actual config class used by the model is defined in the main modeling file [here](https://huggingface.co/ZhengPeng7/BiRefNet/blob/main/birefnet.py#L7). As a result, methods like `save_pretrained` and `_get_non_default_generation_parameters` do not exist on their config object, which means `model.save_pretrained()` fails.

There's not a lot we can do about this in `transformers` itself - you'll have to open an issue on their repo and ask them to make a config class that inherits from `PretrainedConfig`!

I see now - you’re right. I didn’t realize before that not all models on HuggingFace have to inherit from its classes.

Thank you @Rocketknight1 for your prompt reply!

@cycycc they're supposed to, and a lot of features break if that inheritance isn't there, as you have unfortunately discovered 🙃 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
torch compiled flex attention
Author: samsja
Pull request: yes

# fix flex attention implementation

I encounter terrible performance when training Llama 8b transformer with flex_attention. Got 15% mfu on a workload where I usually have 45%. Switching to flash_attention_2 fixed the issue. 

```python
config_model = LlamaConfig.from_pretrained("meta-llama/Meta-Llama-3-8B", attn_implementation="flex_attention")
model = LlamaForCausalLM.from_pretrained(pretrained_model_name_or_path="meta-llama/Meta-Llama-3-8B", config=config_model)
```

It seems that the flex_attention implementation is wrong. It does not compile the flex_attention function from pytorch, which is necessary to reach fa2/3 performance. Even when using torch compile on the model itself I saw low mfu (15%) which suggested that torch compile break before reaching flex_attn call

This PR fixes it by adding a torch compile on the flex attention. Mfu increase to 42%.

![Screenshot from 2025-02-28 18-33-39](https://github.com/user-attachments/assets/38c6d64d-9230-4744-8a37-9fb9ab49ce52)


The flex attention [blog](https://pytorch.org/blog/flexattention/) post explains that the speed up is mainly coming from the use of torch compile, and other reference 

It seems that the llama transformers implementation is still underperforming compared to flex_attn on the torch titan codebase (reach 45% mfu that I cannot reach when using transformers llama).

IMO, it would be nice to be able to reach good performance on transformers llama as it is the default implementation of a lot of codebases. I think that there are a lot of graphs that break when using torch compile, probably because of some parameters that are not used during LLM training (attention_mask, etc).

PS the code that I used for benchmark is on this pr : https://github.com/PrimeIntellect-ai/prime/pull/224


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?


## Who can review?


Models:

- text models: @ArthurZucker






Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

ah nice will close this PR then
Speed up image processors - cast to array before BatchFeature
Author: amyeroberts
Pull request: no
Labels: name: Good Second Issue

name: Feature request

### Feature request

Complete work initially started in #28221

### Motivation

Converting the model outputs to an array before creating the `BatchFeature` object can speed up image processing, especially for nested outputs, such as for videos. 

### Your contribution

Open to the community to add

Comments:
Hi @amyeroberts , If this Issue is still open then I would like to work on it. If so, can you provide me some more context to work on. 

Hi @yaswanth19, apologies for the delay in response. 

The idea is similar to what's done here in [idefics 3](https://github.com/huggingface/transformers/blob/e7c8af7f338eab42bd6615f77da5b69094b4836e/src/transformers/models/idefics3/image_processing_idefics3.py#L877), where the inputs are converted to numpy arrays before passing to BatchFeature. 

I'm going to be off for a while, so if you're still keen to take this on, @yonigozlan is the best person to ping for first review 🤗 

Hi @amyeroberts @yonigozlan , is this issue still open? If so, could I work on it?

@csking101 I am working on it and will raise the PR soon

> @csking101 I am working on it and will raise the PR soon

Sure, thanks for informing me!

Hi, is this issue solved?

@Aravind-11 Nope AFAIAK, I started some work and raised a PR but later closed it. Checkout my above PR for a good starting point.
Fix _load_state_dict_into_meta_model with device_map=None
Author: hlky
Pull request: yes

# What does this PR do?

https://github.com/huggingface/diffusers/actions/runs/13604967458/job/38035399161?pr=10898

```python
>       device_map_regex = "|".join(sorted(device_map.keys(), reverse=True))
E       AttributeError: 'NoneType' object has no attribute 'keys'
```

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36488). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Support SDPA & Flash Attention 2 for LayoutLMv3
Author: stancld
Pull request: no
Labels: name: Feature request

### Feature request

Support SDPA & Flash Attention 2 for LayoutLMv3.

### Motivation

Speed up document transformer training and inference.

### Your contribution

I can add it.

Comments:
++ This will drastically improve the speed of LayoutLmv3.
I see the PR is already raised
Add Wan model into Transformers
Author: VladOS95-cyber
Pull request: no
Labels: name: New model

### Model description

Wan2.1, a comprehensive and open suite of video foundation models that pushes the boundaries of video generation. Wan2.1 offers these key features:

SOTA Performance: Wan2.1 consistently outperforms existing open-source models and state-of-the-art commercial solutions across multiple benchmarks.
Supports Consumer-grade GPUs: The T2V-1.3B model requires only 8.19 GB VRAM, making it compatible with almost all consumer-grade GPUs. It can generate a 5-second 480P video on an RTX 4090 in about 4 minutes (without optimization techniques like quantization). Its performance is even comparable to some closed-source models.
Multiple Tasks: Wan2.1 excels in Text-to-Video, Image-to-Video, Video Editing, Text-to-Image, and Video-to-Audio, advancing the field of video generation.
Visual Text Generation: Wan2.1 is the first video model capable of generating both Chinese and English text, featuring robust text generation that enhances its practical applications.
Powerful Video VAE: Wan-VAE delivers exceptional efficiency and performance, encoding and decoding 1080P videos of any length while preserving temporal information, making it an ideal foundation for video and image generation.

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

Implementation: https://github.com/Wan-Video/Wan2.1
Weights: https://huggingface.co/Wan-AI/Wan2.1-T2V-14B

Comments:
@qubvel @zucchini-nlp @ArthurZucker. I think I could work on it if nobody from HF team is working on it already.

@VladOS95-cyber Wan is being supported by [Diffusers 🧨](https://github.com/huggingface/diffusers).

@hlky very good, missed that
`padding_side` is of type `bool` when it should be `Literal['right', 'left']`
Author: winstxnhdw
Pull request: no
Labels: name: bug

### System Info

main branch

### Who can help?

@amyeroberts 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://github.com/huggingface/transformers/blob/main/src/transformers/tokenization_utils_base.py#L2816

### Expected behavior

Should be `Literal['right', 'left']` as mentioned in the docs.

https://huggingface.co/docs/transformers/en/main_classes/tokenizer#transformers.PreTrainedTokenizerFast.__call__.padding_side


Comments:
@winstxnhdw oops, that's a typo. Can you open a PR to fix that?

Fixed in #36326
please support register_full_backward_pre_hook and register_full_backward_hook
Author: rubbberrabbit
Pull request: no
Labels: name: Feature request

### Feature request

support  pytorch.module register_full_backward_pre_hook and register_full_backward_hook

### Motivation

I am use huggingface transformer and pytoch, I am trying to add some hooks in the state of model backward computation, it works well in original torch module. But with huggingface model I got 

> UserWarning: For backward hooks to be called, module output should be a Tensor or a tuple of Tensors but received <class 'transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions'>

detail:

It seems that hugging face change the backward function output, my hook example like that
`
    def pre_backward_time_hook(module, grad_output):
        # do something like print the grad_output
        print(grad_output)
        return None
`
it fit the request of hook in register_full_backward_pre_hook

hook(module, grad_output) -> tuple[Tensor] or None

but got 

> UserWarning: For backward hooks to be called, module output should be a Tensor or a tuple of Tensors but received <class 'transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions'>

by the way register_backward_pre_hook works well cause it wont be checked by the output type.I am not sure if I misuse it or cause by hugging face model defination, glad to know the solution.

### Your contribution

N

Comments:
Add AIMv2 to Transformers
Author: AlanPonnachan
Pull request: yes
Labels: name: New model

name: Multimodal

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #35351

This PR adds [AIMv2](https://arxiv.org/abs/2411.14402) support in Transformers. AIMv2 showed better performance than SigLIP.

## TODO
- [x] Completed almost every steps mentioned in this [guide](https://huggingface.co/docs/transformers/main/en/add_new_model)



## Who can review?

@qubvel
@Rocketknight1

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
 @qubvel @Rocketknight1 Could you help review this PR ?  Let me know if you have any suggestions. Thank you in advance for your time and assistance.

@AlanPonnachan Are you still working on this and if you don't have the bandwidth then I can take it from here and refine the code.

@yaswanth19 you can try to do modular converter part. I am facing some issues on that part. Further I can help you
Fix loading zero3 weights
Author: muellerzr
Pull request: yes
Labels: name: bug

# What does this PR do?

https://github.com/huggingface/transformers/pull/36335 broke zero3 weight loading because we can't do TP for it and it needs to be how the methods were done originally. 

Related fixings: https://github.com/huggingface/transformers/issues/36441


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @SunMarc 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36455). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

when is this gonna be merged ?

thanks @muellerzr 
failing tests seem to be coming from main, they first appeared when unsilenced in https://github.com/huggingface/transformers/pull/36449

Yeah @ArthurZucker needs to merge it b/c of failings :)
Check `TRUST_REMOTE_CODE` for `RealmRetriever` for security
Author: ydshieh
Pull request: yes

# What does this PR do?

Similar to #27776 and other similar fixes.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36511). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
GRPO Reward Weight Scheduler
Author: leonardtang
Pull request: no
Labels: name: Feature request

### Feature request

It would be great to support dynamic weights for aggregating rewards -- i.e. different weightings based on how deep into training we have progressed.

### Motivation

There are often rewards that we can use for local updates that don't make sense globally in terms of their magnitude. 

For example, one potential reward is to rank the set of rollouts and assign the #1 ranking a max reward of 1 and the last ranking a min reward of 0. This is useful *locally* when true rewards are sparse, but becomes distracting in the limit of training (the #1 ranking always gets a reward of 1).

It is possible to schedule the reward function itself, but that seems not as clean. Also, the logs for that reward function would be misleading. 

### Your contribution

Should be easy for me to submit a PR for this, but thought it's worth flagging explicitly here for feedback

Comments:
@leonardtang I think TRL library would be a better place to ask this query as that library supports GRPO.

TRL: https://github.com/huggingface/trl 

> @leonardtang I think TRL library would be a better place to ask this query as that library supports GRPO.
> 
> TRL: https://github.com/huggingface/trl 

Ha of course, great point. Will do

Closing since this discussion is moving to another repo!
add a param to control cache in streamer when return output
Author: ExcitingFrog
Pull request: no
Labels: name: Feature request

### Feature request

https://github.com/huggingface/transformers/blob/dcbdf7e962c4b36140cc9ee76f870016121e69e5/src/transformers/generation/streamers.py#L102    put() under class TextStreamer will save serval tokens in cache, then return serveral token str in response.  need a paramer to control token cache on or off.

code example
```python
        >>> generation_kwargs = dict(inputs, streamer=streamer, max_new_tokens=20)
        >>> thread = Thread(target=model.generate, kwargs=generation_kwargs)
        >>> thread.start()
        >>> generated_text = ""
        >>> for new_text in streamer:
        ...     generated_text += new_text
        >>> generated_text
```

### Motivation

special token and common token will return in same time in my project, for example  `123
`  return in one time.we need to handle this situation. we create a new class and remove part of cache code.
i wonder to know if necessary to add a param to control streamer cache or another way to handle it.



### Your contribution

i can submit a pr , it it is necessary to change it.

Comments:
cc @gante
Fix kwargs UserWarning in SamImageProcessor
Author: MSt-10
Pull request: yes
Labels: name: Vision

# What does this PR do?
Fixes a user warning when using the SamProcessor. 

```
transformers/image_processing_utils.py:41: UserWarning: The following named arguments are not valid for `SamImageProcessor.preprocess` and were ignored: 'point_pad_value'
```

The [SamProcessorKwargs](https://github.com/huggingface/transformers/blob/51083d1bac7905aa8316b75f7897bdd4e5302044/src/transformers/models/sam/processing_sam.py#L45) contain the argument `point_pad_value` but this is only needed by the SamProcessor and not by the SamImageProcessor, hence a warning is generated when calling the SamImageProcessor [here](https://github.com/huggingface/transformers/blob/51083d1bac7905aa8316b75f7897bdd4e5302044/src/transformers/models/sam/processing_sam.py#L110). 
I removed the argument from the dictionary similarly to how e.g. `input_points` are handled since they are also not needed by the SamImageProcessor.


## Who can review?
@qubvel 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

Thanks for the PR @MSt-10!

run-slow: sam

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['models/sam']
quantizations: [] ...
Add MultipleChoice & QuestionAnswering heads to ModernBERT
Author: warner-benjamin
Pull request: yes

This PR adds the ForQuestionAnswering and ForMultipleChoice heads to ModernBERT. It also updates the existing heads to use the `self.loss_function` loss implementations where one exists.

At the moment I have removed the full model sequence packing inputs from the `ModernBertForMultipleChoice` head, as I don't think we can guarantee they would be correct post flattening. I think it would be better to have `ModernBertBase` handle unpadding for this head to reduce complexity and edge cases.

cc @ArthurZucker, @tomaarsen 



Comments:
Hi @warner-benjamin, we already have a PR for the QuestionAnswering head at #35566, which should have been merged by now, but I'm slow at reviewing

We definitely appreciate the `MultipleChoice` head, though - if you can trim this PR to just add that, I'd be happy to review/accept it!

Given the status of #35566, would it be easier to just merge this one? @Rocketknight1 

@orionw up to you! If you prefer to close yours and just merge this instead that's fine.