noamaanMulla-03/transformers-issues huggingface dataset:
Fix torchao usage
Author: jiqing-feng
Pull request: yes

Fix #37035 

Fix torchao usage including config and save-load and tests.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @SunMarc @ydshieh . I have fixed the torchao config by saving the layout into a list ([layout_name, layout_kwargs]). Please review the new changes. Now the torchao tests can all pass.

For the new changes, SunMarc is much more qualified for the review 🙏
Adding ArlowGPT
Author: yuchenxie4645
Pull request: yes

This PR adds support for upcoming model & tokenizer for ArlowGPT. 

---

##  Add Arlow Model & Tokenizer Support

This PR adds support for the **Arlow** model and its corresponding **ArlowTokenizer**.

### Features:
- Flash Attention 2 for fast memory-efficient training  
- Rotary Position Embeddings (RoPE) with `rope_theta=100000.0`  
- Grouped Query Attention (GQA)  
- Cross-Attention for future multimodal extensions  
- RMSNorm, SiLU activations, tied embeddings  
- Supports full causal language modeling (ArlowForCausalLM)  
- `ArlowTokenizerFast` (fast tokenizer, vocab size 131072)
- `ArlowTokenizer` (Non fast tokenizer)

### Includes:
- `ArlowConfig`, `ArlowModel`, `ArlowForCausalLM`, `ArlowPreTrainedModel`  
- `ArlowTokenizerFast` as fast tokenizer (`tokenization_arlow_fast.py`)  
- `ArlowTokenizer` as fast tokenizer (`tokenization_arlow.py`)  
- Auto mapping + lazy loading registration  

@ArthurZucker

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

@ArthurZucker 

I got the modeling and config for the upcoming ArlowGPT model files ready. The model isn't ready, hence why as no existing model with said architecture currently exists, I uploaded a dummy config.json to pass the check for repo consistency. I fixed all other errors as well, and I wanna get this merged ASAP!

@ArthurZucker 

@ArthurZucker @Rocketknight1 I think this is ready to be merged.
Add ArlowGPT
Author: yuchenxie4645
Pull request: no
Labels: name: New model

##  Add Arlow Model & Tokenizer Support

Adds support for the **Arlow** model and its corresponding **ArlowTokenizer**.

### Features:
- Flash Attention 2 for fast memory-efficient training  
- Rotary Position Embeddings (RoPE) with `rope_theta=100000.0`  
- Grouped Query Attention (GQA)  
- Cross-Attention for future multimodal extensions  
- RMSNorm, SiLU activations, tied embeddings  
- Supports full causal language modeling (ArlowForCausalLM)  
- `ArlowTokenizerFast` (fast tokenizer, vocab size 131072)
- `ArlowTokenizer` (Non fast tokenizer)

### Includes:
- `ArlowConfig`, `ArlowModel`, `ArlowForCausalLM`, `ArlowPreTrainedModel`  
- `ArlowTokenizerFast` as fast tokenizer (`tokenization_arlow_fast.py`)  
- `ArlowTokenizer` as fast tokenizer (`tokenization_arlow.py`)  
- Auto mapping + lazy loading registration  

@ArthurZucker

PR Link: [Here](https://github.com/huggingface/transformers/pull/36899)

### Open source status

- [x] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

https://github.com/huggingface/transformers/pull/36899

Comments:
Avoid build crashes when torch.version.xpu doesn't exist and fix Llama4 processor tests
Author: Rocketknight1
Pull request: yes

Quick update to #37126 which breaks on my local machine because `torch.version.xpu` doesn't exist. I'm not sure which versions that exists for and which doesn't, but we can just use `getattr()` to avoid crashes in those cases either way.

Comments:
In future we may want to refactor this to use `is_torch_xpu_available()` or some other check, but this should fix things for now cc @yao-matrix

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37346). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Adding the Llama4 processor test fixes to this PR because they're making the CI red, so I can't merge without them. cc @ydshieh @ArthurZucker @zucchini-nlp 

One more error here, this time in `test_image_processing`. Doesn't seem to be related to the changes I made, cc @zucchini-nlp @qubvel can you see the problem? Are we saving a broken imageprocessor somehow?

@Rocketknight1 seems to be that we try to load slow processor while fast was saved. I am not sure if we support loading this way, can check real quick

@zucchini-nlp just going to add another skip for now to make the CI clear!

Oh wait, slow processor doesn't even exist. Makes sense to skip, should not be tested, thanks! We'll make this test more generalizable, in subsequent PRs :)

Tests green, merging to clear the CI!

thx, get_attr is safer, will follow up on a more safter way to check.
Multiple llama4 fixe
Author: ArthurZucker
Pull request: yes

# What does this PR do?

- Fixed training with flex
- Fixed inference with flex, no cache
- Fixed full compile compatibility 
- Reduced overhead for the cache cc @Cyrilvallez 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Update model-card for DINOv2
Author: shubham0204
Pull request: yes

# What does this PR do?

This PR updates the model-card for the `dinov2` model, as described in #36979, in an attempt to standardize all model-cards.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).



## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@stevhliu 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37104). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu All requested changes have been made in the latest commit 👍🏻 

@stevhliu I have added a quantization example as requested 👍🏻
convert float for yarn related arguments in rope_scaling
Author: bzantium
Pull request: yes

# What does this PR do?
Remove rope related warnings when loading DeepSeek-V3 checkpoints.
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #37134 

## Who can review?
@ArthurZucker @Rocketknight1 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Yep, LGTM! cc @arthurzucker is this the right place, or should we convert ints to floats when initializing the config class too so this isn't necessary?

Hi @Rocketknight1 @ArthurZucker , thanks for your excellent work on merging deepseek-v3. Are we planning to merge this PR?
Warnings when loading Deepseek-V3 without custom code
Author: Rocketknight1
Pull request: no
Labels: name: bug

### System Info

I tried loading Deepseek-V3 with the library code following #35926 but I got the following errors when loading the config:

```
`rope_scaling`'s factor field must be a float >= 1, got 40
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
```

I don't get these errors when loading with `trust_remote_code=True`. Are we sure the RoPE implementation in Transformers matches the original model?

Comments:
cc @bzantium, were you seeing this issue as well?

Yes I saw this as well! Not necessarily deepseek related

yes, this is because of `rope_config_validation` with `_validate_yarn_parameters` which is not in original code.
https://github.com/huggingface/transformers/blob/3b07ca78bb696825feee3e976795fab58f2b6d0c/src/transformers/models/deepseek_v3/configuration_deepseek_v3.py#L236
If you want to remove this warning, you can add code to convert dtype of such elements in rope_scaling into float. If you want me to give PR, I will do!
to: @Rocketknight1
A bit of cleaning 🧹🧹
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

I stumbled upon the 2 unused functions in `from_pretrained` and they are not used anywhere in the codebase.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37215). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix deepspeed job
Author: ydshieh
Pull request: yes

# What does this PR do?

Wrong `working_directory` and got 

> OCI runtime exec failed: exec failed: unable to start container process: chdir to cwd ("/transformers") set in config.json failed: no such file or directory: unknown

Also install some packages at specific versions.

The pytest command is running with these changes

https://github.com/huggingface/transformers/actions/runs/14269422275/job/39998982497

I will leave you people to update the deepspeed docker image however: it's quite old.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37284). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @S1ro1 (not sure if relevant however)
Llama 4: eager attention results in wrong casual mask shape
Author: Qubitium
Pull request: no
Labels: name: bug

### System Info

Latest transformers 4.51.0



### Who can help?



@ArthurZucker @SunMarc  @MekkCyber  I know you guys are probably burning an all-nighter due to Llama 4. =)




### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

While adding GPTQModel support Llama 4 which does transformers inference at the layer/module level, we found issues where when eager attention is used, casual mask shapes are off by exactly factor of 2 or 4 for Llama 4:

https://github.com/ModelCloud/GPTQModel/pull/1508

![Image](https://github.com/user-attachments/assets/882c784c-40cc-41b5-a9c4-5e0611cc6c7c)

![Image](https://github.com/user-attachments/assets/7c64121c-3438-406a-86e2-21dd71eeef70)

We are able to bypass this for now by setting `batch=1`, removing padding, and not passing `attention_mask` for inference. 

### Expected behavior

No error. 

Comments:
Thanks will have a look asap! 

I think this is fixed, closing
[llama 4] dynamic rope decorator
Author: gante
Pull request: yes

# What does this PR do?

Same as #37249, but applied to Llama 4 

Also updates the docstrings of the Hybrid Caches.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37365). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Skip non-selected experts for mixtral and qwen2_moe
Author: Coco58323
Pull request: yes

Fixes #32283

This PR avoids redundant computation for some MoE models (mixtral and qwen2_moe).
The current implementation loops all the experts and inevitably loads experts' weight, which brings extra IO costs.
@ArthurZucker 




Comments:
@ArthurZucker, I encountered a conflict between torch.fx and a dynamic for loop in my implementation. I haven't found a concise solution for this issue yet. Could you help or just give up for this?

https://github.com/huggingface/transformers/pull/30209

> #30209

Thanks for the info. I am not quite familiar with 'torch.fx'. Seems like there is a trade-off between enabling FX tracing and skipping experts.

same problem.

Thanks for the PR 🤗 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32429). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[core] remove `GenerationMixin` inheritance by default in `PreTrainedModel`
Author: gante
Pull request: yes

# What does this PR do?

In v4.45, we set in motion the removal of `GenerationMixin` inheritance by default in `PreTrainedModel`, our base model class. You can recap the full list of reasons in the [original PR](https://github.com/huggingface/transformers/pull/33203), but TL;DR removes circular dependencies and make non-generative models more lightweight.

This PR is the final step: removes the `GenerationMixin` inheritance. Note that this change is NOT breaking in most contexts:
✅ Loading `generate`-capable `transformers` models (https://github.com/huggingface/transformers/pull/33203 added direct inheritance, https://github.com/huggingface/transformers/issues/36180 added a meta test to ensure we always add `generate` tests to `generate`-capable models)
✅ Loading non `generate`-capable models
✅ Loading `generate`-capable Hub models with `AutoModelXXX` (https://github.com/huggingface/transformers/pull/33203 added the logic to ensure we add `GenerationMixin` even if the original model is missing it, as well as corresponding tests)
❌ Loading `generate`-capable Hub models directly, i.e. not using `AutoModelXXX`, if and only if the model doesn't inherit from `GenerationMixin` (= old implementation). In this case, an informative warning is thrown, suggesting to load the model with `AutoModelXXX`. This warning was present since v4.45.

Relevant tests:
- `py.test tests/models -k test_generation_tester_mixin_inheritance`
- `py.test tests/models/auto/ -k test_custom_model_patched_generation_inheritance`
- `py.test tests/utils/test_modeling_utils.py::ModelUtilsTest::test_can_generate`

__________________________

After merging, let's keep an eye on issues. Although I think I've got the only breaking case well documented, Hub code is always a wildcard.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37173). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix(qwen): fix shape error when using tp
Author: KimmiShi
Pull request: yes

# fix qwen shape error when using tp


<!-- Remove if not applicable -->

Fixes #36875


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@zucchini-nlp 



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers.

Hey @KimmiShi , thanks for opening this! LMK when the PR is ready, and do not forget to indicate it is `Ready for review` :)

> Hey @KimmiShi , thanks for opening this! LMK when the PR is ready, and do not forget to indicate it is `Ready for review` :)

Thanks @zucchini-nlp , I've turned it into `Ready`.

Let's update as many as possible and merge 🤗
Qwen2-VL-7B-Instruct shape error when using TP=4
Author: KimmiShi
Pull request: no
Labels: name: Tensor Parallel

name: bug

### System Info

- `transformers` version: 4.49.0
- Platform: Linux-5.4.250-2-velinux1u1-amd64-x86_64-with-glibc2.31
- Python version: 3.10.15
- Huggingface_hub version: 0.29.2
- Safetensors version: 0.5.3
- Accelerate version: 1.4.0
- Accelerate config:    not found
- DeepSpeed version: 0.16.3
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A800-SXM4-80GB


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

pass

### Expected behavior

Hi, I am trying to use transformers TP to inference a reward model in my RL project. 

I followed [this page](https://huggingface.co/docs/transformers/main/perf_infer_gpu_multi) and inited qwen vl model  with `tp_plan='auto'`, however I came across this error when calling generate:

```
File "/root/miniconda3/lib/python3.10/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py", line 790, in forward
[rank0]:     attn_output = attn_output.view(bsz, q_len, self.hidden_size)
[rank0]: RuntimeError: shape '[1, 359, 3584]' is invalid for input of size 321664
```

`attn_output.view(bsz, q_len, self.hidden_size)` is apparantly wrong since in TensorParallel num_heads are splited,so we should not expect `self.hidden_size` in last dim, instead it should be `(bsz, q_len, self.hidden_size/tp_size)`.

Comments:
@KimmiShi yes, I believe we should be reshaping without indicating `hidden_dim` as in qwen2-vl. Since the attention module in qwen-2lv was not yet refactored, it didn't get the same changes as in llama for example, where the reshape follows as `attn_output = attn_output.reshape(*input_shape, -1).contiguous()`

Would you like to open a PR?

> [@KimmiShi](https://github.com/KimmiShi) yes, I believe we should be reshaping without indicating `hidden_dim` as in qwen2-vl. Since the attention module in qwen-2lv was not yet refactored, it didn't get the same changes as in llama for example, where the reshape follows as `attn_output = attn_output.reshape(*input_shape, -1).contiguous()`
> 
> Would you like to open a PR?

@zucchini-nlp OK, I also want to make sure that TP should be able to do inference except this bug in `attention module in qwen-2lv`