noamaanMulla-03/transformers-issues huggingface dataset:
Add retry hf hub decorator
Author: muellerzr
Pull request: yes

# What does this PR do?

This PR adds a `@hub_retry` decorator, which should be used on tests that give us any form of `requests`-like error when a download from the hub fails for *any reason*. It'll then wait a beat then try to rerun it again. I currently have it just located in the 
`ModelTesterMixin` class, but as time goes by and see where other places have it pop up, we should use it in those mixins/classes as well.



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ydshieh 

Comments:
This is a useful feature. I wonder if there is a possibility to use this for monkey patching `from_pretrained`. That way, it would be easier to use the retry mechanism in other packages. As an example, in PEFT we don't have a base class for all tests, so the trick with `__init_subclass__` would not work.

@BenjaminBossan I was also thinking if we can instead patching `from_pretrained` in `conftest.py` rather than this change in `__init__`, but I can's judge it's better than the current version.

Also, it looks this could be applied to any function (test, from_pretrained) directly, so 

> monkey patching from_pretrained

is just `hub_retry(XXX_class.from_pretrained)` maybe

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35213). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> is just `hub_retry(XXX_class.from_pretrained)` maybe

Right, this could work. Do you know if there is a single method that can be patched that would work for all transformers auto models?

Not a single method though.

`from_pretrained` for `_BaseAutoModelClass` will work for all `auto model`, but there are also auto tokenizer/processors etc.
And There are also `PreTrainedModel` if `auto` is not used (then similarly for tokenizer/processors etc)

For now going with what's in here (to just get it merged)
ValueError: Architecture deepseek2 not supported
Author: czq99972
Pull request: no
Labels: name: bug

### System Info

The current Transformers framework doesn't support the gguf quantized model files from deepseek2. Can you please advise when this support might be added? @SunMarc @MekkCyber 



### Who can help?

@SunMarc @MekkCyber 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

File "/home/work/miniforge3/envs/vllm/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1006, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/work/miniforge3/envs/vllm/lib/python3.11/site-packages/transformers/configuration_utils.py", line 570, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/work/miniforge3/envs/vllm/lib/python3.11/site-packages/transformers/configuration_utils.py", line 661, in _get_config_dict
    config_dict = load_gguf_checkpoint(resolved_config_file, return_tensors=False)["config"]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/work/miniforge3/envs/vllm/lib/python3.11/site-packages/transformers/modeling_gguf_pytorch_utils.py", line 103, in load_gguf_checkpoint
    raise ValueError(f"Architecture {architecture} not supported")
ValueError: Architecture deepseek2 not supported

### Expected behavior

1

Comments:
Hey @czq99972, @SunMarc, @MekkCyber! I can take it as soon as I finish current implementation for Mamba arch, but it wouldn't be so long. I think I will be able to start working on deepseek2 on this week. Link to main issue thread: https://github.com/huggingface/transformers/issues/33260

Hey ! Deepspeedv2 gguf can be supported with gguf files once it is integrated in transformers: https://github.com/huggingface/transformers/pull/31976 ! 

Any update on deepseek v2 support? @VladOS95-cyber 

hey @wavy-jung, I see that Deepseekv2 architecture is not supported yet, this PR [#31976](https://github.com/huggingface/transformers/pull/31976) is still in progress


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

我现在依然遇到这个问题，请问有好的解决方案嘛
Issue: Error with _eos_token_tensor when using Generator with GenerationMixin
Author: Algoistricky2004
Pull request: no
Labels: name: bug

### System Info

e-readability-summarization/src/inference$ python run_2.py 
Traceback (most recent call last):
  File "/home/surenoobster/Documents/project/src/inference/run_2.py", line 87, in <module>
    output = generator.generate(
             ^^^^^^^^^^^^^^^^^^^
  File "/home/surenoobster/anaconda3/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/surenoobster/Documents/project/src/inference/generation_2.py", line 572, in generate
    stopping_criteria = self.model._get_stopping_criteria(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/surenoobster/anaconda3/lib/python3.12/site-packages/transformers/generation/utils.py", line 1126, in _get_stopping_criteria
    if generation_config._eos_token_tensor is not None:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'GenerationConfig' object has no attribute '_eos_token_tensor'

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I'm not able to get it , getting problem with device

### Expected behavior

The Generator should initialize the required token tensors correctly to ensure compatibility with GenerationMixin and avoid errors.

Comments:
Hey!

The attributes like `_eos_token_tensor` were added for `torch.compile` compatibility and are created on-the-fly when calling `generate()`. Since in your code you are using private generation methods for custom code, I'd recommend you to call this mthod just before preparing stopping criteria

https://github.com/huggingface/transformers/blob/5fa35344755d8d9c29610b57d175efd03776ae9e/src/transformers/generation/utils.py#L1823-L1827

cc @gante 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[modular] allow multiple modular files in the same model folder
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. It fixes the issue mentioned in https://github.com/huggingface/transformers/pull/36261. There are 2 small issues:
- we were relying on the folder name to get the model name -> of course this fails if several modular files are in the same folder, so we now parse the full filename instead
- Small prefix issue, in case the two models (the new one, and the one imported) have a common name pattern (i.e. in this case, `rt_detr_resnet` and `d_fine_resnet` both end with `resnet`, the prefix matching would lead to inconsistencies


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36287). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @ArthurZucker! What do you propose to do in this situation then? For example, in case of D-FINE, we created resnet and main models modulars in the same folder like it is done for RT-DETR, but there, resnet is implemented from scratch and we are reusing it in D-FINE implementation that leads to some problems with modular conversion script which @Cyrilvallez is trying to resolve here.
add recommendations for NPU using flash_attn
Author: zheliuyu
Pull request: yes

# What does this PR do?

Currently, Ascend NPU does not support using [flash_attn](https://github.com/Dao-AILab/flash-attention) by setting attn_implementation='flash_attention_2'. 

Refer this [doc](https://www.hiascend.com/document/detail/zh/Pytorch/600/ptmoddevg/trainingmigrguide/performance_tuning_0027.html), NPU's flash attention is integrated in ```torch_npu.npu_fusion_attention``` and ```torch.nn.functional.scaled_dot_product_attention```. So if users want to use flash attention in NPU, they can set attn_implementation='sdpa'.

For NPU, "the package flash_attn seems to be not installed" is not a standard error, so I want to add a note to guide users who want to use flash_attn in NPU so that they can choose the right method.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## notes
NPU needs torch>=2.1 to enable SDPA: https://github.com/huggingface/transformers/pull/35165


Comments:
For Google search：

By the way, the ruff installed in transformers-ci is v0.5.1, and ruff has been updated to v0.9.7.  The reformatting results of the two versions are very different.

If transformers-ci reports that ```ruff format failed```,  you can check the local ruff's version.
Please add support for TransPixar AI
Author: jozefchutka
Pull request: no
Labels: name: New model

### Model description

TransPixar AI - Revolutionary AI-powered transparent visual effects generation system developed by Adobe and HKUST researchers

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

https://transpixarai.org/
https://huggingface.co/spaces/wileewang/TransPixar

Comments:
Hi @jozefchutka, are the modeling code and weights for TransPixar AI available anywhere?

Hi @Rocketknight1 would any of these sources help?
https://huggingface.co/papers/2501.03006
https://wileewang.github.io/TransPixar/
https://github.com/wileewang/TransPixar
https://huggingface.co/wileewang/TransPixar/blob/main/cogvideox_rgba_lora.safetensors

Hi @jozefchutka, I'm still a bit confused about the model itself. I can find some LoRA weights on their Github, but because these are adapters for existing model formats, there's nothing we can add to support them in `transformers` itself, beyond the support of the actual base architecture.

I am afraid I do not have knowledge deep enough to understand what is missing. Afaik, https://wileewang.github.io/TransPixar/ is the source of the model, can you please look around if you can find whats needed?
Make Mamba Export Compatible with ONNX
Author: AyoubMDL
Pull request: no
Labels: name: Feature request

### Feature request

When exporting Mamba to ONNX while using the cache, an if-else statement prevents proper inference and alignment with the Hugging Face model. Specifically, the issue arises due to the condition at [this line](https://github.com/huggingface/transformers/blob/88d10517b4c2c1aceeee89787db2578653667beb/src/transformers/models/mamba/modeling_mamba.py#L254) and the alternative execution path at [this line](https://github.com/huggingface/transformers/blob/88d10517b4c2c1aceeee89787db2578653667beb/src/transformers/models/mamba/modeling_mamba.py#L262).

A potential solution is to ignore the cache position and modify the update_conv_state method in Mamba’s cache. Instead of the current implementation at [this line](https://github.com/huggingface/transformers/blob/88d10517b4c2c1aceeee89787db2578653667beb/src/transformers/cache_utils.py#L1875), always update the last position with the new state after the roll:
```python
conv_state[:, :, [-1]] = new_conv_state.to(device=conv_state.device, dtype=conv_state.dtype)
```
This ensures that the else branch is always taken in Mamba’s forward pass, making the model exportable and inference-compatible. Notably, this approach is already used in Mamba2Cache, as seen in [this implementation](https://github.com/huggingface/transformers/blob/88d10517b4c2c1aceeee89787db2578653667beb/src/transformers/models/mamba2/modeling_mamba2.py#L198).

Would it be possible to align Mamba’s behavior with Mamba2Cache to support ONNX export properly?

### Motivation

Mamba’s current conv state update prevents proper ONNX export due to an if-else condition that disrupts inference consistency.

### Your contribution

Proposed a fix that updates the conv state without using the cache position.

Comments:
cc @sunmarc @mekkcyber

cc @gante
Loading model on meta device with `PretrainedModel.from_pretrained` does not work with .bin checkpoints, or when specifying `torch_dtype="auto"`
Author: fxmarty-amd
Pull request: no
Labels: name: Core: Modeling

name: Usage

name: bug

### System Info

transformers latest
pytorch 2.4

### Who can help?

@arthurz

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hello,

I encounter an issue where a model that should be loaded on meta device with PyTorch context manager `with torch.device("meta"):` is effectively not loaded on meta device:

```python
from transformers import AutoModelForCausalLM

original_model_id = "fxmarty/tiny-llama-fast-tokenizer"
    
with torch.device("meta"):
    model = AutoModelForCausalLM.from_pretrained(original_model_id, torch_dtype="auto")

# Meta device loading is bugged in Transformers:
model_state_dict = model.state_dict()
for key in model_state_dict.keys():
    print(key, model_state_dict[key].device)
    assert model_state_dict[key].device.type == "meta"
```

raises
```
Traceback (most recent call last):
  File "<tmp 2>", line 13, in <module>
    assert model_state_dict[key].device.type == "meta"
AssertionError
```

However, removing `torch_dtype="auto"`, or using instead `fxmarty/small-llama-testing` the code behaves as expected.

I'll have a look at it sometime if I can, but just reporting in case it helps

### Expected behavior

The model should be loaded on meta device, not cpu

Comments:
Hey Felix! 😀

Indeed; do you have a fix in mind?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I am reverting the fix, as it was breaking use cases where you want to init the model on meta to then assign params

But closing because you can passe a state dict directly to form pretrained
[WIP] Set default processing device to best available device in fast image processors
Author: yonigozlan
Pull request: yes

# What does this PR do?

Process images on the best available device instead of choosing cpu by default, as discussed with Arthur. 
This could be impactful as fast image processors are muuch faster on GPU than cpu, however right now users have to manually specify the device on which to process the image, which is not intuitive as slow image processor don't support a device kwarg.

However this introduces several questions, mainly, if a device is not passed, and for example cuda:0 is chosen as the best device, once processing is done, should we keep the output tensors on cuda or forced them back to cpu?

Forcing them back would be a waste if we do want to use these inputs on cuda, but if we don't force them back to cpu, we will have backward compatibility problems such as:

```python
import torch
import requests

from PIL import Image
from transformers import RTDetrV2ForObjectDetection, RTDetrImageProcessorFast

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

image_processor = RTDetrImageProcessorFast.from_pretrained("PekingU/rtdetr_v2_r50vd")
model = RTDetrV2ForObjectDetection.from_pretrained("PekingU/rtdetr_v2_r50vd")

inputs = image_processor(images=image, return_tensors="pt")

with torch.no_grad():
     outputs = model(**inputs)
```

will raise an error as the model and inputs are not on the same device.

I guess for this PR to work, we would need to do something similar for models, where the best available device would be chosen by default when instantiating them.

Would love to know your thoughts @ArthurZucker @Rocketknight1 @ydshieh 🤗




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36268). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> however right now users have to manually specify the device on which to process the image, which is not intuitive as slow image processor don't support a device kwarg.

For me, explicit placement is better than implicit (i.e. auto choose the device), and it aligns better with what have done by torch.
Also love to see what other maintainers think 🙏 


Yes I understand your point. I have opened a "warning only" version of this PR here https://github.com/huggingface/transformers/pull/36314, but I'm leaving this open as I'm curious to know what others think :)

Also closing this following discussion on https://github.com/huggingface/transformers/pull/36314
[i18n-zh] Translating `kv_cache` into zh-hans
Author: neofung
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @stevhliu and @MKhalusova for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.md) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.md) (waiting for initial PR to go through)
- [ ] [installation.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.md).

## Tutorial section
- [ ] [pipeline_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.md)
- [ ]  [autoclass_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/autoclass_tutorial.md)
- [ ]  [preprocessing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.md)
- [ ]  [training.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.md)
- [ ]  [accelerate.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.md)
- [ ]  [model_sharing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.md)
- [ ]  [multilingual.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.md)

## Generation
- [x] [kv_cache.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/kv_cache.md)

<!--
Keep on adding more as you go 🔥
-->


Comments:
Hey, thanks for your interest in translating the docs! 🤗 I'm closing this issue in favor of [#26803](https://github.com/huggingface/transformers/issues/26803) so that we can centralize our translation efforts a bit and other contributors have more visibility over the progress.