noamaanMulla-03/transformers-issues huggingface dataset:
Fix device issue for tapas (with `as_tensor`)
Author: ydshieh
Pull request: yes

# What does this PR do?

#37216 added a new test `test_can_load_with_global_device_set` which, at the end of the test, we restore the default device by calling

> torch.set_default_device(default_device)

However, it has unexpected impact on `as_tensor`. By calling `torch.set_default_device`, `as_tensor(T)` has `cpu` device even if `T` is on `cuda`. Previously, `as_tensor(T)` has the same device as T has.

This causes `tapas` model has 30 test failures. This PR set device explicitly in `tapas` code to avoid the issue, but I will open an issue on torch's repository later.



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Will merge. For other affected tests, let's come up with a solution in another PR.
More appropriate cuda warmup in resource-constrained hardware
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. See https://github.com/huggingface/transformers/issues/37436 for the edge-case issue


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37550). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Make Ignored Columns ValueError More Informative
Author: wbuchanan
Pull request: yes

# What does this PR do?
Included forward method signature columns in the ValueError so end users will know what columns are expected to be passed to the model in addition to those which are ignored.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? [issue 33119](https://github.com/huggingface/transformers/issues/33119#issuecomment-2321681582)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@SunMarc I made the modification discussed previously in the thread linked above.


Comments:
cc @sunmarc!
Fix TimesFm doc issue
Author: Cyrilvallez
Pull request: yes

# What does this PR do?




Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37552). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Detect and use device context manager or global device in `from_pretrained`
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. See https://github.com/huggingface/transformers/pull/37213 as well.
Note that this will only ever be used when no `device_map` is provided, otherwise the `device_map` always take precedence.
This behavior should be aligned with what PyTorch does with contetx manager/globally setting the default device. It it also what will happen to the buffers (they are not initialized on meta during loading, so they will go to whatever default device torch knows based on context/global device), so makes total sense IMO (see the example).

This script:

```python
import torch
from transformers import AutoModelForCausalLM

with torch.device(2):
    model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct", torch_dtype=torch.float16)

unique_devices = {v.device for v in model.parameters()} | {v.device for v in model.buffers()}
print(unique_devices)
```

returns
```python
>>> {device(type='cuda', index=2)}
```

And before, only the non-persistent buffers would be moved (because we entirely control the rest)
```python
>>> {device(type='cuda', index=2), device(type='cpu')}
```


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37216). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This deepseed test is failing most likely from this PR as it started failing today: https://github.com/huggingface/transformers/actions/runs/14483575317/job/40624977571
update `test_can_load_with_global_device_set` with a hack
Author: ydshieh
Pull request: yes

# What does this PR do?

`torch.set_default_device` has some unexpected impacts on other tests even if we try to restore the origin value by calling it at the end of the test. See #37551 for one example that is fixed there.

This PR uses `torch._GLOBAL_DEVICE_CONTEXT` to do something more close to the internal mechanism in `torch.set_default_device` to perform the cleanup, see

https://github.com/pytorch/pytorch/blob/e229ce34c4ab8cd4e2800227615be32fb362b1e6/torch/__init__.py#L1205-L1218



Running 

> RUN_SLOW=1 python3 -m pytest -v tests/models/mask2former/test_modeling_mask2former.py

will pass on this PR while it fails on the base commit (b33edf1b) of this PR.

> FAILED tests/models/mask2former/test_modeling_mask2former.py::Mask2FormerModelTest::test_torch_export - AttributeError: module 'torch._tensor' has no attribute 'split'

>FAILED tests/models/mask2former/test_modeling_mask2former.py::Mask2FormerModelIntegrationTest::test_export - AttributeError: module 'torch._tensor' has no attribute 'split'








Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37553). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

yeah, i didn't see it is decorated with `require_torch_accelerator` and I tried to avoid overhead added to CircleCI. But now I just use subprocess.
Add FastImageProcessor for EfficientNet
Author: chewyuenrachael
Pull request: yes

# Add FastImageProcessor for EfficientNet

## What does this PR do?

This PR implements a `FastImageProcessor` for the `EfficientNet` model using `BaseImageProcessorFast`.

- Added `EfficientNetImageProcessorFast` to support fast GPU-based preprocessing using PyTorch/Torchvision ops
- Verified default values (e.g., size, crop_size, normalization) match the existing slow processor
- Cleaned out unused attributes (`default_to_square`, `do_convert_rgb`) after comparing both implementations
- Updated the test file `test_image_processing_efficientnet.py` to support both fast and slow processors using `image_processor_list` pattern
- Verified that all relevant tests pass with `RUN_SLOW=1`

This PR contributes to [[#36978](https://github.com/huggingface/transformers/issues/36978)](https://github.com/huggingface/transformers/issues/36978)

## Before submitting

- [x] I have read the [[contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request)](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request)
- [x] This PR was initiated via a GitHub issue: #36978
- [x] I have matched default attributes between the slow and fast image processors
- [x] I have updated test coverage to loop over both slow and fast processors
- [x] I have verified all tests pass with `RUN_SLOW=1`
- [x] I ran `black` and `make fixup` to format the code
- [ ] Documentation did not need updating (no new public methods introduced)
- [x] All new methods follow the docstring conventions

## Who can review?

This contribution affects **vision models**.  
Tagging: @qubvel @amyeroberts for review.

Thanks in advance! 🤗

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Fix pixel attention mask padding in smolvlm
Author: ManuelFay
Pull request: yes

# What does this PR do?

The attention mask (binary) is initialized with zeros in float64, while in make_pixel_mask, the ones and zeros are in int64.

```python
def make_pixel_mask(
    image: np.ndarray, output_size: Tuple[int, int], input_data_format: Optional[Union[str, ChannelDimension]] = None
) -> np.ndarray:
    """
    Make a pixel mask for the image, where 1 indicates a valid pixel and 0 indicates padding.
    Args:
        image (`np.ndarray`):
            Image to make the pixel mask for.
        output_size (`Tuple[int, int]`):
            Output size of the mask.
    """
    input_height, input_width = get_image_size(image, channel_dim=input_data_format)
    mask = np.zeros(output_size, dtype=np.int64)
    mask[:input_height, :input_width] = 1
    return mask
```


Fixes # (issue)

This causes either int64, or float64 mask values depending on if inputs are padded.
This makes MPS devices bug since they don't support float64.

By the way @andimarafioti , we could just cast as np.bool if masks are binary but I guess there is a reason it was done in np.int64 ?

## Who can review?

@orrzohar @andimarafioti @zucchini-nlp


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37497). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
All models can be initialized on meta device
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Some old models could not be initialized on meta device because they used dynamic `.item()` or `.tolist()` in the `__init__`, which does not work with meta initialization as the underlying tensor will be on meta.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37563). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
enable 5 cases on XPU
Author: yao-matrix
Pull request: yes

**command**
`pytest -rA tests/models/speecht5/test_modeling_speecht5.py::SpeechT5ForTextToSpeechIntegrationTests::test_batch_generation`

`pytest -rA  tests/models/glm/test_modeling_glm.py::GlmIntegrationTest::test_model_9b_bf16`
`pytest -rA tests/models/glm/test_modeling_glm.py::GlmIntegrationTest::test_model_9b_eager`
`pytest -rA tests/models/glm/test_modeling_glm.py::GlmIntegrationTest::test_model_9b_fp16`
`pytest -rA tests/models/glm/test_modeling_glm.py::GlmIntegrationTest::test_model_9b_sdpa`

**results**

all pass on XPU


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Seems okay to me but cc @ydshieh for tests!
enable test_offloaded_cache_implementation test case on XPU
Author: yao-matrix
Pull request: yes

78 cases passed, 1 case(tests/models/dbrx/test_modeling_dbrx.py::DbrxModelTest::test_offloaded_cache_implementation_0_offloaded
) failed, will fix the 1 failed case in a separate PR. 



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @ydshieh!

Approved, thanks. 

But just to be sure, I will wait until you change the PR from draft to `Ready for review` 🙏 before I merge.

@Rocketknight1  @ydshieh , thx you very much!

failed cases seem not relevant to my PR.
enable several cases on XPU
Author: yao-matrix
Pull request: yes

w/ this PR:
1. autoawq cases: 9 pass, 1 skip(should skip, since xpu doesn't support exllama, all optimized op will go through ipex backend), 2 fail
2. peft_integration cases: 2 pass, 1 fail(for bnb issue)
3. test_sdpa_can_dispatch_on_flash: 19 pass, 1 fail

fail cases:
tests/models/diffllama/test_modeling_diffllama.py::DiffLlamaModelTest::test_sdpa_can_dispatch_on_flash
tests/peft_integration/test_peft_integration.py::PeftIntegrationTester::test_peft_from_pretrained_kwargs
tests/quantization/autoawq/test_awq.py::AwqTest::test_quantized_model_bf16
tests/quantization/autoawq/test_awq.py::AwqTest::test_quantized_model_multi_gpu

We will follow 4 failure cases and submit fixing separate PRs.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @ydshieh!
enable 3 mpt test cases on XPU
Author: yao-matrix
Pull request: yes

PASSED tests/models/mpt/test_modeling_mpt.py::MptIntegrationTests::test_generation
PASSED tests/models/mpt/test_modeling_mpt.py::MptIntegrationTests::test_generation_8k
FAILED tests/models/mpt/test_modeling_mpt.py::MptIntegrationTests::test_generation_batched

FAILED case is our bnb integration issue, will submit separate PR if needed once fixed


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

LGTM 👍
[chat template] fix security vulnerability
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As discussed internally, let's just use `urlparse`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37523). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[qwen-vl] Standardize config
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

BC is kept and the models can be loaded as before. All attributes are still available though general config (`config.vocab_size`) but in model code we use `config.text_config` now

Separating out the text config from the general config allows us to support multimodality and text as separate entities, with their own base classes and configs. Related to #37033

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37268). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
enable 6 rt_detr_v2 cases on xpu
Author: yao-matrix
Pull request: yes

below 6 cases PASSED

tests/models/rt_detr_v2/test_modeling_rt_detr_v2.py::RTDetrV2ModelTest::test_inference_equivalence_for_static_and_dynamic_anchors_0_float32
tests/models/rt_detr_v2/test_modeling_rt_detr_v2.py::RTDetrV2ModelTest::test_inference_equivalence_for_static_and_dynamic_anchors_1_float16
tests/models/rt_detr_v2/test_modeling_rt_detr_v2.py::RTDetrV2ModelTest::test_inference_equivalence_for_static_and_dynamic_anchors_2_bfloat16
tests/models/rt_detr_v2/test_modeling_rt_detr_v2.py::RTDetrV2ModelTest::test_inference_with_different_dtypes_0_float32
tests/models/rt_detr_v2/test_modeling_rt_detr_v2.py::RTDetrV2ModelTest::test_inference_with_different_dtypes_1_float16
tests/models/rt_detr_v2/test_modeling_rt_detr_v2.py::RTDetrV2ModelTest::test_inference_with_different_dtypes_2_bfloat16

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

xpu 🔥 

> BTW, would your team prefer we use `require_torch_accelerator` in the first place when a new test is added? (so you don't have to open PRs like this, but this may give some surprise on your CI from time to time)
> 
> Thanks as always

@ydshieh  great! that actually what i want to propose to you. If you can apply this, it will be wonderful, we will keep monitoring the CI failures and fix them. 

Sure 👍
Fix qwen2audio wanr -> warn
Author: alex-jw-brooks
Pull request: yes

# What does this PR do?

Fixes a typo for `warnings.wanr` -. `warnings.warn` in the qwen2 audio processor. Should not really be reachable because of the deprecation decorator, but maybe still nice to fix anyway!


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

 @eustlb

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Small fix on context manager detection
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Following feedback in https://github.com/huggingface/transformers/pull/37216!
cc @SunMarc
In the meantime, I switched the test to ensure we _can_ load on meta so that it works for 3rd party libs until we remove it (because it was not the case before https://github.com/huggingface/transformers/pull/37216, as we need to explicitly pass a `device_map=torch.device("meta")`, which is weird and an anti-pattern)
BTW, do you have concrete examples of 3rd party libs using such meta context managers? 🤔

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37562). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

If you check all occurrences of` with torch.device("meta"):` in github, you will see that a few scripts actually do that with `from_pretrained`. 

https://github.com/StrongResearch/isc-demos/blob/ea531ff5e32643b5d03d7eccf0406d5a332e4519/deepseek/fsdp.py#L72

https://github.com/fxmarty/accelerated-pytorch-transformers-generation/blob/cd00bdbf4f549dbc1c25d834822aafa3718cefb8/scripts/run_llama.py#L168

https://github.com/aypan17/latentqa/blob/e40cc447ba89e0a935c83548e8f1f767b09015b5/lit/utils/infra_utils.py#L261
Ensure positive warm-up size
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

cc @qubvel @ydshieh for viz 🤗

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37581). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.