noamaanMulla-03/transformers-issues huggingface dataset:
[caches] Raise exception on offloaded static caches + multi device
Author: gante
Pull request: yes

# What does this PR do?

Follow-up to this comment: https://github.com/huggingface/transformers/pull/37922#issuecomment-2854023534

Tests for offloaded static caches are failing on multiple devices because the implementation of these caches doesn't work with multiple devices. This PR raises an informative exception and skips the tests when this happens.

Although I think it is fixable (see added TODO), I don't think it's worth the effort for now.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37974). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @Cyrilvallez: not urgent imo, offloaded static caches are not working with multidevice, this PR throws an appropriate exception

Thanks for the ping! Will try to find some time for it, but it was mostly designed to work with TP indeed (i.e. single device)
[tests] Test all cache implementations
Author: gante
Pull request: yes

# What does this PR do?

The main purpose of this PR is to convert a few slow tests targeted at one cache implementation into fast tests that run on ALL cache implementations. 

Secondarily, makes `RUN_SLOW=1 py.test tests/utils/test_cache_utils.py` green 🟢 These tests also become much, much faster (3 mins -> 1 min, on my machine), despite covering a larger number of features.

This is a follow up to https://github.com/huggingface/transformers/pull/37684, which paved the way for this PR. After this PR is merged, I can go back to https://github.com/huggingface/transformers/pull/37394 and properly test things!

👉  torch.compile was benchmarked with gemma2/hybrid and qwen3/static, no speed regressions.
👉  no regressions in `RUN_SLOW=1 py.test tests/models/llama/test_modeling_llama.py`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37873). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker yeah, generalist cache + compile tests will be up next! :D
Re-Enable `Trigger CircleCI via GitHub Actions when "ready for review" (#37885)`
Author: ydshieh
Pull request: yes

# What does this PR do?

With `CircleCI-Public/trigger-circleci-pipeline-action@v1.2.0`, we could specify `target-branch` 🎉 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38041). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix linalg.norm for CovnNextV2
Author: qubvel
Pull request: yes
Labels: name: bug

# What does this PR do?

Bug introduced in 
 - https://github.com/huggingface/transformers/pull/37041

While we are normalizing on multiple dimensions
1.  `torch.linlag.norm` much slower than `torch.norm`
2. `torch.linlag.norm` **is not equivalent** to `torch.norm`

Instead, `torch.linlag.vector_norm` should be used

Fixes: https://github.com/huggingface/transformers/issues/37748

cc @Rocketknight1 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38015). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: convnextv2

This comment contains run-slow, running the specified jobs: 

models: ['models/convnextv2']
quantizations: [] ...

Yes, that's tricky!
[Bug Fix] Renamed loss to losses to suppress UnboundLocalError
Author: her0e1c1
Pull request: yes

# What does this PR do?

Fixes # (issue)

I use the keywords of `eval_accumulation_steps` and `UnboundLocalError ` but none of the issues are found. So I just decide to make a PR (because this is a quick fix)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
- trainer: @muellerzr and @SunMarc



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31365). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix wrong example in grounding dino
Author: developer0hye
Pull request: yes

# What does this PR do?

Fix wrong example in grounding dino. the more objects can be detected with this modification.


Test Input Image:

![image](https://github.com/user-attachments/assets/d226f186-14da-401d-a36f-e0b1e69fd0e9)

With `text_labels` in example, it only can detect cats when using grounding-dino-base,

<img width="881" alt="스크린샷 2025-05-02 112340_bad" src="https://github.com/user-attachments/assets/8ffda0ed-6e9f-427f-b523-eb9ad4227ab3" />

After modification via an [official guide](https://huggingface.co/IDEA-Research/grounding-dino-base#how-to-use), it can detect remote controls also.

<img width="889" alt="스크린샷 2025-05-02 112340_good" src="https://github.com/user-attachments/assets/eaa59255-6f12-4bf9-8204-3b32a21ec62a" />



- https://huggingface.co/IDEA-Research/grounding-dino-base#how-to-use
 


## Before submitting
- [v] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@amyeroberts, @qubvel

Library:
@zucchini-nlp

Documentation: @stevhliu

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @qubvel!

Hey @developer0hye, thanks for opening a PR, can you please check your version of transformers? Both variants should be supported and produce same results on the latest version. Thanks

Hi @qubvel, my trasnformers version is 4.49.0.

Can you please check with the latest version?

@qubvel 

https://colab.research.google.com/drive/1C_YonKtFUrqKr10Nj7utMCv8et4qHoVR#scrollTo=XzB_KvT0zf4C

I got same results on colab with `trasnformers==4.51.3`.
added mllama doc
Author: Nikil-D-Gr8
Pull request: yes

What does this PR do?
As suggested in this issue[ #issue-2947704577](https://github.com/huggingface/transformers/issues/36979#issuecomment-2817025078) - this PR updates the documentation of the CLIP model, which will now be aligned with the standardized format for all the docs.
Worked on mllama, Used AI , so please let me know even if u need a complete rewrite.

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


Please let me know if there are any changes to be done, do share references if any for those changes
Documentation: @stevhliu

 -->


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@stevhliu Thanks a lot for the help mate,  used ai a bit here and there thinking it can do a better job, looks like it just gave u more work,  new to all this , will keep it organic from now,thanks, let me know if i can make any more edits tho

is it fine now?
@stevhliu
fix(conversion): Fix size mismatch error during TF->PT model loading
Author: arjunaskykok
Pull request: yes
Labels: name: TensorFlow

# What does this PR do?

Loading a PyTorch model from a saved TensorFlow checkpoint using
`from_pretrained(..., from_tf=True)` could fail with a
`RuntimeError: size mismatch`. The error indicated that weights like
position embeddings were expected to have the shape of word embeddings
(e.g., [vocab_size, hidden_size]).

This issue was triggered by recent changes that defaulted to initializing
the PyTorch model with meta tensors (`init_empty_weights`) during this
conversion process.

The root cause was in the tied weight handling logic within
`load_tf2_state_dict_in_pytorch_model` in `modeling_tf_pytorch_utils.py`.
Multiple distinct parameters initialized as meta tensors can share the same
`data_ptr() == 0`. The existing logic incorrectly identified these as tied
weights and reused the tensor loaded for the first parameter encountered
with `data_ptr() == 0` (often the word embeddings) for subsequent parameters
that also had `data_ptr() == 0`.

This fix modifies the tied weight check to explicitly skip cases where
`pt_weight.data_ptr() == 0`, preventing the incorrect reuse of tensors
for distinct meta parameters and resolving the size mismatch error.

Includes a unit test in `test_modeling_utils.py` to specifically verify
this scenario using `from_pretrained(..., from_tf=True)` with meta initialization.

Fixes #37786


## Who can review?

@Rocketknight1 @gante 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@arjunaskykok to make our CI happy: 
- run `make fixup` on your terminal, inside the `transformers` root folder (this runs automated code formatting)
- commit the changes

@gante Will do! But need more time, since I'm on Windows and using 'make' is a bit tricky on Windows.

& 'C:\Program Files (x86)\GnuWin32\bin\make.exe' fixup
-n was unexpected at this time.
make: *** [modified_only_fixup] Error 255

Thanks a lot @arjunaskykok !! Also LGTM and now the CI is happy after merging recent main changes. Merging this!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38014). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Whether transformers Trainer support pipeline parallelism?
Author: liuheng0111
Pull request: no

Hi, deepspeed support pipeline parallelism and  tensor parallelism, i want to know whether transformers Trainer support pipeline parallelism? Is there any reference code for using pipeline parallelism to train large language models?

Comments:
And another question is whether tensor parallel training is supported?

### Does HuggingFace `transformers.Trainer` Support Pipeline Parallelism?

The `transformers.Trainer` class from the HuggingFace Transformers library **does not natively support DeepSpeed's pipeline parallelism**. It is primarily designed as a high-level training API for ease of use and supports less complex parallelism scenarios, including:

- Single-node multi-GPU **data parallelism**
- DeepSpeed **ZeRO optimizer** stages (ZeRO-1/2/3)
- **Mixed precision training** (fp16/bf16)

While you can enable DeepSpeed functionality inside the `Trainer` using the `--deepspeed ds_config.json` flag, **pipeline parallelism requires explicit model partitioning into stages**, which HuggingFace `Trainer` does not handle automatically.

> ✅ **Conclusion:** HuggingFace `Trainer` does **not** support pipeline parallelism.  
> ❗ To use pipeline parallelism, you must write custom training logic using DeepSpeed’s lower-level API.

---

### ✅ Reference Code for Pipeline Parallelism with DeepSpeed

You can find a working example of pipeline parallelism using DeepSpeed in the following official repository:

**🔗 Repo:** [DeepSpeedExamples/training/pipeline_parallelism/train.py](https://github.com/microsoft/DeepSpeedExamples/blob/master/training/pipeline_parallelism/train.py)

#### Key Pipeline Logic in the Code:

1. **Model Layer Definition:**
   ```python
   def join_layers(vision_model):
       layers = [
           *vision_model.features,
           vision_model.avgpool,
           lambda x: torch.flatten(x, 1),
           *vision_model.classifier,
       ]
       return layers```
2. **Wrapping with PipelineModule:**
```python
model = PipelineModule(
    layers=layers,
    loss_fn=torch.nn.CrossEntropyLoss(),
    num_stages=args.pipeline_parallel_size,
    partition_method='parameters',
    activation_checkpoint_interval=0
)
```
3.**DeepSpeed Initialization:**
```python
engine, _, dataloader, _ = deepspeed.initialize(
    args=args,
    model=model,
    model_parameters=model.parameters(),
    training_data=trainset
)
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
In `Trainer`, `evaluation_strategy` defaults to `no`, but `save_strategy` defaults to `steps`. Why?
Author: UrosOgrizovic
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.9.2
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.7.4
- PyTorch version (GPU?): 1.9.0+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


### Who can help

@sgugger

## Information

I'm using `BERTForMaskedLM`, but it doesn't matter, because it's a general `Trainer` issue.

Perhaps `save_strategy` and `evaluation_strategy` should both have the same default value?

## To reproduce

Steps to reproduce the behavior:

1. In `Trainer`, set `load_best_model_at_end` to `True` without assigning any values to `save_strategy` or `evaluation_strategy`
2. Try to run training

Here's the error trace:
```
ValueError: --load_best_model_at_end requires the save and eval strategy to match, but found
- Evaluation strategy: IntervalStrategy.NO
- Save strategy: IntervalStrategy.STEPS
```

## Expected behavior

Since `load_best_model=True` requires `evaluation_strategy == save_strategy`, it seems reasonable to me that both `evaluation_strategy` and `save_strategy` should be set to `"no"` by default.

If I am wrong and the current behavior is the intended one, I'd like to understand why that's the case. :)


Comments:
The value of those default is mainly historical, and we can't change it as it would be a breaking change. So even if it does not necessarily make sense when using `--load_best_model_at_end` (which is exactly why that error message was added, to make sure you are not surprised after spending lot of compute in training), we are a bit stuck with it.

I encountered this issue when my libraries wasn't in the "right" versions,
I found that this issue is gone when the libraries are:
pyorch - 1.4
Adapter transformers - 2.0.0
(note that transformers and adapter-transformers has the same domain name, so make sure that you use adapter-transformers)



Another solution is to add the training parameters
          evaluation_strategy = "no",  
           save_strategy = 'no'
where the values can be - ['no', 'steps', 'epoch']

solutions can be like this:
evaluation_strategy='epoch',
save_strategy='epoch',

In my case, adding `padding='max_length'`, `max_length=512` solved the problem

`tokenized_inputs = tokenizer(examples["tokens"], truncation=True, is_split_into_words=True, padding='max_length', max_length=512)`


如果是llama-factory微调的时候出现这个问题：
--eval_strategy steps 
--eval_steps 100 
--save_steps 100 
--save_strategy steps 
是eval_strategy不是evaluation_strategy
enable generation fsdp/utils cases on XPU
Author: yao-matrix
Pull request: yes


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@ydshieh , pls help review, thx

Thank you. Have 2 nit questions regarding if we can move or have some definitions in `src/transformers/testing_utils.py` that we can reuse.

@ydshieh , updated the PR per your comments, thx.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38009). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🌐 [i18n-KO] Translated codegen.md to Korean
Author: maximizemaxwell
Pull request: yes

# What does this PR do?

Translated the codegen.md file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179
## Before reviewing

  - [x]  Check for missing / redundant translations (번역 누락/중복 검사)
  - [x] Grammar Check (맞춤법 검사)
  - [x] Review or Add new terms to glossary (용어 확인 및 추가)
  - [x]  Check Inline TOC (e.g. [[lowercased-header]])
  - [ ]  Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

@junejae, @Jwaminju, @010kim, @4N3MONE, @boyunJang
## Before submitting

   - [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
   - [x]  Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
    Pull Request section?
   - [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
    to it if that's the case.
   - [x]  Did you make sure to update the documentation with your changes? Here are the
    [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
    [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
   - [ ]  Did you write any new necessary tests?

## Who can review? (Final)

@stevhliu May you please review this PR?

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36698). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

LGTM ! 번역 고생하셨습니다 :)
Fix an error in STOPPING_CRITERIA_INPUTS_DOCSTRING
Author: bilibili12433014
Pull request: yes

# What does this PR do?
There is an error in the comment `STOPPING_CRITERIA_INPUTS_DOCSTRING` for `StoppingCriteria`: 
```
`True` indicates we should continue. 
```
Should be
```
`False` indicates we should continue.
```

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@gante @stevhliu 



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Original content around where I edit:
```
    Return:
        `torch.BoolTensor`. (`torch.BoolTensor` of shape `(batch_size, 1)`), where `True` indicates we stop generation
            for a particular row, `True` indicates we should continue.
```
`True` appeads twice here and one of them must be wrong.

This is my first try to contribute to other's repo, if something I did is wrong, please tell me.
If further information is needed, tell me too.
Thanks.

I think something I did is wrong, and I would like to recreate a new pull request.