noamaanMulla-03/transformers-issues huggingface dataset:
Added resource class configuration option for `check_circleci_user` job
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
I was reading `CircleCI` docs and [here](https://circleci.com/docs/concepts/#resource-class) i found that not mentioning a resource class for a job will provide a default one which is the `medium` Class and if we see the resources used by this [check_circleci_user job](https://app.circleci.com/pipelines/github/huggingface/transformers/101143/workflows/c8c67fcb-87dd-4f82-a5bc-81b7da1cd712/jobs/1347251/resources). It's CPU and RAM usage is **zero**, so it causes un-necessary extra (double) costs as, given [here](https://circleci.com/pricing/price-list/). 
Also, from the [insights](https://app.circleci.com/insights/github/huggingface/transformers/workflows/setup_and_quality/jobs?branch=b1d8acd4-8701-4c76-8a87-7ff2320aa95f&job-name=check_circleci_user&reporting-window=last-30-days) we can see that the check is running so many times > 3000/month. So, i think specifying a `small` resource_class for this job will save some credits/cost.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ydshieh @amyeroberts 

Comments:
The argument "dim" is gone from LlamaRotaryEmbedding initializer. Intentional?
Author: jeffhataws
Pull request: no
Labels: name: bug

### System Info

4.48.0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

We see the following error with the latest version 4.48.0 of transformers when initializating LlamaRotaryEmbedding:

```
2025-01-10 14:08:19,223::DEBUG: [stdout]         cache_ids = cache_ids.view(-1, 1)
2025-01-10 14:08:19,223::DEBUG: [stdout] >       embed = LlamaRotaryEmbedding(dim=d_head, max_position_embeddings=2048, base=10000)
2025-01-10 14:08:19,223::DEBUG: [stdout] E       TypeError: LlamaRotaryEmbedding.__init__() got an unexpected keyword argument 'dim'
2025-01-10 14:08:19,223::DEBUG: [stdout] 
2025-01-10 14:08:19,223::DEBUG: [stdout] transformers_neuronx_test/unit/1_core/test_rotary.py:108: TypeError
```
To test simply instantiate LlamaRotaryEmbedding with dim equal something:
```
from transformers.models.llama.modeling_llama import LlamaRotaryEmbedding
embed = LlamaRotaryEmbedding(dim=96, max_position_embeddings=2048, base=10000)
```

### Expected behavior

The dim argument was there in previous version. Is the argument no longer needed?

Comments:
cc @ArthurZucker 

Hey! We used a deprecation cycle with #32135. We had a warning saying that you should pass the config starting 4.45 (only removed support in 4.48)! 🤗 

Thanks. Then it is working as expected. We will update.
About GA loss in the latest transformers version
Author: hiyouga
Pull request: no
Labels: name: bug

### System Info

transformers 4.48.0

### Who can help?

@ArthurZucker and @muellerzr 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I feel deeply apologize for everyone because in my previous PR #35438 , I accidentally inherited the typo in #34915 that ought to be fixed in #35113 and #35121. This caused the training loss of models with `loss_kwargs` to become very large once the gradient accumulation is enabled. I think we should merge #35651 ASAP and provide a stable version with the correct implementation.


### Expected behavior

The GA loss should be correctly scaled.

Comments:
These things happen, and getting back into the swing of things post holidays takes a minute so I overlooked testing the slow tests locally as well.

I'll have some new tests today added which run as part of the daily CI + PR CI so we can be flagged sooner of it. 🤗 

We appreciate all you've done with your investigations into it immensely! 

We'll merge it soon + make it part of the patch this week

Thanks @hiyouga for your honesty, and really don't worry! It is a lot more our responsibility than yours! 🤗 Your contributions are most welcome, make sure to keep them coming!!!
Dino V2 pre-training
Author: schmidt-ai
Pull request: no

### Feature request

I'm a newcomer to `transformers`; I found it for its implementation of Dino V2. I take it that the scope of the `models` is mainly inference and (supervised) fine-tuning; so maybe this request is fundamentally out of scope.

Would it be feasible to implement the core pre-training loop (self-distillation w/ student and teacher) and loss functions (token loss, iBot loss, KoLeo) of Dino V2 in `transformers`?

### Motivation

This would enable users to end-to-end train Dino V2 on their own pre-training dataset.

### Your contribution

Happy to help.

Comments:
Indeed it's a bit out of scope for `transformers`, but can very well be shared on [the forum ](https://discuss.huggingface.co/)! 🤗 

Awesome, thanks!

I apologize for responding to a closed issue, but for anybody who needs this feature I want to briefly mention that I have modified facebook's code to train a huggingface ViT using the DINO v1 Training method here: https://github.com/PaulKMandal/huggingface-dino

@PaulKMandal  is there DINOv2?

@schmidt-ai Did you find a way to do it ?

> I apologize for responding to a closed issue, but for anybody who needs this feature I want to briefly mention that I have modified facebook's code to train a huggingface ViT using the DINO v1 Training method here: https://github.com/PaulKMandal/huggingface-dino

really excellent work，have you implemt the DINOV2  by transformers？
Fix whisper compile
Author: jiqing-feng
Pull request: yes

This change could fix the whisper compile error.
Besides, we don't need cache here, this change also avoids the log warning [here].

To reproduce the error:
```python
import torch
from transformers import pipeline
from datasets import load_dataset

pipe = pipeline("automatic-speech-recognition", model="openai/whisper-small")
pipe.model.forward = torch.compile(pipe.model.forward)
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
sample = ds[0]["audio"]
prediction = pipe(sample.copy(), batch_size=8)["text"]
print(prediction)
```

Traceback:
```
...
    if truth_fn(mod):
  File "/home/jiqing/transformers/src/transformers/cache_utils.py", line 405, in __len__
    return len(self.key_cache)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1931, in __getattr__
    raise AttributeError(
torch._dynamo.exc.InternalTorchDynamoError: AttributeError: 'DynamicCache' object has no attribute 'key_cache'
```

Require test?

Comments:
cc @eustlb

Hi @LysandreJik @eustlb   please let me know where can I add the test. Thanks!

Waiting for review ...
Deleting quantization_config broken
Author: psinger
Pull request: no
Labels: name: bug

### System Info

Version 4.47.0

### Reproduction

```
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("h2oai/h2o-danube3-500m-chat", load_in_4bit=True)
del model.config.quantization_config
model.config
```

`TypeError: Object of type dtype is not JSON serializable`

### Expected behavior

I am unable to delete the `quantization_config` from an existing model. Whenever I do it, it just completely breaks the whole config.

I also tried setting `is_quantized=False` but it does not change anything.

Is there another way of achieving this?

I am aware that there is a `.dequantize` function, but in this case Im changing dtypes on my own and want to exclude that `quantization_config` particularly when saving the model.

Comments:
I think this might be a question for the forums/Discord, but pinging @sunmarc @MekkCyber just in case

Hey @psinger, what's the reason you want to delete the quantization_config after the model is loaded ?

Because I am manually dequantizing and am not relying on the HF functionality for it. And then before pushing to hub, I want to remove the quantization_config from the config.

Thanks for the clarification @psinger, I think you can make it work as follows : 
```
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "h2oai/h2o-danube3-500m-chat", load_in_4bit=True
)

config_dict = model.config.to_dict()
if "quantization_config" in config_dict:
    del config_dict["quantization_config"]

from transformers import PretrainedConfig

model.config = PretrainedConfig.from_dict(config_dict)
print(model.config)
```

Does it fix the issue @psinger ? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix some tests
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Fixes some tests:

- Glm (add a decorator to avoid oom)
- Cohere (fix dtype issue)
- Cohere2 (fix model name + fix expected completion + use decorator to avoid oom), see also https://github.com/huggingface/transformers/pull/35681 which was discovered
- DiffLlama (removes _support_flex_attention)


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35682). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

From what I see, `@tooslow` is just a skip, while according to a discussion offline with @ydshieh, the runners may sometimes run on A10 (but this is not deterministic by model type ATM), so `@require_torch_large_gpu` is not strictly equivalent (it will _sometimes_ run the test if the runner is on a10 instead of t4)

Yes, CI will run A10 on some special events. Let's go with this new `require_torch_large_gpu` and see how it goes.

I have to double check if the CI bot has the access to those repo. decorated with `require_read_token`. Update here later

Update: `CohereForAI/c4ai-command-r7b-12-2024` ok

Thanks for double-checking @ydshieh! I applied your comments. Merging then as we agree!
check is added for the report_to variable in TrainingArguments
Author: alpertunga-bile
Pull request: yes

# What does this PR do?

None, string and string array check is added for ```TrainingArgument``` structure's ```report_to``` variable in [get_reporting_integration_callbacks](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/integration_utils.py#L2149) function

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #34445 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. [Link](https://github.com/huggingface/transformers/issues/34445#issuecomment-2560053046)
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@SunMarc @muellerzr

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35403). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Added liger_kernel compatibility with `PeftModel`
Author: ambroser53
Pull request: yes

# What does this PR do?

Fixes #34016

## Who can review?

@BenjaminBossan 

Comments:
> Did you test if the kernel works with PEFT models after this change?

Yes I've been using it for a long time locally

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35680). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
OmDet Turbo processor standardization
Author: qubvel
Pull request: yes
Labels: name: Vision

name: run-slow

name: Processing

# What does this PR do?

Standardize post-processing for OmDet Turbo model (see #34926 for more info)

1) Rename `score_threshold` to `threshold` argument in `post_process_grounded_object_detection`
2) Rename `classes` to `text_labels` argument in `post_process_grounded_object_detection` and make it **optional**
3) Rename `classes` key to `text_labels` in post-processed output dictionary
4) Add `labels` (class indexes) key to post-processed output dictionary

The signature is going to be:
```python
    def post_process_grounded_object_detection(
        self,
        outputs: "OmDetTurboObjectDetectionOutput",
        text_labels: Optional[Union[List[str], List[List[str]]]] = None,           # <--------- renamed: classes -> text_labels + Optional now
        threshold: float = 0.3,                                                    # <--------- renamed: score_threshold -> threshold
        nms_threshold: float = 0.5,
        target_sizes: Optional[Union[TensorType, List[Tuple]]] = None,
        max_num_det: Optional[int] = None,
    )
```

The output is going to be:
```python
[
  {
      "boxes": torch.tensor of shape (N, 4),
      "labels": torch.tensor of shape (N,),                                       # <-------- new key, int indices, consistent with other object detection outputs
      "scores": torch.tensor of shape (N,),
      "text_labels": list of str names of len N (previously classes) or `None`    # <-------- renamed: classes -> text_labels
  },
  ...
]
```
"classes" key is also available, it will return `text_labels` and issue a warning

## TODO:

 - [x] Add deprecation for dict key


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34937). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @yonigozlan for review if you have bandwidth

@ArthurZucker please review when you have bandwidth
Grounding DINO Processor standardization
Author: qubvel
Pull request: yes
Labels: name: Vision

name: run-slow

name: Processing

# What does this PR do?

Simplification and standardization of zero-shot object detection image processor for Grounding DINO

### Add Input Text Preprocessing

Grounding DINO processor handle candidate labels/classes differently compared to other zero-shot object detection models. Specifically:

- **Grounding DINO** requires labels to be merged into a single string with a period (`.`) as the separator:  
  ```python
  text = "a cat. a dog."
  ```

- **OWLv2/OwlVit/OmDet** expects labels to be provided as a list of classes:  
  ```python
  text = ["a cat", "a dog"]
  ```

This PR introduces support for passing a list of labels to Grounding DINO processors, aligning its behavior with other processors. A preprocessing method has been added to format labels as required. The change is BC, labels can still be provided in both formats: either as a single string (`"a cat. a dog."`) or as a list of labels (`["a cat", "a dog"]`).

### Simplification of Postprocessing

Previously, the `post_process_grounded_object_detection` method required `input_ids` to be explicitly passed. This PR makes `input_ids` optional (maintaining backward compatibility) by enabling them to be extracted directly from the model output if not provided.

### Standardization of post-processing output
Added `text_labels` field to post-processed output, which represents string object names (across all other zero shot object detection processors). 

see #34926 for more details on standardization of post-processing 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34853). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @EduardoPach if you have the bandwidth to look into this 🤗 

Thanks @EduardoPach!

@molbap please review if you have bandwidth

@ArthurZucker please review when you have bandwidth
Giving dictionary to Audio Classification Pipeline overwrites dictionary data
Author: wilke0818
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.48.0
- Platform: macOS-14.6-arm64-arm-64bit
- Python version: 3.12.4
- Huggingface_hub version: 0.27.1
- Safetensors version: 0.5.2
- Accelerate version: 1.2.1
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.5.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: None/NA

### Who can help?

@Rocketknight1 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
from transformers import pipeline
import torch
import numpy as np

model_name = 'audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim'
#model_name = 'pollner/distilhubert-finetuned-ravdess'
top_k = 5
device = 'cuda' if torch.cuda.is_available() else 'cpu'
classification_pipeline = pipeline(
                "audio-classification",
                model=model_name,
                # top_k=top_k, 
                # function_to_apply='none',
                device=device,
            )

# dummy signal
sampling_rate = 16000
signal = np.zeros((sampling_rate), dtype=np.float32)

input_signal = {
    'array': signal,
    'sampling_rate': sampling_rate
}
print(input_signal)
print(classification_pipeline(input_signal))
print(input_signal)
```

### Expected behavior

I would imagine, but could be wrong, that the internal logic should not be mutating inputs, especially because this occurs when inputting a dictionary but not when just inputting a numpy array.

Comments:
Yes, thank you for the report! A fix is open at #35754

@wilke0818 a fix has been merged! Can you install the latest version from `main` and confirm that it fixes the issue for you?
ci: fix xpu skip condition for test_model_parallel_beam_search
Author: dvrogozh
Pull request: yes

`return unittest.skip()` used in the `test_model_parallel_beam_search` in skip condition for xpu did not actually mark test to be skipped running under pytest:
* 148 passed, 1 skipped

Other tests use `self.skipTest()`. Reusing this approach and moving the condition outside the loop (since it does not depend on it) allows to skip for xpu correctly:
* 148 skipped

Secondly, `device_map="auto"` is now implemented for XPU for IPEX>=2.5 and torch>=2.6, so we can now enable these tests for XPU for new IPEX/torch versions.

Fixes: 1ea3ad1ae ("[tests] use `torch_device` instead of `auto` for model testing (#29531)")
CC: @faaany @ydshieh 

Comments:
Use AMD CI workflow defined in hf-workflows
Author: ivarflakstad
Pull request: yes

# What does this PR do?

Uses external workflows defined in hf-workflows for self hosted AMD runners. This removes a possible attack vector on self hosted CI runners on github.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35058). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

LGTM. But is it still in draft mode?

Also, does everything run smooth with this new `hf-workflows` now? Do you need a merge to test?

> LGTM. But is it still in draft mode?
> 
> Also, does everything run smooth with this new `hf-workflows` now? Do you need a merge to test?

Yes, I kept it in draft until the PR in hf-workflows was merged so that I could test 👍