noamaanMulla-03/transformers-issues huggingface dataset:
Fix: check TrainerState file exists before loading during resume
Author: Petecheco
Pull request: yes

# What does this PR do?

When resuming training from a checkpoint, the Trainer attempts to load \`trainer_state.json\` to recover the train batch size. However, if the file does not exist, a \`FileNotFoundError\` is raised, causing resume to fail.

This patch adds a check using \`os.path.isfile\` before loading the state, consistent with other parts of the codebase. If the file is missing, a warning is logged and batch size recovery is skipped, allowing training to continue.

Fixes a potential crash and improves robustness when resuming from incomplete or custom checkpoint directories.

Related PR #27568 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

trainer: @zach-huggingface, @SunMarc and @qgallouedec
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.



Comments:
fix moe routing_weights
Author: llbdyiu66
Pull request: yes

About `Ernie4_5_MoEStatics`, e_score_correction_bias only affects selected_experts, but not routing_weights.

Comments:
Also, you need to modify the `modular_ernie4_5_moe.py` file and apply the changes with `python utils/modular_model_converter.py ernie4_5_moe` or `make fix-copies` instead.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: ernie4_5_moe

run-slow: ernie4_5_moe

This comment contains run-slow, running the specified jobs: 

models: ['models/ernie4_5_moe']
quantizations: [] ...

Thank you, just updated the integration test and it will be merged after the docs have been built :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39581). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[WIP] Add OneformerFastImageProcessor
Author: Player256
Pull request: yes

# What does this PR do?
Adds Fast Image Processor for Oneformer model

Fixes # (issue)
#36978


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you write any new necessary tests?


## Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.




Comments:
cc @yonigozlan 

I would definitely like some review from the maintainers on this PR. I think my code does need some revision.

Hey @Player256 ! I haven't had the bandwidth to look at this PR yet, but I'll try to get to it probably early next week. Sorry for the delay!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, conditional_detr, deformable_detr, detr, grounding_dino, oneformer, rt_detr, yolos

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38343). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hey thanks @yonigozlan I think you handled most of my code. Thank you for that. Though I'd love to work on another issue.
[idefics3] fix for vLLM
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, the recent integration tests are failing because Idefics had an edge case. We can't assume `num_cols == num_rows` because idefics can output non-square images

See https://github.com/vllm-project/vllm/pull/20543#discussion_r2212501398 for more

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39470). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: idefics3, smolvlm

No, I didn't add tests for helper functions as they are not same for all models. Some models didn't need any helpers, some needed more. The only tests we have are on vLLM side

I can add a tiny test though I don't think anyone will touch these methods 😆
[phi-4] add processor tests
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Now that the image processor is merged, we can add processor tests as well. Note: the tests will be failing for now, I found that audio and image configs are saved with the same filename and thus overwrite each other 😮 

I will be first fixing the audio processor save filename in a separate PR

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38124). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

model ckpt won't be added in the Hub so closing
The similarity between image and text in siglip2 is very low
Author: sigma-alpha-beta
Pull request: no

transformers：4.53.2
I use the following code to test the similarity between image and text, but the similarity is only 0.37, the output is `tensor([[3.7129e-01, 4.2452e-04, 1.8075e-07]])`
code:
```
from PIL import Image
import requests
from transformers import AutoProcessor, AutoModel
import torch

model = AutoModel.from_pretrained("google/siglip2-so400m-patch16-256")
processor = AutoProcessor.from_pretrained("google/siglip2-so400m-patch16-256")

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
# image = Image.open("cat.png").convert('RGB')
text = ["two cats", "a cat", "a dog"]

inputs = processor(text=text, images=image, padding="max_length", max_length=64, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
    logits_per_image = outputs.logits_per_image
    probs = torch.sigmoid(logits_per_image)
print(probs)
```

Comments:
Hi @sigma-alpha-beta, models can just be wrong sometimes, it's not a bug! Unless this is an example from the docs which is supposed to have a different output, I'm closing this for now.
Wandb isn't logging config in offline mode
Author: souththzz
Pull request: no
Labels: name: bug

### System Info

- transformers version: 4.52.4
- Platform: Linux-5.4.0-215-generic-x86_64-with-glibc2.31
- Python version: 3.11.11
- Accelerate version: 1.8.1
- Wandb version: 0.20.1

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

The key code simply enables the offline mode of wandb.

```
os.environ["WANDB_PROJECT"] = "test"
os.environ["WANDB_MODE"] = "offline"

trainer = Trainer(
    model=model,
    args=TrainingArguments(
        report_to="wandb",
        run_name="tmp-10-offline",
    ),
    train_dataset=ds_train,
    eval_dataset=ds_eval,
)
```

It can record loss, but there is no parameter configuration.

![Image](https://github.com/user-attachments/assets/4cc1f876-6921-44c3-9ec9-b9fb953ea008)

### Expected behavior

Log parameter config in offline mode, just like in online mode.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing since I think this is fixed by https://github.com/huggingface/accelerate/pull/3648!
Fixed num_quantizers to use variable codebook_size
Author: EishaanKhatri
Pull request: yes
Labels: name: bug

name: Audio

# What does this PR do?

Adds the modification for "num_quantizer in EncodecConfig should accept variable codebook size #34521"
It adds on code provided by a user originally in a comment and tested on use cases.


Fixes # (issue)

It fixes the issue of not being able to use the variable coodbook_size in config_encodec.

## Before submitting
- [Yes ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ Yes] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ No] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [No Need for Changes ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [Not Needed ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 
@maintainer
Hi, I am new to all this, please help me tag an appropriate person


 

Comments:
cc @hollance who added the original model, in case he'd like to review! If there's no reply in a couple of days, @EishaanKhatri please ping me again and I'll find a reviewer inside Hugging Face

Thanks

With Regards
Eishaan Khatri
22mc3014
RGIPT


On Thu, Oct 31, 2024 at 6:59 PM Matt ***@***.***> wrote:

> cc @hollance <https://github.com/hollance> who added the original model,
> in case he'd like to review! If there's no reply in a couple of days,
> @EishaanKhatri <https://github.com/EishaanKhatri> please ping me again
> and I'll find a reviewer inside Hugging Face
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/pull/34532#issuecomment-2449847301>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/BH6YZVDLF7PCNHFAKUFFPOLZ6IWE7AVCNFSM6AAAAABQ6HM722VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBZHA2DOMZQGE>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


No reply from the original author - cc @ylacombe for audio models maybe?

Hey @Eishaan-Khatri, thanks for opening this PR!

Could you link to the comment you refer to in the PR description? I'd be curious to also see a code snippet on the use-case mentioned in the PR. Thanks!

Thanks @EishaanKhatri for the PR! I'm closing it, as it is being address in another PR, see https://github.com/huggingface/transformers/pull/39431#discussion_r2209787221
Fix important models CI
Author: molbap
Pull request: yes

# What does this PR do?

Minor. Fixes a few tests failing currently on the important test action, on llama/gemma/mistral/mixtral.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39576). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: llama, gemma, mistral, mixtral

This comment contains run-slow, running the specified jobs: 

models: ['models/gemma', 'models/llama', 'models/mistral', 'models/mixtral']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: kyutai_speech_to_text
[Trackio] Allow single-gpu training and monitor power
Author: qgallouedec
Pull request: yes

# What does this PR do?

<img width="1146" height="962" alt="Screenshot 2025-07-22 at 5 18 42 PM" src="https://github.com/user-attachments/assets/45fd0d90-a150-4bf4-9a26-c30b49449b1a" />



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39595). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Amazing!!! Thank you so much @qgallouedec :hugs: :earth_africa:
Transformers serve VLM
Author: LysandreJik
Pull request: yes

Add VLM support for `transformers serve`.

In case you're interested in playing around with it, Open WebUI is a good tool to work with.

Launching `transformers serve` with VLM support:

```
cd <PATH_TO_TRANSFORMERS>
git checkout transformers-serve-vlm
transformers serve --enable_cors
```

To install Open WebUI:
```
pip install open-webui
open-webui serve
```

To set it up to work with the transformers serve backend:
- Click on the top right icon
- Settings
- Connections
- Add connection
- Set URL to "http://localhost:8000/v1"
- Set Key to non-null string
- Verify connection
- All good!

You should now find a limited (but growing) list of available models:

<img width="779" height="416" alt="image" src="https://github.com/user-attachments/assets/1e709b26-e444-4708-80ed-f8c680266132" />

Enjoy chatting with the model!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39454). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Looking great! qq from our side (without having done much research): how difficult would it be to simply spawn up the chat completion endpoint/function directly from our backend? It would be great to have `pip install open-webui[transformers]` option where our users can get started right away with all the batteries included. We'd appreciate any guidance here!

@tjbck from my experience, with the exception of changing the OpenAI address manually, integration with `open-webui` is seamless! Open your app, launch `transformers serve --enable-cors`, and everything just works 💛 

If we can ask for features on your side, it would be nice to be able to specify arbitrary models from the model selection dropdown (as opposed to being limited to the list from `/v1/models`). Then any model from the HF Hub could be immediately accessed! 

Let us know if we can help from our side 🫶 



Thanks for dropping by @tjbck!

As Joao said above, it should be pretty seamless. You might want to specify some specific extras when installing `transformers`:

```
transformers[torch,serving]  # for raw transformers + torch, so for all LLMs basically, and required deps to launch serve
transformers[torch,serving,vision]  # for VLMs as well
transformers[torch,serving,audio]  # for STT (and eventually TTS)
```

As Joao highlights there is a question around which LLMs/VLMs are shown as `transformers` supports ~250k LLMs and ~4k VLMs from the Hugging Face Hub, displaying them all in the top-left hand corner would be overwhelming 😀 

Let us know in case you run into any issues, we'd be happy to add in features that would make your life simpler.
Move openai import
Author: ebezzam
Pull request: yes

# What does this PR do?

Moving openai import which causes [failing test](https://app.circleci.com/pipelines/github/huggingface/transformers/139153/workflows/81ebf2c1-8e6b-4250-b673-6008589eb88a/jobs/1843853?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link&utm_content=summary)

Broke due to https://github.com/huggingface/transformers/pull/39454

cc @ArthurZucker 


Comments:
Thank you!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39613). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix typo in generation configuration for Janus model weight conversion
Author: thisisiron
Pull request: yes

# What does this PR do?

Fixes an AttributeError caused by attempting to access generation_kwargs from a GenerationConfig object in `convert_janus_weights_to_hf.py`.

When running the conversion script, the following error occurs:

```bash
Traceback (most recent call last):
  File "/home/infidea/xxx/code/tmp/transformers/src/transformers/models/janus/convert_janus_weights_to_hf.py", line 504, in <module>
    main()
  File "/home/infidea/xxx/code/tmp/transformers/src/transformers/models/janus/convert_janus_weights_to_hf.py", line 492, in main
    convert_model(
  File "/home/infidea/xxx/code/tmp/transformers/src/transformers/models/janus/convert_janus_weights_to_hf.py", line 413, in convert_model
    model.generation_config.generation_kwargs["boi_token_id"] = tokenizer.vocab.get("<begin_of_image>")
AttributeError: 'GenerationConfig' object has no attribute 'generation_kwargs'
```

Changed the code from:
```
model.generation_config.generation_kwargs["boi_token_id"] = tokenizer.vocab.get("<begin_of_image>")
```

to:
```
model.generation_config.boi_token_id = tokenizer.vocab.get("<begin_of_image>")
```
This resolves the error by directly assigning the custom token ID as an attribute of the GenerationConfig object.


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts, @qubvel

Comments:
Hey @thisisiron, thanks for opening a PR! I'm not sure it's backward compatible with current models on the Hub
cc @zucchini-nlp 

run-slow: janus

This comment contains run-slow, running the specified jobs: 

models: ['models/janus']
quantizations: [] ...

Hi @zucchini-nlp,
In order to preserve the original behavior, shouldn't we include the following code?
```python
if not hasattr(model.generation_config, 'generation_kwargs'):
    model.generation_config.generation_kwargs = {}
model.generation_config.generation_kwargs["boi_token_id"] = tokenizer.vocab.get("<begin_of_image>")
```

When I run the following command:
```
python convert_janus_weights_to_hf.py \
    --repo_id deepseek-ai/Janus-Pro-1B \
    --output_dir ./tmp/hub_code_out \
    --safe_serialization

```
I encounter the error mentioned at the top.

---

Just out of curiosity, how should I make it backward compatible?

> In order to preserve the original behavior, shouldn't we include the following code?

Yep, correct! That will work and save the model without errors

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: janus

run-slow: janus

This comment contains run-slow, running the specified jobs: 

models: ['models/janus']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39432). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add Swin2SR ImageProcessorFast
Author: thisisiron
Pull request: yes

# What does this PR do?

#36978
Add fast image processor for Swin2SR

cc @yonigozlan 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

> Missed a few things in my first review. Also since there is very little processing happening here, the fast processors doesn't look faster than the slow on cpu, so we might need to modify or skip `test_fast_is_faster_than_slow`

@yonigozlan, Then would it be better to skip it?


> @yonigozlan, Then would it be better to skip it?
In this case yes I think that's the best thing to do

@yonigozlan, I’ve added the skip to the test.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37169). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix: Update pixel_values parameter in hf_model input
Author: thisisiron
Pull request: yes

# What does this PR do?

## Changes
- Modified hf_model input parameter from original_pixel_values to pixel_values to use correct variable name
- The code shows that original_model takes original_pixel_values as input while hf_model uses pixel_values.
```python
logits = hf_model(
-       pixel_values=original_pixel_values,
+       pixel_values=pixel_values, 
    input_ids=input_ids,
    attention_mask=attention_mask,
    use_image_text_matching_head=True,
)
```

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts, @qubvel

Comments:
@qubvel Thank you for your quick response. As you mentioned, while pixel_values and original_pixel_values are identical, the code changes were made to maintain naming consistency.

The following is a portion of the code for passing input to the model appropriately:
```python
 with torch.no_grad():
    if "opt" in model_name:
        original_logits = original_model({"image": original_pixel_values, "text_input": [""]}).logits
        logits = hf_model(pixel_values, input_ids).logits
    else:
        original_logits = original_model(
            {"image": original_pixel_values, "text_input": ["\n"], "text_output": ["\n"]}
        ).logits
        labels = input_ids.masked_fill(input_ids == tokenizer.pad_token_id, -100)
        logits = hf_model(pixel_values, input_ids, labels=labels).logits
```

```python
original_outputs = original_model.generate(
    {"image": original_pixel_values, "prompt": prompt}, use_nucleus_sampling=True, max_length=50
)
outputs = hf_model.generate(
    pixel_values,
    input_ids,
    do_sample=True,
    num_beams=5,
    max_length=30,
    min_length=1,
    top_p=0.9,
    repetition_penalty=1.0,
    length_penalty=1.0,
    temperature=1,
)
```

If this isn't necessary, I'll close the request.