noamaanMulla-03/transformers-issues huggingface dataset:
RuntimeError when loading InternVL3-14B model: Embedding size mismatch
Author: wkzcml-1
Pull request: no

## Problem Description
When trying to load the [InternVL3-14B](https://huggingface.co/OpenGVLab/InternVL3-14B) model using the `transformers` library, I encountered the following error:
```
RuntimeError: Error(s) in loading state_dict for Embedding:
	size mismatch for weight: copying a param with shape torch.Size([151674, 5120]) from checkpoint, the shape in current model is torch.Size([151936, 4096]).
```

## Additional Environment Information
### Transformers Package Details:​
```
Name: transformers
Version: 4.52.0.dev0
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /home/tiger/.local/lib/python3.11/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: peft
```
### Python version
```
Python 3.11.2
```
### Code Snippet Used
```python3
model = InternVLForConditionalGeneration.from_pretrained(
    internvl3_14B_dir, 
    trust_remote_code=True
)
```

Comments:
Hi @wkzcml-1, your code snippet refers to a local directory that we don't have access to. Can you give us a complete snippet that we can copy and paste to see the error, preferably using a model on the Hub instead?

Hi @wkzcml-1 ! The link you provided is not the official transformers implementation, that would be [this one](https://huggingface.co/OpenGVLab/InternVL3-14B-hf), hope it solves your issue!

> Hi [@wkzcml-1](https://github.com/wkzcml-1) ! The link you provided is not the official transformers implementation, that would be [this one](https://huggingface.co/OpenGVLab/InternVL3-14B-hf), hope it solves your issue!

Thank you. By the way, I couldn't find the [OpenGVLab/InternVL3-14B-hf](https://huggingface.co/OpenGVLab/InternVL3-14B-hf) link in [OpenGVLab's Collections](https://huggingface.co/collections/OpenGVLab/internvl3-67f7f690be79c2fe9d74fe9d), which might cause inconvenience for users.

> Hi [@wkzcml-1](https://github.com/wkzcml-1), your code snippet refers to a local directory that we don't have access to. Can you give us a complete snippet that we can copy and paste to see the error, preferably using a model on the Hub instead?

Thank you. The reason is that the model I used is not the official Transformers implementation.
Same data but has large different while evaluating in the training stage vs evaluate it standalone from read the finetuned-model
Author: irmathebest
Pull request: no

Hi team. 
I have stuck on this problem for a whole week and still cannot figure out why. 
Env: python 3.8, transformer -- 4.28
I am using the XLMRobertA Base for finetuning the model for a multi-class classification. 
However, 
when in the training step, I run trainer.evaluate() it shows the accuracy is 68% while in the evaluate standalone, which it reads the base model and then make the prediction and evaluate it, the accuracy drops to 30%. Is there any reason why it happens, or it's a bug? 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
ed_video = input_tokens.index(video_token_id, st) ValueError: 151656 is not in list
Author: yaomingzhang
Pull request: no
Labels: name: bug

### System Info

During the fine-tuning process of Qwen-OMNI

### Who can help?

During the fine-tuning process of Qwen-OMNI,
using transformer version: transformers-f742a644ca32e65758c3adb36225aef1731bd2a8,

I encountered embedding video issues:

![Image](https://github.com/user-attachments/assets/7c6a77e1-b578-4eee-9810-1ce328cf3661)
After switching to transformers-3a1ead0aabed473eafe527915eea8c197d424356,
the inference process no longer throws errors. However, during fine-tuning, I encountered:

![Image](https://github.com/user-attachments/assets/6a317e2f-aec9-4bc8-9350-c866abe9765c)

help me

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

help me

### Expected behavior

no error

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[typo] qwen2_5_vl in modeling_utils.py
Author: JJJYmmm
Pull request: yes

# What does this PR do?

Fixes https://github.com/QwenLM/Qwen2.5-VL/issues/1195 just a small typo🤗
Related PR #37033 @zucchini-nlp

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
[VLM] fix loading issues
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, the recent refactor broke loading for some models. The tests were ran before the last commit when a restriction on model names was added, thus the issue went unnoticed while merging 🥲 



fixes https://github.com/huggingface/transformers/issues/38056, and fixes https://github.com/huggingface/transformers/issues/38070

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38051). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Typo in modeling_utils.py causing checkpoint loading error with Qwen2.5-VL
Author: tanghme0w
Pull request: no
Labels: name: bug

### System Info

Version: 4.52.0.dev0

In modeling_utils.py, line 236, qwen2_5_vl is misspelled as qwem2_5_vl

<img width="487" alt="Image" src="https://github.com/user-attachments/assets/9a0a6c2b-fc6f-4c17-9a5c-926c67cea597" />

This will cause the load of _checkpoint_conversion_mapping in line 4113 being skipped and thereby causing param names mismatch during checkpoint load.

<img width="864" alt="Image" src="https://github.com/user-attachments/assets/704a26b2-a035-48ac-9ac7-58a034cd5bc9" />


### Reproduction

simply run the from_pretrained method of Qwen2_5_VLForConditionalGeneration to reproduce.

···
from transformers import Qwen2_5_VLForConditionalGeneration

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-32B-Instruct", torch_dtype="auto", device_map="auto"
)
···


### Expected behavior

Will report missing parameters and the model will be initialized with random weights.

Comments:
lol bro🤣

Duplicate of https://github.com/huggingface/transformers/issues/38056, and will be fixed soon
Qwen/Qwen2.5-VL-7B-Instruct not work [2025-05-10]
Author: kekxv
Pull request: no
Labels: name: bug

### System Info

work for `pip install git+https://github.com/huggingface/transformers@7a3e208892c06a5e278144eaf38c8599a42f53e7`
not work `main`

https://github.com/QwenLM/Qwen2.5-VL/issues/1192

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://github.com/QwenLM/Qwen2.5-VL/issues/1192

### Expected behavior

work qwen2.5-vl

Comments:
Thanks for reporting, looking at it. We merged a huge refactor, should have been it, maybe related to https://github.com/huggingface/transformers/pull/38051
[New model] RT-DETR
Author: rafaelpadilla
Pull request: no
Labels: name: New model

### Model description

The paper "DETRs Beat YOLOs on Real-time Object Detection" proposes a Real-Time DEtection TRansformer (RT-Detr), a hybrid encoder and a transformer decoder for object detection tasks.
Repo has +380 stars and paper shows improvement in inference speed and AP in comparison with other models, outperforming YOLOv8 detectors in both speed and accuracy.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Paper: https://arxiv.org/abs/2304.08069
Code: https://github.com/lyuwenyu/RT-DETR/

Comments:
I'll be working on porting this model.

This was one was added in #29077
Add MobileViT v2
Author: SunHaozhe
Pull request: no
Labels: name: New model

### Model description

[MobileViT](https://openreview.net/forum?id=vh-0sUt8HlG) is a computer vision model that combines CNNs with transformers that has already been added to Transformers. 

[MobileViT v2](https://arxiv.org/abs/2206.02680) is the second version; it is constructed by replacing multi-headed self-attention in MobileViT v1 with the proposed separable self-attention. 

Does Hugging Face have plan to add MobileViT v2 to Transformers?

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The official implementation is from Apple at this link: [https://github.com/apple/ml-cvnets](https://github.com/apple/ml-cvnets)

The timm library also implemented it and has pre-trained weights at this link: [https://github.com/huggingface/pytorch-image-models/blob/82cb47bcf360e1974c00c35c2aa9e242e6b5b565/timm/models/mobilevit.py](https://github.com/huggingface/pytorch-image-models/blob/82cb47bcf360e1974c00c35c2aa9e242e6b5b565/timm/models/mobilevit.py)


Comments:
Hi @SunHaozhe , I would like to work on implementing this model.

This was added in #22820
Add GPT-2-climate
Author: saeedashraf
Pull request: no
Labels: name: New model

### Model description

GPT-2 was pretrained on a climate change-related corpus consisting of over 500 thousand abstracts of top climate scientists' articles from trustable sources covering large temporal and spatial scales. The climate-gpt-2 model could further be used for downstream tasks in the climate change domain, including Classification, Fact-checking, and text generation (climate change-related texts).

paper: https://www.climatechange.ai/papers/neurips2022/27

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation


@seashr

Comments:
Hey @saeedashraf are we expecting to work on integrating this in Hugging Face? If so then I'll be interested in helping out.

Hi Manish,

Ye ... we would like to fully integrate this.


On Mon, Jan 9, 2023 at 5:47 PM Manish ***@***.***> wrote:

> Hey @saeedashraf <https://github.com/saeedashraf> are we expecting to
> work on integrating this in Hugging Face? If so then I'll be interested in
> helping out.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/20747#issuecomment-1375935797>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFLUPIJHVKSB5GHO53JETW3WRQ6LTANCNFSM6AAAAAAS47TOKU>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


Okay, so the GPT model itself is available in HuggingFace. Do we wish to incorporate this dataset? or just the training objective?

I'll close this issue since it does not seem any weights or code are available. Feel free to re-open if that is the case.
[`chat`] generate parameterization powered by `GenerationConfig` and UX-related changes
Author: gante
Pull request: yes

# What does this PR do?

The main goal of this PR is to enable user-friendly `generate` parameterization. This also facilitates performance-related customization, which will be the focus of a follow-up PR.

After the deprecation cycle, new users typing `transformers chat -h` will be redirected to a (new) [docs intro section to generation arguments](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38047/en/llm_tutorial#common-options), instead of seeing a wall of CLI arguments. For `transformers` power users, `chat` is now usable and can be parameterized without prior knowledge about the CLI. These changes were inspired by the idea that [CLIs should be a conversation](https://clig.dev/#conversation-as-the-norm) and that [they shouldn't drown users in information](https://clig.dev/#saying-just-enough)

More specifically, with this PR:
1. We can accept almost any `generate` flag as a positional argument, present and future, as opposed to being limited to a set of hardcoded flags;
2. We can pass a `generation_config.json`, for power users to pass complex `generate` arguments that may be difficult to specify in a CLI;
3. User chat commands are clearly distinguished from potential chat entries -- they now start with `!`
4. `!status`, a new command, can be used to print state-related information, such as the current `generate` flags
5. `!set` can now be used to set arbitrary `generate` flags
6. `!reset` was removed -- it was providing minimal benefits (relaunching the CLI with the previous command is the same) but it was requiring us to maintain and pass the input state around
7. help is now printed if there is a typo in a user command (e.g. `!stats` -> not a valid command -> prints error and help)
8. (non-chat specific) There is a new [intro section to `generate` args in the docs](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38047/en/llm_tutorial#common-options), allowing a soft-landing into the parameterization of the text generation universe

Example usage:
```
transformers chat Qwen/Qwen2.5-0.5B-Instruct do_sample=False max_new_tokens=10
```

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38047). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix OneFormer integration test
Author: qubvel
Pull request: yes

# What does this PR do?

Fix OneFormer integration test

Comments:
run-slow: oneformer

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['models/oneformer']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38016). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

slow tests passed: https://github.com/huggingface/transformers/actions/runs/14905302034

cc @ydshieh when you have bandwidth 

wow, you merged after 1 nano second I approved 😮 

@ydshieh auto-merge magic 🪄
docs: fix md style
Author: imba-tjd
Pull request: yes

trivial

![image](https://github.com/user-attachments/assets/05135ccc-85a8-458f-9a4b-2222423731f4)


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Simple but still worth fixing. Thank you!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38057). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
refactor: background thread for CB
Author: McPatate
Pull request: yes

Building up on Arthur's work, adding a background thread for efficient processing of incoming inputs in an asynchronous manner.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37926). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix mt5 test on AMD devices
Author: ivarflakstad
Pull request: yes

The result of this test on AMD is 0.000103, which the test correctly catches as being larger than 0.0001, but it's close enough for the purposes of this test so I simply increased the limit to 0.0002.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38081). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
transformers require torch >= 2.1.0 to run fp8 model, but im using 2.7.0
Author: O5-7
Pull request: no
Labels: name: bug

### System Info

python = 3.9
torch = 2.7.0+cu128
transformers =  4.51.3


### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

download Qwen3-1.7B-FP8
run  [quick start](https://huggingface.co/Qwen/Qwen3-1.7B-FP8#quickstart) with local model

the result is :

```
  File "D:\anaconda3\lib\site-packages\transformers\models\auto\auto_factory.py", line 571, in from_pretrained
    return model_class.from_pretrained(
  File "D:\anaconda3\lib\site-packages\transformers\modeling_utils.py", line 279, in _wrapper
    return func(*args, **kwargs)
  File "D:\anaconda3\lib\site-packages\transformers\modeling_utils.py", line 4228, in from_pretrained
    hf_quantizer.validate_environment(
  File "D:\anaconda3\lib\site-packages\transformers\quantizers\quantizer_finegrained_fp8.py", line 36, in validate_environment
    raise ImportError(
ImportError: Using fp8 quantization requires torch >= 2.1.0Please install the latest version of torch ( pip install --upgrade torch )
```

Im using torch=2.7.0 , I will try lower version lately

### Expected behavior

no error

Comments:
Hi @O5-7, torch 2.7 should work fine. This kind of thing is usually an environment error, which is quite tricky for us to diagnose! If you can make some code that reproduces the error consistently even on other platforms (e.g. Colab) with torch 2.7, please let us know.

> Hi [@O5-7](https://github.com/O5-7), torch 2.7 should work fine. This kind of thing is usually an environment error, which is quite tricky for us to diagnose! If you can make some code that reproduces the error consistently even on other platforms (e.g. Colab) with torch 2.7, please let us know.

Thank you. I found out that `importlib` caused this issue, which returns the wrong version of Torch.
In `transformers\quantizers\quantizer_finegrained_fp8.py` line 37 :

```
version.parse(importlib.metadata.version("torch")) < version.parse("2.1.0"):
```

test script:
```python
import torch
import transformers
from importlib.metadata import version

print(torch.__version__)
print(transformers.__version__)

print(version("torch"))
print(version("transformers"))
```

and result:
```
2.6.0+cu124
4.51.3
1.12.1
4.51.3
```

I installed different versions of Torch and upgraded importlib, but importlib still returns 1.12.1. and there is no envs in my PC had installed torch=1.12.1
I'm not sure whether it's an issue with my env or with importlib or with torch.
And I dont have hardware to support follow test, becasue the `compute capability` of my GPU is too low for FP8

Hi @O5-7, I'm afraid this is definitely some kind of environment error! I'm not sure why `importlib.metadata.version` is returning the wrong value, but it's not something we can reproduce at our end.

I'm closing for now, but if you can figure out the cause, maybe we can update our version checks to avoid this in future!
[Trainer] tot update steps is incorrect
Author: efsotr
Pull request: no
Labels: name: bug

### System Info

latest main

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

num_examples is the number of examples in the dataloader and is not the number of batches in the dataloader, and is not the number of batches for one process. so the remainer is incorrect. 
when steps_in_epoch % args.gradient_accumulation_steps == 0 and args.gradient_accumulation_steps > 1,  total_updates is one more than expected.
https://github.com/huggingface/transformers/blob/7bb619d710ea3bcddeedb2e7999dff4e124aee85/src/transformers/trainer.py#L2497-L2503

num_examples from:
https://github.com/huggingface/transformers/blob/7bb619d710ea3bcddeedb2e7999dff4e124aee85/src/transformers/trainer.py#L1756-L1768


### Expected behavior

correct code is
```python
 remainder = steps_in_epoch % args.gradient_accumulation_steps 
 if remainder == 0: 
     remainder = args.gradient_accumulation_steps 
 update_step = -1 
 total_updates = steps_in_epoch // args.gradient_accumulation_steps + int(remainder < args.gradient_accumulation_steps)
```

Comments:
cc @SunMarc
uninstall `kernels` from docker images
Author: ydshieh
Pull request: yes

# What does this PR do?

`kernels` may give different outputs (within 1e-5 range) even with the same model (weights) and the same inputs.

https://huggingface.slack.com/archives/C01NE71C4F7/p1747055897244909

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38083). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Should we just remove hub-kernels in integration

I would love to do that too. Let's see how @danieldk says.

I will merge as it is at this moment.