noamaanMulla-03/transformers-issues huggingface dataset:
Fix long lagging when streaming text without spaces and NJK chars
Author: CNSeniorious000
Pull request: yes

- Caused by #22664
- Resolves #28849

As shown in the first screen record in [this comment](https://github.com/huggingface/transformers/pull/22664#issuecomment-1509469001) from @bcol23, `TextStreamer` will always refused to flush the buffer until it encounters a space or an CJK char.

An easier reproduction is just ask a model to

```
"Say a long long string of English letters without any spaces."
```

If you ask any instruct model with this prompt, and use `TextStreamer` to show the output, you can't see the chars streaming out because they are not CJK chars and `text[self.print_len : text.rfind(" ") + 1]` fail to find a new space anywhere.

https://github.com/huggingface/transformers/blob/69bc848480d5f19a537a70ce14f09816b00cd80f/src/transformers/generation/streamers.py#L107-L115

# What does this PR do?

This PR made a tweaks on the logic, making it usefully better:

1. In the situations where multiple tokens construct one char, it is only one char. So text[:-1] is always safe (which means that they will never change)
2. Not only CJK chars but also simple ascii letters and punctuations are safe too.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@gante because you reviewed #22664. Thank you!


Comments:
Is the CI flaky yet? I can't tell how this PR affects it.
Gemma3 fp16 fix
Author: mobicham
Pull request: yes

# What does this PR do?
Fixes float16 inference with Gemma 3 models: simply clip the input in `Gemma3RMSNorm`

Fixes https://github.com/huggingface/transformers/issues/36822

## Who can review?
@ArthurZucker @Rocketknight1



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).
One more fix for reviewer assignment
Author: Rocketknight1
Pull request: yes

There was another bug I missed - the Github API doesn't always give start absolute paths from the root with `/`. This updated regex handles that!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36829). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Support tracable dynamicKVcache
Author: tugsbayasgalan
Pull request: yes

# What does this PR do?

Recently, https://github.com/huggingface/transformers/pull/35873 was landed to support static KV cache with export. As a side effect, it now also made it easy to support dynamic KV cache. Since all HF models accept KVCache as extra input, we can represent dynamic KV cache as a pytree container. Credit goes to @xadupre who attempted this approach in https://github.com/pytorch/pytorch/issues/147326. 

cc @guangy10 @IlyasMoutawwakil 



Comments:
cc @gante @zucchini-nlp as well!

Awesome work ! This is much cleaner than my attempt in https://github.com/huggingface/transformers/pull/35792 where I tried to make the cache a tensor subclass, that didn't work because dynamic cache doesn't always contain tensors (empty lists at first), but simply adding directives for pytree to de-construct and re-construct the class doesn't come with that constraint 😍.

can you please run `make style` to pass styling tests 🤗

@gante It is bit hard for me to disable the linter on my side. Running make style doesn't seem to help. If possible, could you land lint improvement after landing my PR :) 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36311). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add Space to Bitsandbytes doc
Author: MekkCyber
Pull request: yes

# What does this PR do?
Add the space link to Bitsandbytes doc, and overview quantization doc

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36834). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add internlm3 dense
Author: HIT-cwh
Pull request: yes

# Adding InternLM3

Implementation of the InternLM3 models. Model weights can be requested [here](https://huggingface.co/internlm/internlm3-8b-instruct).

Models can then be loaded via:

```
tokenizer = transformers.InternLM3Tokenizer.from_pretrained("internlm/internlm3-8b-instruct")
model = transformers.InternLM3ForCausalLM.from_pretrained("internlm/internlm3-8b-instruct")
```

Example:

```
model = model.eval()

system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文."""
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Please tell me five scenic spots in Shanghai"},
 ]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")

generated_ids = model.generate(tokenized_chat, max_new_tokens=1024, temperature=1, repetition_penalty=1.005, top_k=40, top_p=0.8)

generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(tokenized_chat, generated_ids)
]
prompt = tokenizer.batch_decode(tokenized_chat)[0]
print(prompt)
response = tokenizer.batch_decode(generated_ids)[0]
print(response)
```

Requirements:
```
transformers >= 4.48
```

## TODO

1. Complete docs
2. Pass lint

Comments:
Hey @HIT-cwh! Thanks for the contribution this is very cool!
However, I don't really see any difference with Llama in the model architecture? Looks like you can directly use the existing Llama architecture with the model for full Transformers support! Let me know if I'm wrong! 🤗

Super happy to add `.md` file presenting your model and how to use with `transformers`! 

> Super happy to add `.md` file presenting your model and how to use with `transformers`!

Done [here](https://github.com/huggingface/transformers/pull/35694/commits/f1f9e745bbb28d0b7fb041e208b64b8d8dcbc281) !

Hey @ArthurZucker @Cyrilvallez ! Thank you for your response! We still have some training tricks (like deepnorm) that haven't been updated yet, and they will be added in the near future. Additionally, we have already adapted the `InternLM3ForCausalLM` class in some downstream toolchains. We believe having a separate class would be more convenient for distinguishing between models.

super sorry for the late review and thanks a lot for contribution!

Let's add this 🔥 

Hi @ArthurZucker ! Sorry for the late reply! Really appreciate your detailed review and suggestions.

Just pushed a few commits with some training tweaks (like adding DeepNorm). Would you mind taking another look when you get a chance?

cool~
Remove our AdamW implementation
Author: Rocketknight1
Pull request: yes

Transformers added an `AdamW` implementation before Torch supported it. However, Torch supports it now so there's not really much point in maintaining our own version!

This PR deletes our `AdamW` class, but imports `torch.optim.AdamW` in the same file, to ensure that imports that depended on it still work.

Fixes #35504

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36177). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ready for core maintainer review @ArthurZucker @Cyrilvallez 

cc @ArthurZucker I cut it out entirely (it was raising a deprecation warning every time it was being used anyway). I refactored the references to it to use `torch.optim.AdamW` instead.

cc @arthurzucker @cyrilvallez I think this should be ready to go, but I'd like a core maintainer approval first!

The plan is to just totally remove our AdamW class and redirect all the legacy references to it to use `torch.optim.AdamW` instead. The class **has already been throwing a deprecation warning for some time**, so I think people have had enough notice by now.
Fix test isolation for clear_import_cache utility
Author: sambhavnoobcoder
Pull request: yes

## Problem
The `test_clear_import_cache()` test in `tests/utils/test_import_utils.py` is affecting the state of subsequent tests, causing failures in the test suite. This is happening because the test modifies the global module cache without properly restoring it.

## Solution
Updated the test to:
1. Save the initial state of all transformers modules before running the test
2. Run the test to verify `clear_import_cache()` functionality
3. Restore the initial state after the test completes, including proper handling of `_LazyModule` instances

The core functionality of `clear_import_cache()` remains unchanged - this fix only addresses test isolation.

Edit : #36334

## Screenshots
<img width="780" alt="Screenshot 2025-02-22 at 4 43 58 PM" src="https://github.com/user-attachments/assets/c539e8f9-c800-4366-a869-35dd5c859e01" />

cc @dvrogozh

Comments:
cc @ydshieh to this PR and the related issue at #36334

Apologies for the delay! This was a minor test isolation issue related to the environment—completely slipped my mind. I've fixed it now, and all test cases, including the previously failing ones, are passing. Hopefully, this gets merged soon so we can wrap up this issue quickly.

<img width="1407" alt="Screenshot 2025-03-14 at 6 51 31 PM" src="https://github.com/user-attachments/assets/22b972c4-c27d-487a-8dc1-5b1a9794960e" />



thank you for the comments @ydshieh . i have looked at all of them and resolved all of them . please tell me if you need anything else here . i'll make that happen as well . 

Thank you a lot @sambhavnoobcoder for the fix and thank @dvrogozh for opening the issue 🙏 ❤️ 

These changes have been made as well . i'll gladly look into any more comments on this PR if any more improvements are required .
Marian RNN conversion support
Author: FricoRico
Pull request: no
Labels: name: Feature request

### Feature request

Add support for converting Marian Transformer RNN models.

### Motivation

Firefox is doing great work at training new models. Their teacher models are able to be converted to PyTorch models using [existing conversion tooling](https://github.com/huggingface/transformers/blob/main/src/transformers/models/marian/convert_marian_to_pytorch.py).

However their student models which are way smaller, and more efficient, are structured as Transformer-RNN. This is currently not supported by the conversion tools. Would it be even possible to add support for this?

### Your contribution

At this point I'm just a parrot trying to seek help and information on this topic. It is far outside my current knowledge on what this exactly means. Perhaps if someone could point me in the right direction I could figure this out.

Comments:
Hi @FricoRico, I don't know if the Marian modeling code in Transformers supports Transformer-RNN architectures either! This means that you'd need to either:

1) Convert the models as "custom code" models: https://huggingface.co/docs/transformers/en/custom_models
2) Write a full PR to add the Transformer-RNN architecture to Transformers

@Rocketknight1 Yeah you are right, ideally the Marian tools would also need to be expanded to support Transformer-RNN for inference. But I guess step one would be to even allow to export the models in the first place. But perhaps I'm over simplifying things.

In general, we need modeling code first in order to support conversion of model checkpoints, rather than the other way around! The model code provides the "architecture" that runs a particular set of weights.

I guess the two will have to go hand-in-hand then? Because you will need to have a model to test with? The original models are in `npz` format.

Usually, yes! You'd do something like:

1) Make a PR branch that adds the modeling architecture
2) Add a conversion script to that branch
3) Convert the models and add them to the Hub (they won't work in normal Transformers until the PR is merged)
4) Add tests to the PR branch to make sure everything's working okay
5) Get reviews + merge the PR

As much as I'd love to write an implementation, I'm by far not knowledgable enough to even comprehend where to start or what to do. Perhaps one day in the future I could invest time to dive deep into the subject. Hopefully there is someone out there more skilled than me that is also interested in this implementation.

Alternatively perhaps it is possible to train MarianMT student models in a regular (non-RNN) Transformer model. But I can't find much information about this, probably for good reasons.
Wav2vec model further training [RuntimeError: you can only change requires_grad flags of leaf variables]
Author: xinghua-qu
Pull request: no
Labels: name: bug

### System Info

```shell
Latest version
```


### Who can help?

@patrickvonplaten, @anton-l
I use wav2vec model as part of my own pytorch model. 
`self.configuration = Wav2Vec2Config()
 self.wav2vec_feature = Wav2Vec2Model(self.configuration)
self.wav2vec_feature = self.wav2vec_feature.train()`
However, it raises error when I call wav2vec model in my own forward function.
The error is:
`File "/home/tiger/.local/lib/python3.7/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 440, in forward
    hidden_states.requires_grad = True`



### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

`configuration = Wav2Vec2Config()
 model = Wav2Vec2Model(configuration)
model = model.train()`

put this 'model' into a forward function in pytorch.

### Expected behavior

```shell
The wav2vec model inserted in my own model is expected to be trainable.
```


Comments:
Hey @xinghua-qu,

Could you please provide a fully reproducible code snippet here? E.g. something along the lines:

```python
from transformers import Wav2Vec2Model
....
```

More than happy to look into solving it then - thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@patrickvonplaten I am facing the same issue! Here is the snippet:
`from transformers import Wav2Vec2ForSequenceClassification
model = Wav2Vec2ForSequenceClassification.from_pretrained("facebook/wav2vec2-base-960h")`
when I freeze the FE `self.model.freeze_feature_extractor()` training is fine otherwise I get:
`RuntimeError: you can only change requires_grad flags of leaf variables.`

Having the same error, any news on this?
fix: update vocab size of language model's config on resize - mllama
Author: Ssukriti
Pull request: yes

# What does this PR do?

The PR updates the config of the mllama language model to update its vocab size. This is useful when vocab size of `MllamaForConditionalGeneration` class is updated when resizing embeddings using `model.resize_token_embeddings(embedding_size)` . **More details included in the issue below.**

The PR includes one potential fix. There are several ways to do this. Suggestions welcome!

Fixes # (issue)
https://github.com/huggingface/transformers/issues/36590

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Models:
- vision models: @amyeroberts, @qubvel




Comments:
@zucchini-nlp the Pr is ready for review changes have been made as discussed earlier (in resolved conversation above). 

1. all resize embeddings tests are passing, which indicates no change in current behavior 
2. added new unit tests for the function set_vocab_size 
3. mllama bug https://github.com/huggingface/transformers/issues/36590 is fixed. The llava class does not use vocab size in loss, hence the bug did not impact it. On quick look, I didnt find other applicable multimodel classes that could be impacted either. Hence, fix is only for mllama in this PR

@zucchini-nlp sure we can go with that approach instead . I created a new PR https://github.com/huggingface/transformers/pull/36840 as had to undo the changes here. 

If the solution looks ok, can continue with unit tests , thank you!
In the _speculative_sampling function, it seems that the "squeeze" method is being used incorrectly.
Author: ZipECHO
Pull request: no

The squeeze function does not accept input of multiple dimensions.

```
q_i = q[:, torch.arange(candidate_length), new_candidate_input_ids].squeeze(0, 1)
p = new_logits.softmax(dim=-1)
p_i = p[:, torch.arange(candidate_length), new_candidate_input_ids].squeeze(0, 1)
```


https://github.com/huggingface/transformers/blob/706703bba6c920b10aa7e7ee8163b06a8a03c450/src/transformers/generation/utils.py#L4745C1-L4747C86


Comments:
Hi @ZipECHO, this is actually a call to the `Tensor.squeeze()` method and not the `torch.squeeze()` function. `Tensor.squeeze()` will accept `*args`, so `x.squeeze(0, 1)` works fine!

This seems to be a problem caused by an outdated torch version, which does not exist in the torch=2.x version and later.
Fix attention_mask dimension issue in GPT2Model
Author: sukwoojang
Pull request: yes

# What does this PR do?

This PR fixes a bug in GPT2Model where the 'attention_mask' was incorrectly reshaped from 4D to 2D.
the fix ensures that 'attention_mask.view()' is only applied when the dimension is 2, preventing shape mismatches.

Fixes #35290


## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      [issue link](https://github.com/huggingface/transformers/issues/35290)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker, since this PR affects text models.
@Rocketknight1, since this PR modifies attention mask handling.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

Hi @sukwoojang, can you check the `test_gpt2_double_lm_head_model` test that's failing here? It may need to be updated based on these changes!
Move the warning to the documentation for DataCollatorWithFlattening
Author: qgallouedec
Pull request: yes

A warning shouldn't indicate normal functioning. I suggest moving this warning to the doc.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36707). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix swanlab global step
Author: Zeyi-Lin
Pull request: yes

# What does this PR do?

A tiny fix to make the curves displayed by the `swanlab` logger align with the actual steps, provide a better experience for developers

Before this PR, if you set `logging_steps` to 5, the points in the line chart on SwanLab would still be spaced 1 step apart (without passing the `step` parameter in `swanlab.log`) 💦; after the modification, the points are spaced 5 steps apart, consistent with `logging_steps` 🎉

![image](https://github.com/user-attachments/assets/5c4c0e4a-b681-41c6-86c8-52628d57e59d)


---

This pull request includes a minor change to the logging functionality in the `on_log` method within the `integration_utils.py` file. The change ensures that the global step is included in the logging calls for better tracking and consistency.

Logging improvements:

* [`src/transformers/integrations/integration_utils.py`](diffhunk://#diff-70576373928eee203a92800d5ea1f6270d45462a9bde494cde65338710a0a331L2294-R2297): Modified the `on_log` method to include the `step` parameter in calls to `self._swanlab.log` for both single value scalars and non-scalar logs.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

cc @sunmarc @muellerzr @mekkcyber (feel free to yell at me if i should have pinged someone else)

> cc @SunMarc @muellerzr @MekkCyber (feel free to yell at me if i should have pinged someone else)

thanks! I updated my description in [the comment](https://github.com/huggingface/transformers/pull/36728#issue-2920707398) to make the significance of this PR clearer 😄.