noamaanMulla-03/transformers-issues huggingface dataset:
Remove irrelevant comment in `_update_causal_mask`
Author: fxmarty
Pull request: yes

Forgot to push a commit to https://github.com/huggingface/transformers/pull/32227, my bad!

Feel free to merge this one if approved!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32363). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Use AutoModelForCausalLM to load the Qwen2.5 model, and then use the save_pretrained method to save it to another folder. The model weight will double! ! ! Why is this?
Author: lalalabreeze
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.44.2
- Platform: Linux-3.10.0-1160.el7.x86_64-x86_64-with-glibc2.31
- Python version: 3.11.9
- Huggingface_hub version: 0.25.1
- Safetensors version: 0.4.5
- Accelerate version: 0.34.2
- Accelerate config:    not found
- PyTorch version (GPU?): 2.4.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A800-SXM4-80GB

### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

config = AutoConfig.from_pretrained("Qwen/Qwen2.5-3B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B", config=config)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B")

model.save_pretrained(output_dir, safe_serialization=True)
tokenizer.save_pretrained(output_dir)

### Expected behavior

![image](https://github.com/user-attachments/assets/42211fe5-6dc0-4661-a9ed-ad2e1aa1c5a5)


Comments:
The model weights are `torch.bfloat16`, but by default models are loaded with `torch.float32`, which is twice as big. If you set `torch_dtype=torch.bfloat16` when loading then you will load with the same dtype, and resave with the same size!
Torchao weights only + prequantized compability
Author: SunMarc
Pull request: yes

# What does this PR do ? 

This PR pins the torch version to 2.5 since torchao needs it in order to load the pre-quantized model with weights_only=True. We also fix a few minor bug. I've added some tests for the serialization logic.

cc @jerryzh168 cc @MekkCyber 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34355). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for this PR @SunMarc, really helpful !

added a weights_only check in case the user don't want to upgrade torch
properly fix and RUN_SLOW
Author: ArthurZucker
Pull request: yes

# What does this PR do?
I think I am a bit tired 😅 fixes  #33929 again after #33226 was merged.
```python
 RUN_SLOW=1 pytest tests/models/siglip -k test_inference_interpolate_pos_encoding                                              20s py312 22:53:02
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.12.3, pytest-8.2.2, pluggy-1.5.0
rootdir: /Users/arthurzucker/Work/transformers
configfile: pyproject.toml
collected 606 items / 605 deselected / 1 selected                                                                                                                                 

tests/models/siglip/test_modeling_siglip.py .                                                                                                                               [100%]

================================================================================ warnings summary =================================================================================
src/transformers/deepspeed.py:24
  /Users/arthurzucker/Work/transformers/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
    warnings.warn(

../../.pyenv/versions/3.12.3/envs/py312/lib/python3.12/site-packages/google/protobuf/internal/well_known_types.py:91
  /Users/arthurzucker/.pyenv/versions/3.12.3/envs/py312/lib/python3.12/site-packages/google/protobuf/internal/well_known_types.py:91: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    _EPOCH_DATETIME_NAIVE = datetime.datetime.utcfromtimestamp(0)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================== 1 passed, 605 deselected, 2 warnings in 5.02s
``` 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33965). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> I think I am a bit tired 😅 

🫂 

cc @xenova as you broke them 💞 

Related to #33958
Fix CI by tweaking torchao tests
Author: SunMarc
Pull request: yes

# What does this PR do?
Fixes the torchao tests so that we don't get an error on `tests_hub` CI. The issue was that we were running `TorchAoConfig` when importing the test class. However, since we don't have torchao installed, it raised an error. 

Also, I added more check in torchaoconfig to help the user. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34832). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix CI slack reporting issue
Author: ydshieh
Pull request: yes

# What does this PR do?

#34812 causes the slack report fail to be sent.

This PR fixes this issue (a bit hacky)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34833). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Bump Replicate guard to torch 2.5
Author: qubvel
Pull request: yes

# What does this PR do?

With torch 2.4 `transformers` imports fail on

```
from torch.distributed.tensor import Replicate
```

this PR update import guard to torch >= 2.5

Fixes #34831

## Who can review?



Comments:
Closed in favor of https://github.com/huggingface/transformers/pull/34816

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34838). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
VLMs: enable generation tests - last batch
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Last 3 VLMs that had no generation tests now support it. After this PR and https://github.com/huggingface/transformers/pull/34174/files is merged we'll have only one model (BLIP) which has no generation tests.

I see only one way of making it, by overwriting all tests for generation in BLIP, The reason is because BLIP adds 'eos' token while tokenizing and then in `self.generate()` removes the last token manually by cropping input ids. Thus we never can check the length of output sequences correctly. 

We could make tokenizer remove 'eos' by post processing outputs in processor code, but that seems too much BC breaking so I am not sure yet if we want it. Plus BLIP doesn't seem to be commonly used anymore when we have better VLMs so prob it doesn't worth spending time refactoring the code

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34484). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Plus BLIP doesn't seem to be commonly used anymore when we have better VLMs so prob it doesn't worth spending time refactoring the code

Yeah, I have this impression as well. Let's hold on the test refactor for BLIP, unless we start getting issues :)
Change logging level from warning to info for `max_steps` overriding `num_train_epochs`
Author: qgallouedec
Pull request: yes

# What does this PR do?

Passing `max_step` is a supported feature, and therefore should not raise a warning. I suggest lowering the level to info.

Alternatively, we could have an additional parameter like `ignore_max_step_warning` but this seems overengineered.

Considering that the use of `max_steps` is already documented:

https://github.com/huggingface/transformers/blob/ce1d328e3b73cf6d1d993fc0d487b7dc8a14d7ee/src/transformers/training_args.py#L307

lowering the level to info seems to be the best solution.


Technically, we can ignore this warnings with 

```python
TrainingArguments(
    ...,
    max_steps=100,
    num_train_epochs=-1
)
```

But it's not intuitive at all IMO.

---

As for warnings, I'd say that a good practice is that they should always be removable by the user:

- either by modifying their code, in the event of a feature being misused, deprecated, etc. E.g example:
    https://github.com/huggingface/transformers/blob/ce1d328e3b73cf6d1d993fc0d487b7dc8a14d7ee/src/transformers/models/speech_to_text/feature_extraction_speech_to_text.py#L235-L238

- or by acknowledging, in the case of very important information upgraded to warning; we add a parameter for acknowledging. E.g:
example: `allow_missing_key` in `load_pytorch_checkpoint_in_tf2_model`

Related: https://github.com/huggingface/trl/pull/2350

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@muellerzr and @SunMarc



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34810). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Unrelated errors, merging
Add Nemotron GGUF Loading Support
Author: farrosalferro
Pull request: yes

# What does this PR do?
Add Nemotron GGUF loading support

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. Link: [Community contribution: Adding GGUF support for more architectures #33260](https://github.com/huggingface/transformers/issues/33260)
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?
@SunMarc @LysandreJik @ArthurZucker 


Comments:
Could you please take a look at this PR? @SunMarc @lyaronskaya @ArthurZucker .
Thank you!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34725). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
When will transformers trainer supports fp8
Author: wangxiaoyang-dev
Pull request: no
Labels: name: Feature request

### Feature request

Hope transformers trainer can support fp8 .

### Motivation

Currently, accelerate has support fp8, so we hope transformers trainer can support fp8 to .

### Your contribution

yes, I can help by submitting a PR .

Comments:
cc @SunMarc 

Everything that it supported in accelerate is available on transformers since the backend of trainer is using accelerate @wangxiaoyang-dev 

> Everything that it supported in accelerate is available on transformers since the backend of trainer is using accelerate @wangxiaoyang-dev

Thanks.
pretrained_model_name_or_path why not absolute path from /
Author: suhjohn
Pull request: no
Labels: name: Feature request

### Feature request

I don't understand why only relative paths are supported for pretrained_model_name_or_path. 

### Motivation

```
OSError: Incorrect path_or_model_id: '/models/colpaligemma-3b-pt-448-base'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
```

Above is a stupid error that shouldn't happen.

### Your contribution

No

Comments:
Hi @suhjohn! Can you please provide a code example to reproduce the error? Thanks

Absolute paths are supported for this. You either made a mistake in the directory name, the directory doesn't contain a valid model for the class you're using, or you're using Unix-style paths on Windows.
Fix ds nvme
Author: eljandoubi
Pull request: yes

# What does this PR do?

Fixes #34429 

## Who can review?

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr


Comments:
@SunMarc @muellerzr Do you have any feedback?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34444). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

pinging @muellerzr !

Hey @SunMarc. When using NVMe offloading, recursively calling `deepspeed.zero.Init` causes a pinning error.  Because Transformers and accelerate do not support multiple models, `deepspeed.zero.Init` should be called at most once per process.

Thanks for the additional context ! I think that we do support multiple models with deepspeed https://huggingface.co/docs/accelerate/en/usage_guides/deepspeed_multiple_model . cc @muellerzr 

Anyway, we might use a context manager solution to prevent embedded calls to DeepSpeed.zero.Init.

@ArthurZucker @SunMarc @muellerzr Is there any update on the pull request?

@SunMarc can you underscore them? 

Just missing `make style`
Heuristic scheduling for UAG
Author: jmamou
Pull request: no

https://github.com/huggingface/transformers/blob/5de58d595547f57dd4df2f3572158eb9cbdd24c9/src/transformers/generation/candidate_generator.py#L238-L261

When using [Universal Assisted Generation (UAG)](https://huggingface.co/docs/transformers/en/generation_strategies#universal-assisted-decoding), draft and target vocabularies are different.

In `heuristic` scheduling, the `update_candidate_strategy` function adjusts the maximum number of draft tokens for the next iteration by verifying whether num_matches equals self.num_assistant_tokens.

However, in UAG:
- num_matches: Represents the number of matched tokens according to the **target** vocabulary.
- self.num_assistant_tokens: Reflects the number of tokens generated by the **draft** model, based on the **draft** vocabulary.



Comments:
[run_slow] idefics2
Author: andimarafioti
Pull request: yes
Labels: name: run-slow

# What does this PR do?

Fixes #32288


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Models:

- vision models: @amyeroberts


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32840). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@andimarafioti Do you have permission to merge in `transformers`? If not I can merge for you
Refactor StarCoder2 using modular
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Refactor StarCoder2 to remove all the copied-from and use modular instead.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34015). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix: take into account meta device
Author: tibor-reiss
Pull request: yes

Fixes #34091

The models given in the issues' example have different torch_dtype which results in a different handling in `check_support_param_buffer_assignment`.
fxmarty/small-llama-testing: torch_dtype=float32
fxmarty/tiny-llama-fast-tokenizer: torch_dtype=float16


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@LysandreJik @fxmarty @muellerzr


Comments:
cc @SunMarc for initial review

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34134). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Thanks for adding this @tibor-reiss! Can you also add a test so that this doesn't happen in the future ?

@SunMarc I was not sure where to add the test, so please feel free to move it around. Additionally, the TINY_* models are float32, so I went with the models from the issues' example (they are quite small) - let me know if there is a better way.

Friendly ping @fxmarty-amd
Generate: move `prepare_inputs_for_generation` in encoder-decoder llms
Author: gante
Pull request: yes

# What does this PR do?

Part of step 6 in https://github.com/huggingface/transformers/issues/32685
Follow-up to https://github.com/huggingface/transformers/pull/33870

This PR:
- Adds a minor change to `GenerationMixin.prepare_inputs_for_generation` to use `decoder_input_ids` in encoder-decoder models
- Deletes almost all `prepare_inputs_for_generation` in encoder-decoder llms 🔪 😎 

Comments:
@zucchini-nlp this PR may have a conflict with your encoder-decoder+compile PR 👀 

Ran the following slow tests before merging:
- Llama 
- BART
- T5 (same failures as main)
Watermarking: fix order
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/34630

cc @sumedhghaisas2 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34849). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Beam search incorrectly sampled 2*num_beams tokens instead of num_beams tokens.
Author: transcend-0
Pull request: no
Labels: name: bug

name: Generation

### System Info

Python 3.10.11
transformers > 4.40 (all versions released so far)
torch 2.0.1
Linux version 4.15.0-55-generic x86_64

### Who can help?

@gante 

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

the source code in transformers.generation.utils.py (line 3299 in v4.45.0)

![image](https://github.com/user-attachments/assets/31902081-d37c-4dbb-8cc9-ffc532e6cf29)



### Expected behavior

Why is `max(2, 1 + n_eos_tokens)` used here instead of `max(1, 1 + n_eos_tokens)` ?

This results in most of the time, beam search first samples **_2*num_beams_** tokens, then takes the **_top num_beams_**, rather than directly sampling **_num_beams_** tokens. This operation is different from the beam search we generally consider.


Comments:
TL;DR if all beams pick `eos_token_id` as the next token, generation would stop. However, depending on the parameterization, it may still be possible to continue beam search and get a better score (see `beam_scorer.is_done` and how this attribute is set). We want to guarantee there are at least `num_beams` valid continuations for the next iteration, so we can continue beam search to potentially get a better score.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.