noamaanMulla-03/transformers-issues huggingface dataset:
NaN model parameter found in  meta-llama/Llama-3.2-11B-Vision under 4.46.1 version
Author: cuong-dyania
Pull request: no
Labels: name: bug

### System Info

We tried with H100/A100 GPU machine both got the same issues.
transformers==4.46.1
torch==2.3.1+cu121


### Who can help?

 @ArthurZucker

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction
```python
import transformers
import torch
from transformers import (
    AutoConfig,
    AutoModelForCausalLM)

print(transformers.__version__)
print(torch.__version__)

model_name_or_path ="meta-llama/Llama-3.2-11B-Vision"
def check_for_nan_parameters(model):
    for name, param in model.named_parameters():
        if torch.isnan(param).any():  # Check if any value in the parameter tensor is NaN
            print(f"NaN found in parameter: {name}")
            return True
    print("No NaNs found in model parameters.")
    return False


model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
        torch_dtype=torch.bfloat16,device_map ='auto')

print(check_for_nan_parameters(model))
```

# The above code will return 
#NaN found in parameter: model.layers.0.input_layernorm.weight
# True

### Expected behavior

We should not see NAN in model parameters. The above codes should return False.

We tried the above codes for transformers==4.45.2 and did not see any NaN in model parameters. 

Comments:
Hey! you have a warning that says:
```python
Some parameters are on the meta device because they were offloaded to the cpu.
NaN found in parameter: model.layers.0.input_layernorm.weight
```
the weights are just on the `meta` device for fast loading, until you run a forward pass 😉 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Adding CohereForSequenceClassification to Cohere
Author: liuhaoran124578
Pull request: no
Labels: name: Feature request

### Feature request

I’ve noticed that the Cohere models do not include a ForSequenceClassification implementation. Is it due to some issues that prevent this from being done, or was there simply not enough time to implement this requirement?

### Motivation

Recently, I've been learning how to use large models for human dialogue preference classification. I noticed that Cohere has a model called aya-expanse-8b. I wanted to use this model but found that there is no corresponding ForSequenceClassification implementation available for it. However, I noticed that such an implementation is available for Gemma2.

### Your contribution

I have attempted to implement it by following the approach used for Gemma2, but I have not yet validated it.
If it’s merely a time issue preventing the implementation of ForSequenceClassification in transformers, I’d be more than willing to contribute by adding this feature for Cohere.

Comments:
I think we'd be happy to accept this PR, yes!

> I think we'd be happy to accept this PR, yes!

thx😊
Add Falcon3 documentation
Author: mokeddembillel
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Adding Falcon3 model documentation

## Who can review?
@ArthurZucker
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
🚨🚨🚨 Delete conversion scripts when making release wheels
Author: Rocketknight1
Pull request: yes

This PR updates `release.py` to delete model conversion scripts. These scripts are generally included with specific model classes to convert checkpoints in non-Transformers formats. Often these scripts have to open insecure file types, because those were the file types the model was released with (e.g. `pickle` or old Torch `.bin` checkpoints). This results in vulnerability scanners flagging us, and can cause compliance issues for users. 

We don't see this as a serious attack vector in practice because users would have to be induced to download a malicious file and call an obscure conversion script on it, but excluding these files from release wheels should help with compliance issues!

Fixes #34840

Comments:
cc @LysandreJik @ArthurZucker for core maintainer review! I checked with `make pre-release` locally and it worked as intended.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35296). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
DOC: Typo In Audio Classification Documentation
Author: Uvi-12
Pull request: no

The term "finetune" is used throughout the documentation, but the correct form should be "fine-tune".

Comments:
I think both are acceptable! Words that are constantly used in the same hyphenated pairing in English often get merged into a single word over time (e.g. "e-mail" was the only correct form 20 years ago but now "email" is completely fine). See the [relevant Wikipedia section](https://en.wikipedia.org/wiki/Hyphen#Prefixes_and_suffixes) for more.
remove `benchmark` job in `push-important-models.yml`
Author: ydshieh
Pull request: yes

# What does this PR do?

As discussed offline: the benchmark workflow is updated and is triggered directly with push to main / pull_request events.
There is no need to be called in  `.github/workflows/push-important-models.yml` (and it's currently failing due to the format mismatching)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35292). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix image preview in multi-GPU inference docs
Author: jla524
Pull request: yes

# What does this PR do?

Fixes image preview in multi-GPU inference docs

before:
<img width="1475" alt="image" src="https://github.com/user-attachments/assets/d644af47-9584-4b7d-9114-6240eead0d50" />

after:
<img width="1475" alt="image" src="https://github.com/user-attachments/assets/8910c657-a1e8-4b57-a7c3-9167a30fc4f6" />


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu

Comments:
Fix remove unused parameter in docs
Author: zzzzzsa
Pull request: yes

# What does this PR do?

Remove unused parameter in batch inference example in llava doc.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@stevhliu

Comments:
Add Cohere2 docs details
Author: alexrs-cohere
Pull request: yes

# What does this PR do?
Adds more details in the `cohere2` model docs.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker, @stevhliu 

Comments:
[docs] Improve register_pipeline
Author: stevhliu
Pull request: yes

Provides additional clarification that `register_pipeline` registers a pipeline's implementation, task type, and supported backend to a models `config.json` file.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35300). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add config.json explicit change when using custom pipelines.
Author: tcapelle
Pull request: yes

# Improves the docs for custom pipeline.

Something I would have loved to know!

Documentation: @stevhliu 

Comments:
Feel free to out it where it makes more sense, but it should be somewhere.
When you are developing locally you need this functionality so it shouldn't be only when pushing to the hub.
Sent from Proton Mail Android
-------- Original Message --------On 12/16/24 08:49, Steven Liu  wrote:
@stevhliu commented on this pull request.

Thanks for the feedback!
Maybe we can add it to the paragraph above the code snippet demonstrating register_pipeline. I think something like the below would be nice. What do you think?
The register_pipeline function registers the pipeline details to a models config.json file.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>
[
{
***@***.***": "http://schema.org",
***@***.***": "EmailMessage",
"potentialAction": {
***@***.***": "ViewAction",
"target": "https://github.com/huggingface/transformers/pull/35284#pullrequestreview-2506753406",
"url": "https://github.com/huggingface/transformers/pull/35284#pullrequestreview-2506753406",
"name": "View Pull Request"
},
"description": "View this Pull Request on GitHub",
"publisher": {
***@***.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]

Updated in #35300!

Lol but you totally omitted the content of the JSON file.
Sent from Proton Mail Android
-------- Original Message --------On 12/17/24 19:28, Steven Liu  wrote:
Updated in #35300!

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>
[
{
***@***.***": "http://schema.org",
***@***.***": "EmailMessage",
"potentialAction": {
***@***.***": "ViewAction",
"target": "https://github.com/huggingface/transformers/pull/35284#issuecomment-2549279277",
"url": "https://github.com/huggingface/transformers/pull/35284#issuecomment-2549279277",
"name": "View Pull Request"
},
"description": "View this Pull Request on GitHub",
"publisher": {
***@***.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]

Oops my bad, I totally forgot! I'll ping you in the new PR that fixes this :)
Export-to-ExecuTorch via `optimum` integration
Author: guangy10
Pull request: no
Labels: name: Feature request

name: ExecuTorch

### Feature request

Integrate `ExecuTorch` to `optimum, enabling an new ["Export to ExecuTorch"](https://github.com/huggingface/transformers/issues/32253) workflow

### Motivation

Enable a new e2e workflow for on-device ML use-cases via [ExecuTorch](https://github.com/pytorch/executorch)

### Your contribution

TBD

Comments:
Open a master task in Optimum: https://github.com/huggingface/optimum/issues/2128

We can close this one
Torch is installed but still getting this error "RuntimeError: At least one of TensorFlow 2.0 or PyTorch should be installed. "
Author: kartikihx
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.47.0
- Platform: Linux-5.11.0-1021-azure-x86_64-with-glibc2.31
- Python version: 3.10.16
- Huggingface_hub version: 0.27.0
- Safetensors version: 0.4.5
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.5.1+cu124 (False)
- Tensorflow version (GPU?): 2.18.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. I created a new environment in conda & activated it with python=3.10
2. Installed jupyter, transformers,tensorflow,torch
3. Tried running this code in jupyter
```
from transformers import pipeline
sentiment_pipeline = pipeline("sentiment-analysis")
```
4. Got the error "RuntimeError: At least one of TensorFlow 2.0 or PyTorch should be installed."

### Expected behavior

Hi @sanchit-gandhi @Rocketknight1 & HF Team,
I am not able to run transformer pipelines due to the error mentioned in the reproduction steps.
Please advise, thank you!

Comments:
This is unusual - I haven't seen any reports of this failing for anyone else, which suggests it might be an environment issue. Can you try a fresh environment with conda or something like that and check if the issue recurs?

You most probably have installed the packages in a different environment than the current one your using to execute the code. Check it once.

I re-ran and got a new error asking me to install tf-keras. After installing that, I am now able to import transformers successfully
about gradient accumulation
Author: Murasame-sway
Pull request: no
Labels: name: bug

### System Info

none

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hello, I would like to ask whether the latest stable version has fixed the problems related to gradient accumulation? Whether to still only install from source code to avoid this problem

### Expected behavior

none

Comments:
cc @muellerzr 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix loading with only state dict and low_cpu_mem_usage = True
Author: SunMarc
Pull request: yes

# What does this PR do ? 

This PR enables to load a model by passing a state_dict only with `low_mem_cpu_usage=True`. 

Added a test that was not passing before this PR ! 

Required for DDUF (reference: https://github.com/huggingface/diffusers/pull/10037)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35217). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Finish short form / long from generation integration in Whisper 
Author: kamilakesbi
Pull request: no
Labels: name: Feature request

### Feature request

Unify the output format to Whisper Generate method for short form/long form generation. 

### Motivation

In PR #30984, short-form and long-form generation in Whisper were unified so that both benefit from generation with fallback. 

However, the output to the generate method's format still varies depending on whether we're doing short form or long form generation, as we can see in this [line](https://github.com/kamilakesbi/transformers/blob/cb4201e8ed5b087933b6ee48f464d26b7b73f5a7/src/transformers/models/whisper/generation_whisper.py#L722). 

- For **short form generation** the output format can be either a torch tensor containing the sequence of token ids or an instance of `ModelOutput` with additional information (attention masks, hidden states, ...) if `return_dict_in_generate` is set to True (we can now also use `return_segments` with short form generation). 

- For **long form generation** the output is either a torch tensor with the sequence of token ids, or a dict containing the `sequences` of token ids and a list of all `segments` if `return_segments` is set to True. Note that if both `return_dict_in_generate` and `return_segments` are set to true, the additional information (attention masks, hidden states) will be contained in `segments`. However, at the moment we can't get an instance of `ModelOutput` as output with long form generation. 

### Should we work on this ? 

Ideally, we should also unify the output format for the Whisper `generate` method so that users don't have to distinguish between short and long form audio.  They should only have to specify wether they want to perform sequential generation (non chunked) or parallel generation (chunked) with the pipeline. 

The aim of PR #30984 was to implement all the modifications to allow generation with fallback for short form audios without breaking **Backward Compatibility** on main. If we further unify the output format, we would break backward compatibility and have to adapt several tests. 

cc @sanchit-gandhi @ArthurZucker Do you think we should complete the unification of Whisper Generation by unifying the output format? 

Comments:
Closing this as it's been done in https://github.com/huggingface/transformers/pull/34135
Fix EncoderDecoder cache candidate edge case
Author: ylacombe
Pull request: yes

# What does this PR do?
Whisper introduced `EncoderDecoder` cache, which in some cases - e.g distil whisper, can have empty cache layers. This aims to fix this.

Note that this allows speculative decoding for Whisper again, but that the two linked issues still fails because the decoding with assistant decoder is not as fast, at least on my machine: `test_speculative_decoding_whisper_distil` , `test_speculative_decoding_distil`


cc @ArthurZucker @eustlb


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33926). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Closing as it was fixed in https://github.com/huggingface/transformers/pull/34135 ! 🤗
Fix Whisper shortform EOS
Author: ylacombe
Pull request: yes
Labels: name: run-slow

# What does this PR do?

Since short and longform merging, Whisper removed EOS tokens when doing shortform transcription, which is something not happening in the original implementation. It fixes the `test_default_multilingual_transcription_short_form` and `test_generate_with_prompt_ids` tests

A side effect is that average logprob was miscomputed.


cc @eustlb


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33917). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Closing as it was fixed in https://github.com/huggingface/transformers/pull/34135 ! 🤗
[Whisper] fix docstrings typo
Author: eustlb
Pull request: yes

# What does this PR do?

Fix mini typo in rendering links to GenerationMixin's generate :) 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35319). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
feat: add `benchmarks_entrypoint.py`
Author: McPatate
Pull request: yes
Labels: name: run-benchmark

Adding `benchmarks_entrypoint.py` file, which will be run from the benchmarks CI.

This python script will list all python files from the `benchmark/` folder and run the included `run_benchmark` function, allowing people to add new benchmarks scripts.



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34495). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I'll update the README and then we can merge!
InternVL is ExecuTorch Compatible
Author: guangy10
Pull request: no
Labels: name: Feature request

name: ExecuTorch

### Feature request


### Feature request

Enable OpenGVLab/InternVL2-1B to ["Export to ExecuTorch"](https://github.com/huggingface/transformers/issues/32253) workflow


### Motivation

See details in #32253

### Your contribution

model enablement

Comments: