noamaanMulla-03/transformers-issues huggingface dataset:
DBRX model passes probabilities and not logits to the load balancer
Author: rudolfwilliam
Pull request: no
Labels: name: bug

### System Info

The problem has nothing to do with my system.

### Who can help?

@ArthurZucker

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Look into the code and verify that it is not compliant with the [intended load balancing](https://arxiv.org/pdf/2101.03961):

The signature of the load balancer in [modeling_dbrx](https://github.com/huggingface/transformers/blob/main/src/transformers/models/dbrx/modeling_dbrx.py) looks as follows:

```python
def load_balancing_loss_func(
    gate_logits: torch.Tensor,
    num_experts: int,
    top_k: int,
    attention_mask: Optional[torch.Tensor],
) -> torch.Tensor:
```

And also, the `gate_logits` are treated as such, meaning that the softmax is computed later in the code. However, the values (called `weights`) that are passed into that function are the expert probabilities and not the logits:

```python
weights = self.layer(hidden_states).softmax(dim=-1, dtype=torch.float32)
```

Thus, the load balancer takes the softmax of the logits twice, which is incorrect.

### Expected behavior

`gate_logits` should be logits and not probabilities.

Comments:
This code hasn't been changed since the model was added in #29921 by @abhi-mosaic and @eitanturok. If either of those see this ping, can you verify? Especially @eitanturok since I believe he was/is working at Databricks.
Fix the shape of ModernBertForMaskedLM's output hidden_states
Author: sheryc
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

Fix the shape of `ModernBertForMaskedLM`'s output `hidden_states` when both having `output_hidden_states=True` and using Flash Attention 2.

Before fix: The output hidden states are linearized to shape similar to `(batch_size * sequence_length, hidden_size)` since the input ids are unpadded before passing to `ModernBertModel`.

Expected behavior: All output `hidden_states` should have shape `(batch_size, sequence_length, hidden_size)` according to the docs.

This PR applies unpadding to the output hidden_states to make the shapes correct.



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. (There's no issue for this.)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation). (This PR makes the behavior following the current documentation.)
- [ ] Did you write any new necessary tests? (No need for extra tests.)


@ArthurZucker

Comments:
Modernbert 3D attention mask
Author: meetdoshi-iitb
Pull request: no
Labels: name: Feature request

### Feature request

Support request for passing a custom 3D attention mask to a modernbert model.
Currently it only supports 2D attention mask of bs, seq_len [modeling_modernbert.py#L859](https://github.com/huggingface/transformers/blob/774dc274ac966f4bccbcd90d55bba23f6cca37ae/src/transformers/models/modernbert/modeling_modernbert.py#L859)
Unlike bert which supports 3D attention mask as well [modeling_bert.py#L988](https://github.com/huggingface/transformers/blob/774dc274ac966f4bccbcd90d55bba23f6cca37ae/src/transformers/models/bert/modeling_bert.py#L988)

### Motivation

Linked issues: [transformers/issues/27640](https://github.com/huggingface/transformers/issues/27640)

### Your contribution

NA

Comments:
cc @arthurzucker for text models / attention

Hi @meetdoshi-iitb @Rocketknight1 @ArthurZucker ! I'm interested in working on this feature and have opened a PR for your review.

Will have a look!
Fix ModernBERT tokenizer issue with is_split_into_words flag
Author: ved1beta
Pull request: yes

Fix ModernBERT tokenizer issue with is_split_into_words flag

Fixes #37883

This PR adds validation to when using the ModernBERT tokenizer with pre-tokenized inputs (is_split_into_words=True),
The fix follows the same pattern used in other tokenizers like BART, which already have this validation in place.
@ArthurZucker  @itazap 

Comments:
Looks good! Thank you for working on this! 🤗 

hey do i need to pass tests  ?  I'm not sure why they are failing for this change ,
[chat template] add a testcase for kwargs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Just adds a testcase, I didn't know this was supported and I don't want it to break in the future

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39415). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

>  is it an add_system_prompt=True

Yep, that is the kwarg. I've seen today a question that qwen2-vl uses `add_vision_ids` in its chat template, and didn't know it was actually supported. I see it was added after Qwen release though, no idea why we had `add_vision_ids=True` in the documentation

P.S. can you link qwen's issue/comment please

It's an internal dialogue with someone from Cornell doing experiments on video+image+text
Fix L270 - hasattr("moe_args") returning False error
Author: wjdghks950
Pull request: yes

Hi, I've been trying to convert the Llama-4 weights to HF using the `convert_llama4_weights_to_hf.py` and I've been having an error that causes the `num_experts` to be set to 0 due to the problem in L270 as referenced below:

https://github.com/huggingface/transformers/blame/81799d8b556b3c810ed314187674bc439c0582b4/src/transformers/models/llama4/convert_llama4_weights_to_hf.py#L270

The `if hasattr(params, "moe_args")` in L270 always returns false since "moe_args" is a key of the params dict (not an attribute). I changed the line to `if params.get("moe_args", None):` and the error is gone. I do recommend the change. Thank you.

Tag: @ArthurZucker


Comments:
cc @pcuenca since I see you in the `git blame` for those lines! Does this differ between Llama/LlamaGuard?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llama4

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38715). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
LlamaAttention forward function type hint is incorrect
Author: nhatkhtn
Pull request: no
Labels: name: bug

### System Info

For the current version (4.52.4), in the `LlamaAttention` class, the type hint for the forward function

https://github.com/huggingface/transformers/blob/aa798b7ac9ff5018b3578eb927dc438671ab6a3e/src/transformers/models/llama/modeling_llama.py#L231

and what it actually returns

https://github.com/huggingface/transformers/blob/aa798b7ac9ff5018b3578eb927dc438671ab6a3e/src/transformers/models/llama/modeling_llama.py#L264

mismatch. Looking at the git blame, this is probably the result of the attention refactor introduced in #35235. 

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Look at the code

### Expected behavior

The type hint should be correct to avoid confusion

Comments:
Hello! I noticed a small mismatch in the return type of the LlamaAttention forward method in Transformers version 4.52.4. It does not match what is actually being returned. This might have happened during the recent attention refactor, possibly in pull request . I just wanted to point it out. I would be happy to open a quick pull request to fix it if that helps. Thank you for all the great work you are doing!


@nhatkhtn ?


Hi @nhatkhtn, yes, this looks like an incorrect type hint. Would you be willing to make a PR to fix it?

Yeah sure...

@Rocketknight1 i have done the changes...I am pushing the changes
Plz review


Hi @ArkVex can you link the PR?

#38795 @Rocketknight1 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Fixed with #38998 

Thanks!
Fix missing definition of diff_file_url in notification service
Author: ahadnagy
Pull request: yes

# What does this PR do?

Yesterday's PR to track the performed tests in CI (https://github.com/huggingface/transformers/pull/39198) resulted in a small error, this PR fixes that:
https://github.com/huggingface/transformers/actions/runs/16309221900/job/46061828202


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface, @SunMarc and @qgallouedec
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39445). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
add test scanner
Author: molbap
Pull request: yes
Labels: name: Tests

# What does this PR do?

Another day, another tool: when adding a model, tests fail as they should, and sometimes `test_modeling_common` feels irreconcilable with the peculiarities of our brand new model. But how can we be sure we're not breaking everything by adding a seemingly innocent `skip`?

Well I'm not 100% sure, but this:

- scans all test_modeling_common methods
- looks for times where a method is skipped
- returns a summary json you can load as a DataFrame/inspect

For instance `test_inputs_embeds` is skipped in a whooping 39% proportion! 

<img width="1880" height="772" alt="image" src="https://github.com/user-attachments/assets/8a0977a2-dc48-4649-9f2c-40b0f029984d" />

So if you're skipping a test and it has never been skipped, maybe ponder a moment. I'm pushing this because I always end up eyeballing in my IDE the number of hits a given test has repo-wide, this should be slightly more robust. Attaching an example scan result as well.

cc @ArthurZucker @ydshieh !
[all_tests_scan_result.json](https://github.com/user-attachments/files/21230848/all_tests_scan_result.json)
 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39419). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Actually will add tests from `GenerationTesterMixin` as well before merging :eyes: 

haha, that was my initial API, figured might as well scan for all methods because it was quick. added the entrypoint back. For all `test_modeling_common` takes < 0.2s on my machine, probably double that with generation tests. 

For the single method:

```bash
(work) pablo@Brighter:~/git/transformers$ time python utils/scan_skipped_tests.py --test_method_name test_inputs_embeds
🔬 Parsing 331 model test files once each...

== test_inputs_embeds ==
Ran on    : 199/323 models
Skipped on: 124/323 models (38.4%)
 - aimv2: Aimv2 does not use inputs_embeds
 - align: Inputs_embeds is tested in individual model tests
 - altclip: Inputs_embeds is tested in individual model tests
 - audio_spectrogram_transformer: AST does not use inputs_embeds
 - beit: BEiT does not use inputs_embeds
 - bit: Bit does not use inputs_embeds
 - blip: Blip does not use inputs_embeds
 - blip_2: Inputs_embeds is tested in individual model tests
 - bridgetower: 
 - canine: CANINE does not have a get_input_embeddings() method.
 - ...

📄 JSON saved to /home/pablo/git/transformers/scan_test_inputs_embeds.json

real	0m0.179s
user	0m0.172s
sys	0m0.006s

```




Added `GenerationMixin` tests as well, so this covers 224 tests + allows to scan for a specific method.
Add StableAdamW Optimizer 
Author: SunMarc
Pull request: yes

# What does this PR do?

Supersedes https://github.com/huggingface/transformers/pull/36606 as the github CI is broken

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39446). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

To provide more details, the workflow `run_tests` is failing here: 

```python
________________ OPTModelTest.test_pipeline_text_classification ________________
[gw5] linux -- Python 3.9.21 /usr/local/bin/python3

self = <tests.models.opt.test_modeling_opt.OPTModelTest testMethod=test_pipeline_text_classification>

    @is_pipeline_test
    def test_pipeline_text_classification(self):
>       self.run_task_tests(task="text-classification")

tests/test_pipeline_mixin.py:694: 
```


which generates `Failed: Timeout >120.0s` for `hf_hub_download()` when trying to download from HuggingFace hub the tokenizer for the test of text classification:

```python
+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++
~~~~~~~~~~~~~~~~~~~~~ Stack of Thread-1 (140546618164928) ~~~~~~~~~~~~~~~~~~~~~~
  File "/usr/local/lib/python3.9/threading.py", line 937, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/site-packages/tqdm/_monitor.py", line 60, in run
    self.was_killed.wait(self.sleep_interval)
  File "/usr/local/lib/python3.9/threading.py", line 581, in wait
    signaled = self._cond.wait(timeout)
  File "/usr/local/lib/python3.9/threading.py", line 316, in wait
    gotit = waiter.acquire(True, timeout)
~~~~~~~~~~~~~~~~~~~~~ Stack of <unknown> (140547710781120) ~~~~~~~~~~~~~~~~~~~~~
  File "/usr/local/lib/python3.9/site-packages/execnet/gateway_base.py", line 411, in _perform_spawn
    reply.run()
  File "/usr/local/lib/python3.9/site-packages/execnet/gateway_base.py", line 341, in run
    self._result = func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/execnet/gateway_base.py", line 1160, in _thread_receiver
    msg = Message.from_io(io)
  File "/usr/local/lib/python3.9/site-packages/execnet/gateway_base.py", line 567, in from_io
    header = io.read(9)  # type 1, channel 4, payload 4
  File "/usr/local/lib/python3.9/site-packages/execnet/gateway_base.py", line 534, in read
    data = self._read(numbytes - len(buf))
+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++
```

The download seems to start downloading, but then the SSL connection times out during the file transfer. So, maybe increase the timeout or add some retry to handle network connectivity issues or slow download speeds within the node running the workflow?

Thanks both!
Fix logger warnings in Gemma model test files
Author: ridima11
Pull request: yes

This PR fixes logger warning issues in the test files for Gemma, Gemma2, Gemma3, and Gemma3n models under tests/models/. Updated the use of deprecated logger.warn() to logger.warning() to remove Deprecation Warnings during test runs.


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma, gemma2, gemma3, gemma3n

cc @ydshieh
docs: add missing numpy import to minimal example
Author: IliasAarab
Pull request: yes

# What does this PR do?

Add missing numpy import to minimal example


## Before submitting
- [ X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Documentation: @stevhliu




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39444). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[cache] make all classes cache compatible finally
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, and let's get rid of `_supports_cache/_supports_quantized_cache` flags. We will assume all models from now on support cache and initialize a `DynamicCache` (model-specific cache in case of mamba) by default

For the static cache, we can't yet assume all models support it because even if the model technically can use `StaticCache`, it can't always compile fullgraph. We have an auto-compilation for static cache enabled, so maybe the compilation should check for smth like `_can_compile_fullgraph` and not `_supports_static_cache`?

I checked all models are updated and the generation tests are passing. Note, the PR depends on #38751 which cleans up non-generative models from `past_key_values`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38635). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This should be ready for review. Most model changes are same, so I suggest to review Bert first from which many others copy and a few audio models



(plz ping me again when it's ready for a re-review :) )

@gante ready for review :)

Compile failing tests are not related

Merged a blocking PR, will finalize this one today

Deprecation cycle added for Bert-like models. All comments addressed. I will merge after core maintainers review, @Cyrilvallez @ArthurZucker 

For easier review, I recommend start from Bert, most models copy from it

Will merge tomorrow morning if no comments left, otherwise the PR is growing bigger and bigger 

run-slow: albert, arcee, aria, audio_spectrogram_transformer, autoformer, aya_vision, bamba, bark, bart, beit, bert, bert_generation, big_bird, bigbird_pegasus, biogpt, bitnet

This comment contains run-slow, running the specified jobs: 

models: ['models/albert', 'models/arcee', 'models/aria', 'models/audio_spectrogram_transformer', 'models/autoformer', 'models/aya_vision', 'models/bamba', 'models/bark', 'models/bart', 'models/beit', 'models/bert', 'models/bert_generation', 'models/big_bird', 'models/bigbird_pegasus', 'models/biogpt', 'models/bitnet']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: albert, arcee, aria, audio_spectrogram_transformer, autoformer, aya_vision, bamba, bark, bart, beit, bert, bert_generation, big_bird, bigbird_pegasus, biogpt, bitnet

Merging, hopefully not so many tests fails after 🤞🏻
[YosoConfig] Missing `architectures` field
Author: SankethHonavar
Pull request: no

While contributing to the project and running the `add_missing_architectures.py` utility, I found that `YosoConfig` was missing the `architectures` field.

This issue tracks the fix to add the following field to:

📄 `src/transformers/models/yoso/configuration_yoso.py`

```python
architectures = ["YosoForMaskedLM"]


Comments:
Closing this issue, as I now realize the fix I proposed was based on a misunderstanding.
The `add_missing_architectures.py` script and class-level `architectures` field aren't standard or needed in this repo.
Thank you.