noamaanMulla-03/transformers-issues huggingface dataset:
Misleading documentation for `is_decoder` configuration parameter
Author: d-kleine
Pull request: no

## Issue Description

The current documentation for the `is_decoder` configuration parameter is misleading and causes confusion. The parameter name suggests it determines whether a model is a decoder (like GPT-2, Llama, etc.), but in reality, it specifically controls whether a model functions as a decoder component within an encoder-decoder architecture.

This is confusing because autoregressive models like GPT-2 and Llama 3 have `is_decoder=False` by default, despite being decoder-only architectures.

## Current Behavior

The documentation currently states:
> is_decoder (bool, optional, defaults to False) — Whether the model is used as decoder or not (in which case it's used as an encoder).

This description doesn't clarify that this parameter is specifically for encoder-decoder architectures and cross-attention functionality.

## Reproduction

```python
import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

seed = 42
torch.manual_seed(seed)

tokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2")
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

model = AutoModelForTokenClassification.from_pretrained(
    "openai-community/gpt2",
    pad_token_id=tokenizer.eos_token_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

print("Default is_decoder:", model.config.is_decoder)
```

Output:
```
Default is_decoder: False
```

## Expected Behavior

The documentation should clearly explain that:

1. The config param `is_decoder=True` specifically enables cross-attention layers and related functionality for models used as decoders _only_ in **encoder-decoder architectures**.
2. Standalone autoregressive models (like GPT-2, Llama) use `is_decoder=False` by default despite being decoder-only architectures.

## Suggested Documentation Update

- Update documentation as described in *Expected Behavior*
- Maybe remove `is_decoder` param from decoder-only and encoder-only transformer architecture config settings?

Comments:
Hi @d-kleine, I think we'd definitely support a documentation update PR here! I'm less sure about removing the param - you can experiment with it, but I think that might be breaking for a lot of workflows. Would you be willing to make the documentaiton update PR first, and then we can think about future steps afterwards if you want?
doc: Clarify `is_decoder` usage in PretrainedConfig documentation
Author: d-kleine
Pull request: yes

# What does this PR do?
Enhance the documentation for the `is_decoder` parameter to specify its relevance to encoder/decoder architectures and clarify its impact on different model types.

<!-- Remove if not applicable -->

Fixes #36482 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu @Rocketknight1 

Comments:
Hey @shethaadit, would you mind stop adding reviews without any substantial feedback to PRs please? Thanks! 🤗 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36724). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add ArlowTokenizer to Transformers
Author: yuchenxie4645
Pull request: yes

This PR adds a new tokenizer, ArlowTokenizer, to the Hugging Face Transformers library. ArlowTokenizer is a BPE-based tokenizer designed for large language models, similar to GPT-2 and GPT-3. This architecture's got nothing to do with the 2 tokenizer versions I released on huggingface, I plan to train my V3 on 350 billion tokens once this PR gets approved. I basically rewrote it so that it is extremely similar to pretrainedfast tokenizer but made it so that it had the name ArlowTokenizer.

---

Changes Introduced:

New File: src/transformers/tokenization_arlow.py; 
Defines the ArlowTokenizer class.
Updated src/transformers/__init__.py
Registers ArlowTokenizer for direct import (from transformers import ArlowTokenizer).
Adds lazy import _import_structure["tokenization_arlow"] = ["ArlowTokenizer"].

---

Details about tokenizer: 

Supports GPT-style special tokens, including:

 (BOS)

 (EOS)

 (Unknown token)

 (Padding token)

 (Masking token)

, 
 (Additional special tokens for AI system messages).

---

Documentation Updates:

Created ArlowGPT.md in docs/source/en/model_doc/:
Describes ArlowTokenizer.
Provides tokenization guidelines.

I verified that this indeed works with a dummy tokenizer. I had the test in transformers/testrun where it had the scripts and dummy tokenizer I used for testing. 

**A future PR may introduce an associated model architecture (ArlowForCausalLM).**

---

## Before submitting
- [n] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [y] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [n] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [y] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [y] Did you write any new necessary tests?

---

@ArthurZucker 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).
Add arlow tokenizer
Author: yuchenxie4645
Pull request: yes

This PR adds a new tokenizer, ArlowTokenizer, to the Hugging Face Transformers library. ArlowTokenizer is a BPE-based tokenizer designed for large language models, similar to GPT-2 and GPT-3. This architecture's got nothing to do with the 2 tokenizer versions I released on huggingface, I plan to train my V3 on 350 billion tokens once this PR gets approved. 

---

Changes Introduced

New directory src/transformers/models/arlow where it exists a __init__.py and tokenization_arlow.py (fast). 
Edited __init.py__ in models to include arlow in an import.
Updated src/transformers/__init__.py to append and define the model + the tokenizer for arlow.
Registers ArlowTokenizer for direct import (from transformers import ArlowTokenizer).

I verified that this indeed works with a dummy tokenizer. I had the test in transformers/testrun where it had the scripts and dummy tokenizer I used for testing. 

**A future PR may introduce an associated model architecture (ArlowForCausalLM), template is already completed.**

## Before submitting
- [n] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [y] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [n] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [y] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [y?] Did you write any new necessary tests?

@ArthurZucker 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).
ArlowTokenizer
Author: yuchenxie4645
Pull request: yes

This PR adds a new tokenizer, ArlowTokenizer, to the Hugging Face Transformers library. ArlowTokenizer is a BPE-based tokenizer designed for large language models, similar to GPT-2 and GPT-3. This architecture's got nothing to do with the 2 tokenizer versions I released on huggingface, I plan to train my V3 on 350 billion tokens once this PR gets approved. I basically rewrote it so that it is extremely similar to pretrainedfast tokenizer but made it so that it had the name ArlowTokenizer.

Changes Introduced

New directory src/transformers/models/arlow where it exists a init.py and tokenization_arlow.py (fast).
Edited init.py in models to include arlow in an import.
Updated src/transformers/init.py to append and define the model + the tokenizer for arlow.
Registers ArlowTokenizer for direct import (from transformers import ArlowTokenizer).

I verified that this indeed works with a dummy tokenizer. I had the test in transformers/testrun where it had the scripts and dummy tokenizer I used for testing. 

**A future PR may introduce an associated model architecture (ArlowForCausalLM), template already added.**

## Before submitting
- [n] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [y] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [n] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [y] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [y?] Did you write any new necessary tests?

@ArthurZucker 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).
Trainer evaluation delay
Author: OllieBroadhurst
Pull request: yes

# What does this PR do?

Attempts to implement #16327 by adding two new `TrainingArguments` arguments, one to delay evaluation until a certain number of epochs have passed and another for steps (which overrides the former). The aim is to delay evaluation to free up more time for training at the initial stages.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. [Link](https://github.com/huggingface/transformers/issues/16327)
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
Let me know your thoughts @sgugger


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16356). All of your documentation changes will be reflected on that endpoint.

@OllieBroadhurst using this after 3 years, you saved me almost a day - Big thanks!!.
bug in save checkpoint
Author: NirHeaven
Pull request: no
Labels: name: bug

### System Info

use main branch to sft qwen25_vl，when save ckpt, got error:OSError: [Errno 39] Directory not empty:


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

use main branch do full sft

### Expected behavior

you will get a error: OSError: [Errno 39] Directory not empty:

Comments:
due to this pr: https://github.com/huggingface/transformers/pull/35580

We'll revert asap if needed! super sorry for the inconvenience 😢 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fixing typo in gemma3 image_processor_fast and adding a small test
Author: ZebWilson
Pull request: yes

# What does this PR do?

Removes typo present in Gemma3 Model's ImageProcessorFast.
Added 1 small test for the verification of the same.
Fixes https://github.com/huggingface/transformers/issues/36739


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)?
Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

PR Discussion - https://github.com/huggingface/transformers/issues/36739
Added small test for Gemma3 model with do_pan_and_scan disabled.

## Who can review?

@zucchini-nlp 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).
Fix gemma3_text tokenizer in mapping
Author: LysandreJik
Pull request: yes

The `gemma3_text` tokenizer was not in the tokenizer mapping, leading to errors when trying to load text-only models such as `google/gemma-3-1b-it`

cc @zucchini-nlp @ArthurZucker 

To reproduce the error before this PR:
```py
from transformers import pipeline
pipe = pipeline("text-generation", model="google/gemma-3-1b-it")

messages = [
    [
        {
            "role": "system",
            "content": [{"type": "text", "text": "You are a helpful assistant."},]
        },
        {
            "role": "user",
            "content": [{"type": "text", "text": "Write a poem on Hugging Face, the company"},]
        },
    ],
]

output = pipe(messages, max_new_tokens=15)
print(output)
```

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36793). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix hqq due to recent modeling changes
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR fixes Hqq quantizer due to recent modeling changes which make use of `load_state_dict` instead of `set_module_tensor_to_device`. The issue was that hqq had rewrote `load_state_dict` and it is causing issue with models with a bias. 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36771). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Wrong return type in `load_balancing_loss_func` function of `modeling_mixtral.py`.
Author: PhilipMay
Pull request: no
Labels: name: Documentation

Hi.

this return type here is wrong:

https://github.com/huggingface/transformers/blob/78b2929c0554b79e0489b451ce4ece14d265ead2/src/transformers/models/mixtral/modeling_mixtral.py#L127

It is not `float` but `torch.Tensor`.

I think this should be fixed as it is confusing and can lead to errors in use. 

What do you think?
PR is at #33641 

Comments:
Thanks a lot for your PR @PhilipMay :raised_hands: 

i think tokens_per_expert = torch.sum(expert_mask.float() * expert_attention_mask, dim=0) / torch.sum(
            expert_attention_mask, dim=0
        )    miss count the topk dim. 

@af-74413592 @PhilipMay seems to be `dim=(0,1)` to me as well
you can check the result with a simple example and set e.g.:
```python
num_experts = 4
top_k = 2
gate_logits = tuple(torch.randn(2, 16, 4))
attention_mask = torch.ones((2, 16))
```
Update SHA for `tj-actions/changed-files`
Author: ydshieh
Pull request: yes

# What does this PR do?

Update sha for `tj-actions/changed-files`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36795). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
When gradient checkpointing is enabled, flash_attn_kwargs cannot be passed into the decoder_layer
Author: efsotr
Pull request: no
Labels: name: bug

### System Info

transformers 4.47.1

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://github.com/huggingface/transformers/blob/241c04d36867259cdf11dbb4e9d9a60f9cb65ebc/src/transformers/models/llama/modeling_llama.py#L896C1-L931C54
```python
for decoder_layer in self.layers[: self.config.num_hidden_layers]:
    if output_hidden_states:
        all_hidden_states += (hidden_states,)


    if self.gradient_checkpointing and self.training:
        layer_outputs = self._gradient_checkpointing_func(
            decoder_layer.__call__,
            hidden_states,
            causal_mask,
            position_ids,
            past_key_values,
            output_attentions,
            use_cache,
            cache_position,
            position_embeddings,
        )
    else:
        layer_outputs = decoder_layer(
            hidden_states,
            attention_mask=causal_mask,
            position_ids=position_ids,
            past_key_value=past_key_values,
            output_attentions=output_attentions,
            use_cache=use_cache,
            cache_position=cache_position,
            position_embeddings=position_embeddings,
            **flash_attn_kwargs,
        )
```

### Expected behavior

x

Comments:
Hey! Yep, this is a known issue and it is rather a limitation from the way we implement gradient checkpointing AFAIK. 
You can try this branch: #34987 with `GradientCheckpointLayer` should be working fine! Tell me if that helps? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

https://github.com/huggingface/transformers/pull/34987 can't work. 
It does not pass `flash_attn_kwargs` into `decoder_layer.__call__` when gradient checkpointing is enabled. However, we can resolve this by replacing `decoder_layer.__call__` with `partial(decoder_layer.__call__, **flash_attn_kwargs)`.
Loading optimizations
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Profiling of CPU memory usage showed that calling `gc.collect()` has no effect (the `del` statement is enough and cleaning will happen automatically as it should), i.e. the peak cpu memory usage is never larger than 1 state dict (even with older `.bin` sharded checkpoints).
Removing the statement speeds up loading by 15-20% as `gc.collect()` is costly and should in general not be called.

As for the caching allocator, profiling shows that having an `allocation_factor > 1` does not help at all neither for TP nor usual loading on multiple GPUs. It is then much easier to always use 1, as we will never encounter cases of blowing GPUs. Also, code was not allocating anything in case allocation size was larger than gpu size, which could lead to big slowdown if reaching this state only because of the factor 2, as the model may still fit.
Also, improved granularity of the allocator by checking each param dtype, which may be different with composite models/keep_in_fp_32_modules.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36742). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[don't merge] trigger
Author: ydshieh
Pull request: yes

# What does this PR do?

[don't merge] trigger

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36451). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Mistral3 tests
Author: yonigozlan
Pull request: yes

# What does this PR do?

Fix modeling and processor tests after last-minute changes


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36797). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.