noamaanMulla-03/transformers-issues huggingface dataset:
do_sample does not default to False
Author: edmondja
Pull request: no
Labels: name: bug

### System Info

My colleague Amine made me notice that if do_sample is not explicitly defined, unlike what the documentation says, it defaults to True.


### Who can help?

_No response_

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction


Here is a script showing do_sample=False must be specified :  
```
from transformers import AutoProcessor, Gemma3ForConditionalGeneration
from PIL import Image
import requests
import torch

model_id = "google/gemma-3-4b-pt"

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"
image = Image.open(requests.get(url, stream=True).raw)

model = Gemma3ForConditionalGeneration.from_pretrained(model_id).eval()
processor = AutoProcessor.from_pretrained(model_id)

prompt = "<start_of_image> in this image, there is"
model_inputs = processor(text=prompt, images=image, return_tensors="pt")

input_len = model_inputs["input_ids"].shape[-1]

with torch.inference_mode():
    generation = model.generate(**model_inputs, max_new_tokens=100
    #, do_sample=False # SPECIFYING IT IS MANDATORY AS THE DOCUMENTATION IS WRONG
    )
    generation = generation[0][input_len:]

decoded = processor.decode(generation, skip_special_tokens=True)
print(decoded)
```

### Expected behavior

Always the same text must be generated as do_sample=False by default according to the documentation -> https://huggingface.co/docs/transformers/main_classes/text_generation

Comments:
Normally it's set in the `configuration_utils.py` in the right way, but we can manage to define the `self.do_sample` in an other way to ensure this bug doesn't happen again. I'll try to work on that and thank you for reporting the issue.

Hey @edmondja @Lynsoo 👋 

Model-specific parameterization takes precedence over global defaults. Each model has a `generation_config.json` with model-specific defaults which, in the case of `gemma3`, sets `do_sample=True` (see [here](https://huggingface.co/google/gemma-3-4b-it/blob/main/generation_config.json#L4))

Hey @gante, 
I get it now, it was weird to see that it was accurately defined in `configuration_utils.py` yet it wasn't giving `False` as default. I think it would be better to specify that in the documentation, so that other users don't get confused. 
Like in this line : "do_sample ( `bool`,  optional, defaults to `False`) — Whether or not to use sampling ; use greedy decoding otherwise.", we can add that it depends on the model. 

Agreed. We're working towards making this behavior being more explicit, including model-level `GenerationConfig` classes 🤗 

That's amazing then. Thank you.
[docs] logits docstring
Author: gante
Pull request: yes

# What does this PR do?

Fixes #37918 

this one was copied pasted across classes, and not corrected in the case with beam search -- we output the logits for all beams

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37929). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add LlamaForSequenceClassification example to docs
Author: suryaprasanthcse
Pull request: yes

# What does this PR do?
Ref #36979  
Added a minimal working example for `LlamaForSequenceClassification` in the LLaMA documentation.

**Motivation**:  
This helps users understand how to use LLaMA for classification tasks, as requested in the issue.

**Changes**:  
- Added Python code example showing:  
  - Model loading  
  - Tokenization  
  - Prediction  

## Before submitting
- [x] This PR fixes a typo or improves the docs
- [x] Did you read the contributor guideline?
- [x] Did you make sure to update the documentation?

## Who can review?
@ArthurZucker @stevhliu

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Thanks for adding an example! It should go in the `forward` below. Check [this](https://github.com/huggingface/transformers/blob/5f4ecf2d9f867a1255131d2461d75793c0cf1db2/src/transformers/models/llama/modeling_llama.py#L799) out for an example of how to format it!

https://github.com/huggingface/transformers/blob/5f4ecf2d9f867a1255131d2461d75793c0cf1db2/src/transformers/models/llama/modeling_llama.py#L885

@ArthurZucker @stevhliu This documentation PR is now ready for review.  
- All requested fixes completed  
- Branch synced with latest main  
- Tests passing  

Would you please approve the workflows when convenient? Thank you!

@stevhliu @ArthurZucker 
All requested fixes are complete:
- Example now in `forward()` docstring
- Typos corrected
- Formatting matches guidelines

Ready for final review! 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37897). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Training Qwen2.5 VL with dynamic image size using more balanced Sampler for each GPU mem usage
Author: OpenJarvisAI
Pull request: no
Labels: name: Feature request

### Feature request

HI, currently training Qwen2.5 VL with peft, the dataloader using lazy load.

But since Qwen2.5 VL uses dynamic input size, each training sample actually will have very divesity training length (very image tokens are not same)

This make training very tricky, the GPU mem usage extremly imbalanced.
 

### Motivation

Is there a way to support it?

### Your contribution

Is there a way to support it?

Comments:
@OpenJarvisAI didn't really get the issue, is the training failing due to different first dim shapes per input (because it doesn't reflect true batch size)?

We had issues with multi-gpu training of qwen-vl where the batches wouldn't get split correctly per gpu (https://github.com/huggingface/transformers/issues/33666). Are you training with distributed multiple GPUs as well?

Am using multiple-GPUs but just one machine with 8 GPUs.

My issues is that, each GPU mem usage it not very balanced, this is a common issue when training MLLM models, however, for LLaVA like models, since each image token is fixed, this can be pre-calculated before iteration all dataset.

When dealing with QwenVL, since the input is dynamic and image input size is not known, when training with a Lazy dataset loader (which is now a common usage), this caused GPU mem hard to balanced, and since each GPU mem is not balanced, it could be easily OOM if one GPU had more heavier load.

AM thinkking is there a way to solve this issue, but not pre-calculated each image dimension (this could be cubersome if images are a lot)
add shared experts for upcoming Granite 4.0 language models
Author: mayank31398
Pull request: yes

This PR adds support for shared experts in GraniteMoE model class for upcoming Granite 4.0 language models.
@ArthurZucker

Comments:
@ArthurZucker can you merge this?
all checks have passed

We're adding an additional feature (shared experts) that doesn't break past checkpoints, and is an extension of our own model class. Would every extension entail a new model class? 

 Yes 🤗 I am sorry but that is the way we have been handling every single model so far! 

Not sure how to get the tests to pass , some are not due to the changes I've made.

@ArthurZucker the PR is ready, please review.
The failing tests seem unrelated

Thanks for approving, please merge as soon as possible :)

I have updated with the main branch , made corresponding changes and all checks have passed :) 

Is the public release of [Granite-4.0-Tiny-Preview](https://huggingface.co/ibm-granite/granite-4.0-tiny-preview) in any way relevant to this PR? (like does it warrant any follow-up work, additional validation/testing/CI, etc.)
AutoTrain Unsloth support
Author: urroxyz
Pull request: no
Labels: name: bug

### System Info

Any system (incl. Google Colab)

Certain versions (incl. 4.50.0 and up) produce this behavior

The version that Colab has installed by default works fine

### Reproduction

```py
#@title 🤗 AutoTrain
#@markdown Install requirements and fix dependencies
# Install requirements
!pip install -q autotrain-advanced unsloth unsloth-zoo

# Fix dependencies
!pip install -q --force-reinstall --no-deps xformers "trl<0.9.0" triton==3.0.0 "git+https://github.com/huggingface/transformers@v4.50.0"
```

Removing the final git installation prevents the errors, however Gemma 3 cannot be loaded without it. Training Llama 3.2, for example, works fine.

```py
#@markdown Train the model

# -*- coding: utf-8 -*-
import os
import yaml

# Step 2. Define full training parameters

os.environ["HF_TOKEN"] = hf_token
os.environ["HF_USERNAME"] = hf_username

# Step 3. Create unified YAML configuration for training
conf = f"""
task: llm-generic
base_model: google/gemma-3-4b-pt
project_name: my-autotrain-gemma-llm
log: tensorboard
backend: local

data:
  path: data/
  train_split: train
  valid_split: null
  chat_template: null
  column_mapping:
    text_column: text

params:
  block_size: 1024
  lr: 0.0001
  warmup_ratio: 0.03
  weight_decay: 0.01
  epochs: 3
  batch_size: 4
  gradient_accumulation: 8
  mixed_precision: fp16
  peft: true
  quantization: int4
  lora_r: 16
  lora_alpha: 32
  lora_dropout: 0.05
  unsloth: true

sequence_len: 2048
max_steps: null

hub:
  username: ${{HF_USERNAME}}
  token: ${{HF_TOKEN}}
  push_to_hub: true
"""

# Write the unified training configuration to file.
with open("conf.yaml", "w") as f:
    f.write(conf)

print("Full training configuration saved to conf.yaml")

# Step 4. Start the full training via autotrain (will take hours)
!autotrain --config conf.yaml
```

### Expected behavior

Train with Unsloth, as opposed to without it

Comments:
I'm not sure if it will help, because I'm using my own custom Python training code with **Unsloth** and **Gemma 3**, which is quite customized. It can be made to work if you're patient, but it's not entirely straightforward. 

The most important thing, however, is that you still need to use: 
`pip install git+https://github.com/huggingface/transformers@v4.49.0-Gemma-3
`
Yes, I know they changed it about 12 days ago in [this commit](https://huggingface.co/google/gemma-3-4b-it/commit/31fa622bc2d63f14ab20aca8dfe5f9dab3eed4ea), but it simply doesn't work for me otherwise — I keep running into obscure errors.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Qwen3 Chat Template `enable_thinking` tag reversed?
Author: jeromeku
Pull request: no
Labels: name: bug

### System Info


`transformers version: 4.52.0.dev0`

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Confused by `enable_thinking` when applying chat template for Qwen3 models.

Specifically
```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-30B-A3B")

messages = [
    {"role": "user", "content": "What is the capital of France?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True
)

print("With thinking mode enabled:")
print(text)
"""
With thinking mode enabled:

user
What is the capital of France?


assistant
"""

text_no_thinking = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False
)

print("\nWith thinking mode disabled:")
print(text_no_thinking)
"""
With thinking mode disabled:

user
What is the capital of France?


assistant
<think>

</think>
"""
```
Running the above, it seems `enable_thinking` is reversed -- it adds the `think` tags when `enable_thinking` is `False`.  

Printing the chat template confirms this:
```python
print(tokenizer.chat_template)
"""
...
{%- if add_generation_prompt %}
    {{- '
assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is false %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}
"""
```
 
### Expected behavior

`enable_thinking` should insert `think` tags when set to `True`, i.e., the conditional in the template should be:
```
{%- if enable_thinking is defined and enable_thinking is true %}
```

Comments:
```
<think>
</think>
```
is an empty thinking block, so this is correct.

See this blog from @cfahlgren1 too: https://huggingface.co/blog/qwen-3-chat-template-deep-dive
[D-FINE] Update names
Author: NielsRogge
Pull request: yes

# What does this PR do?

As [all D-FINE models](https://huggingface.co/collections/ustc-community/d-fine-68109b427cbe6ee36b4e7352) were renamed on the hub, this PR updates the Transformers docs accordingly.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37957). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for updating 👍
More fault tolerant notification service
Author: ivarflakstad
Pull request: yes

Recently I noticed my beloved CI updates weren't showing up in slack.

Turns out notification service was failing in two places. I think the root issue stems from the github API, but regardless I think it makes sense to let the notification service report as much as it can.

One where there was a mismatch between the available artifacts downloaded and the reported jobs on github ([here](https://github.com/huggingface/transformers/actions/runs/14745603051/job/41450810919#step:5:208))

And one where we ran out of `stacktraces` when processing artifact stats ([here](https://github.com/huggingface/transformers/actions/runs/14768732794/job/41518505314#step:5:65)). Preferrably this would be fixed by ensuring the result from `handle_stacktraces` always had the amount of results as `artifact["summary_short"].split("\n")` where `line.startswith("FAILED ")`, but I was hoping we could fix that later.

I also added some variables because the nested dictionary accesses was a bit much for my eyes, but that is optional.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37924). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Use T4 single GPU runner with more CPU RAM
Author: ydshieh
Pull request: yes

# What does this PR do?

Currently this runner type has only 32G CPU RAM, and the new one has 64G RAM.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37961). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[generate] Fix `vocab_size` access for multimodal models
Author: kurzdev
Pull request: yes

# What does this PR do?
Fixes #37930

Prior to this PR, `compute_transition_scores`, `_get_candidate_generator` and `_get_logits_processor` still used `config.vocab_size` to access a model's vocab size. This caused issues with multimodal models, as the vocab_size is stored within their `text_config`.
This PR addresses this issue by migrating these remaining accesses to `vocab_size` to `config.get_text_config().vocab_size`.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@zucchini-nlp @gante

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37937). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Ready to Merge][HFQuantizer] Squelch pydantic warnings
Author: kylesayrs
Pull request: yes

## Purpose ##
* Reduce pydantic warnings

```
  ...lib/python3.12/site-packages/transformers/utils/quantization_config.py:1336: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.11/migration/
    quantization_config = self.quantization_config.dict()
```

## Changes ##
* Replace `.dict()` with `.model_dump()`

## Reviewers ##
* @SunMarc, @ArthurZucker, @Rocketknight1 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

> Thanks, why is this only for CompressedTensorsConfig?

This is because in `CompressedTensorsConfig`, they is also `quantization_config` key that comes from `compressed_tensors`

 `from compressed_tensors.quantization import QuantizationConfig`

Ok! LGTM then!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37726). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks @kylesayrs :hugs:
enable internvl UTs on XPU
Author: yao-matrix
Pull request: yes

`pytest -rA tests/models/internvl/test_modeling_internvl.py`

137 cases PASSED, 4 failed cases are same as CUDA, will investigate in separate efforts.

FAILED tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_resize_embeddings_untied_with_deepspeed
FAILED tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_resize_embeddings_untied_with_deepspeed_multi_gpu
FAILED tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_resize_tokens_embeddings_with_deepspeed
FAILED tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_resize_tokens_embeddings_with_deepspeed_multi_gpu



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@ydshieh , pls help review, thx

cc @IlyasMoutawwakil 

@ydshieh @IlyasMoutawwakil , could you help review again? thx

> LGTM ! Only thing I'm a bit opinionated about is the use of "require_deterministic_for_xpu" because it behaves differently than other "require_xxx". We can probably try getting rid of it if every the problem with "warn_only=True" was fixed ? [#30774 (comment)](https://github.com/huggingface/transformers/pull/30774#issuecomment-2114772828)

it's a new info for me, let me go back to check with our pytorch team, and if OK, i will make update. And at the same time, we are working w/ our pytorch team to root cause this difference btw xpu and cuda.

> Thank you. Only 2 nits regarding formatting 🙏

@ydshieh thx, done per comments.