noamaanMulla-03/transformers-issues huggingface dataset:
Add Whole Word Masking and Padding Strategy to DataCollatorForLanguageModeling
Author: rjgleaton
Pull request: yes

# What does this PR do?

Main contribution is adding Whole Word Masking to DataCollatorForLanguageModeling. This is to fix the issues mentioned in #12998 where the previous implementation only returned input_ids and labels. This implementation should pass through all values from the input mappings. Because I feel it's more intuitive and easier to maintain I've added an arg to DataCollatorForLanguageModeling that allows for whole word masking. The DataCollatorForWholeWordMasking collator has been largely deprecated and is now just a rapper for the language modeling collator where mlm and whole_word_mask are forced to true.

This implementation depends on offset mappings for determining if a given pair of tokens belong to the same word. As such, it only natively works for FastTokenizers. I'm not sure there's a tokenizer agnostic way to do this with non-fast tokenizers atm. 

Additionally I've adding an argument for specifying PaddingStrategy to DataCollatorForLanguageModeling. This is a minor QOL improvement that makes it more consistent with other collators.


Fixes #12998


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger, @Rocketknight1 


Comments:
Add voxtral
Author: eustlb
Pull request: yes
Labels: name: New model

# What does this PR do?

Adds mistral's audio-text-to-text model 🤗

## EVALS

Ran transcriptions mode eval on librispeech test-clean ([this gist](https://gist.github.com/eustlb/fc899dd9ff7fe357065c35cb0120d145))
- 3B: 1.9% wer
- 24B: 1.6% wer


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39429). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> My review is only for the Tokenization part:
> 
> Besides the suggestion, LGTM thanks for the work ! If it's possible it would be awesome to add tests as I did for images. LMK if I can be of any help.

Also for `mistral-common` to work properly with audio it would need some dependencies installed. Is it possible to update the dependency group `mistral-common` from `mistral-common[opencv]` to `mistral-common[image,audio]` ? It should install dependencies for all the modalities we support.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, voxtral

Nice, let's go 🚀
Fix typing order
Author: Tavish9
Pull request: yes

# What does this PR do?

- This PR fix the typing order change by #38797

Fixes #39462 

**Importing order matters**
Before #38797, `Union[str, Dict]` was okay for type `Union[dict, str]`. 
When changing from `Dict` from `dict`, `Union[str, dict]` would predominate `Union[dict, str]` if it's imported early than `TrainingArguments`.



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@zach-huggingface, @SunMarc and @qgallouedec

Comments:
Hi, I want to add tests, but as I said, `Importing order matters`, I cannot test by importing every file in the repo.

Commonly used files: `Trainer`, `PreTrainedModel`, `AutoModel` etc... (too many)

What's your opinion here for the test? 

Let's just restrict ourselves to the most used files like `Trainer`, `AutoModel` in the first place and we can potentially add more in the future. 

Already done.

I fix the order in the test and we don't need to test importing every module. 

Before:

<img width="1928" height="78" alt="image" src="https://github.com/user-attachments/assets/ede8291e-659b-4ca4-8017-c4ea6a55fe0d" />

After:

<img width="1929" height="312" alt="image" src="https://github.com/user-attachments/assets/a4691e09-c702-4130-808e-fcbbc2ac4c35" />


Wait, I found the tests are skipped

```bash
SKIPPED [1] tests/utils/test_hf_argparser.py:487: test requires deepspeed
```

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39467). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Replace `deepspeed` with `accelerator` to not be skipped :)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: modernbert
no use now
Author: zRzRzRzRzRzRzR
Pull request: yes

not using now.use https://github.com/huggingface/transformers/pull/39393

Comments:
run-slow: auto, glm4_moe

This comment contains run-slow, running the specified jobs: 

models: ['models/auto', 'models/glm4_moe']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, glm4_moe
Adding Space-Time-MiniLM-v0
Author: haidarjomaa
Pull request: no
Labels: name: New model

### Model description

The model is an encoder, used for sentence embedding. It uses a custom attention mechanism to incorporate space-time information into the embeddings. This would be very helpful especially in retrieval scenarios.
This has not been done before, and it is not available yet to be used. We aim to have this the start of a series of models that do this task.
The All-MiniLM-L2-v6 is the closest since our model was built on top of this one's knowledge.
It uses the same tokenizer as MiniLM.

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Can you add paper link and model implementation link ? It would be helpful.

Sure, here is the model on Hugging Face:
https://huggingface.co/HaidarJomaa/Space-Time-MiniLM-v0

On Fri, 18 Jul 2025, 12:20 pm Punitvara, ***@***.***> wrote:

> *punitvara* left a comment (huggingface/transformers#39479)
> <https://github.com/huggingface/transformers/issues/39479#issuecomment-3088753807>
>
> Can you add paper link and model implementation link ? It would be helpful.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/39479#issuecomment-3088753807>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/A7FE2XBUM3LJ3WX4QAL2AW33JC367AVCNFSM6AAAAACBYCSFXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAOBYG42TGOBQG4>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
[doc builder job] temporary pyarrow pin
Author: gante
Pull request: yes

# What does this PR do?

Adds a temporary pin on the doc builder CI job (pyarrow==20.0.0) to work around a datasets exception in our doc builder.
[Failing job](https://github.com/huggingface/transformers/actions/runs/16365952006/job/46243081358?pr=38545)

```
(...)
  File "/home/runner/work/transformers/transformers/.venv/lib/python3.10/site-packages/datasets/features/__init__.py", line 18, in <module>
    from .features import Array2D, Array3D, Array4D, Array5D, ClassLabel, Features, Sequence, Value
  File "/home/runner/work/transformers/transformers/.venv/lib/python3.10/site-packages/datasets/features/features.py", line 634, in <module>
    class _ArrayXDExtensionType(pa.PyExtensionType):
AttributeError: module 'pyarrow' has no attribute 'PyExtensionType'. Did you mean: 'ExtensionType'?
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39496). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Slack CI bot: set default result for non-existing artifacts
Author: ahadnagy
Pull request: yes

# What does this PR do?

Since https://github.com/huggingface/transformers/pull/39198 the notification service works based on the downloaded artifacts instead of a pre-compiled list. This resulted in key errors in the AMD CI where some of the tests pipelines are not run and the artifacts are not produced. This PR sets a default for those cases.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface, @SunMarc and @qgallouedec
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39499). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[dependencies] Update `datasets` pin
Author: gante
Pull request: yes

# What does this PR do?

(follow-up to #39496 with a more thorough fix)

With the release of `pyarrow==21.0.0`, whose most recent version is installed in `datasets` by default, `datasets<2.15.0` starts throwing exceptions.

This means we need `datasets>=2.15.0` or `pyarrow<21.0.0`. Maximum version pins may limit other packages, so I'm adding a minimum version pin.

Relevant commit in `datasets`: https://github.com/huggingface/datasets/commit/c096bd288d07ed86f340ae090e5d4d9c5351f76f (`pyarrow==21.0.0` isn't compatible before this commit)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39500). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[chat template] return assistant mask in processors
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/38521. I checked with fast tokenizers' implementation of `word_to_char` and saw no difference in the time taken, so I think this can be the permanent solution

Otherwise we can add in `BatchFeature` support for `EncodingFast` features, though I don't think anyone needs them and I have never seen user requesting it

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38545). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ready for review!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: csm, shieldgemma2
Error for `return_assistant_tokens_mask` in MLLM processor
Author: MilkClouds
Pull request: no
Labels: name: bug

### System Info

latest version of transformers, after #37602 

### Who can help?

@zucchini-nlp 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Same as #36713 .

Run scripts in https://huggingface.co/HuggingFaceM4/idefics2-8b-chatty/discussions/9.
Besides, I'm not sure the problem is on the transformers source or example script.

### Expected behavior

Expected:
```
# script output
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
```

Real output:
```
Traceback (most recent call last):
  File "/mnt/home/claude/GitHub/transformers/src/transformers/feature_extraction_utils.py", line 90, in __getattr__
    return self.data[item]
           ~~~~~~~~~^^^^^^
KeyError: 'char_to_token'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/home/claude/GitHub/closed-world-agents/main.py", line 82, in <module>
    inputs = processor.apply_chat_template(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/claude/GitHub/transformers/src/transformers/processing_utils.py", line 1614, in apply_chat_template
    start_token = out.char_to_token(i, assistant_start_char)
                  ^^^^^^^^^^^^^^^^^
  File "/mnt/home/claude/GitHub/transformers/src/transformers/feature_extraction_utils.py", line 92, in __getattr__
    raise AttributeError
AttributeError
```

Comments:
I see, seems like the assistant tokens construction uses a tokenizer-specific helper. And since the processor returns a `BatchFeature`, it wouldn't work. Oke, I will fix it and we will need a test, so it doesn't break anymore

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

not stale, PR waiting for review
Fix: Skip weight initialization for quantized int8 models
Author: imjbassi
Pull request: yes

Fixes #39366 by skipping weight initialization when all model parameters are non-floating-point types (e.g., int8 from W8A8 quantized models). This avoids a RuntimeError from PyTorch's `normal_()` function, which cannot handle integer dtypes.

Adds a conditional check inside `_load_pretrained_model` to skip `self.initialize_weights()` when appropriate.

Tested to ensure model loads without crashing for quantized cases.

Comments:
cc @sunmarc @MekkCyber
modeling_flax_gemma.FlaxGemmaModule failed with incompatible shapes when running with GemmaConfig
Author: nhatleSummer22
Pull request: no

Hi, I got this error when running the above model with GemmConfig:

"/python3.11/site-packages/jax/_src/numpy/ufuncs.py", line 1280, in multiply
    return lax.mul(x, y) if x.dtype != bool else lax.bitwise_and(x, y)

```
from transformers.models.gemma import modeling_flax_gemma

from transformers import GemmaConfig

config = GemmaConfig()
model = modeling_flax_gemma.FlaxGemmaModule(config, dtype=jnp.float32)

input_ids = jnp.zeros((32, 128), dtype=jnp.int32)

variables = model.init(
        jax.random.key(0),
        input_ids=input_ids,
      )

def model_apply(input_ids):
        return model.apply(variables, input_ids=input_ids)
model_apply(input_ids)
```

I am using transformers 4.53.2 and jax 3.10. Could you please take a look? Thanks!

Comments:
Hi, our Flax support is deprecated, unfortunately! This code will likely be removed in the near future, and we're not maintaining/supporting it at the moment.
Fix BatchEncoding.to() for nested elements
Author: eginhard
Pull request: yes

# What does this PR do?

Extend `BatchEncoding.to()` to also work for nested elements.

When using voice presets in Bark, the processor returns a `BatchEncoding` of 
- `{ "input_ids": torch.Tensor, "attention_mask": torch.Tensor, "history_prompt": BatchFeature}`

Currently, only tensor elements are moved, so running on `cuda` the following code fails with `RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper_CUDA__index_select)`:

```python
import scipy
import torch

from transformers import AutoProcessor
from transformers import BarkModel

model = BarkModel.from_pretrained("suno/bark-small")
device = "cuda:0" if torch.cuda.is_available() else "cpu"
model = model.to(device)

sampling_rate = model.generation_config.sample_rate
processor = AutoProcessor.from_pretrained("suno/bark-small")
voice_preset = "v2/en_speaker_6"

# prepare the inputs
text_prompt = "Let's try generating speech, with Bark, a text-to-speech model"
inputs = processor(text_prompt, voice_preset=voice_preset)

# generate speech
speech_output = model.generate(**inputs.to(device))
scipy.io.wavfile.write("bark_out.wav", rate=sampling_rate, data=speech_output[0].cpu().numpy())
```
A workaround was to manually do `inputs["history_prompt"].to(device)`. This PR fixes this by moving all nested elements with a callable `to()`.

Fixes #34634


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case: https://github.com/huggingface/transformers/issues/34634#issuecomment-2996386321
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Rocketknight1

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38985). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @itazap for tokenizers, feel free to merge it if you're happy

@Rocketknight1, @itazap any update on this PR? 

I can confirm it would address https://github.com/huggingface/transformers/issues/34634 and a Bark test mentioned in https://github.com/huggingface/transformers/pull/39478 🙂 

Good point - I took another look and I think this is safe, so I'm going to merge! If it breaks anything, anyone finding this PR can yell at me :sweat_smile:
Corrections to PR #38642 and enhancements to Wav2Vec2Processor __call__ and pad docstrings
Author: renet10
Pull request: yes

# What does this PR do?

The modifications to Wav2Vec2Processor __call__ and pad docstrings adds back the reference of PreTrainedTokenizer __call__ and pad removed incorrectly in PR #38642.  As the __call__ and pad methods are important public methods for Wav2Vec2Processor, this PR also enhances the docstrings with the invocation procedure post removal of "as_target_processor()" and adds information on return values.

Fixes # 38609

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38822). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu is there anything else required of me before these changes are finalized?  Ty.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: wav2vec2
Add fast image processor SAM
Author: yonigozlan
Pull request: yes

# What does this PR do?
Split from https://github.com/huggingface/transformers/pull/32317 as this PR is becoming huge


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39385). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, sam
Improve @auto_docstring doc and rename `args_doc.py` to `auto_docstring.py`
Author: yonigozlan
Pull request: yes

# What does this PR do?

As the title says :).

Cc @Cyrilvallez @stevhliu 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39439). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Feel free to merge after fixing the conflict!