noamaanMulla-03/transformers-issues huggingface dataset:
[saving] Simplify general logic
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42766). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42766&sha=4bd1b2
Default auto 🚨 🚨 
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Superseeds #34919

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42805). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: beit, bigbird_pegasus, blip_2, data2vec, edgetam, gpt_oss, internvl, sam2, sam3, timm_wrapper

I understand that this is a big change that is intended to break some stuff. E.g. for opt-125m, the auto dtype is now float16 instead of float32. However, this PR also makes it so that the `dtype` attribute is no longer updated when converting the model. Is that also intended?

```python
import torch
from transformers import AutoModelForCausalLM

model_id = "facebook/opt-125m"

# previous commit (8a2a83d574fd461697a29410a36737ed112f8ba7)
# this passes
model = AutoModelForCausalLM.from_pretrained(model_id)
assert model.dtype == torch.float32
model.half()
assert {p.dtype for p in model.parameters()} == {torch.float16}
assert model.dtype == torch.float16, f"not float16, got {model.dtype} instead"  # passes

# after this commit (6217adc6c8f0be7b5374e6a46129ad2214e4c6ed)
model = AutoModelForCausalLM.from_pretrained(model_id)
assert model.dtype == torch.float16  # <= used to be float32
model.float()
assert {p.dtype for p in model.parameters()} == {torch.float32}
assert model.dtype == torch.float32, f"not float32, got {model.dtype} instead"  # fails
# AssertionError: not float32, got torch.float16 instead
```

Ha damn, I just merged https://github.com/huggingface/transformers/pull/42825 but seeing this now... Will open a new one, it indeed looks like relying on the config is a bad idea in those cases!
[`T5Gemma2`] Fix bidirectional mask for encoder
Author: vasqu
Pull request: yes

Wrong `is_causal` flag causes unexpected behavior where when we shortcut mask creation (all True) we trigger SDPA's `is_causal=True` behavior.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42820). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: t5gemma2
Correctly create tied key mapping in post_init, and dynamic tie weight
Author: Cyrilvallez
Pull request: yes

As we rely more and more in `self.all_tied_weight_keys` everywhere (i.e. the list of tied keys obtained during `post_init`) for multiple manipulations (device_map computation, cuda warmup, post-processing of `from_pretrained`...), it becomes very important that the (few) models containing regex patterns for their `_tied_weights_keys` mapping have the patterns expanded to fit in `all_tied_weight_keys` as well, instead of containing simple patterns that are skipped in different ways for all downstream application.
This PR fixes that, by expanding correctly at `post_init` time, so the mapping are correct params everywhere.
Also allows for recomputing this mapping in `tie_weights` dynamically, so that it is correct if calling `tie_weights` after having modified the config

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42270). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: esm, hubert, idefics, openai, sew, sew_d, unispeech, unispeech_sat, wav2vec2, wavlm

@Cyrilvallez @ArthurZucker 
I have a concern: with the changes in this PR, a model like Qwen/Qwen3-0.6B — which is a tie-weight model — is now treated as a non–tie-weight model. In its config.json, "tie_word_embeddings": true is set, and lm_head.weight and embed_tokens.weight actually share identical values. However, after this PR, the model is no longer recognized as a tie-weight model.

As a result, when training Qwen/Qwen3-0.6B, the gradients update completely different parameters before and after this PR. I believe the original behavior is more consistent with the model’s intended design.
Reference model: https://huggingface.co/Qwen/Qwen3-0.6B

Hey @lugimzzz! You can check-out my answer to the same question [here](https://github.com/huggingface/transformers/issues/42832#issuecomment-3646657225)! Let me know if you want more clarification!
Improve BatchFeature: stack list and lists of torch tensors
Author: yonigozlan
Pull request: yes

I have been wanting to change that for a while, it shouldn't be a breaking change, but align what we support in `BatchFeature` between numpy arrays and torch tensors.
The issue was that np.array() works on lists of array and even nested list of arrays), but torch.tensor() doesn't, so if we tried to call batch feature, with lists of tensors, we would get an error. I haven't added support for nested lists of tensors as I haven't seen the need anywhere.
Also the errors we were getting were very generic, and not very useful, this should help with that.

Needed for https://github.com/huggingface/transformers/pull/41388


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42750). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: beit, bridgetower, cohere2_vision, convnext, deepseek_vl, deepseek_vl_hybrid, depth_pro, dinov3_vit, donut, dpt, efficientloftr, efficientnet, eomt, flava, fuyu, gemma3

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42750&sha=e2ceac
Fix #38709: Add support for return_last_hidden_state in get_video_features
Author: babyfox1306
Pull request: yes

This PR adds support for returning the last_hidden_state in the get_video_features method of XCLIPModel, as requested in #38709.

## ✅ Summary of changes

Added a new class `XCLIPVideoFeatureOutput` inheriting from `BaseModelOutput`, including:
- `pooled_output`
- Optional `last_hidden_state`
- Optional `hidden_states`
- Optional `attentions`

Modified `get_video_features`:
- New argument `return_last_hidden_state` (default False)
- Now supports `return_dict=True` and optional returns
- Keeps backward compatibility (default behavior unchanged)
- Updated the docstring to reflect the new usage options

Added unit tests to cover:
- Default output
- Output with `return_last_hidden_state=True`
- Output with `return_dict=True`
- Full output with hidden states and attentions

## 💡 Example Usage

```python
# Default usage (returns pooled_output only)
pooled_output = model.get_video_features(pixel_values)

# Get last_hidden_state instead of pooled_output
last_hidden_state = model.get_video_features(
    pixel_values,
    return_last_hidden_state=True
)

# Get all outputs
outputs = model.get_video_features(
    pixel_values,
    return_last_hidden_state=True,
    return_dict=True,
    output_hidden_states=True,
    output_attentions=True
)

print(outputs.last_hidden_state.shape)
print(outputs.hidden_states[-1].shape)
```

This change improves the flexibility of the XCLIPModel without breaking any existing functionality, and it adds support for feature extraction and analysis use cases.

Comments:
Enforce implementation of sibling.size_in_bytes for model_info.siblings, for getting model files-sizes without downloading it
Author: benayat
Pull request: no
Labels: name: Feature request

### Feature request

Now, we can use model_info.siblings->size_in_bytes for getting individual file sizes, but many models have only Nan values there.

### Motivation

The first thing I do before running a model is checking if it'll fit my hardware. Instead of counting sizes manually in the UI, a feature for getting the model size ahead of downloading would be a great help.

### Your contribution

I can't, since I don't know enough yet.

Comments:
This information is displayed in the Hub UI - is there some way to check it via `huggingface-hub`? cc @Wauplin maybe?

Hi @benayat , the `model_info.siblings` field is indeed unreliable for certain repos (depends on repo size and Hub cache). The correct way to list all files with their details (including sizes) is to use `huggingface_hub.list_repo_tree`. More info in the docs: https://huggingface.co/docs/huggingface_hub/v1.2.3/en/package_reference/hf_api#huggingface_hub.HfApi.list_repo_tree

That's good enough for me. thanks.
Add instructions for appending NLLB language codes
Author: hunterschep
Pull request: yes

### What does this PR do?

This PR adds a short section to the NLLB documentation explaining how to add new language codes using the existing `add_special_tokens` / `extra_special_tokens` API in Transformers v5+.

This follows reviewer feedback that the current tokenizer API already supports this workflow and that a dedicated method isn't needed. The new docs make this workflow more discoverable for users who fine-tune NLLB on languages outside the FLoRes-200 set.

### Background / Related PRs

Supersedes and replaces #42756.
That PR attempted to add new tokenizer methods; based on reviewer feedback, this PR is documentation-only.

Thanks to @itazap for the review and guidance.

### Checklist

* [x] Documentation update only
* [x] No code changes
* [x] Builds cleanly with `doc-builder`
* [x] Reflects current v5 tokenizer behavior (`extra_special_tokens`, `replace_extra_special_tokens`, deduping logic, etc.)

Comments:
Hi! This PR only modifies the NLLB documentation.

CircleCI `tests_hub` is failing with:

```text
     NameError: name 'BACKEND' is not defined
     in tests/quantization/gptq/test_gptq.py::GPTQTest
```
This looks unrelated to the doc changes and may be a CI or `main`-branch issue.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42779). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Reapply modular examples
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Makes it much easier to see diff as I'm working on modular

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42846). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Gemma
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Just stumbled upon this... No idea why this was there...

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42847). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
image_transforms: fix tensor annotations
Author: cebtenzzre
Pull request: yes

# What does this PR do?

pyright complains when I call these functions. This is because their inputs and outputs are ostensibly tensors, but have been annotated with TensorType, which is an enum.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@yonigozlan @amyeroberts

Comments:
@yonigozlan I just added some overloads for the numpy case. Either way the existing annotations are wrong because they let me do this:
```python
center_to_corners_format(TensorType.PYTORCH)  # ok: TensorType passed
```
but not this:
```python
center_to_corners_format(torch.Tensor())  # error: "Tensor" is not an instance of "TensorType"
```
`PreTrainedTokenizerBase.model_input_names` is a singleton
Author: stephantul
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.55.4
- Platform: macOS-14.8.1-arm64-arm-64bit
- Python version: 3.12.8
- Huggingface_hub version: 0.34.3
- Safetensors version: 0.6.2
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0 (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker 
@itazap 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

The `model_input_names` property of each tokenizer class is initialized as a singleton list on import. This means that if any program modifies this list, all instances of that tokenizer will have modified behavior until the program exits.

```python
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("bert-base-uncased")
print(tok.model_input_names)
# ['input_ids', 'token_type_ids', 'attention_mask']
tok.model_input_names.append("HELLO THIS IS AN INPUT")

tok_2 = AutoTokenizer.from_pretrained("bert-base-uncased")
print(tok_2.model_input_names)
# ['input_ids', 'token_type_ids', 'attention_mask', 'HELLO THIS IS AN INPUT']
```

### Expected behavior

I think this can be solved by turning it into a property instead of a list. The property will have a setter. On initialization this property is set to the default for that class (which is already defined). This would "reset" the property for each individual instance, which is the expected behavior.

Comments:
Could we not just make a copy of the current `model_input_names` using the when we're initializing the instance? Just set it to 
`self.__class__.model_input_names.copy()` in the init method . Your fix would most likely work as well but I think this might be easier . It at least worked for your test script not sure if it will work for the others tho . 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
error: argument --include_num_input_tokens_seen/--include-num-input-tokens-seen: Truthy value expected: got non_padding but expected one of yes/no, true/false, t/f, y/n, 1/0 (case insensitive).
Author: guofy-ai
Pull request: no
Labels: name: bug

### System Info

Failed to parse argument "include_num_input_tokens_seen" using HfArgumentParser, code is unusable.

error: argument --include_num_input_tokens_seen/--include-num-input-tokens-seen: Truthy value expected: got non_padding but expected one of yes/no, true/false, t/f, y/n, 1/0 (case insensitive).

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

torchrun --save_steps=100 --cutoff_len=4096 --include_num_input_tokens_seen=non_padding

### Expected behavior

can not use args include_num_input_tokens_seen=non_padding

Comments:
cc @sunmarc is this possibly because a `Union[str, bool]` type is triggering `str_to_bool` and then failing when the input is a string that can't be converted?

Hi @Rocketknight1 , I took a look at it and would like to attempt a fix for this. Can you assign this to me ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[`Quantization FP8`] Native `from_config` support
Author: vasqu
Pull request: no
Labels: name: Feature request

### Feature request

Related to https://github.com/huggingface/transformers/pull/42028#discussion_r2592235170

Since FP8 is becoming more and more standard, it would be nice to create fp8 native models via config or more like using `from_config`. Atm, quant configs are not respected apparently - either that or we need to update the docs to show how to use it properly.

### Motivation

Fp8 is becoming increasingly important

### Your contribution

👀 

Comments:
cc @MekkCyber 

Yup the quant config is not respected in `from_config`, the logic only lives inside `from_pretrained`. We will explore how to fix that.

I opened a small PR that makes the current behavior explicit by raising an error when FP8 is requested via `from_config()`, instead of silently ignoring the quantization config.

PR: https://github.com/huggingface/transformers/pull/42865

This doesn’t add FP8 support yet, but it avoids confusing silent fallback and should make it clearer that FP8 currently needs to go through `from_pretrained()`.
enable static cache on vision encoder decoder
Author: jiqing-feng
Pull request: yes

After the issue #39746 fixed, the vision encoder decoder model can support static cache and can got more significant speed-up

```python
import time
import requests
import torch
import PIL.Image
from transformers import pipeline

model_id = "nlpconnect/vit-gpt2-image-captioning"
image_to_text = pipeline("image-to-text", model=model_id, device="cpu", torch_dtype=torch.float16)
image_url = "https://ankur3107.github.io/assets/images/image-captioning-example.png"
image = PIL.Image.open(requests.get(image_url, stream=True, timeout=3000).raw)
generation_config = image_to_text.model.generation_config
generation_config.cache_implementation = "static"

for _ in range(10):
    output = image_to_text(image, generate_kwargs={"generation_config": generation_config})

start = time.time()
output = image_to_text(image, generate_kwargs={"generation_config": generation_config})
end = time.time()
print(f"eager mode pipeline latency {end - start}")

image_to_text.model.forward = torch.compile(image_to_text.model.forward)

for _ in range(10):
    output = image_to_text(image, generate_kwargs={"generation_config": generation_config})

start = time.time()
output = image_to_text(image, generate_kwargs={"generation_config": generation_config})
end = time.time()
print(f"compile mode pipeline latency {end - start}")
```

Comments:
Hi @zucchini-nlp , could you please review this change? Thanks!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: vision_encoder_decoder

run-slow: vision_encoder_decoder

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39773). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix tensor device when loading state dict
Author: jiqing-feng
Pull request: yes

The tensor device is wrong when `device_map="auto"`
```python
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4", device_map="auto")
```
The [tensor_device](https://github.com/huggingface/transformers/blob/v4.53.3/src/transformers/modeling_utils.py#L759) turns out `"cpu"` which is not expected. I fix the correct tensor device and param device based on the device_map.

Comments:
It's veri likely that the failed CI is introduced by this PR, but I can pass these tests locally.

Hi @jiqing-feng I believe the device map logic is handled with `accelerate` - I don't know if there's a bug here! cc @SunMarc 

> ```python
> ```python
> from transformers import AutoModelForCausalLM
> model = AutoModelForCausalLM.from_pretrained("hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4", device_map="auto")
> ```

> ```

@jiqing-feng same question as Marc, what issue is triggered?

It hasn't triggered any issues so far. It's just not reasonable. The `tensor_device` will be `torch.device("cuda", index=0)` if I pass `device_map={"": 0}`. So I suppose it should also be `cuda` if I pass `device_map=auto` instead of `cpu`. 
For more details, if I pass `device_map="auto"` in the `from_pretained` function, the [device_map](https://github.com/huggingface/transformers/blob/v4.53.3/src/transformers/modeling_utils.py#L760) here will be
```
OrderedDict([('model.embed_tokens', 1), ('model.layers.0', 1), ('model.layers.1', 1), ('model.layers.2', 1), ('model.layers.3', 2), ('model.layers.4', 2), ('model.layers.5', 2), ('model.layers.6', 2), ('model.layers.7', 2), ('model.layers.8', 2), ('model.layers.9', 3), ('model.layers.10', 3), ('model.layers.11', 3), ('model.layers.12', 3), ('model.layers.13', 3), ('model.layers.14', 3), ('model.layers.15', 4), ('model.layers.16', 4), ('model.layers.17', 4), ('model.layers.18', 4), ('model.layers.19', 4), ('model.layers.20', 4), ('model.layers.21', 5), ('model.layers.22', 5), ('model.layers.23', 5), ('model.layers.24', 5), ('model.layers.25', 5), ('model.layers.26', 5), ('model.layers.27', 6), ('model.layers.28', 6), ('model.layers.29', 6), ('model.layers.30', 6), ('model.layers.31', 6), ('model.norm', 6), ('model.rotary_emb', 6), ('lm_head', 7)])
```
It will not trigger the following logic, so the tensor_device will be `cpu`, which is not reasonable. WDYT?

UT fixed. Hi @SunMarc . Please review this PR. Thanks!

Hi @SunMarc . Would you please review this PR? Thanks!

BTW, the failed tests are not related to my changes. It comes from torchcodec, which I already raised an issue here: https://github.com/pytorch/torchcodec/issues/795 . The final solution is that I upgraded my base docker image to `ubuntu:24.04`.

Close until it causes a bug.
Update quantization overview for XPU
Author: jiqing-feng
Pull request: yes

Update quantization overview for XPU.
Keep in draft until optimum-quanto PR: [395](https://github.com/huggingface/optimum-quanto/pull/395) merged.

Comments:
run-slow: aqlm_integration

cc @IlyasMoutawwakil @MekkCyber 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: aqlm_integration, ggml, quanto_integration

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40331). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.