noamaanMulla-03/transformers-issues huggingface dataset:
[Offloading] Support full disk offloading
Author: kylesayrs
Pull request: yes

## Purpose ##
* Support full disk offloading of an entire model
  * This is useful for testing and cases like [vllm-project/llm-compressor](https://github.com/vllm-project/llm-compressor) which heavily utilizes disk offloading to compress large models
* Add missing handling for when tied weights are disk offloaded
* This PR can be merged independently from https://github.com/huggingface/accelerate/pull/4083

## Changes ##
* Add tied tensor targets to the disk index when they do not appear in checkpoint files
  * This handles the case where lm_head does not appear the checkpoint
* Skip `torch.equal` check, which is not supported for meta tensors
  * This handles the case where embeddings are tied, but both appear in the checkpoint and both are disk offloaded
* Force `accelerate_dispatch` to be called if the model is full disk offloaded
  * This causes hooks to be added to the model, which are required for disk offloading

## Testing ##
* Tested changes with `test_disk_offloading.py` in conjunction with https://github.com/huggingface/accelerate/pull/4083
  * This test should not be merged into main because it requires changes from accelerate

<details open><summary>test_disk_offloading.py</summary>

```python
import torch
import pytest

from transformers import AutoModelForCausalLM


# Qwen3-0.6B has tied weights, two copies in checkpoint
# Llama-3.2-1B-Instruct has tied weights, only one copy in checkpoint
# tinysmokeqwen3 has untied weights
@torch.no_grad()
@pytest.mark.parametrize("model_id,cpu_mem", [
    ("Qwen/Qwen3-0.6B", 0),
    ("Qwen/Qwen3-0.6B", 6e8),
    ("meta-llama/Llama-3.2-1B-Instruct", 0),
    ("meta-llama/Llama-3.2-1B-Instruct", 1e9),
    ("nm-testing/tinysmokeqwen3", 0),
    ("nm-testing/tinysmokeqwen3", 1.5e6),
])
def test_tied_in_checkpoint(model_id, cpu_mem):
    true_model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
    model = AutoModelForCausalLM.from_pretrained(
        model_id,
        device_map="auto",
        max_memory={"cpu": cpu_mem},
        offload_folder="./offload_folder",
    )

    inputs = {key: value.to(true_model.device) for key, value in model.dummy_inputs.items()}

    true_outputs = true_model(**model.dummy_inputs)
    outputs = model(**model.dummy_inputs)
    assert torch.nn.functional.mse_loss(true_outputs.logits, outputs.logits) < 1e-2
```

</details>

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46749)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46749). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Import diffusion_gemma in models init
Author: boringcrypto
Pull request: yes

# What does this PR do?

Import diffusion_gemma in models init


Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46841)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46841). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
docs(trainer): add JIT checkpointing to trainer recipes
Author: efazal
Pull request: yes

Adds documentation for JIT checkpointing (`enable_jit_checkpoint`) to the Checkpointing section of `trainer_recipes.md`, as requested by @SunMarc in #41723.

Covers what JIT checkpointing is, how it differs from periodic checkpointing, usage, and orchestrator grace period configuration for Kubernetes and Slurm.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46826)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46826). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix: convert Tensor q_offset/kv_offset to int in flex_attention_mask
Author: hashwnath
Pull request: yes

## Summary
- When `q_offset` or `kv_offset` arrive as scalar `torch.Tensor` objects (e.g. from `cache_position[0]` or `StaticLayerCache.get_seq_length()`), they get captured by the `inner_mask` closure inside `add_offsets_to_mask_function()`. Adding a plain int to a scalar Tensor returns a Tensor instead of a scalar, so the boolean mask result has shape `(1,)` instead of being a scalar bool. This extra dimension propagates through `create_block_mask`, producing a 5D `BlockMask` that crashes with `ValueError: too many values to unpack (expected 4)`.
- The fix calls `.item()` on Tensor offsets at the `flex_attention_mask()` boundary, converting them to native Python ints before they enter the closure.

## Test plan
- [x] Traced the call chain: `create_causal_mask` -> `flex_attention_mask` -> `add_offsets_to_mask_function` -> `create_block_mask`
- [x] Confirmed `_preprocess_mask_arguments` reshapes to 0-dim Tensor but does not convert to int
- [x] Verified `.item()` is the correct conversion (matches the reporter's confirmed workaround)

Fixes #46682

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46828)

  Tracked this down. When `q_offset` arrives as a scalar Tensor (which happens with `cache_position[0]` or`StaticLayerCache.get_seq_length()`), it gets captured by the mask closure inside `add_offsets_to_mask_function`. Adding a plain int to a scalar Tensor returns a Tensor, not a scalar, so the boolean result gains an extra dimension. This propagates through `create_block_mask` and produces a 5D BlockMask instead of 4D.

There's already a reshape to 0-dim in `_preprocess_mask_arguments` (line 863), but that still leaves it as a Tensor. Converting to a native Python int via `.item()` at the `flex_attention_mask` boundary is the clean fix.

Should be resolved already by #46802!

Also blocked for just pasting raw Claude output to us. We are a world-leading AI company and we know how to run a code agent so we do not need you to type "Claude find issues and fix them" for us
Fix beam search for mamba models
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Fixes https://github.com/huggingface/transformers/issues/46612

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46819). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: rwkv, xlstm

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46819)
Switch decorator order
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. For proepr skip, `skip` must be AFTER `parameterized.expand` (currently, they appear as PASSED, because the body of the test is then only `pass`)

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: chameleon, fsmt, gemma4, glm_moe_dsa, qwen3_omni_moe, xlstm

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46853)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46853). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix expanded dim in the cache's cumulative length
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Follow-up to https://github.com/huggingface/transformers/pull/46802. Rather than reshaping when creating the mask, I believe it makes more sense to never use the added dim, i.e. use `cumulative_length` as a scalar tensor all the time, closer to a full python int.
Note that I made sure that it does not break our compile pipeline, both for StaticLayer and StaticSlidingWindowLayer.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46856). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46856)
docs: Add NeMo Automodel community integration docs
Author: adil-a
Pull request: yes

# What does this PR do?

Adds NeMo Automodel under the community integrations tab.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @stevhliu


Comments:
Hi @stevhliu thanks so much for the review. I've addressed most of your comments. 

NeMo Automodel provides extended support for *both* pretraining (which is large scale) and finetuning (which is small, low-compute scale). Given that the Transformer community integration docs contain two separate categories for pretraining and finetuning, I would highly prefer if we can keep the docs separate here (I agree they're largely the same). I would like to avoid undercutting ourselves by, for example, putting pretraining and finetuning together, the doc lives under the finetuning category, and a user comes in who wants to do continued pretraining and can't find NeMo Automodel under the pretraining category. The onus is on the NeMo team to ensure consistency between both docs, please have no worry about this. I'd highly appreciate it if you could take this into consideration. Many thanks once again!

Thank you very much @stevhliu for your time and consideration. I have applied your recommendations. This should now be good to go :)

How can we launch CI to get the PR merged?

thanks again for your contribution!
[CB] Fix issues with FA read / writes
Author: remi-or
Pull request: yes

# Summary

This PR fixes an issue where the padding tokens from flash attention would write NaN in the cache at the same place where they read their cache from. 
This should not be a problem, expect NaNs screw up the softmax of real tokens for which the QK tensor is computed in the same block as the padding tokens. For some reason, this issue only manifests when the cache is tiny (16 blocks) hence why it stayed silent for so long. Also really hard to debug.

This is fixed by separating the trash index (where padding token read / write their KV) into two indices: read trash and write trash. This fixed the issue across all reproductions scripts I had written.

## Performance ✅ 
 
| label | main acc | new PR acc | main (tok/s) | new PR (tok/s) | diff |
|----------------------|----------|------------|--------------|----------------|--------|
| gsm8k_default | 0.822 | 0.822 | 2470.10 | 2468.19 | -0.08% |
| gsm8k_sampling | 0.782 | 0.778 | 1971.01 | 1968.82 | -0.11% |
| gsm8k_compile | 0.821 | 0.821 | 2487.11 | 2485.12 | -0.08% |
| gsm8k_no_fast_decode | 0.822 | 0.822 | 2374.41 | 2377.39 | +0.13% |
| gsm8k_bare_bones | 0.821 | 0.821 | 1905.06 | 1898.28 | -0.36% |
| ifeval_default | 0.444 | 0.445 | 9675.81 | 9667.16 | -0.09% |
| few_blocks | - | - | 677.06 | 676.26 | -0.12% |
| multi_return_seq | - | - | 1638.08 | 1642.73 | +0.28% |
| rollouts_1024 | - | - | 3535.73 | 3538.73 | +0.08% |
| rollouts_2048 | - | - | 3375.33 | 3371.30 | -0.12% |
| rollouts_4096 | - | - | 2991.36 | 2991.89 | +0.02% |
| rollouts_8192 | - | - | 2356.92 | 2357.09 | +0.01% |
| rollouts_16384 | - | - | 1506.42 | 1506.57 | +0.01% |

No perf change.

## Tests ✅ 

Took the opportunity to write some tests for the sliding window indices. All tests pass.

## AI Review ✅ 

Reviewed and addressed. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46765). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46765)
Fix Moonshine training-loss double-shift (train against labels, not labels[..., 1:])
Author: Incheonkirin
Pull request: yes

## What does this PR do?

Moonshine right-shifts `labels` into `decoder_input_ids`, then computes the loss with `self.loss_function` (`ForCausalLMLoss`), which shifts again, so it trains against `labels[..., 1:]`. Switch to a plain `CrossEntropyLoss`, matching `WhisperForConditionalGeneration`/`BartForConditionalGeneration` and the `VisionEncoderDecoderModel` fix in #40863 for the same issue (#40111).

Only the training loss (when `labels` are passed) changes; logits and `generate()` are unchanged, and `-100` stays ignored. Adds a regression test, with and without `-100`.


Comments:
cc @Incheonkirin I'm seeing the CI complain about the modular conversion. Can you rerun `make fix-repo`?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46784). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

`make fix-repo` propagated the changes to `moonshine-streaming` too - can we copy the test over there too to make sure we don't break it?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: moonshine, moonshine_streaming

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46784)

Yep, LGTM now!
Enhance return type annotation in apply_chat_template in processing_u…
Author: boringcrypto
Pull request: yes

Update return type annotation for apply_chat_template to include multiple possible types. Currently the return type is str, but the function can return many types, tripping up PyLance or any type checking.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46843)

> ⚠️ **Code quality check failed** — all test jobs were skipped. Fix the code quality issues and push again to run tests.

Hey! Type hint looks good, but can you do `pip install -e .[quality]` followed by `make fix-repo`? I think the indentation change is making the style tools complain
Fix loss and epoch accounting for native (accelerate) SP backend
Author: qgallouedec
Pull request: yes

See https://github.com/huggingface/transformers/pull/46775#issuecomment-4785502492, cc @SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46855). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46855)
Feature/add axk1
Author: kmswin1
Pull request: yes
Labels: name: New model

Add A.X K1 model architecture
What does this PR do?
This PR adds support for A.X K1, a large-scale Mixture-of-Experts (MoE) language model developed by [SK Telecom](https://huggingface.co/skt). A.X K1 contains 519B total parameters with 33B active parameters per token, enabling strong reasoning performance with practical inference efficiency.

Paper: [A.X K1 Technical Report (arXiv:2601.09200)](https://arxiv.org/abs/2601.09200)
Model: [skt/A.X-K1](https://huggingface.co/skt/A.X-K1)
License: Apache 2.0

Architecture overview
A.X K1 is a decoder-only Transformer with the following key design choices:

Sparse MoE: 61 layers (1 dense + 60 MoE), with 192 routed experts + 1 shared expert per MoE layer. 8 experts + 1 shared expert are activated per token.
Multi-Latent Attention (MLA): Compressed KV representations for efficient long-context inference.
Post-MLP RMSNorm: An additional RMSNorm applied after the MLP/MoE block in each layer, improving training stability in large-scale sparse settings.
Multi-Token Prediction (MTP): An auxiliary training objective that also enables speculative decoding at inference time.
Hybrid Reasoning (Think / Non-Think): Supports user-controllable reasoning depth within a single unified model.
Context length: 131,072 tokens
Vocab size: 163,840 (BBPE-based tokenizer optimized for EN, KO, ZH, JA, ES and code)

This PR includes

Model implementation (AXK1ForCausalLM, AXK1Model, AXK1Config)
Tokenizer integration (BBPE-based, 163,840 vocab)
Documentation and model card
Tests

Before submitting

I confirm that this is not a pure code agent PR.
This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
Did you write any new necessary tests?

Who can review?
@ArthurZucker @Cyrilvallez — text model architecture addition

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=45426&sha=262a93
Feature/add axk1
Author: kmswin1
Pull request: yes

Add A.X K1 model architecture
What does this PR do?
This PR adds support for A.X K1, a large-scale Mixture-of-Experts (MoE) language model developed by [SK Telecom](https://huggingface.co/skt). A.X K1 contains 519B total parameters with 33B active parameters per token, enabling strong reasoning performance with practical inference efficiency.

Paper: [A.X K1 Technical Report (arXiv:2601.09200)](https://arxiv.org/abs/2601.09200)
Model: [skt/A.X-K1](https://huggingface.co/skt/A.X-K1)
License: Apache 2.0

Architecture overview
A.X K1 is a decoder-only Transformer with the following key design choices:

Sparse MoE: 61 layers (1 dense + 60 MoE), with 192 routed experts + 1 shared expert per MoE layer. 8 experts + 1 shared expert are activated per token.
Multi-Latent Attention (MLA): Compressed KV representations for efficient long-context inference.
Post-MLP RMSNorm: An additional RMSNorm applied after the MLP/MoE block in each layer, improving training stability in large-scale sparse settings.
Multi-Token Prediction (MTP): An auxiliary training objective that also enables speculative decoding at inference time.
Hybrid Reasoning (Think / Non-Think): Supports user-controllable reasoning depth within a single unified model.
Context length: 131,072 tokens
Vocab size: 163,840 (BBPE-based tokenizer optimized for EN, KO, ZH, JA, ES and code)

This PR includes

Model implementation (AXK1ForCausalLM, AXK1Model, AXK1Config)
Tokenizer integration (BBPE-based, 163,840 vocab)
Documentation and model card
Tests

Before submitting

I confirm that this is not a pure code agent PR.
This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
Did you write any new necessary tests?

Who can review?
@ArthurZucker @Cyrilvallez — text model architecture addition

Comments:
Round the ue8m0 FP8 scale before quantizing so dequant matches the stored inverse
Author: Incheonkirin
Pull request: yes

`Fp8Quantize._quantize_one` quantizes the weight with the unrounded block scale, then for `scale_fmt="ue8m0"` (DeepSeek-V4 style) rounds only the stored `weight_scale_inv` up to a power of two. At load time `Fp8Dequantize` computes `weight * weight_scale_inv`, so the dequant scale can be up to a full octave (~2x) larger than the scale the weight was actually divided by. Nothing flags it: the shapes and dtypes are valid, the weights are just wrong.

DeepSeek's own DeepGEMM reference rounds the scale first and quantizes with that same rounded scale (`deep_gemm/utils/math.py`: `sf = ceil_to_ue8m0(sf); x_fp8 = x / sf`), so quant and dequant agree. This change matches that ordering: round `inv_scales` to the power-of-two grid, re-derive `scales = 1 / inv_scales`, then quantize. The rounding direction is unchanged; only the order moves. The `scale_fmt="float"` path never reassigns `scales`, so non-ue8m0 checkpoints quantize bit-identically.

Tests: `test_fp8_ue8m0_quantize_dequantize_round_trip` checks the round-trip (main ~0.6, with this change ~0.022, matching the e4m3 floor); `test_fp8_float_scale_fmt_quantization_unchanged` asserts the `scale_fmt="float"` path stays bit-identical to the original formula across 100 random 128x128 blocks. CPU-only.


Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46763)

cc @sunmarc for quants

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46763). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.