Add support for MiniMax-M2
Author: rogeryoungh
Pull request: yes
Labels: name: New model

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

This PR adds MiniMax-M2 model to Hugging Face Transformers from MiniMaxAI.

Relevant Links:

- [MiniMax M2 & Agent: Ingenious in Simplicity](https://www.minimax.io/news/minimax-m2)
- https://huggingface.co/MiniMaxAI/MiniMax-M2

<!-- Remove if not applicable -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with 

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->

@ArthurZucker @Cyrilvallez


Comments:
Failing tests on the hub are unrelated timeouts. Thanks for the update!

@ArthurZucker lets gets this quickly merged. 2 weeks is infinity in the current model wars cycle.

@Qubitium sorry for the delays, will be taking over for the most part. Crunching some important features for v5 so some PRs got lost under it, apologies

I've encountered a strange problem:

```python
>>> model = AutoModelForCausalLM.from_pretrained(MODEL_PATH, device_map="auto")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "../transformers/src/transformers/models/auto/auto_factory.py", line 373, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/modeling_utils.py", line 278, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/modeling_utils.py", line 3967, in from_pretrained
    device_map = _get_device_map(model, device_map, max_memory, hf_quantizer)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/integrations/accelerate.py", line 407, in _get_device_map
    device_map = infer_auto_device_map(
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/integrations/accelerate.py", line 736, in infer_auto_device_map
    current_buffer_size = compute_module_total_buffer_size(module, hf_quantizer)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/integrations/accelerate.py", line 260, in compute_module_total_buffer_size
    module_sizes, _ = compute_module_sizes(model, hf_quantizer, buffers_only=True)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/integrations/accelerate.py", line 240, in compute_module_sizes
    dtype_size = hf_quantizer.param_element_size(model, name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/quantizers/base.py", line 182, in param_element_size
    return model.get_parameter_or_buffer(param_name).element_size()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1964, in __getattr__
    raise AttributeError(
AttributeError: 'MiniMaxM2DecoderLayer' object has no attribute 'get_parameter_or_buffer'
```

I found a solution #42342, which involves adding a `get_parameter_or_buffer` function to the MiniMaxM2RotaryEmbedding. However, for MiniMax-M2, `e_score_correction_bias` also needs to be changed to `nn.Parameter`.

```diff
@@ -106,7 +106,10 @@ class MiniMaxM2SparseMoeBlock(nn.Module):
         self.jitter_noise = config.router_jitter_noise
         self.gate = nn.Linear(config.hidden_size, config.num_local_experts, bias=False)
         self.experts = MiniMaxM2Experts(config)
-        self.register_buffer("e_score_correction_bias", torch.zeros(config.num_local_experts))
+        self.e_score_correction_bias = nn.Parameter(
+            torch.zeros(config.num_local_experts),
+            requires_grad=False,
+        )
 
     def forward(self, hidden_states: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
         batch_size, sequence_length, hidden_dim = hidden_states.shape
@@ -168,6 +171,20 @@ class MiniMaxM2RotaryEmbedding(nn.Module):
         self.register_buffer("inv_freq", inv_freq, persistent=False)
         self.original_inv_freq = inv_freq
 
+    # Add a compatibility method so callers expecting PreTrainedModel-like API don't crash.
+    def get_parameter_or_buffer(self, name: str):
+        # Prefer direct attribute access (parameters and buffers are attributes)
+        if hasattr(self, name):
+            return getattr(self, name)
+        # Fallback: search named parameters and buffers (non-recursive to keep semantics)
+        for n, p in self.named_parameters(recurse=False):
+            if n == name:
+                return p
+        for n, b in self.named_buffers(recurse=False):
+            if n == name:
+                return b
+        raise AttributeError(f"{self.__class__.__name__} has no parameter or buffer named '{name}'")
+
     @staticmethod
     def compute_default_rope_parameters(
         config: Optional[MiniMaxM2Config] = None,
```

Hi @rogeryoungh, we are taking care of this in : https://github.com/huggingface/transformers/pull/42289/files#diff-7f40070336f6d7b1ffe08e654cdf930080e3cbd4dbbcbee2996fabe4ffc1c2b3, this should fix the issue without adding a `get_parameter_or_buffer`

I verified that PR #42289 resolved the previous issue.

Now I've encountered a new problem: when loading weights, I get the following error. Can this be resolved using `WeightRenaming`?

```bash
Key                                                       | Status     | Details
----------------------------------------------------------+------------+--------
model.layers.{0...61}.mlp.experts.down_proj_scale_inv     | UNEXPECTED |        
model.layers.{0...61}.mlp.experts.gate_up_proj_scale_inv  | UNEXPECTED |        
model.layers.{0...61}.mlp.experts.gate_up_proj_scales_inv | MISSING    |        
model.layers.{0...61}.mlp.experts.down_proj_scales_inv    | MISSING    |        

Notes:
- UNEXPECTED    :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
- MISSING       :those params were newly initialized because missing form the checkpoint. Consider training on your downstream task.
```

I'm a bit surprised that you get 
```text
model.layers.{0...61}.mlp.experts.gate_up_proj_scales_inv | MISSING    |        
model.layers.{0...61}.mlp.experts.down_proj_scales_inv    | MISSING    |     
```
Don't you have https://github.com/huggingface/transformers/blob/8fe4d2bd67dc8f0c624e2234ce0d07768bc54c1a/src/transformers/models/minimax_m2/modeling_minimax_m2.py#L58-L59

This seems weird to me, how do you load the model? But yes, that can work as well but I'm surprised about the naming because it isn't in the modeling file at all - can you double check?

<img width="1863" height="1111" alt="image" src="https://github.com/user-attachments/assets/1e4f38ec-f581-4820-bfb3-8332b66fbb28" />

This was a part of FP8Linear.

https://github.com/huggingface/transformers/blob/8fe4d2bd67dc8f0c624e2234ce0d07768bc54c1a/src/transformers/integrations/finegrained_fp8.py#L328-L335

Could it be linked with what we discussed yesterday @MekkCyber , just in case?

#42434 should fix your issue @rogeryoungh 

Sorry didn't notice we have native fp8 in this model :facepalm: 

Hi @rogeryoungh ! Can you try with the pr @vasqu shared please 🤗 ! Thank you

When syncing branches, I encountered some issues related to FP8Linear:

```bash
Key                                                      | Status     |  | 
---------------------------------------------------------+------------+--+-
model.layers.{0...61}.mlp.experts.gate_up_proj_scale_inv | UNEXPECTED |  | 
model.layers.{0...61}.mlp.experts.down_proj_scale_inv    | UNEXPECTED |  | 

Notes:
- UNEXPECTED    :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
Some parameters are on the meta device because they were offloaded to the cpu.
```

I also encountered a strange error, it seems that FP8Linear is not working correctly here?

```bash
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "../transformers/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/generation/utils.py", line 2684, in generate
    result = decoding_method(
             ^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/generation/utils.py", line 2877, in _sample
    outputs = self._prefill(input_ids, generation_config, model_kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/generation/utils.py", line 3853, in _prefill
    return self(**model_inputs, return_dict=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/accelerate/hooks.py", line 175, in new_forward
    output = module._old_forward(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/utils/generic.py", line 764, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/minimax_m2/modeling_minimax_m2.py", line 671, in forward
    outputs: MoeModelOutputWithPast = self.model(
                                      ^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/utils/generic.py", line 919, in wrapper
    outputs = func(self, *args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/minimax_m2/modeling_minimax_m2.py", line 508, in forward
    hidden_states = decoder_layer(
                    ^^^^^^^^^^^^^^
  File "../transformers/src/transformers/modeling_layers.py", line 94, in __call__
    return super().__call__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/accelerate/hooks.py", line 175, in new_forward
    output = module._old_forward(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/minimax_m2/modeling_minimax_m2.py", line 412, in forward
    hidden_states = self.mlp(hidden_states)
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/accelerate/hooks.py", line 175, in new_forward
    output = module._old_forward(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/minimax_m2/modeling_minimax_m2.py", line 124, in forward
    hidden_states = self.experts(hidden_states, top_k_index, top_k_weights)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/.venv/lib/python3.12/site-packages/accelerate/hooks.py", line 175, in new_forward
    output = module._old_forward(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/minimax_m2/modeling_minimax_m2.py", line 80, in forward
    gate, up = nn.functional.linear(current_state, self.gate_up_proj[expert_idx]).chunk(2, dim=-1)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c10::Float8_e4m3fn
```

Sorry about that @rogeryoungh ! can you check with this pr: https://github.com/huggingface/transformers/pull/42654, it's a small fix

I used the following code to load the model from the configuration file:

```python
# ... modify default config and save
config = AutoConfig.from_pretrained(
    save_folder,
    trust_remote_code=True,
)
model = AutoModelForCausalLM.from_config(config)

# full code: https://github.com/rogeryoungh/MiniMaxM2TinyModelGenerator/blob/main/generator.py
```

But I received the following error:

```bash
Traceback (most recent call last):
  File "../MiniMaxM2TinyModelGenerator/generator.py", line 49, in <module>
    model = AutoModelForCausalLM.from_config(config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/auto/auto_factory.py", line 237, in from_config
    return model_class._from_config(config, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/modeling_utils.py", line 250, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/modeling_utils.py", line 1438, in _from_config
    model = cls(config, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/minimax_m2/modeling_minimax_m2.py", line 617, in __init__
    self.model = MiniMaxM2Model(config)
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/minimax_m2/modeling_minimax_m2.py", line 458, in __init__
    self.rotary_emb = MiniMaxM2RotaryEmbedding(config=config)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/models/minimax_m2/modeling_minimax_m2.py", line 160, in __init__
    self.rope_type = self.config.rope_parameters["rope_type"]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "../transformers/src/transformers/configuration_utils.py", line 198, in __getattribute__
    return super().__getattribute__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'MiniMaxM2Config' object has no attribute 'rope_parameters'
```


@rogeryoungh I think the auto mapping is getting confused here, you should rather directly use `MiniMaxM2Config` from your current PR. At least that seems to work somewhat. Note that this was used for BC loading but not saving so you might need to check if some values are unwanted, e.g. `rope_scaling` will also live beside `rope_parameters`, `layernorm_full_attention_beta`, etc. --> We should have a config that is only having the relevant fields (if that's ok) for us.

Also the automap should not consist in the config unless you want to use remote code and need to point the respective files (which shouldn't be the case anymore after merging this).

Last side note: I think the way you load the generation config it will load all our values (including those that don't divert from defaults). Imo, you can keep the generation config purely as is and just move the raw json.

Edit: If it helps you can also do something along
```python
from transformers import MiniMaxM2Config, MiniMaxM2Config

config = MiniMaxM2Config.from_pretrained("MiniMaxAI/MiniMax-M2") # ideally just MiniMaxM2Config() as those should have the respective values already 
config.num_hidden_layers = 7  # check the size, something up to 7-8B should be ok
model = MiniMaxM2Config.from_pretrained(
    "MiniMaxAI/MiniMax-M2",
    config=config,  # we cut the model into x layers, e.g. here 7
    device_map="auto",
    dtype="auto",
    # ... (quants if you need etc)
)
model.save_pretrained("Small-MiniMax-M2")
```

According to my tests, `from_pretrained` does not reduce the number of layers in the model, indicating that the configuration is not being used.

I found a solution: generate the model twice.

1. Load the model using `from_config`, which saves the model with the correct structure but without the correct FP8Linear layers.
2. Use `from_pretrained` again, which allows me to save the correct quantized results.

My complete code is here: https://github.com/rogeryoungh/MiniMaxM2TinyModelGenerator/blob/main/generator.py

```diff
model = MiniMaxM2Config.from_pretrained(
    "MiniMaxAI/MiniMax-M2",
+    num_layers=3,  # we cut the model into x layers, e.g. here 7
-    config=config,
    device_map="auto",
    dtype="auto",
    # ... (quants if you need etc)
)
```
should work with the appropriate name!

@rogeryoungh I pushed a few commits to 
- sync with main
- adjust some values in the config
- fixup modular

With the current state, a conversion could look like this
```python
from transformers import FineGrainedFP8Config, MiniMaxM2Config, MiniMaxM2ForCausalLM


token = "hf_xxx"


config = MiniMaxM2Config()
quantization_config = FineGrainedFP8Config(
    modules_to_not_convert=[
      "gate",
      "e_score_correction_bias",
      "lm_head"
    ]
)

config.num_hidden_layers = 2  # check the size, something up to 7-8B should be ok
model = MiniMaxM2ForCausalLM.from_pretrained(
    "MiniMaxAI/MiniMax-M2",
    config=config,
    device_map="auto",
    dtype="auto",
    quantization_config=quantization_config,
)
model.save_pretrained("MiniMax-M2-Small", push_to_hub=True, repo_id="hf-internal-testing/MiniMax-M2-Small", token=token)
```

However, I discovered that while there are no error logs, the conversion doesn't seem to match the intended shapes. Discussing this internally and following back when I got news cc @MekkCyber 

Edit: Fixed, the quant config needs to be within the config for native fp8 weights, not passed as separate quant config

```python
from transformers import FineGrainedFP8Config, MiniMaxM2Config, MiniMaxM2ForCausalLM


token = "hf_xxx"


config = MiniMaxM2Config()
config.quantization_config = FineGrainedFP8Config(
    modules_to_not_convert=[
      "gate",
      "e_score_correction_bias",
      "lm_head"
    ]
).to_dict()

config.num_hidden_layers = 2  # check the size, something up to 7-8B should be ok
model = MiniMaxM2ForCausalLM.from_pretrained(
    "MiniMaxAI/MiniMax-M2",
    config=config,
    device_map="auto",
    dtype="auto",
)
model.save_pretrained("MiniMax-M2-Small", push_to_hub=True, repo_id="hf-internal-testing/MiniMax-M2-Small", token=token)
```

Edit 2: Our CI Runner can't handle fp8 because it needs an hopper GPU and we only have an ampere :cry: updated to a base version that is randomized
```python
import torch

from transformers import MiniMaxM2Config, MiniMaxM2ForCausalLM, set_seed


token = "hf_xxx"

config = MiniMaxM2Config()
config.head_dim = 128
config.hidden_size = 1024
config.intermediate_size = 2048
config.num_attention_heads = 12
config.num_experts_per_tok = 2
config.num_hidden_layers = 12
config.num_key_value_heads = 2
config.num_local_experts = 16
config.tie_word_embeddings = True

model = MiniMaxM2ForCausalLM(config)
set_seed(42)
model = model.cpu()
with torch.no_grad():
    for name, p in sorted(model.named_parameters()):
        torch.nn.init.normal_(p, mean=0.0, std=0.2)
model.to(torch.float16)
model.save_pretrained("MiniMax-M2-Small", push_to_hub=True, repo_id="hf-internal-testing/MiniMax-M2-Small", token=token)
```

run-slow: minimax_m2

This comment contains `run-slow`, running the specified jobs: 

models: ["models/minimax_m2"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20131951083)

✅ No failing test specific to this PR 🎉 !


run-slow: minimax_m2

This comment contains `run-slow`, running the specified jobs: 

models: ["models/minimax_m2"]
quantizations: []

Deleted the github action comment because I was just checking something which I already reverted

run-slow: minimax_m2

This comment contains `run-slow`, running the specified jobs: 

models: ["models/minimax_m2"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20791435476)

✅ No failing test specific to this PR 🎉 !


Sorry for the delays, just coming back from the holidays @rogeryoungh. I've synced with the latest changes 
- PR for native transformers integration is at https://huggingface.co/MiniMaxAI/MiniMax-M2.1/discussions/15 (will need to double check if the values are correct for the new model here, it's currently the same as for the base M2 model)
- Likely needing #43154 to have correct Moe behavior with fp8 weights

Will come back to this tomorrow but wanted to give a first insight, should be close to getting merged :hugs: 

I found that the current code is not running correctly and throws an error at `self.experts(hidden_states, top_k_index, top_k_weights)`. But I'm not sure if it's a problem with MiniMaxM2TopKRouter or FP8Expert.

I found a way to fix it:

```diff
@@ -59,7 +59,7 @@ class MiniMaxM2TopKRouter(nn.Module):
         _, top_k_index = torch.topk(scores_for_choice, self.top_k, dim=-1, sorted=False)
         top_k_weights = routing_weights.gather(1, top_k_index)
         top_k_weights /= top_k_weights.sum(dim=-1, keepdim=True)
-        router_scores = top_k_weights
+        router_scores = torch.zeros_like(routing_weights).scatter_(1, top_k_index, top_k_weights)
         return router_logits, router_scores, top_k_index
```

@rogeryoungh it will be fixed by #43154, but yes your solution would work as well but we removed that way of routing because scatter caused issues iirc (see #42456)

Will finalize that PR and update this branch then

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42028&sha=cca9ee

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, minimax_m2

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, minimax_m2

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, minimax_m2, finegrained_fp8

run-slow: minimax_m2

This comment contains `run-slow`, running the specified jobs: 

models: ["models/minimax_m2"]
quantizations: []

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42028&sha=b86771

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20824746312)

✅ No failing test specific to this PR 🎉 !


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, minimax_m2, finegrained_fp8

Ok, now everything works and is synced with the latest changes on main. Hub PRs are open at
- Minimax M2: https://huggingface.co/MiniMaxAI/MiniMax-M2/discussions/52
- Minimax M2.1: https://huggingface.co/MiniMaxAI/MiniMax-M2.1/discussions/15

Lmk if you have any last changes @rogeryoungh, otherwise I'll merge tomorrow as it works from our side then.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42028). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.