GlmMoeDsaConfig: legacy `num_experts` unconditionally overrides explicit `n_routed_experts` (sibling of #46739)
Author: hermia-ai
Pull request: no

### System info

- `transformers` `main` (verified against `src/transformers/models/glm_moe_dsa/configuration_glm_moe_dsa.py` today)
- Python 3.12, torch 2.11, vLLM 0.25.1
- Affected checkpoint: [`0xSero/GLM-5.2-504B-Nvidia`](https://huggingface.co/0xSero/GLM-5.2-504B-Nvidia) —
  `GlmMoeDsaForCausalLM`, **no `auto_map`**, so this is the in-library config class, not remote code.

### Who can help?

This is a sibling of #46739 (`head_dim` → `qk_rope_head_dim`), which was resolved by #46338 by dropping the alias
and setting the value explicitly in `__post_init__`. The same clobber pattern still exists for `num_experts`.

### Description

`GlmMoeDsaConfig.__post_init__` contains an **unconditional** backward-compat re-route:

```python
# configuration_glm_moe_dsa.py (main), lines 161-163
# BC: re-route `num_experts` to `n_routed_experts`
if (num_experts := kwargs.get("num_experts")) is not None:
    self.n_routed_experts = num_experts
```

If a `config.json` contains **both** keys, the **legacy `num_experts` overwrites the modern, explicit
`n_routed_experts`** — the deprecated key wins. I'd expect a BC shim to be a *fallback* (apply only when
`n_routed_experts` was not explicitly provided), not an override.

This is exactly the shape of #46739, which you fixed in #46338 for `head_dim`.

### Reproduction

```python
from transformers import AutoConfig
M = "0xSero/GLM-5.2-504B-Nvidia"   # config.json has n_routed_experts: 168 AND num_experts: 256

AutoConfig.from_pretrained(M)                        # -> n_routed_experts = 256   ← JSON says 168!
AutoConfig.from_pretrained(M, num_experts=168)       # -> n_routed_experts = 256   ← kwarg cannot help
AutoConfig.from_pretrained(M, n_routed_experts=168)  # -> n_routed_experts = 168
# after deleting num_experts from config.json:       # -> n_routed_experts = 168   ← correct
```

The `num_experts=168` kwarg cannot fix it because the clobber runs during `__post_init__`, before
`from_pretrained` applies kwargs as attributes.

**Expected:** `n_routed_experts` = 168 (the explicit value, and the one matching the checkpoint's tensors).
**Actual:** 256, silently.

### Why it matters in practice

This is not hypothetical. Pruned/distilled GLM-5.2 derivatives in the wild ship **both** keys, because the pruning
tool updates `n_routed_experts` while the stale `num_experts` is inherited from the base config. The resulting
config reports an expert count contradicting **both its own JSON and its own tensors** (verified via
`model.safetensors.index.json`: exactly 168 contiguous routed experts, indices 0..167).

Downstream, vLLM builds `FusedMoE` for 256 experts that do not exist → 64 experts/rank instead of 42 →
**~111 GB/GPU instead of ~80 GB/GPU** → `torch.OutOfMemoryError` on a 94.97 GiB card
(`vllm/model_executor/layers/quantization/modelopt.py:1458`, `create_weights`). vLLM's log shows
`Local/global number of experts: 64/256` where `42/168` is correct. vLLM itself is innocent — it reads
`config.n_routed_experts` correctly and is simply handed a wrong config.

Deleting the stale key makes the model load at 79.1 GB/GPU and serve correctly. Verified on 4× RTX PRO 6000
Blackwell (sm_120).

### Suggested fix

Make the BC re-route a fallback rather than an override, e.g.:

```python
# BC: fall back to legacy `num_experts` only when `n_routed_experts` was not explicitly provided
if "n_routed_experts" not in kwargs and (num_experts := kwargs.get("num_experts")) is not None:
    self.n_routed_experts = num_experts
```

(Or follow the #46338 precedent more directly.) A warning when both keys are present and disagree would also have
saved a lot of debugging here — the current failure is a confusing OOM several layers downstream, with nothing
pointing at the config.

Happy to open a PR if useful.


Comments:
Hi! I'd like to work on this issue if no one is already working on it. If that's okay, I'll prepare a PR.

Hmmn, this looks real but I'm not sure about the suggested fix, because I think config fields don't end up in `kwargs`. Something similar might work, though!

I’d like to investigate this if it is still available. I’ll first trace how PretrainedConfig passes n_routed_experts and legacy num_experts into GlmMoeDsaConfig.__post_init__, then add a focused regression test for configurations containing both fields. I’ll follow the existing #46338 precedent only if it matches the actual constructor behavior. Is this okay to pursue?

Thanks @Rocketknight1 — you're right, and it takes down the fallback snippet I suggested (and the one now in #47398).

`n_routed_experts` is a declared field on the `@strict` dataclass (`configuration_glm_moe_dsa.py:102`), so when a `config.json` sets it, it's bound straight to `self.n_routed_experts` and never reaches `**kwargs`. Only the legacy `num_experts` — which isn't a field — lands in `kwargs`. So, against `main` today:

- `if (num_experts := kwargs.get("num_experts")) is not None:` (lines 161-163) fires whenever the stale key is present and overwrites the already-correct field value.
- `if "n_routed_experts" not in kwargs and ...` (my suggestion, and what #47398 currently does) is a no-op guard — `"n_routed_experts"` is *never* in `kwargs`, so the condition is always True and `num_experts` still wins.

The deeper point is exactly the one you made: by the time `__post_init__` runs, an explicit `n_routed_experts` is indistinguishable in `kwargs` from the default, so no kwargs-based guard can separate "user set it" from "defaulted." That signal only survives in the raw config dict.

Following the #46338 precedent (drop the alias, let the modern field be authoritative) looks like the right shape here: honor `n_routed_experts` whenever it's present in the source dict and treat `num_experts` as a pure fallback for legacy configs that carry only the old key — which likely means handling it where the raw dict is still visible rather than inside `__post_init__`.

I have `0xSero/GLM-5.2-504B-Nvidia` (the checkpoint that ships both keys) on disk and am happy to validate any candidate patch end-to-end — `AutoConfig.from_pretrained(...).n_routed_experts` should read 168, not 256. cc @shark8pepper-eng


Opened a PR for this: #47409

The fix applies to both `DeepseekV32Config` and `GlmMoeDsaConfig` — the BC fallback from `num_experts` to `n_routed_experts` now only triggers when `n_routed_experts` is still at its class default (meaning it wasn't explicitly provided).

Previously, when a config.json had both `n_routed_experts: 168` and `num_experts: 256`, the legacy key would silently overwrite the explicit value. Now the explicit `n_routed_experts` wins. Legacy-only configs (just `num_experts`) continue to work unchanged.