Checkpoint conversion silently loads transposed fused-expert weights when the swapped dims are equal (Transpose(check_dims=True), qwen3_vl_moe)
Author: pjordanandrsn
Pull request: no


### System info

transformers 5.12.1 (code-identical on `main` as of 2026-07-02: `core_model_loading.py`
`Transpose.convert`, `conversion_mapping.py` qwen3_vl_moe entries), torch 2.11.0. CPU-only repro.

### Bug

`Transpose(1, 2, check_dims=True)` — used by the qwen3_vl_moe conversion to accept both the legacy
pre-grouped expert layout `(E, H, 2I)/(E, I, H)` and the v5 F.linear layout `(E, 2I, H)/(E, H, I)`
— decides whether a checkpoint tensor still needs transposing by shape comparison:

```python
# core_model_loading.py, Transpose.convert
expected_shape = kwargs["model"].get_parameter(kwargs["full_layer_name"]).shape
# The shapes are the same: do NOT transpose
if tensor.shape == expected_shape:
    return {target_pattern: tensor}
```

When the two swapped dims are equal, both layouts produce the *same* shape and the comparison
carries no signal — the "do NOT transpose" branch wins and a legacy-layout checkpoint loads with
**silently transposed expert weights**. No warning, finite outputs, garbage model.

Two trigger variants:
- `2 * moe_intermediate_size == hidden_size` → poisons `gate_up_proj`
- `moe_intermediate_size == hidden_size` → poisons `down_proj`

Official Qwen3-VL-MoE checkpoints are non-square, so this is latent today — it waits for any
pruned/distilled/community variant with square dims stored in the legacy layout (and any future
arch that reuses `check_dims=True`).

### Reproduction (CPU, ~seconds)

```python
import torch
from safetensors.torch import save_file
from transformers import Qwen3VLMoeConfig, Qwen3VLMoeForConditionalGeneration

def probe(moe_intermediate):  # hidden_size = 64
    cfg = Qwen3VLMoeConfig(
        text_config=dict(vocab_size=1024, hidden_size=64, intermediate_size=128,
                         moe_intermediate_size=moe_intermediate, num_hidden_layers=1,
                         num_attention_heads=4, num_key_value_heads=2, head_dim=16,
                         num_experts=4, num_experts_per_tok=2, decoder_sparse_step=1,
                         tie_word_embeddings=True),
        vision_config=dict(depth=2, hidden_size=32, intermediate_size=64, num_heads=2,
                           out_hidden_size=64, num_position_embeddings=64,
                           deepstack_visual_indexes=[0, 1]))
    torch.manual_seed(0)
    model = Qwen3VLMoeForConditionalGeneration(cfg).to(torch.bfloat16)
    correct, legacy = {}, {}
    for k, v in model.state_dict().items():
        fused = k.endswith(("mlp.experts.gate_up_proj", "mlp.experts.down_proj"))
        correct[k] = v.clone()
        legacy[k] = v.transpose(1, 2).contiguous() if fused else v.contiguous()
    save_file(legacy, "/tmp/vlmoe/model.safetensors"); cfg.save_pretrained("/tmp/vlmoe")
    loaded = Qwen3VLMoeForConditionalGeneration.from_pretrained("/tmp/vlmoe", dtype=torch.bfloat16)
    for k, v in loaded.state_dict().items():
        if k.endswith(("gate_up_proj", "down_proj")) and not torch.equal(v, correct[k]):
            print(f"moe_intermediate={moe_intermediate}: {k} loaded TRANSPOSED (garbage)")

probe(16)  # non-square control: loads correctly
probe(32)  # 2I == H: gate_up_proj loads transposed, silently
probe(64)  # I == H:  down_proj  loads transposed, silently
```

### Expected behavior

Legacy-layout checkpoints convert correctly regardless of dims — or at minimum, the ambiguous
case fails loudly instead of silently loading transposed weights.

### Proposed fix (implemented and tested; happy to open a PR)

The pair `{gate_up_proj, down_proj}` can never be square in **both** members — `2I == H` makes
down `(H, I)` non-square, `I == H` makes gate_up `(2I, H)` non-square — so the module always
contains an unambiguous sibling that reveals the checkpoint's layout. Replacing the two
single-tensor converters with one paired converter (a small `ConversionOps` that collects both
tensors — same machinery as the Ernie expert fuse/split ops in `_INTERNAL_MANY_TO_MANY_CONVERSIONS`
— votes with the unambiguous members, and applies the decision to both) fixes all cases. Tested
against the 6-case matrix {legacy, target-layout} × {2I==H, I==H, non-square} on 5.12.1: stock
loader fails both square-legacy cases, patched loader passes all six. A defensive warning in
`Transpose.convert` when `tensor.shape[dim0] == tensor.shape[dim1]` under `check_dims=True` would
also protect future reuse of the flag.

(Found while auditing fused-expert 4-bit handling across the ecosystem — related layout-ambiguity
reports: unslothai/unsloth-zoo#849, bitsandbytes-foundation/bitsandbytes#1849 context.)


Comments:
Hmmn, it's unclear that this will ever trigger, given that it requires a new checkpoint that does not yet exist to be saved in an old format.

Agreed nothing on the Hub triggers it today — that's why I filed it as latent rather than live. But the trigger isn't "a new checkpoint deliberately saved in an old format": **the old format is the only input this code path exists to accept.**

```python
# conversion_mapping.py, "qwen3_vl_moe" — the entire entry
WeightConverter("mlp.experts.gate_up_proj", "mlp.experts.gate_up_proj",
                operations=[Transpose(1, 2, check_dims=True)]),
WeightConverter("mlp.experts.down_proj", "mlp.experts.down_proj",
                operations=[Transpose(1, 2, check_dims=True)]),
```

Source == target, so these two converters do nothing except ingest already-fused checkpoints stored in the pre-v5 orientation `(E, H, 2I)/(E, I, H)` — anything saved by transformers ≤4.x, or re-exported by tooling that preserves that layout. That input class exists on the Hub today; what doesn't exist yet is a **square-dim member of it**. And square dims are exactly where pruning/distillation lands (`moe_intermediate_size → hidden_size` or `hidden_size/2` is the natural symmetric target), and where any small test config lands trivially — the repro above hits it with `hidden_size=64, moe_intermediate_size=32`.

What makes it worth a guard even at low odds is the failure mode: no warning, no shape mismatch, finite logits — the model loads "successfully" and emits garbage. The minimal fix is two lines in `Transpose.convert`:

```python
if tensor.shape[self.dim0] == tensor.shape[self.dim1]:
    logger.warning_once("check_dims=True cannot infer transpose when the swapped dims are equal — layout is ambiguous.")
```

which also covers any future arch that reuses `check_dims=True`. The paired-converter fix from the issue makes the ambiguous case load *correctly* rather than loudly (the sibling can never be square at the same time, so it always disambiguates). Happy to open either as a PR — which shape would you take?