Repository: transformers
Issue: AttributeError: 'MERTConfig' object has no attribute 'conv_pos_batch_norm'
Labels: bug

### System Info

```
- `transformers` version: 4.48.0
- Platform: Linux-6.8.0-49-generic-x86_64-with-glibc2.35
- Python version: 3.12.5
- Huggingface_hub version: 0.27.1
- Safetensors version: 0.5.2
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes (but makes no difference on cpu)
- GPU type: NVIDIA GeForce RTX 3090
```

### Who can help?

Hello everyone, since updating from `4.47.1` to version `4.48.0` we are unable to load `MERT-v1-330M` on the latest revision.
This works perfectly fine on the previous version.

I haven't seen any mention of a breaking change regarding this, have we missed something or is this a legitimate bug?

```
config.model = tr.AutoModel.from_pretrained(config.model_id, trust_remote_code=True).to(device)
.venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py:559: in from_pretrained
    return model_class.from_pretrained(
.venv/lib/python3.12/site-packages/transformers/modeling_utils.py:4090: in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
../../.cache/huggingface/modules/transformers_modules/m-a-p/MERT-v1-330M/0e06c986db0527c0fd1b47181c40f006805e3de0/modeling_MERT.py:92: in __init__
    self.encoder = HubertEncoderStableLayerNorm(config)
.venv/lib/python3.12/site-packages/transformers/models/hubert/modeling_hubert.py:1023: in __init__
    self.pos_conv_embed = HubertPositionalConvEmbedding(config)
.venv/lib/python3.12/site-packages/transformers/models/hubert/modeling_hubert.py:275: in __init__
    if config.conv_pos_batch_norm:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = MERTConfig {
  "_attn_implementation_autoset": true,
  "_name_or_path": "m-a-p/MERT-v1-330M",
  "activation_dropout": ... "torch_dtype": "float32",
  "transformers_version": "4.48.0",
  "use_weighted_layer_sum": false,
  "vocab_size": 32
}
, key = 'conv_pos_batch_norm'

    def __getattribute__(self, key):
        if key != "attribute_map" and key in super().__getattribute__("attribute_map"):
            key = super().__getattribute__("attribute_map")[key]
>       return super().__getattribute__(key)
E       AttributeError: 'MERTConfig' object has no attribute 'conv_pos_batch_norm'

.venv/lib/python3.12/site-packages/transformers/configuration_utils.py:211: AttributeError
```

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. Import transformers (version 4.48.0) and load mert
```
import transformers as tr

m = tr.AutoModel.from_pretrained("m-a-p/MERT-v1-330M", trust_remote_code=True).to("cuda")

```

### Expected behavior

I should be able to load the model