AttributeError: 'BitsAndBytesConfig' object has no attribute 'get_loading_attributes' with transformers 4.55.0
Author: yukiharada1228
Pull request: no
Labels: name: bug

### System Info

transformers version: 4.55.0
platform: Linux-5.15.0-139-generic-x86_64-with-glibc2.31
python version: 3.10.13
PyTorch version: 2.1.0a0+32f93b1
TensorFlow version: N/A
Flax version: N/A
JAX version: N/A
JAXLib version: N/A
Using GPU in script?: Yes
Using distributed or parallel set-up in script?: No

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    "openai/gpt-oss-20b",
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
)

### Expected behavior

Steps to reproduce the behavior:
1. Install transformers 4.55.0 and bitsandbytes 0.46.1
2. Run the above code snippet
3. AttributeError occurs during model loading

Error message:
Traceback (most recent call last):
  File "/workspace/script/sft_train.py", line 173, in <module>
    main()
  File "/workspace/script/sft_train.py", line 101, in main
    model = AutoModelForCausalLM.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 600, in from_pretrained
    return model_class.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 316, in _wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 4865, in from_pretrained
    config.quantization_config = AutoHfQuantizer.merge_quantization_configs(
  File "/usr/local/lib/python3.10/dist-packages/transformers/quantizers/auto.py", line 224, in merge_quantization_configs
    loading_attr_dict = quantization_config_from_args.get_loading_attributes()
AttributeError: 'BitsAndBytesConfig' object has no attribute 'get_loading_attributes'

Comments:
did you find a fix

Hi @danii,

The error you're encountering (`AttributeError: 'BitsAndBytesConfig' object has no attribute 'get_loading_attributes'`) is likely related to recent changes in the way the `transformers` library handles quantization configurations. Specifically, the `BitsAndBytesConfig` class doesn't have a `get_loading_attributes` method, which seems to be expected by the `AutoModelForCausalLM.from_pretrained()` method.

This is happening because, in more recent versions of `transformers`, the quantization configuration has shifted from using `BitsAndBytesConfig` to `Mxfp4Config` for certain quantization workflows, particularly when using 4-bit quantization with the `bnb` library. This change can cause compatibility issues when trying to load models that were configured using the older approach.

Here are a few potential solutions:

1. **Use `Mxfp4Config` instead of `BitsAndBytesConfig`**: Since `Mxfp4Config` is now the preferred configuration for some of these quantization operations, try replacing `BitsAndBytesConfig` with `Mxfp4Config`. This should fix the issue if the quantization approach has shifted in newer versions.

2. **Workaround (Custom Configuration)**: If you're stuck with `transformers 4.55.0` and can't update, one workaround would be to manually adjust your quantization code or extend the `BitsAndBytesConfig` to include the missing method (`get_loading_attributes`). But this would require delving into the `transformers` source code and understanding how the configuration is being merged.

cc @sunmarc @mekkcyber just in case we do have a compatibility break with recent versions

Hey @yukiharada1228, you are trying to quantize the model using bnb, but the model is already quantized using mxfp4, I opened a pr to raise a more comprehensive error in that case here : https://github.com/huggingface/transformers/pull/39998
In your case I would suggest dequantizing the model first, using 
```python
quantization_config = Mxfp4Config(dequantize=True)
from transformers import GptOssForCausalLM
model = GptOssForCausalLM.from_pretrained(model_name,
                                                quantization_config=quantization_config,
                                                torch_dtype="auto",
                                                device_map="auto"
)
```
then saving it, locally or on the hub, and then using your dequantized version with bnb quantization

Thanks for the detailed explanation, @MekkCyber
Also, really appreciate the PR to improve the error messaging in this case (#39998) — very helpful!

As you pointed out, the model I was trying to quantize with bnb was already quantized using mxfp4, which explains the issue.
I'll follow your suggestion: first dequantize the model using Mxfp4Config(dequantize=True), save it, and then re-quantize it using BitsAndBytesConfig.

Just to double-check — if I load the model with:

quantization_config = Mxfp4Config(dequantize=False)

that means the model stays in its original mxfp4 quantized state, right?

Thanks again for your help!

cc @SunMarc @MekkCyber

Yes exactly loading with quantization_config = Mxfp4Config(dequantize=False) keeps the model quantized, which is the default behaviour, you don't need to (but you can) specify a quantization_config if you want to keep the model quantized

Got it — thanks for confirming!

Since it's clear that the model remains quantized by default, I'll go ahead and explicitly set quantization_config = Mxfp4Config(dequantize=False) just to be safe.

If there’s nothing else to address, is it okay to go ahead and close this issue?

Yes thanks for reporting the issue @yukiharada1228 🤗 , we can go ahead a close this issue