Repository: huggingface/transformers
Pull request: Remove residual quantization attribute from dequantized models

# What does this PR do?
The `quantization_method` attribute was not being deleted during model dequantization, causing methods like `to()` to fail even when the operation should be valid for dequantized models.

Fixes # 39295


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
https://github.com/huggingface/transformers/issues/39295
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@SunMarc 


Changed files:
filename: src/transformers/quantizers/base.py
patch: @@ -248,6 +248,7 @@ def dequantize(self, model):
         del model.hf_quantizer
         del model.config.quantization_config
         del model.config._pre_quantization_dtype
+        del model.quantization_method
         model.is_quantized = False
 
         return model

filename: tests/quantization/bnb/test_4bit.py
patch: @@ -271,6 +271,33 @@ def test_generate_quality_dequantize(self):
 
         self.assertIn(self.tokenizer.decode(output_sequences[0], skip_special_tokens=True), self.EXPECTED_OUTPUTS)
 
+    def test_clear_quantization_trace(self):
+        r"""
+        Test that dequantizing the model won't leave any attribute relative to quantization in the model's configuration
+        """
+        bnb_config = BitsAndBytesConfig(load_in_4bit=True)
+        model_4bit = AutoModelForCausalLM.from_pretrained(
+            self.model_name, quantization_config=bnb_config, device_map="auto"
+        )
+        model_4bit.dequantize()
+
+        self.assertFalse(hasattr(model_4bit, "hf_quantizer"))
+        self.assertFalse(hasattr(model_4bit.config, "quantization_config"))
+        self.assertFalse(hasattr(model_4bit.config, "_pre_quantization_dtype"))
+        self.assertFalse(hasattr(model_4bit, "quantization_method"))
+        self.assertFalse(model_4bit.is_quantized)
+
+    def test_to_device_dequantized(self):
+        r"""
+        Test that dequantizing the model won't prevent converting it to a different dtype
+        """
+        bnb_config = BitsAndBytesConfig(load_in_4bit=True)
+        model_4bit = AutoModelForCausalLM.from_pretrained(
+            self.model_name, quantization_config=bnb_config, device_map="auto"
+        )
+        model_4bit.dequantize()
+        model_4bit.to(dtype=torch.float16)
+
     def test_device_assignment(self):
         if version.parse(importlib.metadata.version("bitsandbytes")) < version.parse("0.43.2"):
             self.skipTest(reason="This test requires bitsandbytes >= 0.43.2")