RuntimeError: Failed to import transformers.integrations.bitsandbytes because of the following error (look up to see its traceback): cannot import name 'get_keys_to_not_convert' from 'transformers.integrations'
Author: fsaudm
Pull request: no
Labels: name: bug

# System Info

Package                  Version
------------------------ ----------
accelerate               0.33.0
bitsandbytes             0.43.3
transformers             4.44.0


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

# Reproduction

1. Run the code:


### Libraries
import os
from dotenv import load_dotenv
import time
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig



load_dotenv()


os.environ['HF_TOKEN'] = os.getenv('HF_TOKEN')
os.environ['HF_HOME'] = '.'
hf_home = os.environ['HF_HOME'] 


model_id = "meta-llama/Meta-Llama-3.1-70B-Instruct"
device = "cuda"
hf_home = os.environ['HF_HOME']



### Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    model_id, 
    cache_dir=hf_home)

### Load model
quantization_config = BitsAndBytesConfig(load_in_8bit=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    cache_dir=hf_home,
    device_map="auto",
    quantization_config=quantization_config,
    )
    
 2. **ERROR**:
 ---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/utils/import_utils.py:1603, in _LazyModule._get_module(self, module_name)
   1602 try:
-> 1603     return importlib.import_module("." + module_name, self.__name__)
   1604 except Exception as e:

File ~/.conda/envs/LLMs/lib/python3.12/importlib/__init__.py:90, in import_module(name, package)
     89         level += 1
---> 90 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1387, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1360, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1331, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:935, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:995, in exec_module(self, module)

File <frozen importlib._bootstrap>:488, in _call_with_frames_removed(f, *args, **kwds)

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/integrations/bitsandbytes.py:13
     12 if is_bitsandbytes_available():
---> 13     import bitsandbytes as bnb
     14     import torch

File ~/LLMs/bitsandbytes.py:30
     29 ## Quantized model
---> 30 model = AutoModelForCausalLM.from_pretrained(
     31     model_id, 
     32     cache_dir=hf_home,
     33     device_map="auto",
     34     quantization_config=quantization_config,
     35     )
     36 #model = model.to(device)
     37 
     38 
     39 # Push model to hub

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py:564, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    563     model_class = _get_model_class(config, cls._model_mapping)
--> 564     return model_class.from_pretrained(
    565         pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    566     )
    567 raise ValueError(
    568     f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
    569     f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
    570 )

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/modeling_utils.py:3827, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
   3826 if hf_quantizer is not None:
-> 3827     hf_quantizer.preprocess_model(
   3828         model=model, device_map=device_map, keep_in_fp32_modules=keep_in_fp32_modules
   3829     )
   3831     # We store the original dtype for quantized models as we cannot easily retrieve it
   3832     # once the weights have been quantized
   3833     # Note that once you have loaded a quantized model, you can't change its dtype so this will
   3834     # remain a single source of truth

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/quantizers/base.py:182, in HfQuantizer.preprocess_model(self, model, **kwargs)
    181 model.quantization_method = self.quantization_config.quant_method
--> 182 return self._process_model_before_weight_loading(model, **kwargs)

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/quantizers/quantizer_bnb_8bit.py:232, in Bnb8BitHfQuantizer._process_model_before_weight_loading(self, model, device_map, keep_in_fp32_modules, **kwargs)
    225 def _process_model_before_weight_loading(
    226     self,
    227     model: "PreTrainedModel",
   (...)
    230     **kwargs,
    231 ):
--> 232     from ..integrations import get_keys_to_not_convert, replace_with_bnb_linear
    234     load_in_8bit_fp32_cpu_offload = self.quantization_config.llm_int8_enable_fp32_cpu_offload

ImportError: cannot import name 'get_keys_to_not_convert' from 'transformers.integrations' (/u/gfs3/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/integrations/__init__.py)

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[4], line 1
----> 1 model = AutoModelForCausalLM.from_pretrained(
      2     model_id, 
      3     device_map="auto",
      4     quantization_config=quantization_config,
      5     )

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py:564, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    562 elif type(config) in cls._model_mapping.keys():
    563     model_class = _get_model_class(config, cls._model_mapping)
--> 564     return model_class.from_pretrained(
    565         pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    566     )
    567 raise ValueError(
    568     f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
    569     f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
    570 )

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/modeling_utils.py:3827, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
   3824     keep_in_fp32_modules = []
   3826 if hf_quantizer is not None:
-> 3827     hf_quantizer.preprocess_model(
   3828         model=model, device_map=device_map, keep_in_fp32_modules=keep_in_fp32_modules
   3829     )
   3831     # We store the original dtype for quantized models as we cannot easily retrieve it
   3832     # once the weights have been quantized
   3833     # Note that once you have loaded a quantized model, you can't change its dtype so this will
   3834     # remain a single source of truth
   3835     config._pre_quantization_dtype = torch_dtype

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/quantizers/base.py:182, in HfQuantizer.preprocess_model(self, model, **kwargs)
    180 model.is_quantized = True
    181 model.quantization_method = self.quantization_config.quant_method
--> 182 return self._process_model_before_weight_loading(model, **kwargs)

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/quantizers/quantizer_bnb_8bit.py:232, in Bnb8BitHfQuantizer._process_model_before_weight_loading(self, model, device_map, keep_in_fp32_modules, **kwargs)
    225 def _process_model_before_weight_loading(
    226     self,
    227     model: "PreTrainedModel",
   (...)
    230     **kwargs,
    231 ):
--> 232     from ..integrations import get_keys_to_not_convert, replace_with_bnb_linear
    234     load_in_8bit_fp32_cpu_offload = self.quantization_config.llm_int8_enable_fp32_cpu_offload
    236     # We keep some modules such as the lm_head in their original dtype for numerical stability reasons

File <frozen importlib._bootstrap>:1412, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/utils/import_utils.py:1593, in _LazyModule.__getattr__(self, name)
   1591     value = self._get_module(name)
   1592 elif name in self._class_to_module.keys():
-> 1593     module = self._get_module(self._class_to_module[name])
   1594     value = getattr(module, name)
   1595 else:

File ~/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/utils/import_utils.py:1605, in _LazyModule._get_module(self, module_name)
   1603     return importlib.import_module("." + module_name, self.__name__)
   1604 except Exception as e:
-> 1605     raise RuntimeError(
   1606         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1607         f" traceback):\n{e}"
   1608     ) from e

RuntimeError: Failed to import transformers.integrations.bitsandbytes because of the following error (look up to see its traceback):
cannot import name 'get_keys_to_not_convert' from 'transformers.integrations' (/u/gfs3/.conda/envs/LLMs/lib/python3.12/site-packages/transformers/integrations/__init__.py)


### Expected behavior

The model should start to be quantized. This was running a couple of hours ago, but when reproducing, it was not working anymore...

Comments:
Fixed!! After breaking my head over this, I figured out that i had a:
- **File Name** Conflict. If you have a file named `bitsandbytes.py` in your working directory (~/LLMs/bitsandbytes.py), it might be interfering with the actual bitsandbytes package from transformers.

- Changed it to `quantizing_llama70.py`


![image](https://github.com/user-attachments/assets/4b54ccde-d67b-4e48-a119-c0a228180ee7)

![image](https://github.com/user-attachments/assets/0fa7c216-f2a3-4ba5-add2-f431f21d4a98)