Cant run same int8 example on local machine that works in Colab
Author: auwsom
Pull request: no
Labels: name: bug

### System Info

This also looks like a bug:
(but consumer hardware Dell workstation with 64GBs RAM and RTX 3060 12GBs )
```
Traceback (most recent call last):
  File "/home/user/.local/bin/transformers-cli", line 5, in <module>
    from transformers.commands.transformers_cli import main
  File "/home/user/.local/lib/python3.10/site-packages/transformers/commands/transformers_cli.py", line 24, in <module>
    from .pt_to_tf import PTtoTFCommand
  File "/home/user/.local/lib/python3.10/site-packages/transformers/commands/pt_to_tf.py", line 21, in <module>
    from datasets import load_dataset
ModuleNotFoundError: No module named 'datasets'
```


### Who can help?

I dont see the Bloom model above, but Im trying to follow this example:
https://huggingface.co/docs/transformers/perf_infer_gpu_one
using the colab for "BLOOM-3B"
@sgugger, @patil-suraj

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi, I'm trying to run the colab: https://colab.research.google.com/drive/1qOjXfQIAULfKvZqwCen8-MoWKGdSatZ4?usp=sharing

Which runs fine using the "load_in_8bit=True" arg, but gives this error locally with any of the models. Ive tried downloading with huggingface_hub, git lfs clone and using normal cache (with the smaller model).

"TypeError: BloomForCausalLM.__init__() got an unexpected keyword argument 'load_in_8bit'"

Somehow AutoModelForCausalLM is passing off to BloomForCausalLM which is not finding load_in_8bit.. I'm going to try installing the latest transformers package..

```
#name = "/home/user/bloom-mnt/huggingface2/models--bigscience--bloom-7b1/snapshots/fdd9eac0805a9fa2d0641982eceda25885251975"
#name = "/home/user/bloom-mnt/huggingface2/models--bigscience--bloom-3b/snapshots/515ae965cc83b9ebbf0054de106c434bd4ec35dc"
name = "/home/user/bloom-mnt/huggingface2/bloom-560m"
#name = "bigscience/bloom-1b7"
text = "Hello my name is"
max_new_tokens = 20

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
device = "cuda:0" if torch.cuda.is_available() else "cpu"
#device = ("cpu")
model_8bit = AutoModelForCausalLM.from_pretrained(name, device_map="auto", load_in_8bit=True)
tokenizer = AutoTokenizer.from_pretrained(name)
model_8bit = model_8bit.to(device)

def generate_from_model(model, tokenizer):
  encoded_input = tokenizer(text, return_tensors='pt')
  output_sequences = model.generate(input_ids=encoded_input['input_ids'].to(device))
  return tokenizer.decode(output_sequences[0], skip_special_tokens=True)

print(generate_from_model(model_8bit, tokenizer))
```

### Expected behavior

success

Comments:
I solved this by using "pip install git+https://github.com/huggingface/transformers.git" for the latest transformers package. I installed the one locally in the last month, but things must change that fast in this space. I leaving the issue for others, but closing.

I'm getting `"AttributeError: /home/user/.local/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cget_col_row_stats"` now, and dont see any search results for this error.

If someone knows about this error above, that would be great. 

This error turned out to be from not having Cuda installed properly. PyTorch comes with its own many times, but bitsandbytes doesnt use it.

Hello,
I stumbled over the same error: "undefined symbol: cget_col_row_stats", but I have cuda installed (Stable Diffusion works) 
Can you tell me what you mean by "installing Cuda properly"?

@OWKenobi Well, on my Ubuntu 2204 system, with RTX3060, the highest Cuda available was 11.7, however the "compatible" Cuda version is 11.6 (look for cuda116 in the Torch package name). 11.6 however requires a slightly older driver 515.65.01. So I had to remove the one from `ubuntu-drivers autoinstall` and manually install 515. Then had to install Cuda 11.6. Hope that helps.

There might be one other AskUbuntu post about this:
https://askubuntu.com/questions/1392998/cuda-installation-uncomprehensible-conflicts