Downloading llama model
Author: andysingal
Pull request: no

### System Info

colab pro

### Who can help?

@younesbelkada @pacman100 @arth

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
installed it manually:
!pip3 install -q langchain openai jsonlines
!pip3 install -q ipykernel jupyter datasets einops wandb
!pip install -q accelerate==0.21.0 peft==0.4.0 bitsandbytes==0.40.2  trl==0.4.7 transformers

import jsonlines
import itertools
import pandas as pd
from pprint import pprint

import torch

import datasets
from datasets import load_dataset
from huggingface_hub import notebook_login

# from llama import BasicModelRunner
from transformers import AutoTokenizer, AutoModelForCausalLM,BitsAndBytesConfig
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_name = "TintinMeimei/NousResearch-Llama-2-7b-chat-hf"
device_map = {"": 0}

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtyp=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map=device_map
)
# this should be set as False for finetuning
model.config.use_cache = False

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
```

ERROR:
```
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
[<ipython-input-6-b7b86b5aff18>](https://localhost:8080/#) in <cell line: 10>()
      8     bnb_4bit_compute_dtyp=torch.bfloat16,
      9 )
---> 10 model = AutoModelForCausalLM.from_pretrained(
     11     model_name,
     12     quantization_config=bnb_config,

1 frames
[/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
   2399         if load_in_8bit or load_in_4bit:
   2400             if not (is_accelerate_available() and is_bitsandbytes_available()):
-> 2401                 raise ImportError(
   2402                     "Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of"
   2403                     " bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or"

ImportError: Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or pip install bitsandbytes` 

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------
```

### Expected behavior

model needs to run?

Comments:
Could you share the issue that you are having? 

> 

just added the error above.

Did you try to install the packages you are missing ?  
```bash
pip install -u accelerate bitsandbytes
```

According to the contribution guidelines could you share the output of `transformers-cli env`? Would help us know on which hardware you are running this. Pretty sure `load_in_8bit` is not available on `MAC`.




> Did you try to install the packages you are missing ?
> 
> ```shell
> pip install -u accelerate bitsandbytes
> ```
> 
> According to the contribution guidelines could you share the output of `transformers-cli env`? Would help us know on which hardware you are running this. Pretty sure `load_in_8bit` is not available on `MAC`.

i did:
```
!pip install -q accelerate==0.21.0 peft==0.4.0 bitsandbytes==0.40.2  trl==0.4.7 transformers
```
doing without specific version uploads previous versions. I am using colab pro notebook

Did you restart the environment after installing the libraries to make sure they are used?
Could you share your notebook for reproducibility? 

Yes, three times . Let me share the code in another notebook since this one
has a lot of code.

On Thu, Aug 24, 2023 at 16:41 Arthur ***@***.***> wrote:

> Did you restart the environment after installing the libraries to make
> sure they are used?
> Could you share your notebook for reproducibility?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/25720#issuecomment-1691478832>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AE4LJNLCI7CSZUMERSTLSZTXW4ZFBANCNFSM6AAAAAA34XC5PA>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


> Did you restart the environment after installing the libraries to make sure they are used? Could you share your notebook for reproducibility?

Here is the colab notebook: https://colab.research.google.com/drive/1PGp9C7iGO7Lw9pUfwJ0RMsB0-J_xYBZn?usp=sharing 

Thanks, pinging @younesbelkada for him to have a look

Hmm looking at the notebook I don't see any reason to not work, does it works on a fresh new environment?

Yeah, already tried it but shows the shared error.

On Fri, Aug 25, 2023 at 11:14 Younes Belkada ***@***.***>
wrote:

> Hmm looking at the notebook I don't see any reason to not work, does it
> works on a fresh new environment?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/25720#issuecomment-1692793168>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AE4LJNL3BT2PMNTHVW6UF5DXXA3SZANCNFSM6AAAAAA34XC5PA>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


Hmm I have just copied the notebook you shared and ran it and it worked fine on my end . Can you maybe delete and restart runtime?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.