KeyError: 'mistral'
Author: andysingal
Pull request: no

### System Info

RTX 3090

### Who can help?

@younesbelkada @Arthur

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
import torch
from transformers import BitsAndBytesConfig
from langchain import HuggingFacePipeline
from langchain import PromptTemplate, LLMChain


quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)
model_id = "ehartford/samantha-mistral-instruct-7b"

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
model_4bit = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto",quantization_config=quantization_config)
tokenizer = AutoTokenizer.from_pretrained(model_id)

pipeline = pipeline(
        "text-generation",
        model=model_4bit,
        tokenizer=tokenizer,
        use_cache=True,
        device_map="auto",
        max_length=500,
        do_sample=True,
        top_k=5,
        num_return_sequences=1,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.eos_token_id,
)

llm = HuggingFacePipeline(pipeline=pipeline)
```
gives error:
```
(…)ral-instruct-7b/resolve/main/config.json: 100%
628/628 [00:00<00:00, 58.6kB/s]
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[<ipython-input-13-e2c6cff1d0cf>](https://localhost:8080/#) in <cell line: 16>()
     14 
     15 from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
---> 16 model_4bit = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto",quantization_config=quantization_config)
     17 tokenizer = AutoTokenizer.from_pretrained(model_id)
     18 

2 frames
[/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py](https://localhost:8080/#) in __getitem__(self, key)
    732 
    733 
--> 734 class _LazyConfigMapping(OrderedDict):
    735     """
    736     A dictionary that lazily load its values when they are requested.
```

### Expected behavior

runs the model

Comments:
Hi @andysingal 
Thanks for the issue, can you share with us which transformers version are you using?
Can you try to use the latest transformers version?

```bash
pip install -U transformers
```

I had the same error.
`pip install -U transformers` 
solved it. thanks


Thanks alot, I unchecked high ram and it worked. I already had the latest
version of Transformers. Thanks

On Thu, Oct 12, 2023 at 7:40 PM Younes Belkada ***@***.***>
wrote:

> Hi @andysingal <https://github.com/andysingal>
> Thanks for the issue, can you share with us which transformers version are
> you using?
> Can you try to use the latest transformers version?
>
> pip install -U transformers
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/26759#issuecomment-1759686346>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AE4LJNKRP3ZM743DBF6FJWTX6725HAVCNFSM6AAAAAA55PDUPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJZGY4DMMZUGY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


Hi team,

If i try transformer version = 4.35, then i get error of 

ImportError: Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or pip install bitsandbytes` 

But if i downgrade to 4:30
then i get
key error  :mistral

Can anyone help me?

@Abhaycnvrg with transformers 4.35 are you running with the latest version of bitsandbytes and accelerate installed? 

yes. i did not give any version number (so it took the latest version of the bitsandbytes with python 3.11.5 and 3.11.1 )


@Abhaycnvrg Could you open a new issue, and provide full details of the error (full traceback) and your running environment (run `transformers-cli env` in the terminal and copy-paste the output)? 

in the other thread, jitender-cnvrg already gave

> Hi team,
> 
> If i try transformer version = 4.35, then i get error of
> 
> ImportError: Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or pip install bitsandbytes`
> 
> But if i downgrade to 4:30 then i get key error :mistral
> 
> Can anyone help me?

I have exactly the same issue, whether I use CPU or cuda.

I can confirm that I am seeing the same issue as mentioned above, downgraded to Transformers 4.30.0 and get the KeyError : mistral 

or the ImportError mentioned above with Transformers 4.35.1

Hey all. Just saying that you have the same issue without a reproducer and a traceback will not help anyone. 
Feel free to open a new issue with a reproducer that does not use an external package (in this case langchain) if the issue is to load something with `transformers`. If the issue is with `langchain` than open an issue on the `langchain` repo 🤗 

[Abhaycnvrg](https://github.com/Abhaycnvrg)  [smarthi](https://github.com/smarthi) those errors you are facing:
Transformers 4.30.0 --> KeyError : mistral
Transformers 4.35.2 --> ImportError: Using load_in_8bit=True requires Accelerate: pip install accelerate
 only happen if you execute your program on CPU, they don't appear once you run your program on GPU.
 
 That's how i got to solve it.
 Cheers.


> [Abhaycnvrg](https://github.com/Abhaycnvrg) [smarthi](https://github.com/smarthi) those errors you are facing: Transformers 4.30.0 --> KeyError : mistral Transformers 4.35.2 --> ImportError: Using load_in_8bit=True requires Accelerate: pip install accelerate only happen if you execute your program on CPU, they don't appear once you run your program on GPU.
> 
> That's how i got to solve it. Cheers.

I am getting both these errors for the different transformers version you mention. I'm trying to run `autotrain-advanced` locally on my Apple Silicon Macbook. Anyone knows a fix for that?

Hi team
I got to solve it by using a very specific huggingface image.
The details are in this thread
https://github.com/huggingface/transformers/issues/27376