mistralai

/

Mistral-7B-v0.1

like 3.29k

I made it work with ooga booga.
model loader "transformers" and "load in 4 bits".
it take ages to load and answer not that quickly but it work.

I made it work with ooga booga.
model loader "transformers" and "load in 4 bits".
it take ages to load and answer not that quickly but it work.

How did you manage to download the model?

I'm using Ubuntu and when I execute the three lines below, I get the error at the bottom:

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")

Downloading (…)okenizer_config.json: 100%|██████████████████████████████████████| 966/966 [00:00<00:00, 4.40MB/s]
Downloading tokenizer.model: 100%|████████████████████████████████████████████| 493k/493k [00:00<00:00, 18.9MB/s]
Downloading (…)/main/tokenizer.json: 100%|██████████████████████████████████| 1.80M/1.80M [00:00<00:00, 15.7MB/s]
Downloading (…)cial_tokens_map.json: 100%|█████████████████████████████████████| 72.0/72.0 [00:00<00:00, 389kB/s]
Downloading (…)lve/main/config.json: 100%|██████████████████████████████████████| 571/571 [00:00<00:00, 2.84MB/s]
Traceback (most recent call last):
File "/home/j/jr_loadmodel", line 39, in
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")
File "/home/j/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 444, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "/home/j/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 940, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "/home/j/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 655, in getitem
raise KeyError(key)
KeyError: 'mistral'

I found the answer:

1. Re-install transformers
2. pip install accelerate==0.20.3

That worked for me

Hi. Could you please make your jupyter available for us to see?
I am trying to run this code in google colab, and it says that the ram is not enough :(. I dont know what to do now.