Process finished with exit code -1073741819 (0xC0000005) on Windows 10 Enterprise and Python 3.11
I'm on Windows 10 Enterprise (64 bit - Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz) and got the Mistral-7B-Instruct-v0.2.
The running code is the following:
**
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" if torch.cuda.is_available() else "cpu"
model = AutoModelForCausalLM.from_pretrained("../Mistral-7B-Instruct-v0.2")
tokenizer = AutoTokenizer.from_pretrained("../Mistral-7B-Instruct-v0.2")
**
Every time I run the code I get the error:
Process finished with exit code -1073741819 (0xC0000005)
Do you have any suggestions?
Maybe try this:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "../Mistral-7B-Instruct-v0.2"
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="cpu")
tokenizer = AutoTokenizer.from_pretrained(model_id)
text = "<s>[INST] What is your favourite condiment? [/INST]"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
I got the same error and what fixed it was specifying the property device_map="cpu" when calling from_pretrained. I also get the same error when I run out of RAM while loading the model as on my machine it consumes about 29 GB of memory. If this code doesn't fix it, check the RAM consumption while it tries to load the model to see if it is that.
Yes, true, it's a memory problem. I have 16 GB of RAM and it's not enough on a Windows machine.
So the model will be entirely loaded in memory....
thx