64G内存跑不动
#6
by
kalifei
- opened
大概在载入到94%的内存时显示not enough memory: you tried to allocate 283115520 bytes。怎么办
from_pretrain中加入参数device_map="auto"试试呢，我的情况是可以解决的
改成这样model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", trust_remote_code=True, torch_dtype=torch.float16)
我32G内存确实跑不动，64G内存无鸭梨跑得动啊