llava-hf

/

llava-v1.6-mistral-7b-hf

like 134

Hi, thanks for your contribution to open-souce LLM community. I recently try llava-v1.6-mistral-7b using newest transformers(4.39), everything go right if I load model to single gpu, a.k.a device_map={'':0}, but if I load the model dispatch to multi-gpu, a.k.a device_map='auto', the inference will raise cuda errors. It may looks like this issue: https://github.com/haotian-liu/LLaVA/issues/1319#issuecomment-2017359760

I want to make sure that it's this model support load to multi gpu?