Text Generation
Transformers
PyTorch
English
mixtral
conversational
Inference Endpoints
text-generation-inference

Multiple GPUs

#19
by jamesw767 - opened

Hello

The original dolphin model loaded fine in Transformers with multiple GPUs leading to a substantial performance game. Unfortunately I can't seem to get this model to load even if I set the memory GPU allocation correct. Do you have any recommendations

jamesw767 changed discussion title from Multiple GPS to Multiple GPUs

Sign up or log in to comment