Can I run this model on two NVIDIA RTX A5000 GPUs with 24 GB each?
#4
by
nashid
- opened
Can I run this model on two NVIDIA RTX A5000 GPUs with 24 GB each?
Yes you can easily run it
@wizmak Hi wizmak would you be able to tell the minimum graphic requirement to run this version of model CodeLlama-34B-Instruct-GPTQgptq-4bit-32g-actorder_True size 20gb
24 GB GPU should be enoguh @FalconLlamalpaca