Server Requirements for 13B model
Hello Folks, sorry to bother anyone with a simple question that I have already pretty much verified with my research, but I have to ask before I pay for a dedicated Server for my project.
Is this adequate to train OLMo-2 13B?
-1 x EPYC 7402 (24 cores / 48 threads)- 192GB DDR4 ECC- 1 x 2TB NVMe SSD- 3 x 3090- 10TB @ 10Gbps
Is it better of I balance the 13B model over the 3 GPUs?
Is there a trainers guide available to the public? And finally are you folks starting a discord channel? Tysm
Hello
@Notqty
, The 3 x 3090 GPUs with their combined 72GB VRAM should handle the model well, though you might need to adjust batch sizes or use gradient accumulation to fit larger sequences. You need to use DDP to balance on 3 GPUs.
You can find training/fine-tuning scripts here: https://github.com/allenai/OLMo
Tysm!!! Can't wait to get the server and OLMo2 setup. Thanks again!