Question
What do you mean by no GPU needed? Are you saying that you can run the model on 64gb RAM with just CPU?
@rombodawg Yeah ok that makes more sense because only CPU for such a big model would be crazy...
yeah exactly, created it using only my CPU and RAM
What do you mean by no GPU needed? Are you saying that you can run the model on 64gb RAM with just CPU?
You need at least 1.6TB of ram/vram to run it
Basically you CAN run it on cpu if you have enough ram, but you either need something like nvidia EOS or alien tech to run it at comfortable speeds on cpu
Game
How many org members can we get into 1 community post lol
Basically you CAN run it on cpu if you have enough ram, but you either need something like nvidia EOS or alien tech to run it at comfortable speeds on cpu
Are Nvidia Eos not GPU's? Is alien tech a joke or is that actually a name for something?
EOS is a supercomputer with like 6000 h100s
"alien tech" I meant in the literal sense of computers from some extraterrestrial race that has achieved computers far faster than we have
Among many new records and milestones, one in generative AI stands out: NVIDIA Eos — an AI supercomputer powered by a whopping 10,752 NVIDIA H100 Tensor Core GPUs and NVIDIA Quantum-2 InfiniBand networking — completed a training benchmark based on a GPT-3 model with 175 billion parameters trained on one billion tokens in just 3.9 minutes.
You'd need uh $721,803,023.91 USD to create your own EOS
PS. You'd need ten to to match the fastest supercomputer in the world though #^#
Among many new records and milestones, one in generative AI stands out: NVIDIA Eos — an AI supercomputer powered by a whopping 10,752 NVIDIA H100 Tensor Core GPUs and NVIDIA Quantum-2 InfiniBand networking — completed a training benchmark based on a GPT-3 model with 175 billion parameters trained on one billion tokens in just 3.9 minutes.
You'd need uh $721,803,023.91 USD to create your own EOS
PS. You'd need ten to to match the fastest supercomputer in the world though #^#
what about 115x the strength of EOS?
thats gonna make me insane
Going off consumer cost that is roughly 1.4 million H100's
100,000,000,000 / 721,803,023 = 138.5
138.5 x 10752 = 1,489,152
Power usage?
1,489,152 x 350w (H100 tdp) = 521,203,200 Watts of tpd in the gpus alone
VRAM?
119.132 Petabytes
119.132 Petabytes of VRAM...wow lol
I'm just wondering if there will be smaller llama 3 versions, all the recent releases have been getting bigger and bigger. My 8gbs of vram is screaming for help.
But companies see
More parameter + More benchmark points
= More investors
Among many new records and milestones, one in generative AI stands out: NVIDIA Eos — an AI supercomputer powered by a whopping 10,752 NVIDIA H100 Tensor Core GPUs and NVIDIA Quantum-2 InfiniBand networking — completed a training benchmark based on a GPT-3 model with 175 billion parameters trained on one billion tokens in just 3.9 minutes.
You'd need uh $721,803,023.91 USD to create your own EOS
PS. You'd need ten to to match the fastest supercomputer in the world though #^#
Hold on let me check my coat pockets
Will a few strands of cat hair and dryer lint suffice?
Among many new records and milestones, one in generative AI stands out: NVIDIA Eos — an AI supercomputer powered by a whopping 10,752 NVIDIA H100 Tensor Core GPUs and NVIDIA Quantum-2 InfiniBand networking — completed a training benchmark based on a GPT-3 model with 175 billion parameters trained on one billion tokens in just 3.9 minutes.
You'd need uh $721,803,023.91 USD to create your own EOS
PS. You'd need ten to to match the fastest supercomputer in the world though #^#Hold on let me check my coat pockets
Will a few strands of cat hair and dryer lint suffice?
We can tie it together and compress it until it becomes so dense that it starts creating energy which we can use for our machines
I'm just wondering if there will be smaller llama 3 versions, all the recent releases have been getting bigger and bigger. My 8gbs of vram is screaming for help.
But companies see
More parameter + More benchmark points
= More investors
microsoft with phi-2 tho :)
I think the companies will be tired of spending millions of dollars every time they want a model...I think the most important thing that will change in the next ten years is how we process and transfer information.
has anybody run inference with this model?
has anybody run inference with this model?
Honestly, this model is just an experiment to see what I could do locally with passthrough. I had a script to interleave every single layer of Phi-2-Super in order to double its size while retaining 98.6% performance...so I copy pasted it 115 times to see if my pc would break...it didn't...it just took 3 days to merge it and 4 days to upload it lol.
I don't know anyone who would be capable of inferencing with this model to be honest. No one in our team has done it.