Text Generation
Transformers
Safetensors
falcon
text-generation-inference

When it says 400GB of memory, does it mean RAM or GPU memory?

#32
by rahulsanjay18 - opened

trying to figure out how i can train this on AWS, and what instance I should choose, and the model card says the following:
"You will need at least 400GB of memory to swiftly run inference with Falcon-180B."

I'm going to assume this is just normal RAM but I wanted to make sure

ram ... also probably it's grossly under reported . you're looking at much larger requirements based on my experiencee with the previous version

Sign up or log in to comment