When it says 400GB of memory, does it mean RAM or GPU memory?
#32
by
rahulsanjay18
- opened
trying to figure out how i can train this on AWS, and what instance I should choose, and the model card says the following:
"You will need at least 400GB of memory to swiftly run inference with Falcon-180B."
I'm going to assume this is just normal RAM but I wanted to make sure
ram ... also probably it's grossly under reported . you're looking at much larger requirements based on my experiencee with the previous version