What is the minimum amount of memory required to run
Parameter size in bfloat16 is about 350GB. You need about 360GB to run generation. There's also a quantified version here
· Sign up or log in to comment
