Text Generation
Transformers
PyTorch
English
llama
Inference Endpoints
text-generation-inference

Hardware Requirements?

#10
by cameronraygun - opened

Can this be ran with 12gb of VRAM or am I gonna need a lot more?

Can this be ran with 12gb of VRAM or am I gonna need a lot more?

a lot more, even at 2 bit

Will there be a version that can be run in under 24GB consumer level hardware?

The model stabilityai/FreeWilly2 is too large to be loaded automatically (275GB > 10GB). For commercial use please use PRO spaces (https://huggingface.co/spaces) or Inference Endpoints (https://huggingface.co/inference-endpoints).

Will there be a version that can be run in under 24GB consumer level hardware?

maybe at 1bit

Sign up or log in to comment