Spaces:

facebook

/

incoder-demo

like 104

Runtime error

Sorry, but we can't merge this because with this max token limit it's possible for very large queries to to cause the GPU to OOM and the demo to crash.

As an alternative, I'd recommend cloning the repo locally and setting the MAX_TOKENS value to something that your GPU can handle, or running on CPU.

What are minimal requirements for GPU and CPU?

And where is link for cloning?

Sorry I just saw this. I'm not totally sure on the minimal requirements, but I'd estimate you'd need 12-20GB of memory on GPU to use the 6B model (in its fp16 version), ~24GB RAM to use the 6B model in its fp32 version on CPU only (as fp16 can't run on CPU), ~5GB on GPU to use the 1B (in fp16), and ~10GB RAM the 1B on CPU-only (in fp32).

The clone command is git clone https://huggingface.co/spaces/facebook/incoder-demo