Working code with full server requirements

#24

by gmjolt - opened May 31, 2023

May 31, 2023

Is there someone who can provide a working code with server instance and gpu requirements needed to run the model smoothly on aws/azure/google?
Thanks

tinkertank

May 31, 2023

See here: https://huggingface.co/tiiuae/falcon-40b/discussions/18

Contains code & requirements, and runs on any A100 80G instance.
(I'm personally using Datacrunch.io spot instances, but any A100 80G instance should do)

FalconLLM

Technology Innovation Institute org Jun 9, 2023

You could have a look to this blogpost from HuggingFace.

Specific to AWS, there is also a tutorial for deploying on SageMaker.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment