Spaces:

Duplicated from  Tonic/ClaireGPT_Chat

Tonic1

/

e5

like 14

Sleeping

Hello @Tonic , we wanted to let you know that we've assigned a GPU to your space, and your GPU grant application has been approved. Congratulations! Please keep in mind that GPU grants are provided on a temporary basis and may be removed if usage is very low.
To learn more about GPUs in Spaces, please check out https://huggingface.co/docs/hub/spaces-gpus. We look forward to seeing the innovative work you produce with this grant. If you have any questions or concerns, please let us know. Thank you for your interest in our platform!

I have done a factory rebuild of the space as well.

ah okay! thanks for this much obliged <3

It is failing with OOM, can you please try implementing quantization? I have assigned an A10Large in the meanwhile with a 1 hour sleep. We can change the GPU back to T4 once we can bring down the memory usage.

yes, done, with all my thanks 🙏🏻

according to this it should fit on a T4 ?

^^

My appologies with my growing pains here, it's my first time !

@ysharma got it to run on an A100 , sorry for the iterations : just in the middle of work + i hope i can catch you before the end of the work day ;-) 🚀

@ysharma : solution i'm thinking for : duplicate my A100 space with orca to another space and use this grant for a mistral model that i know can fit + i can use immediately :-) hope that's okay (if community grant cant cover an A100 which i do understand )

Hi @Tonic , sorry for the delay in responding to your messages. Assigning an A100 for a GPU grant can be a complex process. Have you considered using quantization to reduce the memory usage of your application? Additionally, we have another demo available for Orca-2 that you might find helpful. You can access it here - https://huggingface.co/spaces/ari9dam/Orca-2-13B.
For another demo you'd have to apply for another grant request stating the project ideas in the request.

everything is fine, simply quantized + fully understand, now we will build with this endpoint some cool apps starting with a gradio/discord bot and then move towards fine tuning + more cool demos :-)

is there any chance to extend the sleep time ? by the time i change every app that depends on this endpoint it's already asleep again :-)