Is it possible to deploy this as a model?

by mitchellhynes - opened

Thank you for the space! It's been great to try this since it wasn't working on my M1 Mac.

I need to run this as an endpoint. I gave Sagemaker a try but it's taking hours to deploy just to error and I'm fed up. HF is nice, I'd love to use their inference solution but it requires a model not a space. Models are vastly different from spaces in ways that are unclear, is it possible?

I'd be happy to donate some compute $$$ for this space if you'd point me in a direction for deploying this! And you'd have my endless gratitude ๐Ÿ˜Š

If i understood correctly, to deploy this space you just neet to git clone, then pip install -r requirements.txt, then just execute as python script, its not hard! The models is will be downloaded and stored in model cache folder that will be placed in folder of webui. This space also can run on cpu, but in strange way: it works, but uses only ~9% of free computations that makes it extremely slow, Idk why.

Thanks for the reply. Actually I meant as a Huggingface model ( I've tried downloading and serving the Gradio server on multiple machines now including a ml.t3.medium, but I haven't gotten anything to complete. It takes hours on a CPU.

Huggingface has inference endpoints I want to use for this reason. I could run it on my own computer, but that wouldn't work too well.

still misunderstanding a little, but ok

Hi @mitchellhynes Hmm, I'm not sure. Actually, I don't know much about inference endpoints, so I think you should ask it in the forum. Probably there is someone there with more knowledge about the inference endpoints.

@mitchellhynes do you mean something like this?

Where you can do curl -X POST -H 'Content-type: application/json' --data '{ "data": ["Jill"] }'

@mitchellhynes You can use gradio_client library to call functions in Spaces.

still misunderstanding a little, but ok

Thanks! - though it looks like the inference API doesn't work with that for some reason :/

You can use gradio_client library to call functions in Spaces

I'll dig up my old Gaming PC with a 650TI in it and see if I can get a Gradio server running on that, then use the client :)

It's not as scalable as a serverless endpoint like the inference API but it works for now, I'll try using a LambdaLabs instance when it catches on fire.

Thanks for the help everyone

mitchellhynes changed discussion status to closed

@mitchellhynes I would also recommend, which is a bit cheaper. I managed to run on an A10 with no problems at all. I wrote up how here.

I'm probably not going to keep this up for long, as it's quite expensive, but if you want to play around with Shap-E as an API, you can use[your text here].ply but replace [your text here] with the prompt. It caches all models, so if a prompt has been requested before, the model is just downloaded immediately, but if not, it's generated on the fly which takes a few seconds. E.g. bowl of vegetables.ply is definitely an option! I will consider it if my dusty 650TI is still too slow. An A10 on Vast is still $230 a month which is prohibitively expensive. Maybe it's necessary though

I appreciate the code! I've used before for my other project and it works really well.

Very cool! I can see how Shap-E could augment that ("put an avocado chair on the table")

Just a heads up, the original repo doesn't have the extra convenience things that hysts added, e.g. the conversion to GLB, and the fact that all the models come out rotated 90 degrees. Code for that here.

Sign up or log in to comment