How to find the right cloud ? Help
I am trying to run the model in a few seconds for getting an answer. My difficulty is to provision a GPU fast enough. I have tried on OVH, Google cloud, Amazon and Azure, spend countless hours on it, trying to provision A10s, A100 sor V100s. Either I cannot get approved for enough GPUs (Amazon , Google) or their are not available to be provisioned (A100 and A10 in particular) . The fastest instance I could get with enough memory was a a T4 but it take 10 minutes per answer even with 3b model..too slow
Any advice on which cloud service a can provision a GPU fast enough ? really wondering how other are doing.
A100s are pretty hard to provision, overall. I haven't had any trouble with A10 or V100 on AWS and Azure, for what it's worth. Do not use spot instances. Try switching to a region with better availability, maybe.