Always try to do a spreadsheet and estimate the actual cost.

The last time we did the math it was kind of a toss-up between roll your own bare metal cluster and GKS when labor costs were considered, but those are estimated labor costs and we also didn't want to deal with the hassle. GKS meanwhile destroys Amazon, so all the big clouds are not equal.

We decided we might make the leap when the savings for bare metal over GKE are 2X or more, since that leaves a good margin for under-estimating labor (which is probably the case).

I edited the parent to add a point about rolling your own making progressively more sense the bigger you get. That's because modern devops lets you leverage a skilled admin across a big deployment, so the labor cost becomes somewhat fixed and allows you to leverage the economy of scale and the cost advantages of bare metal (to a point).

But do your own estimates! There are many details. One detail is that bare metal bandwidth (especially egress) is radically, ridiculously, nonsensically cheaper than any of the big cloud providers. I italicized those because damn do the big cloud providers ever rape you on bandwidth. The same is true to a less absurd degree for raw CPU power. It mystifies me when I see people doing deep learning on AWS. Storage on the other hand is far cheaper on Amazon and other major cloud providers, especially when you factor in the cost of achieving high reliability and uptime. If you are warehousing a ton of data, you might never want to leave these (unless you actually get bit enough to build your own racks!).

Edit: caveat on storage: check out Backblaze B2 and using it from a nearby bare metal data center if you are mostly warehousing and not heavily accessing. Again: there are many many devils in these details, so do your own shopping and cost estimates for your unique workload and factor in everything: compute, storage, bandwidth, labor, uptime requirements, agility, rapid auto-scaling needs, geographic location and/or multi-location needs, etc. Hosting/cloud is a huge market with a dizzying array of choices each making sense for different mixtures of customer profiles. Expect to spend a day or two just surfing around and shopping.

FYI the bare metal hosts we've had good luck with are packet.net, datapacket.com, and OVH, in approximately that order. (For OVH we have only used their bare metal hosts and can't comment on their other services.) There are others. We had lots of odd issues with Hetzner but they are dirt cheap partly because they cut corners like using desktop-grade hardware. Hetzner may actually make sense if you need a ton of super-cheap compute and have a load that is very tolerant of node failures, since as far as I can see nobody sells CPU as cheap as they do. If I were doing a ton of deep learning model training I'd consider using Hetzner for raw compute power cheap and I'd treat the nodes operationally like on-demand/spot instances.

You might look at Digital Ocean and Vultr too. DO is adding some managed services and apparently they're decent. In the past the simple "we give you a VM" hosts have occupied an uncanny valley though: not as cheap in price/performance terms as bare metal "rent-a-box" hosting, and not rich enough in services and value-adds to compete with big cloud.