> Kubernetes makes it as simple as it can be without oversimplifying it.

What about something like Hashicorp Nomad and Docker Swarm? The popularity of the former and the maintenance status of the latter aside, I think they achieve much of the same as Kubernetes in ways that are simpler, which is enough for the majority of deployments out there.

For example, most pieces of software that run in containers have a docker-compose.yml file which can then be fed into Docker Compose to launch an environment on a single node, say for local testing and development, or to just explore a piece of software in a throwaway environment. What Docker Swarm does, is take basically the same specification and add the ability to run containers across multiple loads, do networking across them in a reasonably secure way, whilst being able to set up resource limitations etc. as needed, as well as scale the containers across the available nodes and manage storage with volumes, bind mounts or even plugins for something like GlusterFS or just NFS.

Docker Swarm doesn't concern itself with the concept of Pods because you don't always need those - regular containers can be enough without the additional abstraction in the middle. Docker Swarm doesn't concern itself with the concept of a Service, since you can just access containers based on their names through the built in DNS abstraction, especially if you don't need complicated network isolation (which you can also achieve at server level). Docker Swarm doesn't really care about an Ingress abstraction either, since you can just make your own Nginx/Caddy/Apache container and bind it to ports 80 and 443 on all of the nodes where you want to have your own ingress. No PersistentVolume and PersistentVolumeClaim abstractions either, since the aforementioned bind mounts and volumes, or network storage are usually enough. And the resource usage and API are exceedingly simple, you don't even need to worry about service labels or anything like that, since in most cases you'll only care about the service name to access the container through.

If I built my own container orchestrator, I'd strive for that simplicity. Seems like projects like CapRover also recognized that: https://caprover.com/ Same with Dokku: https://dokku.com/

If you're in a company that has never really run advanced A/B tests or doesn't really need to do complex 2-stage DB migrations or blue-green deployments, circuit breaking and other fancy stuff, there's not that much use in going with Kubernetes, unless you really just want to hire for it and also pay someone else to manage it for you.

Personally, with tools like Kompose https://kompose.io/ I'd advise that you start with Docker and Docker Compose at first (locally, or for dev environments) and then branch out to Docker Swarm or Nomad, before eventually migrating over to Kubernetes, if you need to, maybe with something like K3s or K0s clusters at first. Or maybe even Portainer/Rancher, since those make the learning curve of Kubernetes far more tolerable. Or, at the risk of increasing the complexity of your deployments, go with Helm as well because the templates for Deployments and other objects that Helm creates by default are surprisingly useful in avoiding YAML hell.

Of course, some say that Docker Swarm is dead and you shouldn't ever touch it, which is why I mention Nomad (even though HCL is a little bit odd at times), which is also great, with the added quality of supporting non-container deployments. Either way, ideally look for a way to run apps in containers because they feel like the "right" abstraction, whilst finding the best fit for the features that you actually need vs the complexity introduced.

In short: containers are pretty good, running them has lots of options, although some are indeed complicated. Kubernetes is not the simplest way to run them, in my experience.