Compendium of Kubernetes Application Deployment Tools
By: Karl Isenberg
Tags: DevOps, Continuous Deployment, Software Development, Docker, Kubernetes

Deploying applications to Kubernetes can be as simple as writing a few resource definitions in yaml or json and applying them with kubectl, but it can also be a whole lot more automated (and complicated).

A popular meme in application deployment is the combination of Continuous Deployment and GitOps: the automatic deployment of resources after each change to the source code. In order to for you to use GitOps to deploy applications to Kubernetes, you need several things:

Container Image Building to build your source code and local dependencies into container images.

to build your source code and local dependencies into container images. Resource Templating to customize deployment resources for your environment(s).

to customize deployment resources for your environment(s). Package Management to bundle multiple resources into versioned releases and manage package dependencies.

to bundle multiple resources into versioned releases and manage package dependencies. Continuous Deployment to roll out new changes to your environment(s), often using a pipeline of steps and stages.

to roll out new changes to your environment(s), often using a pipeline of steps and stages. Imperative Deployment to manage complex service lifecycles programmatically and reduce manual or fragile scripted steps.

to manage complex service lifecycles programmatically and reduce manual or fragile scripted steps. Autoscaling to manage the replication and resource allocation of your application over time, based on usage and consumption.

In this article, I have listed many tools (both popular and obscure) for each of these stages in application lifecycle management. Since it’s hard to judge popularity or success objectively, I’ve tried to annotate these tools in a way that makes it easy to see which big corporate backers have invested in these projects. Keep in mind, a large cloud backer may have multiple competing investments, so just because it has a known investor doesn’t mean it will survive and thrive in the long term.

Hopefully this list will give you a place to start when searching for solutions to your application deployment problems.

Container Image Building

Moby / buildkit (Docker) — A toolkit for converting source code to build artifacts.

(Docker) — A toolkit for converting source code to build artifacts. kaniko (Google) — A tool to build container images from a Dockerfile, inside a container or Kubernetes cluster.

(Google) — A tool to build container images from a Dockerfile, inside a container or Kubernetes cluster. img (Jess Frazelle) — A standalone, daemon-less, unprivileged Dockerfile and OCI compatible container image builder.

(Jess Frazelle) — A standalone, daemon-less, unprivileged Dockerfile and OCI compatible container image builder. buildah (IBM/Red Hat) — A tool that facilitates building Open Container Initiative (OCI) container images.

(IBM/Red Hat) — A tool that facilitates building Open Container Initiative (OCI) container images. Source-To-Image (IBM/Red Hat) — A tool for building artifacts from source and injecting into container images.

(IBM/Red Hat) — A tool for building artifacts from source and injecting into container images. Tanzu Build Service / kpack / pack (VMware/Pivotal) — A CLI and service for building apps using Cloud Native Buildpacks.

(VMware/Pivotal) — A CLI and service for building apps using Cloud Native Buildpacks. Carvel / kbld (VMware/Pivotal) — A service for building and pushing images into development and deployment workflows.

(VMware/Pivotal) — A service for building and pushing images into development and deployment workflows. Google Cloud Buildpacks (Google) — Builders and buildpacks designed to run on Google Cloud’s container platforms.

(Google) — Builders and buildpacks designed to run on Google Cloud’s container platforms. Makisu (Uber) — A fast and flexible Docker image building tool, that works in unprivileged containerized environments like Mesos and Kubernetes.

Resource Templating

Helm (Microsoft, Google) — A Kubernetes Package Manager

(Microsoft, Google) — A Kubernetes Package Manager Kustomize (Google, Apple) — A CLI to customize raw, template-free YAML files, leaving the original YAML untouched and usable as-is.

(Google, Apple) — A CLI to customize raw, template-free YAML files, leaving the original YAML untouched and usable as-is. Carvel / ytt (VMware/Pivotal) — A YAML templating tool that works on YAML structure instead of text

(VMware/Pivotal) — A YAML templating tool that works on YAML structure instead of text jsonnet / go-jsonnet (Google) — A JSON templating language.

(Google) — A JSON templating language. gomplate (Dave Henderson) — A CLI for golang template rendering, supporting local and remote datasources.

(Dave Henderson) — A CLI for golang template rendering, supporting local and remote datasources. Mustache (Github) — A framework-agnostic JSON templating engine.

Package Management

Helm (Microsoft, Google) — A Kubernetes Package Manager

(Microsoft, Google) — A Kubernetes Package Manager Cloud Native Application Bundles (CNAB) / Porter / Duffle (Microsoft/Deis, Docker) — A package format specification, bundler, and installer for managing cloud agnostic distributed applications.

Continuous Deployment

Imperative Deployment

Kubebuilder (CNCF, Google, Apple, IBM/Red Hat) — An SDK for building Kubernetes APIs (and controllers and operators) using CRDs.

(CNCF, Google, Apple, IBM/Red Hat) — An SDK for building Kubernetes APIs (and controllers and operators) using CRDs. Operator Framework / Operator SDK (IBM/Red Hat/CoreOS) — An SDK for building Kubernetes application operators.

(IBM/Red Hat/CoreOS) — An SDK for building Kubernetes application operators. KUDO (D2IQ) — A framework for building production-grade Kubernetes Operators using a declarative approach.

(D2IQ) — A framework for building production-grade Kubernetes Operators using a declarative approach. Pulumi (Pulumi) — An Infrastructure as Code SDK for creating and deploying cloud software that use containers, serverless functions, hosted services, and infrastructure, on any cloud.

(Pulumi) — An Infrastructure as Code SDK for creating and deploying cloud software that use containers, serverless functions, hosted services, and infrastructure, on any cloud. Carvel / kapp / kapp-controller (VMware/Pivotal) — A CLI and Kubernetes controller for installing configuration (helm charts, ytt templates, plain yaml) as described by App CRD.

(VMware/Pivotal) — A CLI and Kubernetes controller for installing configuration (helm charts, ytt templates, plain yaml) as described by App CRD. Isopod (Cruise) — An expressive DSL and framework for Kubernetes resource configuration without YAML.

Autoscaling

Horizontal Pod Autoscaler (built-in) — A Kubernetes controller that automatically scales the number of pods in a replication controller, deployment, replica set or stateful set based on a configured metric.

(built-in) — A Kubernetes controller that automatically scales the number of pods in a replication controller, deployment, replica set or stateful set based on a configured metric. Vertical Pod Autoscaler (Google) — A set of Kubernetes components that automatically adjusts the amount of CPU and memory requested by pods running in the Kubernetes Cluster.

(Google) — A set of Kubernetes components that automatically adjusts the amount of CPU and memory requested by pods running in the Kubernetes Cluster. Addon Resizer (Google) — A simplified version of vertical pod autoscaler that modifies resource requests of a deployment based on the number of nodes in the Kubernetes Cluster.

(Google) — A simplified version of vertical pod autoscaler that modifies resource requests of a deployment based on the number of nodes in the Kubernetes Cluster. KEDA (Microsoft) — A Kubernetes-based Event Driven Autoscaling component.

(Microsoft) — A Kubernetes-based Event Driven Autoscaling component. Watermark Pod Autoscaler Controller (DataDog) — A custom controller that extends the Horizontal Pod Autoscaler (HPA).

(DataDog) — A custom controller that extends the Horizontal Pod Autoscaler (HPA). Pangolin (Damian Peckett) — An enhanced Horizontal Pod Autoscaler for Kubernetes that scales deployments based on their Prometheus metrics, using a variety of highly configurable control strategies.

(Damian Peckett) — An enhanced Horizontal Pod Autoscaler for Kubernetes that scales deployments based on their Prometheus metrics, using a variety of highly configurable control strategies. Predictive Horizontal Pod Autoscaler (IBM) — A custom pod autoscaler, similar to Horizontal Pod Autoscaler, however with added predictive elements.

(IBM) — A custom pod autoscaler, similar to Horizontal Pod Autoscaler, however with added predictive elements. Horizontal Pod Autoscaler Operator (Banzai Cloud) — A Kubernetes controller that watches Deployments or StatefulSets and automatically creates HorizontalPodAutoscaler resources, based on autoscale annotations.

In The End…

As any DevOps advocate will tell you, it’s not about the tools but about the mindset. No one tool will give you an end to end application lifecycle management experience that delights you, because everyone uses their own permutation of tools, glued together with scripts and integration code.

You can look for tools that do one thing well, being easily replaceable and extensible, or tools that provide the most value, being less to manage, cheaper to integrate, and the best end to end user experience. There’s not really a wrong answer.

Because of those tradeoffs, it pays to look at who is behind each project, how many companies are investing, and how popular the tool is. Popular tools with large, diverse investors are more likely to keep growing as you use them, rather than stagnate and become abandoned, requiring you to replace the tool or replace the investment with your own time and energy.

Hopefully, this taxonomy will be useful and provide you with a starting place as you consider your options. Good luck!

Did I forget your favorite tool? Leave a comment or let me know on Twitter!