Managing Kubernetes Configuration with Puppet

Breadcrumb

Kubernetes is the new cluster manager from Google that makes it easy to deploy, manage and scale containers to very large infrastructures.  With the new Puppet module for Kubernetes, you can now manage Pods, ReplicationControllers, Services and more, all using the familiar Puppet language. 

Enter Kubernetes

The recent KubeCon event showcased the rapid progress made in a short time with Kubernetes, the cluster manager for Linux containers released by Google, and built based on how the company manages its internal infrastructure.  Kubernetes aims to make complex distributed systems simpler by introducing a core set of primitives and APIs for managing them.  Companies as diverse as eBay, SoundCloud and RedHat all talked about their progress in adopting this new approach to building applications, systems and infrastructure. 

Here at Puppet, we’re always interested in new things to manage, and Kubernetes’ focus on well-designed primitives and a solid API encourage integration.  With the new Kubernetes Puppet module, we now have a solid proof of concept for how to use Puppet to manage Kubernetes resources like Replication Controllers, Pods and Services. 

Configuration Management: An Aside

When talking about Puppet and Kubernetes, it’s useful to touch briefly on a higher-level topic: configuration management.  Some people look at configuration management (and tools like Puppet) as a way of managing host-bound resources like files, services, packages, users or groups.  Kubernetes introduces higher-level primitives, like Pods and Replication Controllers, aimed at making the management of distributed and scalable systems drastically easier.  The story goes that you no longer need configuration management with those new primitives. 

But configuration management is really a much broader discipline, dating back formally to the 1950s.  My favourite description of the capabilities of any configuration management system comes from Military Handbook Configuration Management Guidance MIL-HDBK-61B, which talks about:

Identification

Control

Status accounting

Verification and audit

The problems associated with those capabilities are present with systems like Kubernetes too, and only partially addressed by current native tooling — problems like managing configuration drift, having a single well-audited change control mechanism, having a model of your infrastructure outside Kubernetes, etc.  This becomes even more important as deployments hit production, as well as in heterogeneous (read real-world) environments, where multiple generations of technology run side by side. 

Using Puppet with Kubernetes

The new Kubernetes Puppet module allows you to use the Puppet domain specific language to manage resources in Kubernetes — for instance Replication Controllers, Services and Pods.  This means:

It is easier to manage the state of Kubernetes resources over time, using source code that can be versioned alongside your application code. 

You can be sure of the state of your Kubernetes infrastructure by taking advantage of Puppet's built-in reporting and tools like PuppetDB. 

If you’re already using Puppet, the Kubernetes Puppet module provides a convenient way of managing Kubernetes alongside your other infrastructure. 

The module doesn't (yet) install or manage the various components that make up Kubernetes, although this area is well served by platforms like OpenShift, Tectonic from CoreOS or Google Container Engine. 

The module allows for defining Kubernetes resources in Puppet, and then for Puppet to create or update those resources as needed over time. 

The user interface of the Puppet code follows the Kubernetes YAML format exactly.  This allows for a familiar interface for anyone used to the Kubernetes API, and at the same time, provides a low-level building block for creating higher-level types in Puppet. 

Not all of these can be created via the API, but all are available to query using the power of puppet resource.  For instance, if you want to know the current nodes in your Kubernetes cluster you can run:

$ puppet resource kubernetes_node

Puppet resource can also be used to export the state of a running cluster to Puppet manifests, which can be used as the base for maintaining the state of the cluster over time. 

Why Puppet

The use of Puppet to manage Kubernetes improves on the basic YAML file approach in a few ways:

The Puppet language supports logic and abstractions, allowing you to craft business-specific user interfaces.  You can also create higher-level types, and then instantiate multiple copies of those. 

Modifications can be made to the Puppet code, and Puppet will handle updating the relevant resources.  You don't have to describe the full state of the resource, or manually modify individual resources via the command line interface (CLI). 

Puppet has tools for validating code and for writing unit tests, as well as distributing and sharing reusable modules. 

Puppet supports relationships between resources, so you can enforce ordering where necessary.  With the new application orchestration capabilities, you can even order across nodes where part of the infrastructure isn’t using Kubernetes. 

Examples and Feedback

In the second blog post, we’ll step through a complete example of the module in action, converting the canonical Kubernetes Hello World guestbook example to Puppet. 

Do let us know what you think about the module, and tell us your additional ideas for similar integrations or examples you’d like to see.