Kubernetes: the gold standard
By: Simran Kukareja
Tags: Case Study, Kubernetes, Pinterest, Clustering

What is Kubernetes??

Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.

History

Kubernetes was founded by Joe Beda, Brendan Burns, and Craig McLuckie, who were quickly joined by other Google engineers including Brian Grant and Tim Hockin, and was first announced by Google in mid-2014. Its development and design are heavily influenced by Google’s Borg system, and many of the top contributors to the project previously worked on Borg. The original codename for Kubernetes within Google was Project 7, a reference to the Star Trek ex-Borg character Seven of Nine. The seven spokes on the wheel of the Kubernetes logo are a reference to that codename. The original Borg project was written entirely in C++, but the rewritten Kubernetes system is implemented in Go.

Why you need Kubernetes and what it can do

Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn’t it be easier if this behavior was handled by a system?

That’s how Kubernetes comes to the rescue! Kubernetes provides you with a framework to run distributed systems resiliently. It takes care of scaling and failover for your application, provides deployment patterns, and more. For example, Kubernetes can easily manage a canary deployment for your system.

Features of Kubernetes

1.Service discovery and load balancing

Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable.

2.Storage orchestration

Kubernetes allows you to automatically mount a storage system of your choice, such as local storages, public cloud providers, and more.

3.Automated rollouts and rollbacks

You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all their resources to the new container.

4.Automatic bin packing

You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources.

5.Self-healing

Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.

6.Secret and configuration management

Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.

7.Batch execution

Kubernetes jobs with Python

A Job creates one or more Pods and ensures that a specified number of them successfully terminate. As pods successfully complete, the Job tracks the successful completions. When a specified number of successful completions is reached, the task (i.e, Job) is complete. Deleting a Job will clean up the Pods it created.

A simple case is to create one Job object in order to reliably run one Pod to completion. The Job object will start a new Pod if the first Pod fails or is deleted (for example due to a node hardware failure or a node reboot).

8.Horizontal Pod Auto-scaler

The Horizontal Pod Auto-scaler automatically scales the number of Pods in a replication controller, deployment, replica set or stateful set based on observed CPU utilization. Note that Horizontal Pod Auto-scaling does not apply to objects that can’t be scaled.

The Horizontal Pod Auto-scaler is implemented as a Kubernetes API resource and a controller. The resource determines the behavior of the controller. The controller periodically adjusts the number of replicas in a replication controller or deployment to match the observed average CPU utilization to the target specified by user.

What Kubernetes is not

Kubernetes is not a traditional, all-inclusive PaaS (Platform as a Service) system. Since Kubernetes operates at the container level rather than at the hardware level, it provides some generally applicable features common to PaaS offerings, such as deployment, scaling, load balancing, and lets users integrate their logging, monitoring, and alerting solutions. However, Kubernetes is not monolithic, and these default solutions are optional and pluggable. Kubernetes provides the building blocks for building developer platforms, but preserves user choice and flexibility where it is important.

Kubernetes:

Does not limit the types of applications supported. Kubernetes aims to support an extremely diverse variety of workloads, including stateless, stateful, and data-processing workloads. If an application can run in a container, it should run great on Kubernetes.

Does not deploy source code and does not build your application. Continuous Integration, Delivery, and Deployment (CI/CD) workflows are determined by organization cultures and preferences as well as technical requirements.

Does not provide application-level services, such as middleware (for example, message buses), data-processing frameworks (for example, Spark), databases (for example, MySQL), caches, nor cluster storage systems (for example, Ceph) as built-in services. Such components can run on Kubernetes, and/or can be accessed by applications running on Kubernetes through portable mechanisms, such as the Open Service Broker.

Does not dictate logging, monitoring, or alerting solutions. It provides some integrations as proof of concept, and mechanisms to collect and export metrics.

Does not provide nor mandate a configuration language/system (for example, Jsonnet). It provides a declarative API that may be targeted by arbitrary forms of declarative specifications.

Does not provide nor adopt any comprehensive machine configuration, maintenance, management, or self-healing systems.

Additionally, Kubernetes is not a mere orchestration system. In fact, it eliminates the need for orchestration. The technical definition of orchestration is execution of a defined workflow: first do A, then B, then C. In contrast, Kubernetes comprises a set of independent, composable control processes that continuously drive the current state towards the provided desired state. It shouldn’t matter how you get from A to C. Centralized control is also not required. This results in a system that is easier to use and more powerful, robust, resilient, and extensible.

Why Kubernetes Gains popularity??

Originally developed by Google as an offshoot of its Borg project, Kubernetes has established itself as the de facto standard for container orchestration. It’s the flagship project of the Cloud Native Computing Foundation, which is backed by such key players as Google, Amazon Web Services (AWS), Microsoft, IBM, Intel, Cisco, and RedHat.

Kubernetes continues to gain popularity with DevOps practitioners because it allows them to deliver a self-service Platform-as-a-Service (PaaS) that creates a hardware layer abstraction for development teams. Kubernetes is also extremely portable. It runs on Amazon Web Services (AWS), Microsoft Azure, the Google Cloud Platform (GCP), or in on-premise installations. You can move workloads without having to redesign your applications or completely rethink your infrastructure — which helps you to standardize on a platform and avoid vendor lock-in.

Kubernetes Usage Expanding in Large and Small Companies by CNCF Survey

The State of Kubernetes report from VMware showed similar trends among large enterprises. 59% of respondents said they’re running Kubernetes in production. 33% operated 26 clusters or more, and 20% had more than 50 clusters in production. This report reminded readers that’s it’s still “early days.”

More than half of respondents (57%) are operating fewer than 10 Kubernetes clusters, and 60% are running less than half of their containerized workloads on Kubernetes. Early adopters of container technologies relied on other container orchestration tools (or no orchestration at all), and these environments remain in place.

Respondents to the CNCF survey identified further details about their Kubernetes deployments. Cluster ingress is dominated by nginx and HAProxy, with 62% and 22% usage, respectively. Envoy is one to watch, as it’s become the 3rd most popular option, and 29% are evaluating it for further use. Helm was the runaway favorite for packaging Kubernetes applications, with little else registering interest. When separating applications from each other, the preferred options were namespaces, and separate clusters.

The public cloud plays a big role for Kubernetes users. Amazon EKS and Google Kubernetes Engine (GKE) represented the top two management solutions, according to the CNCF report. 62% reported hosting in the public cloud, which was actually down from 2018 (77%). That’s likely due to “hybrid cloud” being a new option in 2019, as it garnered 38% of responses. The State of Kubernetes report painted a different picture, likely due to the respondents coming from solely large enterprises. 64% of respondents deployed Kubernetes on premises, and 31% were using multiple public cloud vendors.

The State of Kubernetes report went a step further and pointed out some of the decision-making challenges enterprises face when getting started with Kubernetes. 83% of respondents reported that more than one team was involved in choosing the product, with C-level executives playing a role 18% of the time. Interestingly, development teams were involved more than any other group. Some companies also appeared to be selecting technologies before deciding which problem to solve, for whom.

With more stakeholders at the table, complications arise. 40% of survey respondents cited a lack of internal alignment as a problem when selecting a Kubernetes distribution. Clearly, Kubernetes creates an opportunity to bring operators and developers closer together, but this has not been fully realized inside many companies.

Another misalignment identified by the State of Kubernetes report related to what impeded developer productivity. 29% of developers said that access to infrastructure was their top problem. Only 6% of executives answered that way. Rather, 49% of executives thought the top challenge for developers was the integration of new technologies. Only 29% of developers agreed.

Both surveys highlighted the benefits users saw by using Kubernetes, and cloud-native technologies. The CNCF respondents recognized faster deployment times, improved scalability and availability, and more cloud portability. The enterprises participating in the State of Kubernetes report realized benefits like improved resource utilization, faster software delivery, and reduced cloud costs, among others.

Kubernetes Market Share and Competitors in Virtualization Management Software

the best indexing techniques combined with advanced data science to monitor the market share of over 12,500 technology products, including Virtualization Management Software. By scanning billions of public documents, we are able to collect deep insights on every company, with over 100 data fields per company at an average. In the Virtualization Management Software category, Kubernetes has a market share of about 21.9%. Other major and competing products in this category include:

Top Industries that use Kubernetes

Top Countries that use Kubernetes

Distribution of companies that use Kubernetes based on company size (Employees)

Distribution of companies that use Kubernetes based on company size (Revenue)

Industrial Use-case

Who uses Kubernetes??

Companies

Developers

Kubernetes Integration

CASE STUDY

Pinning Its Past, Present, and Future on Cloud Native

What is Pinterest??

Pinterest is an American image sharing and social media service designed to enable saving and discovery of information (specifically “ideas”) on the internet using images and, on a smaller scale, animated GIFs and videos, in the form of pin boards. The site was created by Ben Silbermann, Paul Sciarra, and Evan Sharp and had over 400 million monthly active users as of August 2020. It is operated by Pinterest, Inc., based in San Francisco.

What is Challenge in the front of Pinterest??

After eight years in existence, Pinterest had grown into 1,000 microservices and multiple layers of infrastructure and diverse set-up tools and platforms. In 2016 the company launched a roadmap towards a new compute platform, led by the vision of creating the fastest path from an idea to production, without making engineers worry about the underlying infrastructure.

What is Solution over it??

The first phase involved moving services to Docker containers. Once these services went into production in early 2017, the team began looking at orchestration to help create efficiencies and manage them in a decentralized way. After an evaluation of various solutions, Pinterest went with Kubernetes.

What is Impact of this solution on Pinterest??

“By moving to Kubernetes the team was able to build on-demand scaling and new failover policies, in addition to simplifying the overall deployment and management of a complicated piece of infrastructure such as Jenkins,” says Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group at Pinterest. “We not only saw reduced build times but also huge efficiency wins. For instance, the team reclaimed over 80 percent of capacity during non-peak hours. As a result, the Jenkins Kubernetes cluster now uses 30 percent less instance-hours per-day when compared to the previous static cluster.”

“So far it’s been good, especially the elasticity around how we can configure our Jenkins workloads on that Kubernetes shared cluster. That is the win we were pushing for.” --Micheal Benedict, Product Manager for the Cloud & the Data Infrastructure Group at Pinterest

Pinterest was born on the cloud --running on AWS since day one in 2010 --but even cloud native companies can experience some growing pains.

Since its launch, Pinterest has become a household name, with more than 200 million active monthly users and 100 billion objects saved. Underneath the hood, there are 1,000 microservices running and hundreds of thousands of data jobs.

With such growth came layers of infrastructure and diverse set-up tools and platforms for the different workloads, resulting in an inconsistent and complex end-to-end developer experience, and ultimately less velocity to get to production. So in 2016, the company launched a roadmap toward a new compute platform, led by the vision of having the fastest path from an idea to production, without making engineers worry about the underlying infrastructure.

The first phase involved moving to Docker. “Pinterest has been heavily running on virtual machines, on EC2 instances directly, for the longest time,” says Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group. “To solve the problem around packaging software and not make engineers own portions of the fleet and those kinds of challenges, we standardized the packaging mechanism and then moved that to the container on top of the VM. Not many drastic changes. We didn’t want to boil the ocean at that point.”

“Though Kubernetes lacked certain things we wanted, we realized that by the time we get to productionizing many of those things, we’ll be able to leverage what the community is doing.” --Micheal Benedict, Product Manager for the Cloud & the Data Infrastructure Group at Pinterest

The first service that was migrated was the monolith API fleet that powers most of Pinterest. At the same time, Benedict’s infrastructure governance team built chargeback and capacity planning systems to analyze how the company uses its virtual machines on AWS. “It became clear that running on VMs is just not sustainable with what we’re doing,” says Benedict. “A lot of resources were underutilized. There were efficiency efforts, which worked fine at a certain scale, but now you have to move to a more decentralized way of managing that. So orchestration was something we thought could help solve that piece.”

That led to the second phase of the roadmap. In July 2017, after an eight-week evaluation period, the team chose Kubernetes over other orchestration platforms. “Kubernetes lacked certain things at the time — for example, we wanted Spark on Kubernetes,” says Benedict. “But we realized that the dev cycles we would put in to even try building that is well worth the outcome, both for Pinterest as well as the community. We’ve been in those conversations in the Big Data SIG. We realized that by the time we get to productionizing many of those things, we’ll be able to leverage what the community is doing.”

At the beginning of 2018, the team began onboarding its first use case into the Kubernetes system: Jenkins workloads. “Although we have builds happening during a certain period of the day, we always need to allocate peak capacity,” says Benedict. “They don’t have any auto-scaling capabilities, so that capacity stays constant. It is difficult to speed up builds because ramping up takes more time. So given those kind of concerns, we thought that would be a perfect use case for us to work on.”

“So far it’s been good, especially the elasticity around how we can configure our Jenkins workloads on Kubernetes shared cluster. That is the win we were pushing for.” --Micheal Benedict, Product Manager for the Cloud & the Data Infrastructure Group at Pinterest

They ramped up the cluster, and working with a team of four people, got the Jenkins Kubernetes cluster ready for production. “We still have our static Jenkins cluster,” says Benedict, “but on Kubernetes, we are doing similar builds, testing the entire pipeline, getting the artifact ready and just doing the comparison to see, how much time did it take to build over here. Is the SLA okay, is the artifact generated correct, are there issues there?”

“So far it’s been good,” he adds, “especially the elasticity around how we can configure our Jenkins workloads on Kubernetes shared cluster. That is the win we were pushing for.”

By the end of Q1 2018, the team successfully migrated Jenkins Master to run natively on Kubernetes and also collaborated on the Jenkins Kubernetes Plugin to manage the lifecycle of workers. “We’re currently building the entire Pinterest JVM stack (one of the larger monorepos at Pinterest which was recently bazelized) on this new cluster,” says Benedict. “At peak, we run thousands of pods on a few hundred nodes. Overall, by moving to Kubernetes the team was able to build on-demand scaling and new failover policies, in addition to simplifying the overall deployment and management of a complicated piece of infrastructure such as Jenkins. We not only saw reduced build times but also huge efficiency wins. For instance, the team reclaimed over 80 percent of capacity during non-peak hours. As a result, the Jenkins Kubernetes cluster now uses 30 percent less instance-hours per-day when compared to the previous static cluster.”

“We are in the position to run things at scale, in a public cloud environment, and test things out in way that a lot of people might not be able to do.” --Micheal Benedict, Product Manager for the Cloud & the Data Infrastructure Group at Pinterest

Benedict points to a “pretty robust roadmap” going forward. In addition to the Pinterest big data team’s experiments with Spark on Kubernetes, the company collaborated with Amazon’s EKS team on an ENI/CNI plug in.

Once the Jenkins cluster is up and running out of dark mode, Benedict hopes to establish best practices, including having governance primitives established — including integration with the chargeback system — before moving on to migrating the next service. “We have a healthy pipeline of use-cases to be on-boarded. After Jenkins, we want to enable support for Tensorflow and Apache Spark. At some point, we aim to move the company’s monolithic API service. If we move that and understand the complexity around that, it builds our confidence,” says Benedict. “It sets us up for migration of all our other services.”

After years of being a cloud native pioneer, Pinterest is eager to share its ongoing journey. “We are in the position to run things at scale, in a public cloud environment, and test things out in way that a lot of people might not be able to do,” says Benedict. “We’re in a great position to contribute back some of those learnings.”

Conclusion

Kubernetes has not only helped in the vertical and horizontal scaling of containers but have turned the tables for innovative engineering expectations. It has been succeeded in deployment for an initial estimate of servers. Also, it has gained so much popularity in a short span of time. Hence, most of the industry engineers will respond with inspiring stories to get their businesses right on tracks.