How Spotify uses Kubernetes?
Tags: Kubernetes, Spotify

Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.

It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds upon 15 years of experience of running production workloads at Google, combined with best-of-breed ideas and practices from the community.

Spotify and Kubernetes-The Great Integration:

With 170 million active users and 25 million songs, Spotify is the world’s largest music streaming subscription service. Undergirding the 1 billion plays per day is a digital infrastructure that is slowly shifting.

Spotify open sourced its in-house container orchestration service, Helios, in 2014. After several years of use, Spotify decided to make the switch from Helios to Kubernetes, another orchestration service released shortly after that has since become the de facto orchestrator platform.

Backed by thousands of developers, Kubernetes comes with a huge ecosystem behind it, and trying to reach feature parity with an in-house system not widely adopted by other enterprises is difficult, even for a business as large as Spotify, according to James Wen, site reliability engineer for Spotify, speaking at DevFest DC last week.

It became clear that Spotify needed a managed solution rather than operating clusters from scratch. By moving to Kubernetes, Spotify would benefit from:

Cloud-native “magic,” such as autoscaling, better resource utilization and self-healing

Less capacity planning for developers

Less proprietary technology

Faster experimentation and operations

Spotify continues to expand its use of Kubernetes on a monthly basis since its adoption a few years ago. Previously, Spotify had already begun to shift its operation to a containerized infrastructure before it began to consider the potential benefits Kubernetes might offer.

“When we originally looked at Kubernetes, we were in an interesting situation, because we had already had an in-house orchestration solution we had built, and, anecdotally, launched the very same week an open source [alternative] when Kubernetes was launched,” said Haughwout. “So we did a lot of work to essentially make the transition to Kubernetes incredibly easy for developers, and to make it so that we could have hundreds of teams work across shared clusters securely and safely together.”

Despite its early adoption, Spotify began to shift to Kubernetes “in earnest” about a year and a half ago. Kubernetes has since played a key role in Spotify’s DevOps in two key ways. This includes how the platform has helped to “reduce toil,” said Haughwout.

“We want to take away the need for engineers, who have to manage infrastructure, to worry about scaling up and down so they can just simply build and deploy features,” he explained.

The second main benefit Kubernetes has provided is how the adoption of a cloud native infrastructure has enabled the music streaming giant to add a number of new tools and platforms to improve its production pipeline and operations. “One of the big reasons that we’ve been working diligently with the Cloud Native Computing Foundation is to make it easy to adopt a lot of infrastructure,” Haughwout said. “Kubernetes has become a kind of lingua franca of cloud native technology that opens the door for us to get into lots of other technologies.”

During the past year, Spotify has been expanding the number of services it runs on Kubernetes while taking advantage of its highly distributed structures. For example, Haughwout described how Spotify moved data pipelines and machine learning to Kubernetes and “relies on it to build ephemeral environments.”

The road to Kubernetes adoption can be, of course, fraught with challenges. In Spotify’s case, Haughwout recounts the challenges associated with empowering several hundred autonomous engineering teams to “move as quickly as possible.”

“They’re working on building and iterating features and experiments that people use on Spotify every day and we want them to keep working on that,” Haughwout said. “So one of our challenges is how do we migrate them, when you have 299 million monthly users, without interrupting the music stream to things like Kubernetes, without slowing them down.”