Kubernetes Case Study — Pinterest
By: Chetna Manku
Tags: Case Study, Container Orchestration, Kubernetes, Containers, Pinterest

PINTEREST’s KUBERNETES STORY

Pinterest, a social media web and mobile app that allows users to save or “pin” information, has a huge user base who have collectively saved more than 200 billion pins across 4 billion boards.

The popular visual discovery engine — Pinterest has become a part of the Kubernetes community. Pinterest has over 250 million monthly active users for whom it provides over 10 billion recommendations. As they knew these numbers are going to grow day by day, they began to realize the pain of scalability and performance issues.

Pinterest has a huge fleet of servers and over 300 petabytes of data. It has been on the cloud since day 1. However, as the company grew, it became an important mission to release the overhead of managing VMs. This is why they evaluated several platforms and finally ended up choosing Kubernetes.

Challenge

After eight years in existence, Pinterest had grown into 1,000 microservices and multiple layers of infrastructure and diverse set-up tools and platforms. In 2016 the company launched a roadmap towards a new compute platform, led by the vision of creating the fastest path from an idea to production, without making engineers worry about the underlying infrastructure.

As a result of increasing volume and the associated growth of their infrastructure stack, the Pinterest team had several challenges. They stated that their engineers didn’t have a unified experience when launching their workload and that managing huge numbers of virtual machines was creating a huge maintenance load for the infrastructure team.

Furthermore, it was hard to build infrastructure governance tools across the separate systems and to determine which resources could be recycled. The team attempted to address these problems across three key themes: service reliability, infrastructure efficiency and developer productivity.

Key Themes for Pinterest to solve challenge

Solution

Their initial strategy was to move their workload from EC2 instances to Docker containers; hence they first moved their services to Docker to free up engineering time spent on Puppet and to have an immutable infrastructure.

Once these services went into production in early 2017, the team began looking at orchestration to help create efficiencies and manage them in a decentralized way.

And then the next strategy was to move to Kubernetes. Now they can take ideas from ideation to production in a matter of minutes whereas earlier they used to take hours or even days. They have cut down so much of overhead cost by utilizing Kubernetes and have removed a lot of manual work without making engineers worry about the underlying infrastructure.

Impact

“By moving to Kubernetes the team was able to build on-demand scaling and new failover policies, in addition to simplifying the overall deployment and management of a complicated piece of infrastructure such as Jenkins,” says Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group at Pinterest. “We not only saw reduced build times but also huge efficiency wins. For instance, the team reclaimed over 80 percent of capacity during non-peak hours. As a result, the Jenkins Kubernetes cluster now uses 30 percent less instance-hours per-day when compared to the previous static cluster.”

Kubernetes helped Pinterest increase its delivery velocity by bringing down the time-to-market to just minutes. Machine learning is an important use case for Pinterest as it powers its visual discovery engine. Kubernetes allows Pinterest to build, scale, deploy, and iterate its models quickly and easily.

Kubernetes also helps provide Pinterest a clean abstraction as it takes care of several previously manual jobs and lets developers focus on innovation. This shows how Kubernetes helps organizations minimize manual work and removes the clutter.