The forges behind Netflix, LinkedIn and Co.
Tags: discuss, opensource, technology
Published: 2018-12-19 22:33:15

---
title: The forges behind Netflix, LinkedIn and Co.
published: true
description: A discussion about how and why big tech company start developing their own framework and eventually sharing it with the world
tags: #discuss #opensource #technology
cover_image: https://thepracticaldev.s3.amazonaws.com/i/w45tyyo3dzlf2dz0fey4.jpg
---

When people at home bring up Spotify and Netflix, they often discuss the content these platforms provide. But inside the developer community we do not only discuss the newest season of *House of Cards* but rather the frameworks companies like this have contrived over time.

I was amused by this discrepancy and was wondering at which point a tech company turns from content-deliverer to content-handler. Take LinkedIn. The profiles, search queries and communication channels scaled over the years and LinkedIn turned their monolithic application infrastructure into microservices. In order to cope with these challenges, LinkedIn developed [Kafka](https://engineering.linkedin.com/blog/2016/04/kafka-ecosystem-at-linkedin), a event-driven messaging system. They started off with several small queuing pipelines for the interaction between services, but then went on to develop a publisher-subscriber platform. [Instead of maintaining and scaling those pipelines one-by-one.](https://insidebigdata.com/2016/04/28/a-brief-history-of-kafka-linkedins-messaging-platform/)

Netflix met a similar problem: As they have tons of services, even in different zones, and only few edge services that receive requests, this ginormous amount of calls have to be distributed to these services. Load balancing algorithms back then did not deliver satisfying performances, so they came up with own ones and  tied them up with [Ribbon](https://medium.com/netflix-techblog/announcing-ribbon-tying-the-netflix-mid-tier-services-together-a89346910a62).

Infrastructural change is not the only catalysator for innovation. Even if given language frameworks do not meet your requirements, you could think of introducing your own. That's [how and why](https://reactjs.org/blog/2013/06/05/why-react.html) Facebook gave birth to React.

And what about Spotify? While they did not contribute a huge technological framework, they demonstrate a way to incorporate the agile part of software development into the over-all structure of an organization. They decided to not only to use Agile Practices in small teams but on a big scale. So, Spotify introduced [Squads, Chapters, Tribes and Guilds](https://www.growly.io/spotify-engineering-model-with-squads-tribes-chapters-and-guilds/).

It is one thing to come up with these ideas. It is another to share them with others. All mentioned novelties have hit the open source field. The messaging system above is even just refered to as *Apache* Kafka, named after the Open Source License which *Ribbon* is also licensed under. Not to mention the role Facebook's React plays in the Frontend community. The trend in moving to Open Source and common knowledge is on-going. And these companies have a deep interest in giving access to their inventions.

What other companies do you expect to emerge as a pusher for new technologies and frameworks or mindsets? Or maybe you had a solid problem that you or your team solved with a self-invented toolset? And why Open Source?