What is pnpm? Is it really so fast and space-efficient?
Tags: webdev, beginners, tooling, javascript
Published: 2022-10-13 12:57:41

_We announced at ViteConf that our **WebContainers now support pnpm**. It was a major achievement in our commitment to support the Vite ecosystem as many projects running on Vite also use pnpm (examples include Vue, Astro, SvelteKit)._

In this post, I’d like to talk about **what sets pnpm apart from other package managers - and why we need package managers at all**. As a person who best operates through analogy, I will also talk about: 🍣 sushi, 🪦 cemetery, 🎁 birthdays, and 🖼 memes.

## What are package managers?

*I've also written an [“Explain like I’m 5” version of this section](https://dev.to/stackblitz/explain-like-im-five-package-managers-1a7a)!*

Let’s start with the terminology.

![A surrealist image of massive clouds with a shiba-inu dog head that is consuming a city. The clouds are labelled "node modules" and the city is labelled "github"](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0dx6ny86uj9izv6tld3l.jpeg)

### package, dependency, node module

A “**package**” (or “**dependency**”) is one or a few files bundled neatly together that can be downloaded from a package registry. Once it’s downloaded and installed, it resides in the `node_modules` folder in a project’s directory. In this post, I will use the words a Node module, a package, and a dependency interchangeably as many folks in the tech community do. That being said, [the npm docs offer a differing opinion](https://docs.npmjs.com/about-packages-and-modules) so check it out if you’re interested in trivia and precision. 🙂

### package.json

Each package has a file called `package.json` which contains information on the package author, its version, its dependencies, and so on. A package can have many dependencies, which in turn can also have many dependencies (called "sub-dependencies"). This sounds like a lot of files, doesn’t it! It really is, especially if your project requires bigger packages.

In the olden times, you’d manually download, install, and configure the dependencies of your app. Keep in mind that most packages are frequently updated, which means that you’d have to always make sure that your project's dependencies (and their sub-dependencies!) are up to date - or that in a given project you use a correct version of a given package. This sounds not only stressful but also unnecessarily repetitive! This is where a package manager comes in.

### package managers

A package manager is a tool that helps you keep track of all dependencies of your app in a consistent way. It automates the tasks of dependency installation, upgrading, configuration, and removal. It makes sure that all the packages that your app needs are installed, of the correct version, and up-to-date. This frees so much time for coding… and procrastination 🤪

The first package manager for the Node runtime was introduced only in 2010 and it was npm, (which, contrary to popular belief, [doesn’t stand for “Node package manager” and, in fact, is an empty acronym](https://github.com/npm/cli#is-npm-an-acronym-for-node-package-manager)). Twelve years later, npm is now managed by Microsoft and there are two other contenders to the iron throne of the package universe: yarn and pnpm.

## What do package managers do?

A package manager grabs the published source files from the online registry and installs them into a directory (a folder) called `node_modules`. This is the folder where your app will look for the packages when you call the `require()` method. 

You can think about the `node_modules` folder as a box with helpful tools that you can plug into your app so it works in a desired way. Except that all the tools are made of iron (or [osmium](https://sciencing.com/metals-heaviest-8751708.html)!) and the box is massive. What I mean to say is that, sadly, prior to pnpm the `node_modules` folder would usually be bulky and take a lot of disk space. This resulted in a collection of memes such as:

![A popular meme portraying heaviest objects in the universe such as Sun, neutron star, black hole, and Node modules](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gm7t956uxawl53si8cyp.png)

Imagine that you have numerous projects on your local machine, and each project comes with its own node_modules folder. Most likely, many of these projects use a similar stack. In such a case, you end up with the same packages in numerous projects. Duplication and redundancy! Think about all those inspired side projects that never left the boilerplate stage, all the past attempts to check out all the hot new frameworks - all of them are chilling on your precious disk space.


![A comic strip where a computer folder containing Windows brags about being the biggest just to be intimidated by the "Node modules" folder](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k2uizq6rsnnena1dzkl3.png)

## Who cursed the `node_modules` folder?

To understand these jokes, we need to take a stroll down memory lane and look at how package management has traditionally been handled.

To the best of its intentions and abilities, npm handled the dependencies by splitting the installation (which is triggered by, for example, `npm install` ) process into three phases:

1. **Resolving**, when the package manager is checking all the project’s dependencies (and their sub-dependencies) listed in the `package.json` file, finds a version that satisfies the version specifier (for instance, `^1.0.0` would install any future minor/patch versions). Afterwards, it creates a file like `package-lock.json` for npm and `pnpm-lock.yaml` for pnpm. you can think about it like a project’s equivalent of a birthday gift wishlist.
2. **Fetch**, when the package manager takes the list of resolved dependencies and fetches all the packages from the package registry, which is an equivalent of the feverish shopping in a crowded mall two hours before the party for the gifts all of your friends will give to the birthday person.
3. **Linking**, when the package manager writes all the dependencies into the project’s `node_modules` folder, which, finally, is an equivalent of placing all the gifts in the corner of the party room for when they are needed.

In this scenario, each phase needs to end for the next one to begin. This means that if one dependency has twenty dependencies itself or if one package takes forever to be downloaded, you may have to wait for a long time. If we follow the birthday analogy, imagine that the wishlist included a new version of a popular phone and a box of chocolates. You end up queuing [for a few days](https://www.inc.com/business-insider/massive-lines-iphone-x-apple-store-people-queue-days-tim-cook.html) in front of the phone store to only then buy the chocolates. In such a system, you can’t “save” your spot in the queue, go buy the chocolates and join the party, even if that looked like a better use of your time. You see that t**he way npm manages its tasks is just not efficient.**

As we said earlier, it is also the matter of the disk space. You will end up with numerous duplications of the same packages. And even if there are different versions of the same package, it is rarely the case that all of the files have been changed from one version to another. You still end up with some number of copies. We can agree that **the way npm manages the disc space is not efficient either.**

## How is pnpm a solution to all our worries and troubles?

On its docs page you can read that pnpm is a “fast, disk space efficient package manager.” It really is fast - locally, up to three times faster than the alternatives - and space-efficient. But - how?

### pnpm is fast

Do you remember the birthday wishlist analogy? As a gofer, npm first collected all the wishes, then bought all the gifts, then placed them in the corner of the party room. Each phase needed to end for another to begin. This is how the npm gofer organizes its way through life.

Well, if pnpm were the gofer, they’d buy the gift as soon as they’ve read the wish, and designated the spot in the room as soon as they placed the order. They may not even have the physical gift yet to already be able to mark the space in the room where the gift will stand. This happens for each gift separately and independently from others. By design, pnpm doesn’t have the blocking stages of installation - the processes run for each of the packages independently. 

### pnpm is disc space efficient

**Let’s talk about food. I love veggie sushi. I order a lot of veggie sushi**. I also eat a lot of wasabi and I don’t like waste. Even though it would be fair to assume that such portions surely are to be shared, I usually order just for myself. **My order always arrives with two pairs of chopsticks even though I say on the phone that I don’t need them - I have metal ones at home.** I wouldn’t just throw unused chopsticks away and now I have a drawer full of them, and you can also see them laying around in the most surprising places in the kitchen. Similarly, I have my own wasabi tube but would I throw away those little sachets? Never. They chill on the shelf in my fridge in an ever-expanding fashion.

My chopsticks and wasabi could work as an analogy for how npm manages disc space. **“Oh, you’ve already installed React two hundred times? I’m SURE you NEED the 201st copy!”**

To my relief, pnpm checks what is already available on your disk and only adds what you additionally need for your project to run. All the dependencies are located in one global location (for instance, `~/.pnpm-store/` - you can check it by running `pnpm store path` in the terminal) called “a content-addressable store”. In your project’s `node_modules` folder there is a `.pnpm` file that contains the “virtual store” with many so-called “[hard link](https://pnpm.io/faq#why-does-my-node_modules-folder-use-disk-space-if-packages-are-stored-in-a-global-store)s”. it creates one hard link for each file of each package. [I like this explanation that pnpm docs offer](https://pnpm.io/faq#why-does-my-node_modules-folder-use-disk-space-if-packages-are-stored-in-a-global-store):

> So, for example, if you have `foo` in your project as a dependency and it occupies 1MB of space, then it will look like it occupies 1MB of space in the project's `node_modules` folder and the same amount of space in the global store. However, that 1MB is the same space on the disk addressed from two different locations. So in total `foo` occupies 1MB, not 2MB.

This makes the `node_modules` folder more like a portal (like the wardrobe in Narnia) to files located in different nooks of the global store, and not like a bloated storage unit. This diagram illustrates the strategy employed by pnpm:

![A graph explaining how pnpm store works](https://pbs.twimg.com/media/FGx92b4aUAEAfn5?format=jpg&name=4096x4096)

But, does pnpm “know” if there are duplicate files between two versions of the same package? Yes, because, just like git, it identifies the files by a hash id (called also “content integrity” or “checksum”) and not by the filename. This means that two same files will have identical hash id and pnpm will determine that there’s no reason for duplication.

### pnpm has got your back

There are so many ways that pnpm looks out for you. One of them is that it is impossible to [invite silly bugs](https://www.kochan.io/nodejs/pnpms-strictness-helps-to-avoid-silly-bugs.html) by trying to use modules that are not directly specified in the project's `package.json` but, for instance, are required by your project’s dependencies. While it’s not the end of the world if everything works well, what happens if your project’s dependency no longer requires one of the packages and as a result it is no longer available in the the `node_modules` folder? Well, everything crashes and you don’t even know why.

Such bugs may happen in npm and Yarn Classic because of the flat `node_modules` directory because during installation, they hoist (elevate) all of the packages to the `node_modules` , regardless of whether they are directly required by your app or not. As a result, your project gets access to distant “relatives” of its dependencies.

### …but what if my project doesn’t support symbolic links?

And even if your project presents an edge case and doesn’t work well with symbolic linking, worry not! You can still use pnpm but set it to an npm-like mode. While it will not be space-efficient, it will still be faster.

## But… what about Yarn?

There are two editions of Yarn:

- “Yarn Classic”, which comprises versions below v2 and is no longer maintained,
- “[Yarn Berry](https://yarnpkg.com/)”, which is v2 and higher.

Yarn Classic operates similarly to npm with regards to managing the `node_modules` folder. Yarn Berry, on the other hand, offers three solutions:

1. the npm-like mode
2. the “Plug'n'Play” mode
3. the pnpm-like mode, which uses hard links to reduce the disc space ([not available by default](https://yarnpkg.com/configuration/yarnrc#nodeLinker) and not thoroughly documented)

The [Plug’n’Play seems like an intriguing option](https://yarnpkg.com/features/pnp#the-node_modules-problem) but it has been met with the community’s apprehension as the `.zip` files are not so easily accessible during the dev workflow. The earlier-mentioned [post by TakeShape](https://www.takeshape.io/articles/why-we-switched-from-yarn-to-pnpm/) explains some of the other challenges.

## Closing remarks

Considering how space-efficient and fast pnpm is, it is not a surprise that more and more projects make a move towards it and that its popularity is rapidly growing. This year, the weekly download rate for pnpm was [seven times as high as last year](https://npm-stat.com/charts.html?package=pnpm&package=yarn&package=npm&from=2021-01-01&to=2022-09-20)!

If you’re interested in learning about why projects choose to switch to pnpm, check out these posts by [Gestalt](https://gestaltjs.org/docs/contributors/decision-record/2022-09-01-pnpm/) and [TakeShape](https://www.takeshape.io/articles/why-we-switched-from-yarn-to-pnpm/).

## Further reading

Here are further readings that may give you a better understanding of the current landscape of package managers:

- [feature comparison](https://pnpm.io/feature-comparison) between the three package managers
- [an in-depth comparison](https://blog.logrocket.com/javascript-package-managers-compared/) of the three package managers by LogRocket
- package managers speed benchmarks on the [pnpm page](https://pnpm.io/benchmarks) and [yarn page](https://p.datadoghq.eu/sb/d2wdprp9uki7gfks-c562c42f4dfd0ade4885690fa719c818?tpl_var_npm%5B0%5D=%2A&tpl_var_pnpm%5B0%5D=%2A&tpl_var_yarn-classic%5B0%5D=%2A&tpl_var_yarn-modern%5B0%5D=%2A&tpl_var_yarn-nm%5B0%5D=%2A&tpl_var_yarn-pnpm%5B0%5D=no&from_ts=1664821962982&to_ts=1665426762982&live=true)

---

## Come work with us on package managers!

While you’re here, an announcement: **we are looking for a new team member who will work on package managers including pnpm, npm, yarn, and our own [Turbo](https://developer.stackblitz.com/docs/platform/turbo/)**. Interested? Apply today or reach out to me on Twitter!