JavaScript's ecosystem is uniquely paranoid
Tags: javascript, node, npm, opinion
Published: 2020-04-26 04:22:47

---
title: JavaScript's ecosystem is uniquely paranoid
published: true
description: Why is NPM so full of one-liners and other seemingly-trivial packages? Paranoia.
tags: ['javascript', 'node', 'nodejs', 'npm', 'opinion']
canonical_url: https://listed.to/@crabmusket/14061/javascript-s-ecosystem-is-uniquely-paranoid
---
Another week, another [NPM-related snafu](https://github.com/then/is-promise/issues/13). Why does this keep happening to the JavaScript ecosystem? The answer is paranoia. 😱

---

Many are quick to assert that JavaScript just has a too-low barrier to entry and the n00bs are messing it up. Whenever anyone says "JavaScript is the new PHP!" this is probably what they mean. I don't feel the need to provide evidence against this claim; I think it comes from an understandable frustration, expressed through all-too-common tech elitism.

Others say we should blame resume-driven-development, and the ego boost of having published thousands of open-source modules. We must all suffer, the argument goes, because a few obsessive individuals want to be able to boast about how critical _they personally_ are to the JavaScript ecosystem. While this is probably a real trend, why isn't it more prevalent in other open-source ecosystems?

> **Disclaimer before proceeding**: I use JS every day, and I actually really like it. I'm not trying to criticise it, just to explore some unique problems it has. I hope this post doesn't come across as too harsh.

There are probably _many_ contributing factors that have shaped NPM into what it is today. However, I assert that the _underlying_ reason for the bizarre profusion of tiny, absurd-seeming one-liner packages on NPM is **paranoia**, caused by a _unique combination_ of factors.

## JavaScript makes you paranoid

Three factors have caused a widespread cultural paranoia among JavaScript developers. This has been inculcated over years. These factors are: JavaScript's weak dynamic type system; the diversity of runtimes JavaScript targets; and the fact of deploying software on the web.

### 1. Weak dynamic typing

It's well-known that JavaScript's "type system" leaves a lot to be desired. [This well-known talk](https://www.destroyallsoftware.com/talks/wat) is a humourous take on some of the many ways you can shoot yourself in the foot in JavaScript.

Unless your team (and every open-source package your team depends on) always uses `===`, knows _exactly_ when `typeof` is acceptable, is good at defensive programming, and designs APIs that have good type discipline*, you've probably been tripped up by a string that behaved like a number, a 0 that was skipped for being falsy, an `undefined` turning up somewhere surprising, `typeof null === 'object'`, etcetera.

This isn't _entirely_ unique to JavaScript - many languages have dynamic types, and many languages have weak types and implicit coercions. But I would argue JavaScript is quite a dire example. And this is still an important contributing factor, without which the second factor probably wouldn't be as significant.

\*Or, you are TypeScript users. See Appendix 3.

### 2. Browser runtimes

It's not just the case that "JavaScript is missing a standard library". For example, there is a really easy and straightforward "standard" way to check if an object is an array: `thing instanceof Array`.

But wait! Enter the `iframe`! If the array came from a different context, this check will fail, because the `iframe`'s `Array` constructor is a different object from the parent window's `Array`. Do you _really_ know where that value came from?

Enter `Array.isArray` to save the day! But wait! What if your code needs to run in an older browser which doesn't support `isArray`? Is your transpilation+polyfill pipeline reliable enough to handle this? What do you _mean_ you're not using `babel-env-preset` or whatever the package is called now? This is the downfall of many a well-intentioned addition to JavaScript's standard library (like `String.padStart`).

Having to deal with an extreme diversity of runtimes seems unique to JavaScript among mainstream languages. This could be my bias showing (I'm primarily a web developer), but it's certainly true of the difference between web frontend code and web backend code. You just _never know_ where your code is going to run - in Internet Explorer 8, on Opera for Android, or someone's old version of Safari on their iPhone 5 they're clinging to because it would be too expensive to upgrade.

This is bad enough for application developers, who can to _some_ extent draw a line and decide _not_ to support users in certain demographics. (Or, in Kogan's case, [charge those users more](https://www.kogan.com/au/blog/new-internet-explorer-7-tax/).) But it's a nightmare for library developers, who want to make their code usable by as many _other developers_ as possible.

### 3. Bundle size

Do you remember a few months ago when the internet joined in a collective hate-on for the `is-buffer` package? This package, as its name suggests, checks whether something is a Buffer.

Why would one need a package for that? Well, weak typing might make one want to check types like this; moving targets in the runtime might make one worry that one doesn't know _how_ to check the type reliably - but still, why doesn't one just depend on the buffer package?

Enter the final triumvir of this unholy alliance: _bundle size paranoia_, which was ostensibly [the reason](https://github.com/feross/is-buffer/blob/e5a3b31419315bf954f9a7fab2d0d2fceb818f21/README.md#why-not-use-bufferisbuffer) the `is-buffer` package was created. Because JavaScript programs have to be downloaded _frequently_ by users (even multiple times by the same user on the same day, if caching isn't used carefully), and because Google has convinced us that milliseconds of additional page load time will have _dire_ consequences for our users and consequently for our bank accounts, _and because_ bundlers and module systems have not provided adequate support for modularity, we web developers go to _extreme lengths_ to avoid shipping unnecessary bytes to our users.

When the unit of modularity is "NPM package", rather than "file" or even "function", some will go to great lengths to [split their code across NPM packages](https://www.npmjs.com/search?q=keywords:lodash-modularized). (For more on this, see Appendix 1.) This works with old bundlers that can't tree-shake, and it _can_ avoid reuse - though as noted by the [lodash project itself](https://lodash.com/per-method-packages), they are thankfully moving away from this pattern because it may introduce more opportunities to _duplicate_ code than to _deduplicate_ it!

A huge amount of effort has been poured into not just minifying a source bundle, but producing the best possible bundle in the first place. The NPM ecosystem as it stands today has been shaped in part by these efforts.

## Future proof

These three factors combine and interact in unexpected and awful ways.

Over the years there has been rapid evolution in both frontend frameworks and backend JavaScript, high turnover in bundlers and best-practises. This has metastasized into a culture of uncertainty, an air of paranoia, and an extreme profusion of small packages. Reinventing the wheel can sometimes be good - but would you really bother doing it if you had to learn all the arcane bullshit of browser evolution, IE8 compatibility, implementation bugs, etc. _ad infinitum_?

And it's not just that you don't understand how things work _now_, or how they _used_ to work - but that they'll change in the future!

Whenever NPM's package culture is discussed, one of the benefits touted is that if one of your dependencies is ever updated, your own code will now be updated "for free"! Your application will remain correct, because it depends on an abstraction that will remain correct. (Abstractions are good, but see Appendix 2.)

This is a very reasonable expectation, and an important piece of progress in software development. But I believe the paranoia created by the three factors I discussed above have led to the excesses we see in the current NPM ecosystem. This is why we have [is-even](https://www.npmjs.com/package/is-even) and its whole ludicrous web of dependencies, and why we don't have `is-even` in Python.

"Surely," the rational developer exclaims, "there could be no future changes to the `is-even` package. The definition of even numbers isn't going to change any time soon!"

No, the definition of even numbers won't ever change. But sadly, my friend, this is JavaScript - and you can _never really be sure._

---

### Appendix 1. In praise of modules

My thoughts on this issue have been brewing for a while, but [this comment by Sindre Sorhus](https://github.com/sindresorhus/ama/issues/10#issuecomment-117766328), noted small-package developer, really put it all in focus for me.

Sindre makes a very good argument in favour of modules:

> tl;dr You make small focused modules for reusability and to make it possible to build larger more advanced things that are easier to reason about.

However, this is not an argument in favour of _NPM packages_. All the benefits Sindre lists could be achieved by simply designing programs in a modular way. If another developer wants to avoid having to re-implement an interesting but not-entirely-trivial piece of functionality, they should be able to lift a well-defined module (ideally a [single file](https://github.com/kazzkiq/darkmode/blob/2312fdcf4ac49721a6ec0e4727106ba8d2e485f7/src/index.ts)) from one project to another.

A lot of the issues with NPM are caused by... well, NPM, not by some inherent property of small modules. This was the case for last week's `is-promise` debacle (which precipitated me actually writing this blog post).  _Small NPM packages_ are the "problem", not small modules, and the problem, at its root, is caused by paranoia.

### Appendix 2. The meaning of abstractions

What's wrong with this code?

```javascript
const isPromise = require('is-promise');

if (isPromise(thing)) {
  thing.then(successCallback).catch(failureCallback);
}
 ```

(It's from a real application that uses `is-promise`, but I won't name names.)

Did you spot it? `catch` might be undefined. Why? `is-promise` implements the [Promises/A+ spec](https://promisesaplus.com/), which only requires a `then` method. The specific meaning of "is `thing` a promise?" can actually change based on _how you want to use the answer_. The "promise" is not a reliable abstraction here, because JavaScript has so many versions of it, and because promises can be used in many ways.

This is slightly tangential to the paranoia discussed above, but is an outcome of a "don't ask" approach to packages ("don't ask" because the details will horrify you), and probably not unique to JavaScript.

> **UPDATE:** apparently even the package maintainer [failed to notice the distinction](https://github.com/then/is-promise/pull/40) between a `Promise` and something that has a `then` method. This stuff is not trivial.

The pattern of doing this kind of typecheck is all-too-prevalent in the JS ecosystem, which privileges APIs that seem "simple" because you can chuck anything you want into them, but pushes the burden of being compatible with every conceivable input onto the library. Which brings me to my next appendix...

### Appendix 3. TypeScript

Is there a solution to all this? How can we stop the madness?

I don't believe TypeScript is a _solution_. If anything, it's a clear _symptom_ of the problem. But I believe that TypeScript helps do something important: *it makes poorly-typed code annoying to write*.

Yes, you [can design](https://www.typescriptlang.org/docs/handbook/functions.html#overloads) a method that accepts anything from a `string` to a `then`able that will return an object containing a `Float64Array`, but writing the type of that method becomes _ugly_, and implementing it becomes a _pain_ because TypeScript forces you to demonstrate to its satisfaction that you've done it correctly.

Fewer APIs that take and return different types make it less necessary to implement code like `is-buffer`, `is-number`, etcetera. Of course, browser compatiblity and bundle size anxiety will still present problems. But maybe with an increase in JavaScript developers designing code with types, we'll see less demand for typecheck packages and the like.

### Appendix 4. Deno

One of the reasons I'm excited for [Deno](https://deno.land)'s upcoming stable release is that it builds on a philosophy of _fewer, better dependencies_. But even in cases where you need a specific dependency, Deno's URL-based imports make it trivial to:

- Import _just a single file_ without downloading a whole package plus its tests and everything else. Refer back to Appendix 1 for why this is cool.

- Pin each import _to a commit hash_ or other stable identifier.

Yes, many people are concerned about the idea of importing URLs for many legitimate reasons. NPM is a more trusted place to host packages than some random website. But not even NPM can be 100% reliable indefinitely. Deno at least makes you stop and think... _do I trust this source?_