Repository: node
Issue: Proposal for single-mode packages with optional fallbacks for older versions of node
Labels: esm

This is a counter-proposal to nodejs/modules#273.

A bunch of us have brought this up during informal dual-mode discussions before, but the very concept of "dual mode" packages which execute different code for cjs vs esm callers is distasteful. It pollutes a single identifier (the package name or specifier) with two _distinct_ and potentially _irreconcilable_ package identities (which themselves potentially contain duplicated identities and dependencies). From anything other than the perspective of the registry, it's effectively shipping two separate packages for a single runtime - one for esm callers and one for cjs callers.

The original desire for so-called "dual mode" packages derives from two uses:
1. The ability to support `cjs` consumers in the newest versions of node.
2. The ability to retain support for older versions of node, while still supporting esm where possible.

In this proposal, I will tackle the two issues _separately_.

First, case 1:
In the current implementation, a esm-authored package cannot be `require`'d. This means that you cut support for all cjs consumers when you migrate to esm. This kind of hard cut is, IMO, obviously undesirable, and both the "dual-mode" proposal and this proposal seek to remedy this. In the "dual-mode" proposal, this is solved by shipping seperate cjs code alongside the esm code, which is loaded instead of the esm code. In this proposal, the _esm code itself is loaded_. This means that when a package specifies that it has an entrypoint that is esm, _it will only ever be loaded as esm_. The astute in the crowd would note that while yes, that's all well and good, the cjs resolver is synchronous, while we've specified that the esm resolver is asynchronous - a seemingly irreconcilable difference. I arrive to tell you something: this is [not so](https://github.com/weswigham/ecmascript-modules/tree/syncify-esm-instantiate). An esm-based require may be executed async, but appear to be synchronous to the cjs caller - similarly to how `child_process.execSync` works today (and, in fact, using similar machinery). This synchronization only affects instantiation and resolution - the execution phase remains untouched, so if at some point in the future top-level await becomes a thing, depending on variant, either the `require` can conditionally return a promise (if TLA is supposed to be blocking) or happily return the exports object while the module is still asynchronously executing. The only other concern would be the observable affects on user-defined loaders, which, if we follow through on that design with out-of-process (or at least out-of-context) loaders (which are very desirable from an isolation perspective), the solution there, likewise, is simply using an apparently synchronous execution of the async loader, just as with the builtin loader. In-process loaders can also be syncified (and in fact are in my current implementation), but care must be taken to not have a user loader depend on a task which is meant to resolve in the main "thread" of execution (since it will not receive an opportunity to execute, as only the child event loop will be turned) - this means relying on a promise made before the loader was called is a no-go. This _shouldn't_ be a problem (the builtin loader, despite being written to allow async actions, is actually fully synchronous in nature), and, again, is fully mitigated by having loaders in a new context (where they cannot possibly directly depend on such a task).

By allowing `esm` to be `require`'d in cjs, we can close the difference between the esm and cjs resolvers. If the cjs resolver can resolve a file, IMO, in the esm resolver it should resolve to the same thing or issue an error - it should _not_ be possible to get two different identities for the same specifier based on the resolver used (this implies that the esm resolver we use should be a subset of or identical to the cjs resolver). This means implementing knowledge of `"type": "module"` and the like into the cjs resolver, since afaik, it's not already there (though it does already reserve .mjs).

And case 2:
With the above, a package can only have one runtime execution associated with it, however that only requires shipping esm. To support older versions of node, a cjs version of a package must be shipped. An answer falls out naturally from extension priorities: Older versions of `node` do not search for the `.mjs` extension. Making your entrypoint a `.mjs`/`.js` pair (with `.js` as `cjs`), the `.mjs` will be found by versions of node which support esm, while the `.js` will be found by older versions of node. This is ofc constrained by what older versions of node already support - there's no "work to be done" to improve this, just work to ensure that it continues to work and that it does not become blocked by some other modification. naturally, this means in a package with a cjs fallback for an older node, the entrypoint cannot be a `.js` `esm` file - however other `esm` in the project can be (by placing a `package.json` with `{"type": "module"}` in whichever project subfolder has the `esm` source beyond the entrypoint). This, too, has been alluded to by many people in many other issues, yet has not yet been written down.

TL;DR:
* Synchronous `require` of `esm` can be done. Doing so allows an `esm` to actually replace a `cjs` one, and alleviates the need for a "dual mode" resolver system. This also _greatly_ aids the migration story (especially when paired with the dynamic modules spec, which allow for more than just the `default` member to overlap during migration).
* Patch the `esm` (and `cjs`) resolver to be a proper subset of the cjs resolver and resolve to the same cache entry or an error for each specifier which can be represented in both (cjs obviously does not respect URLs in any way, so any URL resolution remains solely the domain of the esm loader)
* `cjs` fallbacks for older versions of node come free with the extension searching that older versions of node already do (so long as we allow a higher priority entrypoint to be found in newer versions of node, this method of backcompat comes "free"). Without that, an alternate esm entrypoint is useful. Neither is tied to this, but it's worth mentioning that it's still easily doable.