atom/atom — pull request #3761 by kevinsawicki (2014-10-08 19:03:07+00:00)
Title: Cache requires across installed package

## The Problem

Atom packages installed `~/.atom/packages` do not share any requires with core or the packages bundled with Atom.

For example, imagine a package has a dependency on [underscore](http://underscorejs.org) 1.7.

When that package does a `require('underscore')`, underscore will be required and loaded out of the packages `node_modules` folder even if Atom core has this same dependency and has already required underscore@1.7.0. Two copies of the exact same module are required and loaded instead of them sharing that same exact dependency.

So if underscore 1.7 takes 10 milliseconds to require and 10 packages you've installed are using it, Atom will spend 100 milliseconds at startup requiring 10 identical copies, instead of requiring it once and sharing that copy among the 10 packages.

This has led package authors to do the following performance hacks:

```js
require(path.join(atom.getLoadSettings().resourcePath, 'node_modules', 'underscore'))
```

This requires it directly out of core and so it will only loaded once since the full path is given to `require`.

This performance trick has the downside of packages breaking when Atom core upgrades or removes dependencies.

## The Solution

Patch node's [Module](https://github.com/joyent/node/blob/25702abc5638bbecffa92e8349d2a871a5f592c9/lib/module.js#L321) to allow packages to share modules.

So if a package is dependent on underscore `>= 1.5` and underscore `1.7` has already been loaded by core or another package, that already required underscore version will be used instead of loading a new copy out of the package's `node_modules` folder.

This is accomplished by doing the following:

* Precompute all the dependencies a package ships with when it is installed so Atom can track which version of modules are available across all packages and core.
* Precompute what modules each folder in a package depends on. That way when any file is required in that folder, Atom can immediately know what module versions it needs and return a version from the cache if a compatible version has already been required.

Packages won't have to do anything to take advantage of this, it will be automatically detected during `apm install` and handled transparently.

Packages will still always get a version of the module that satisfies the dependency range listed in their `package.json` file.

The apm changes are in https://github.com/atom/apm/pull/200

## Results

To verify that this approach improved startup time I decided to measure the load and activate time for several packages. These packages were chosen because they are all dependent on at least one module and they have similar dependencies to the modules bundled with Atom and/or each other.

### Before

| Package | Load+Acvitate Time in Milliseconds |
|:--------|----------------------------------:|
| [atom-color-highlight](https://atom.io/packages/atom-color-highlight) | 176 |
| [hex](https://atom.io/packages/hex) | 12 |
| [minimap](https://atom.io/packages/minimap) | 97 |
| [minimap-color-highlight](https://atom.io/packages/minimap-color-highlight) | 108 |
| [minimap-git-diff](https://atom.io/packages/minimap-git-diff) | 17 |
| [minimap-selection](https://atom.io/packages/minimap-selection) | 4 |
| **Total** | **414** |

### After

| Package | Load+Acvitate Time in Milliseconds |
|:--------|----------------------------------:|
| [atom-color-highlight](https://atom.io/packages/atom-color-highlight) | 31 |
| [hex](https://atom.io/packages/hex) | 2 |
| [minimap](https://atom.io/packages/minimap) | 69 |
| [minimap-color-highlight](https://atom.io/packages/minimap-color-highlight) | 7 |
| [minimap-git-diff](https://atom.io/packages/minimap-git-diff) | 4 |
| [minimap-selection](https://atom.io/packages/minimap-selection) | 3 |
| **Total** | **116** |

/cc @abe33 since I used a bunch of your packages in my testing

Closes #3672