Repository: ProjectEvergreen/greenwood (js)

Issue:
node resolution not handling flat export maps for files and transient dependency imports
## Type of Change
- [ ] New Feature Request
- [ ] Documentation / Website
- [ ] Improvement / Suggestion
- [x] Bug
- [ ] Other (please clarify below)

## Summary
Doing some discovery as part of #523 to validate Greenwood works correctly with a variety of frameworks and libraries, have a couple more improvements to update here regarding integration work in progress with Lion -https://github.com/ProjectEvergreen/greenwood-getting-started/pull/45.

## Details
### "Flat" Export Map
Was getting an error that one Lion's [export maps entries (`./define`)](https://unpkg.com/browse/@lion/calendar@0.16.5/package.json) was not resolving correctly
```
es-module-shims.js:469 Uncaught (in promise) Error: Unable to resolve specifier '@lion/button/define' from http://localhost:1984/?0
    at throwUnresolved (es-module-shims.js:469)
    at resolve (es-module-shims.js:465)
    at es-module-shims.js:424
    at Array.map (<anonymous>)
    at es-module-shims.js:423
    at async loadAll (es-module-shims.js:237)
    at async topLevelLoad (es-module-shims.js:256)
```
![Screen Shot 2021-09-06 at 10 52 23 AM](https://user-images.githubusercontent.com/895923/132234945-854335d0-b671-419c-95bb-7855ac36ffd3.png)

I think the issue is I was not correctly handling "flat" export maps, e.g.

[Lit](https://unpkg.com/browse/lit@2.0.0-rc.4/package.json) - each entry is an object
```json
"exports": {
  ".": {
    "default": "./index.js"
  },
  "./async-directive.js": {
    "default": "./async-directive.js"
  },
  "./decorators.js": {
    "default": "./decorators.js"
  },
  "./decorators/custom-element.js": {
    "default": "./decorators/custom-element.js"
  },

  ...
}
```

[Lion](https://unpkg.com/browse/@lion/calendar@0.16.5/package.json) - each entry is a string
```json
"exports": {
  ".": "./index.js",
  "./define": "./lion-calendar.js",
  "./translations/*": "./translations/*",
  "./test-helpers": "./test-helpers/index.js",
  "./docs/": "./docs/"
}  
```

### Transient Imports
After fixing the above, I then saw that an [`import`](https://unpkg.com/browse/@lion/localize@0.21.1/src/LocalizeManager.js) of a [dependent package](https://unpkg.com/browse/@bundled-es-modules/message-format) was not correctly getting updated in the import map when walking a module.  
```sh
es-module-shims.js:469 Uncaught (in promise) Error: Unable to resolve specifier '@bundled-es-modules/message-format/MessageFormat.js' from http://localhost:1984/node_modules/@lion/localize/src/LocalizeManager.js
    at throwUnresolved (es-module-shims.js:469)
    at resolve (es-module-shims.js:465)
    at es-module-shims.js:424
    at Array.map (<anonymous>)
    at es-module-shims.js:423
```

![Screen Shot 2021-09-06 at 10 54 48 AM](https://user-images.githubusercontent.com/895923/132235736-0880707e-e764-4213-8968-e7af2cd96080.png)

> Not sure if this is just bad export map reading on my end, but presumably all `import` statements needs to be registered in the import map somewhere, right?

----

There might be some other fixes needed as part of working with MWC and the [double scoped namespace bug seen](https://github.com/ProjectEvergreen/greenwood-getting-started/pull/42), but will save that for another issue / PR.

Pull request description:
handle missed cases for files in exports maps and transient dependency imports
<!--
## Submitting a Pull Request
We love contributions and appreciate any help you can offer!
-->

## Related Issue
resolves #715 

## Summary of Changes
1. Update import map if package "flat" export map entry is a file
1. Add transient dependency imports to import map

## TODO
1. [x] Confirm interop with #611 work
1. [x] Add tests for both cases to develop spec
1. [x] Validate in [Getting Started branches](https://github.com/ProjectEvergreen/greenwood-getting-started/pulls?q=is%3Apr+label%3Aexample+label%3A%22needs+upstream%22) after initial [`v0.16.0-alpha.0`](https://github.com/ProjectEvergreen/greenwood/issues?q=label%3Av0.16.0+label%3Aalpha.0) release, then I will undraft here
1. [x] Figure out what to do about https://github.com/ProjectEvergreen/greenwood/pull/716#discussion_r706357210 - re-opened - https://github.com/ProjectEvergreen/greenwood/issues/557#issuecomment-918414861

Patch:
diff --git a/packages/cli/package.json b/packages/cli/package.json
index a6e01db3..2c28fba8 100644
--- a/packages/cli/package.json
+++ b/packages/cli/package.json
@@ -49,6 +49,8 @@
     "unified": "^9.2.0"
   },
   "devDependencies": {
+    "@lion/button": "~0.12.0",
+    "@lion/calendar": "~0.15.0",
     "@mapbox/rehype-prism": "^0.5.0",
     "lit-element": "^2.4.0",
     "lit-redux-router": "^0.17.1",
diff --git a/packages/cli/src/plugins/resource/plugin-node-modules.js b/packages/cli/src/plugins/resource/plugin-node-modules.js
index 90932322..dbabd83e 100644
--- a/packages/cli/src/plugins/resource/plugin-node-modules.js
+++ b/packages/cli/src/plugins/resource/plugin-node-modules.js
@@ -67,6 +67,7 @@ const walkModule = (module, dependency) => {
         if (fs.existsSync(path.join(process.cwd(), 'node_modules', dependency, sourceValue))) {
           const moduleContents = fs.readFileSync(path.join(process.cwd(), 'node_modules', dependency, sourceValue));
           walkModule(moduleContents, dependency);
+          updateImportMap(`${dependency}/${sourceValue.replace('./', '')}`, `/node_modules/${dependency}/${sourceValue.replace('./', '')}`);
         }
       }
     },
@@ -158,14 +159,13 @@ const walkPackageJson = (packageJson = {}) => {
   
           if (packageExport) {
             const packageExportLocation = path.join(process.cwd(), 'node_modules', `${dependency}/${packageExport.replace('./', '')}`);
-
             // check all exports of an exportMap entry
             // to make sure those deps get added to the importMap
             if (packageExport.endsWith('js')) {
               const moduleContents = fs.readFileSync(packageExportLocation);
 
               walkModule(moduleContents, dependency);
-              updateImportMap(`${dependency}/${packageExport.replace('./', '')}`, `/node_modules/${dependency}/${packageExport.replace('./', '')}`);
+              updateImportMap(`${dependency}${entry.replace('.', '')}`, `/node_modules/${dependency}/${packageExport.replace('./', '')}`);
             } else if (fs.lstatSync(packageExportLocation).isDirectory()) {
               fs.readdirSync(packageExportLocation)
                 .filter(file => file.endsWith('.js') || file.endsWith('.mjs'))
@@ -181,7 +181,6 @@ const walkPackageJson = (packageJson = {}) => {
         walkPackageJson(dependencyPackageJson);
       } else {
         const packageEntryPointPath = path.join(process.cwd(), './node_modules', dependency, entry);
-        
         // sometimes a main file is actually just an empty string... :/
         if (fs.existsSync(packageEntryPointPath)) {
           const packageEntryModule = fs.readFileSync(packageEntryPointPath, 'utf-8');
diff --git a/yarn.lock b/yarn.lock
index 552ffc60..c12af244 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -1222,6 +1222,11 @@
     "@babel/helper-validator-identifier" "^7.14.0"
     to-fast-properties "^2.0.0"
 
+"@bundled-es-modules/message-format@6.0.4":
+  version "6.0.4"
+  resolved "https://registry.yarnpkg.com/@bundled-es-modules/message-format/-/message-format-6.0.4.tgz#232da6877adb960f6c8f598c72588a84352de147"
+  integrity sha512-NGUoPxqsBzDwvRhY3A3L/AhS1hzS9OWappfyDOyCwE7G3W4ua28gau7QwvJz7QzA6ArbAdeb8c1mLjvd1WUFAA==
+
 "@csstools/convert-colors@^1.4.0":
   version "1.4.0"
   resolved "https://registry.yarnpkg.com/@csstools/convert-colors/-/convert-colors-1.4.0.tgz#ad495dc41b12e75d588c6db8b9834f08fa131eb7"
@@ -2005,6 +2010,40 @@
     npmlog "^4.1.2"
     write-file-atomic "^2.3.0"
 
+"@lion/button@~0.12.0":
+  version "0.12.0"
+  resolved "https://registry.yarnpkg.com/@lion/button/-/button-0.12.0.tgz#e93d3772bf4f3384b76598b3d58efd9a9f18b897"
+  integrity sha512-kJ/NkSrnc6hPEtV7ICSGx9cH+H/NiBjSUoEPR6pp3+MQTzEwIfoSEPBe88xMflRLqnrq8GhjPKdhH8Ce/G5yRw==
+  dependencies:
+    "@lion/core" "0.17.0"
+
+"@lion/calendar@~0.15.0":
+  version "0.15.0"
+  resolved "https://registry.yarnpkg.com/@lion/calendar/-/calendar-0.15.0.tgz#9655448250805fd4247b1d1dad003b48682d6407"
+  integrity sha512-MaIZpxuF+tUAwPejI5/bns8VbJXftbAABk+i247fnUvI0Xi4tu0agKE2A13o9RXKQLm2IY/ohLWVjAffBuM9+A==
+  dependencies:
+    "@lion/core" "0.17.0"
+    "@lion/localize" "0.19.0"
+
+"@lion/core@0.17.0":
+  version "0.17.0"
+  resolved "https://registry.yarnpkg.com/@lion/core/-/core-0.17.0.tgz#0af5d7873c1e110fd57d54c331fb22dd01702bb3"
+  integrity sha512-s4z7qxzX+bgMPyxRiZcLuBd3BePPvQRkgTlUxKf+4ao0QBgebuNyoFHUWvZifuWH2gFmcORy3aL7+0HYNMkqoQ==
+  dependencies:
+    "@open-wc/dedupe-mixin" "^1.2.18"
+    "@open-wc/scoped-elements" "^1.3.3"
+    lit-element "~2.4.0"
+    lit-html "^1.3.0"
+
+"@lion/localize@0.19.0":
+  version "0.19.0"
+  resolved "https://registry.yarnpkg.com/@lion/localize/-/localize-0.19.0.tgz#7df884554f1a14d9184241c20bc1939139c566d3"
+  integrity sha512-zkFu5j2eiZYxtmweM96psO7rvzMZqdaX6/+wQwABKGmKdg02I4h1kwdfFvRw5MWZNiFlftyxES/yMY/Q2VJZuA==
+  dependencies:
+    "@bundled-es-modules/message-format" "6.0.4"
+    "@lion/core" "0.17.0"
+    singleton-manager "1.4.2"
+
 "@ls-lint/ls-lint@^1.9.2":
   version "1.9.2"
   resolved "https://registry.yarnpkg.com/@ls-lint/ls-lint/-/ls-lint-1.9.2.tgz#689f1f4c06072823a726802ba167340efcefe19c"
@@ -2162,6 +2201,19 @@
   dependencies:
     "@types/node" ">= 8"
 
+"@open-wc/dedupe-mixin@^1.2.18", "@open-wc/dedupe-mixin@^1.3.0":
+  version "1.3.0"
+  resolved "https://registry.yarnpkg.com/@open-wc/dedupe-mixin/-/dedupe-mixin-1.3.0.tgz#0df5d438285fc3482838786ee81895318f0ff778"
+  integrity sha512-UfdK1MPnR6T7f3svzzYBfu3qBkkZ/KsPhcpc3JYhsUY4hbpwNF9wEQtD4Z+/mRqMTJrKg++YSxIxE0FBhY3RIw==
+
+"@open-wc/scoped-elements@^1.3.3":
+  version "1.3.4"
+  resolved "https://registry.yarnpkg.com/@open-wc/scoped-elements/-/scoped-elements-1.3.4.tgz#96e27e13c8b09668ee631e9fccd5623a05af5cc2"
+  integrity sha512-WD+ObocdzcFCpBxnc8bQa7NoATeA+tJrK0/c/yV1Nx4leV+1PmJNNu+WCcuckBEGd0Op6FP8w1TidoqmVVba6g==
+  dependencies:
+    "@open-wc/dedupe-mixin" "^1.3.0"
+    lit-html "^1.0.0"
+
 "@protobufjs/aspromise@^1.1.1", "@protobufjs/aspromise@^1.1.2":
   version "1.1.2"
   resolved "https://registry.yarnpkg.com/@protobufjs/aspromise/-/aspromise-1.1.2.tgz#9b8b0cc663d669a7d8f6f5d0893a14d348f30fbf"
@@ -7429,14 +7481,14 @@ list-item@^1.1.1:
     is-number "^2.1.0"
     repeat-string "^1.5.2"
 
-lit-element@^2.0.1, lit-element@^2.4.0:
+lit-element@^2.0.1, lit-element@^2.4.0, lit-element@~2.4.0:
   version "2.4.0"
   resolved "https://registry.yarnpkg.com/lit-element/-/lit-element-2.4.0.tgz#b22607a037a8fc08f5a80736dddf7f3f5d401452"
   integrity sha512-pBGLglxyhq/Prk2H91nA0KByq/hx/wssJBQFiYqXhGDvEnY31PRGYf1RglVzyLeRysu0IHm2K0P196uLLWmwFg==
   dependencies:
     lit-html "^1.1.1"
 
-lit-html@^1.1.1:
+lit-html@^1.0.0, lit-html@^1.1.1, lit-html@^1.3.0:
   version "1.3.0"
   resolved "https://registry.yarnpkg.com/lit-html/-/lit-html-1.3.0.tgz#c80f3cc5793a6dea6c07172be90a70ab20e56034"
   integrity sha512-0Q1bwmaFH9O14vycPHw8C/IeHMk/uSDldVLIefu/kfbTBGIc44KGH6A8p1bDfxUfHdc8q6Ct7kQklWoHgr4t1Q==
@@ -10813,6 +10865,11 @@ simpledotcss@^1.0.0:
   resolved "https://registry.yarnpkg.com/simpledotcss/-/simpledotcss-1.0.0.tgz#b5ac8fcd73d111938def9c0f43f1839d6eae96be"
   integrity sha512-inneK6Wqh5rUdASxJMAKJORbDeKRySZPlY6H1Jqc6o6t5QbXhd5UZUi5snkKnB0ror0H72in0zlNeeA4Vw+Fiw==
 
+singleton-manager@1.4.2:
+  version "1.4.2"
+  resolved "https://registry.yarnpkg.com/singleton-manager/-/singleton-manager-1.4.2.tgz#4649acafca3eccf987d828ab16369ee59c4a22a5"
+  integrity sha512-3/K7K61TiN0+tw32HRC3AZQBacN0Ky/NmHEnhofFPEFROqZ5T6BXK45Z94OQsvuFD2euOVOU40XDNeTal63Baw==
+
 slash@^2.0.0:
   version "2.0.0"
   resolved "https://registry.yarnpkg.com/slash/-/slash-2.0.0.tgz#de552851a1759df3a8f206535442f5ec4ddeab44"


Test patch:
diff --git a/packages/cli/test/cases/develop.default/develop.default.spec.js b/packages/cli/test/cases/develop.default/develop.default.spec.js
index 446aa1bc..92def9e8 100644
--- a/packages/cli/test/cases/develop.default/develop.default.spec.js
+++ b/packages/cli/test/cases/develop.default/develop.default.spec.js
@@ -26,7 +26,7 @@
  *     index.html
  *   styles/
  *     main.css
- *
+ * package.json
  */
 const expect = require('chai').expect;
 const { JSDOM } = require('jsdom');
@@ -90,6 +90,82 @@ describe('Develop Greenwood With: ', function() {
         `${process.cwd()}/node_modules/simpledotcss/package.json`,
         `${outputPath}/node_modules/simpledotcss/`
       );
+      const lionButtonLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/button/*.js`,
+        `${outputPath}/node_modules/@lion/button/`
+      );
+      const lionButtonLibsPackageJson = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/button/package.json`,
+        `${outputPath}/node_modules/@lion/button/`
+      );
+      const lionCoreTesterLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/core/test-helpers/*.js`,
+        `${outputPath}/node_modules/@lion/core/test-helpers/`
+      );
+      const lionCoreSrcLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/core/src/*.js`,
+        `${outputPath}/node_modules/@lion/core/src/`
+      );
+      const lionCoreLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/core/*.js`,
+        `${outputPath}/node_modules/@lion/core/`
+      );
+      const lionCoreLibsPackageJson = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/core/package.json`,
+        `${outputPath}/node_modules/@lion/core/`
+      );
+      const lionCalendarLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/calendar/*.js`,
+        `${outputPath}/node_modules/@lion/calendar/`
+      );
+      const lionCalendarLibsPackageJson = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/calendar/package.json`,
+        `${outputPath}/node_modules/@lion/calendar/`
+      );
+      const lionCalendarTesterLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/calendar/test-helpers/*.js`,
+        `${outputPath}/node_modules/@lion/calendar/test-helpers/`
+      );
+      const lionLocalizeLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/localize/*.js`,
+        `${outputPath}/node_modules/@lion/localize/`
+      );
+      const lionLocalizeLibsPackageJson = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/localize/package.json`,
+        `${outputPath}/node_modules/@lion/localize/`
+      );
+      const lionLocalizeTesterLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/localize/test-helpers/*.js`,
+        `${outputPath}/node_modules/@lion/localize/test-helpers/`
+      );
+      const lionLocalizeSrcLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@lion/localize/src/*.js`,
+        `${outputPath}/node_modules/@lion/localize/src/`
+      );
+      const owcDepupLibPackageJson = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@open-wc/dedupe-mixin/package.json`,
+        `${outputPath}/node_modules/@open-wc/dedupe-mixin/`
+      );
+      const owcScopedLibPackageJson = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@open-wc/scoped-elements/package.json`,
+        `${outputPath}/node_modules/@open-wc/scoped-elements/`
+      );
+      const messageFormatLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@bundled-es-modules/message-format/*.js`,
+        `${outputPath}/node_modules/@bundled-es-modules/message-format/`
+      );
+      const messageFormatLibsPackageJson = await getDependencyFiles(
+        `${process.cwd()}/node_modules/@bundled-es-modules/message-format/package.json`,
+        `${outputPath}/node_modules/@bundled-es-modules/message-format/`
+      );
+      const singletonManagerLibs = await getDependencyFiles(
+        `${process.cwd()}/node_modules/singleton-manager/index.js`,
+        `${outputPath}/node_modules/singleton-manager/`
+      );
+      const singletonManagerLibsPackageJson = await getDependencyFiles(
+        `${process.cwd()}/node_modules/singleton-manager/package.json`,
+        `${outputPath}/node_modules/singleton-manager/`
+      );
 
       await runner.setup(outputPath, [
         ...getSetupFiles(outputPath),
@@ -101,7 +177,26 @@ describe('Develop Greenwood With: ', function() {
         ...litHtmlLibs,
         ...litHtmlSourceMap,
         ...simpleCss,
-        ...simpleCssPackageJson
+        ...simpleCssPackageJson,
+        ...lionButtonLibs,
+        ...lionButtonLibsPackageJson,
+        ...lionCoreLibs,
+        ...lionCoreTesterLibs,
+        ...lionCoreLibsPackageJson,
+        ...lionCoreSrcLibs,
+        ...lionCalendarLibs,
+        ...lionCalendarLibsPackageJson,
+        ...lionCalendarTesterLibs,
+        ...lionLocalizeLibs,
+        ...lionLocalizeLibsPackageJson,
+        ...lionLocalizeTesterLibs,
+        ...lionLocalizeSrcLibs,
+        ...owcDepupLibPackageJson,
+        ...owcScopedLibPackageJson,
+        ...messageFormatLibs,
+        ...messageFormatLibsPackageJson,
+        ...singletonManagerLibsPackageJson,
+        ...singletonManagerLibs
       ]);
 
       return new Promise(async (resolve) => {
@@ -159,6 +254,15 @@ describe('Develop Greenwood With: ', function() {
         expect(importMap['lit-html/lit-html.js']).to.equal('/node_modules/lit-html/lit-html.js');
         expect(importMap['lit-html/lib/shady-render.js']).to.equal('/node_modules/lit-html/lib/shady-render.js');
 
+        // https://github.com/ProjectEvergreen/greenwood/issues/715
+        // export maps with "flat" entries
+        expect(importMap['@lion/button']).to.equal('/node_modules/@lion/button/index.js');
+        expect(importMap['@lion/button/define']).to.equal('/node_modules/@lion/button/lion-button.js');
+
+        // https://github.com/ProjectEvergreen/greenwood/issues/715
+        // transient dependency import / exports
+        expect(importMap['@bundled-es-modules/message-format/MessageFormat.js']).to.equal('/node_modules/@bundled-es-modules/message-format/MessageFormat.js');
+
         done();
       });
 
@@ -565,6 +669,77 @@ describe('Develop Greenwood With: ', function() {
       });
     });
 
+    // https://github.com/ProjectEvergreen/greenwood/issues/715
+    describe('Develop command node_modules resolution for a transient dependency\'s own imports', function() {
+      let response = {};
+
+      before(async function() {
+        return new Promise((resolve, reject) => {
+          request.get(`${hostname}:${port}/node_modules/@bundled-es-modules/message-format/MessageFormat.js`, (err, res, body) => {
+            if (err) {
+              reject();
+            }
+
+            response = res;
+            response.body = body;
+
+            resolve();
+          });
+        });
+      });
+
+      it('should return a 200 status', function(done) {
+        expect(response.statusCode).to.equal(200);
+        done();
+      });
+
+      it('should return the correct content type', function(done) {
+        expect(response.headers['content-type']).to.equal('text/javascript');
+        done();
+      });
+
+      it('should return the correct response body', function(done) {
+        expect(response.body).to.contain('export default messageFormat;');
+        done();
+      });
+    });
+
+    // https://github.com/ProjectEvergreen/greenwood/issues/715
+    // @lion/calendar/define -> /node_modules/@lion/calendar/lion-calendar.js
+    describe('Develop command node_modules resolution for a flat export map entry from a dependency (not import or default)', function() {
+      let response = {};
+
+      before(async function() {
+        return new Promise((resolve, reject) => {
+          request.get(`${hostname}:${port}/node_modules/@lion/calendar/lion-calendar.js`, (err, res, body) => {
+            if (err) {
+              reject();
+            }
+
+            response = res;
+            response.body = body;
+
+            resolve();
+          });
+        });
+      });
+
+      it('should return a 200 status', function(done) {
+        expect(response.statusCode).to.equal(200);
+        done();
+      });
+
+      it('should return the correct content type', function(done) {
+        expect(response.headers['content-type']).to.equal('text/javascript');
+        done();
+      });
+
+      it('should return the correct response body', function(done) {
+        expect(response.body).to.contain('customElements.define(\'lion-calendar\', LionCalendar);');
+        done();
+      });
+    });
+
     // need some better 404 handling here (promise reject handling for assets and routes)
     describe('Develop command with default 404 behavior', function() {
       let response = {};
diff --git a/packages/cli/test/cases/develop.default/package.json b/packages/cli/test/cases/develop.default/package.json
index 0cf4fc01..ef5ad763 100644
--- a/packages/cli/test/cases/develop.default/package.json
+++ b/packages/cli/test/cases/develop.default/package.json
@@ -1,6 +1,8 @@
 {
   "name": "test-develop-command-default",
   "dependencies": {
+    "@lion/button": "~0.12.0",
+    "@lion/calendar": "~0.15.0",
     "lit-element": "^2.4.0"
   }
 }
\ No newline at end of file