Repository: vercel/nft (javascript)

Issue:
Add support for `zeromq`


Some libraries like `zeromq.js` need runtime libraries with bindings which they ship along. Those binaries exist under `node_modules/zeromq/prebuilds`.

Simply by creating a new Next.js app, installing https://github.com/zeromq/zeromq.js and trying to import it and running `npm run dev` and use one of its classes would throw a `No native build was found for platform=linux arch=x64 runtime=node abi=115 uv=1 libc=glibc node=20.11.0 webpack=true loaded from: /app/.next/server`. I found out that if I print [this `dir` variable](https://github.com/aminya/node-gyp-build/blob/4cd1eb0f1ef2ed7d4c36b82e3d547e33cd83c9a0/index.js#L26) used (among other things) to search for binaries in node-gyp-builder it print `.next/server/...` instead of `node_modules/...` and since the binaries have not been copied it fails.

> [!NOTE]
> I provided a reproducible example and a container, but it's easily reproducible on mac and linux: https://github.com/eulersson/zeromq.js-next.js-errors

I tried using the `outputFileTracingIncludes` option as explained in https://nextjs.org/docs/pages/api-reference/next-config-js/output#caveats but without success.

Here's the files I would like to have in `.next/server`.

```
❯ tree node_modules/zeromq/prebuilds
node_modules/zeromq/prebuilds
├── darwin-arm64
│   └── node.napi.glibc.node
├── darwin-x64
│   └── node.napi.glibc.node
├── linux-x64
│   ├── node.napi.glibc.node
│   └── node.napi.musl.node
├── win32-ia32
│   └── node.napi.glibc.node
└── win32-x64
    └── node.napi.glibc.node

6 directories, 6 files
```

Since I don't know in what area work should be done I posted the same problem as an issue to the various tools involved:

- https://github.com/vercel/next.js/issues/61844
- https://github.com/zeromq/zeromq.js/issues/600
- https://github.com/prebuild/node-gyp-build/issues/65

Thanks for the help and apologies if I have been generating noise by posting so many issues.

Add support for `zeromq`


Some libraries like `zeromq.js` need runtime libraries with bindings which they ship along. Those binaries exist under `node_modules/zeromq/prebuilds`.

Simply by creating a new Next.js app, installing https://github.com/zeromq/zeromq.js and trying to import it and running `npm run dev` and use one of its classes would throw a `No native build was found for platform=linux arch=x64 runtime=node abi=115 uv=1 libc=glibc node=20.11.0 webpack=true loaded from: /app/.next/server`. I found out that if I print [this `dir` variable](https://github.com/aminya/node-gyp-build/blob/4cd1eb0f1ef2ed7d4c36b82e3d547e33cd83c9a0/index.js#L26) used (among other things) to search for binaries in node-gyp-builder it print `.next/server/...` instead of `node_modules/...` and since the binaries have not been copied it fails.

> [!NOTE]
> I provided a reproducible example and a container, but it's easily reproducible on mac and linux: https://github.com/eulersson/zeromq.js-next.js-errors

I tried using the `outputFileTracingIncludes` option as explained in https://nextjs.org/docs/pages/api-reference/next-config-js/output#caveats but without success.

Here's the files I would like to have in `.next/server`.

```
❯ tree node_modules/zeromq/prebuilds
node_modules/zeromq/prebuilds
├── darwin-arm64
│   └── node.napi.glibc.node
├── darwin-x64
│   └── node.napi.glibc.node
├── linux-x64
│   ├── node.napi.glibc.node
│   └── node.napi.musl.node
├── win32-ia32
│   └── node.napi.glibc.node
└── win32-x64
    └── node.napi.glibc.node

6 directories, 6 files
```

Since I don't know in what area work should be done I posted the same problem as an issue to the various tools involved:

- https://github.com/vercel/next.js/issues/61844
- https://github.com/zeromq/zeromq.js/issues/600
- https://github.com/prebuild/node-gyp-build/issues/65

Thanks for the help and apologies if I have been generating noise by posting so many issues.



Discussion:
You can try `npx @vercel/nft@latest print path/to/my/file.js` to see which files are included.

If its missing prebuilds, then it is likely an issue with `@vercel/nft`.

You can create a PR with a test and special case, similar to PR https://github.com/vercel/nft/pull/37.

Or if you want to fix the general case (likely a dynamic require somewhere) that would be great too (albeit much harder).
@styfle Many thanks for the guidance. I will investigate that route this afternoon! :)
@styfle Indeed it certainly does not bring in the binaries:

```
❯ npx @vercel/nft@latest print ./node_modules/zeromq/lib/index.js
FILELIST:
node_modules/@aminya/node-gyp-build/index.js
node_modules/@aminya/node-gyp-build/package.json
node_modules/zeromq/lib/draft.js
node_modules/zeromq/lib/index.js
node_modules/zeromq/lib/native.js
node_modules/zeromq/lib/util.js
node_modules/zeromq/package.json
```
@styfle I will provide a PR tomorrow.
Also see this related PR that mentions node-gyp-build

- https://github.com/vercel/nft/pull/286

That might lead to the more general purpose fix instead of a special case.
I will do the general purpose one. I think I understood the tracing logic a bit.
You can try `npx @vercel/nft@latest print path/to/my/file.js` to see which files are included.

If its missing prebuilds, then it is likely an issue with `@vercel/nft`.

You can create a PR with a test and special case, similar to PR https://github.com/vercel/nft/pull/37.

Or if you want to fix the general case (likely a dynamic require somewhere) that would be great too (albeit much harder).
@styfle Many thanks for the guidance. I will investigate that route this afternoon! :)
@styfle Indeed it certainly does not bring in the binaries:

```
❯ npx @vercel/nft@latest print ./node_modules/zeromq/lib/index.js
FILELIST:
node_modules/@aminya/node-gyp-build/index.js
node_modules/@aminya/node-gyp-build/package.json
node_modules/zeromq/lib/draft.js
node_modules/zeromq/lib/index.js
node_modules/zeromq/lib/native.js
node_modules/zeromq/lib/util.js
node_modules/zeromq/package.json
```
@styfle I will provide a PR tomorrow.
Also see this related PR that mentions node-gyp-build

- https://github.com/vercel/nft/pull/286

That might lead to the more general purpose fix instead of a special case.
I will do the general purpose one. I think I understood the tracing logic a bit.

Patch:
diff --git a/package-lock.json b/package-lock.json
--- a/package-lock.json
+++ b/package-lock.json
@@ -122,12 +122,24 @@
         "vm2": "^3.9.18",
         "vue": "^2.6.10",
         "vue-server-renderer": "^2.6.10",
-        "when": "^3.7.8"
+        "when": "^3.7.8",
+        "zeromq": "^6.0.0-beta.19"
       },
       "engines": {
         "node": ">=16"
       }
     },
+    "node_modules/@aminya/node-gyp-build": {
+      "version": "4.5.0-aminya.5",
+      "resolved": "https://registry.npmjs.org/@aminya/node-gyp-build/-/node-gyp-build-4.5.0-aminya.5.tgz",
+      "integrity": "sha512-TO7GldxDfSeSRNZVmhlm0liS2GX2o2Q/qTlcD3iD4ltTM6dir568LTRZ+ZDsDbLfMAkfhrbU+VuzNYImwYfczg==",
+      "dev": true,
+      "bin": {
+        "aminya-node-gyp-build": "bin.js",
+        "aminya-node-gyp-build-optional": "optional.js",
+        "aminya-node-gyp-build-test": "build-test.js"
+      }
+    },
     "node_modules/@ampproject/remapping": {
       "version": "2.2.1",
       "resolved": "https://registry.npmjs.org/@ampproject/remapping/-/remapping-2.2.1.tgz",
@@ -8654,6 +8666,24 @@
         "node": ">=0.8"
       }
     },
+    "node_modules/cross-env": {
+      "version": "7.0.3",
+      "resolved": "https://registry.npmjs.org/cross-env/-/cross-env-7.0.3.tgz",
+      "integrity": "sha512-+/HKd6EgcQCJGh2PSjZuUitQBQynKor4wrFbRg4DtAgS1aWO+gU52xpH7M9ScGgXSYmAVS9bIJ8EzuaGw0oNAw==",
+      "dev": true,
+      "dependencies": {
+        "cross-spawn": "^7.0.1"
+      },
+      "bin": {
+        "cross-env": "src/bin/cross-env.js",
+        "cross-env-shell": "src/bin/cross-env-shell.js"
+      },
+      "engines": {
+        "node": ">=10.14",
+        "npm": ">=6",
+        "yarn": ">=1"
+      }
+    },
     "node_modules/cross-fetch": {
       "version": "3.1.8",
       "resolved": "https://registry.npmjs.org/cross-fetch/-/cross-fetch-3.1.8.tgz",
@@ -13676,6 +13706,15 @@
         "node": ">= 0.4"
       }
     },
+    "node_modules/interpret": {
+      "version": "1.4.0",
+      "resolved": "https://registry.npmjs.org/interpret/-/interpret-1.4.0.tgz",
+      "integrity": "sha512-agE4QfB2Lkp9uICn7BAqoscw4SZP9kTE2hxiFI3jBPmXJfdqiahTbUuKGsMoN2GtqL9AxhYioAcVvgsb1HvRbA==",
+      "dev": true,
+      "engines": {
+        "node": ">= 0.10"
+      }
+    },
     "node_modules/intl": {
       "version": "1.2.5",
       "resolved": "https://registry.npmjs.org/intl/-/intl-1.2.5.tgz",
@@ -21978,7 +22017,7 @@
     },
     "node_modules/npm/node_modules/lodash._baseindexof": {
       "version": "3.1.0",
-      "extraneous": true,
+      "dev": true,
       "inBundle": true,
       "license": "MIT"
     },
@@ -21994,19 +22033,19 @@
     },
     "node_modules/npm/node_modules/lodash._bindcallback": {
       "version": "3.0.1",
-      "extraneous": true,
+      "dev": true,
       "inBundle": true,
       "license": "MIT"
     },
     "node_modules/npm/node_modules/lodash._cacheindexof": {
       "version": "3.0.2",
-      "extraneous": true,
+      "dev": true,
       "inBundle": true,
       "license": "MIT"
     },
     "node_modules/npm/node_modules/lodash._createcache": {
       "version": "3.1.2",
-      "extraneous": true,
+      "dev": true,
       "inBundle": true,
       "license": "MIT",
       "dependencies": {
@@ -22021,7 +22060,7 @@
     },
     "node_modules/npm/node_modules/lodash._getnative": {
       "version": "3.9.1",
-      "extraneous": true,
+      "dev": true,
       "inBundle": true,
       "license": "MIT"
     },
@@ -22039,7 +22078,7 @@
     },
     "node_modules/npm/node_modules/lodash.restparam": {
       "version": "3.6.1",
-      "extraneous": true,
+      "dev": true,
       "inBundle": true,
       "license": "MIT"
     },
@@ -27331,6 +27370,18 @@
         "node": ">=0.10.0"
       }
     },
+    "node_modules/rechoir": {
+      "version": "0.6.2",
+      "resolved": "https://registry.npmjs.org/rechoir/-/rechoir-0.6.2.tgz",
+      "integrity": "sha512-HFM8rkZ+i3zrV+4LQjwQ0W+ez98pApMGM3HUrN04j3CqzPOzl9nmP15Y8YXNm8QHGv/eacOVEjqhmWpkRV0NAw==",
+      "dev": true,
+      "dependencies": {
+        "resolve": "^1.1.6"
+      },
+      "engines": {
+        "node": ">= 0.10"
+      }
+    },
     "node_modules/redis": {
       "version": "3.1.2",
       "resolved": "https://registry.npmjs.org/redis/-/redis-3.1.2.tgz",
@@ -28730,6 +28781,23 @@
         "url": "https://github.com/sponsors/ljharb"
       }
     },
+    "node_modules/shelljs": {
+      "version": "0.8.5",
+      "resolved": "https://registry.npmjs.org/shelljs/-/shelljs-0.8.5.tgz",
+      "integrity": "sha512-TiwcRcrkhHvbrZbnRcFYMLl30Dfov3HKqzp5tO5b4pt6G/SezKcYhmDg15zXVBswHmctSAQKznqNW2LO5tTDow==",
+      "dev": true,
+      "dependencies": {
+        "glob": "^7.0.0",
+        "interpret": "^1.0.0",
+        "rechoir": "^0.6.2"
+      },
+      "bin": {
+        "shjs": "bin/shjs"
+      },
+      "engines": {
+        "node": ">=4"
+      }
+    },
     "node_modules/shiki": {
       "version": "0.14.7",
       "resolved": "https://registry.npmjs.org/shiki/-/shiki-0.14.7.tgz",
@@ -28752,6 +28820,31 @@
         "nanoid": "^2.1.0"
       }
     },
+    "node_modules/shx": {
+      "version": "0.3.4",
+      "resolved": "https://registry.npmjs.org/shx/-/shx-0.3.4.tgz",
+      "integrity": "sha512-N6A9MLVqjxZYcVn8hLmtneQWIJtp8IKzMP4eMnx+nqkvXoqinUPCbUFLp2UcWTEIUONhlk0ewxr/jaVGlc+J+g==",
+      "dev": true,
+      "dependencies": {
+        "minimist": "^1.2.3",
+        "shelljs": "^0.8.5"
+      },
+      "bin": {
+        "shx": "lib/cli.js"
+      },
+      "engines": {
+        "node": ">=6"
+      }
+    },
+    "node_modules/shx/node_modules/minimist": {
+      "version": "1.2.8",
+      "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
+      "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
+      "dev": true,
+      "funding": {
+        "url": "https://github.com/sponsors/ljharb"
+      }
+    },
     "node_modules/side-channel": {
       "version": "1.0.4",
       "resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.0.4.tgz",
@@ -33578,6 +33671,23 @@
         "tslib": "^1.9.3",
         "zen-observable": "^0.8.0"
       }
+    },
+    "node_modules/zeromq": {
+      "version": "6.0.0-beta.19",
+      "resolved": "https://registry.npmjs.org/zeromq/-/zeromq-6.0.0-beta.19.tgz",
+      "integrity": "sha512-2eU6H7Z4r9LmTkseGdIfEqp0k3rJr5EpXQw2oC0bUqy3wpoj1M83IU/c3qHPXp0z8IklNMoVmVm130d3g2Xf3g==",
+      "dev": true,
+      "hasInstallScript": true,
+      "dependencies": {
+        "@aminya/node-gyp-build": "4.5.0-aminya.5",
+        "cross-env": "^7.0.3",
+        "node-addon-api": "^7.0.0",
+        "shelljs": "^0.8.5",
+        "shx": "^0.3.4"
+      },
+      "engines": {
+        "node": ">= 10.2"
+      }
     }
   }
 }
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -128,7 +128,8 @@
     "vm2": "^3.9.18",
     "vue": "^2.6.10",
     "vue-server-renderer": "^2.6.10",
-    "when": "^3.7.8"
+    "when": "^3.7.8",
+    "zeromq": "^6.0.0-beta.19"
   },
   "engines": {
     "node": ">=16"
diff --git a/src/analyze.ts b/src/analyze.ts
--- a/src/analyze.ts
+++ b/src/analyze.ts
@@ -127,6 +127,9 @@ const staticModules = Object.assign(Object.create(null), {
   'node-gyp-build': {
     default: NODE_GYP_BUILD
   },
+  '@aminya/node-gyp-build': {
+    default: NODE_GYP_BUILD
+  },
   'nbind': {
     init: NBIND_INIT,
     default: {
@@ -614,18 +617,39 @@ export default async function analyze(id: string, code: string, job: Job): Promi
               }
             break;
             case NODE_GYP_BUILD:
-              if (node.arguments.length === 1 && node.arguments[0].type === 'Identifier' &&
-                  node.arguments[0].name === '__dirname' && knownBindings.__dirname.shadowDepth === 0) {
+              // handle case: require('node-gyp-build')(__dirname)
+              const withDirname =
+                node.arguments.length === 1 &&
+                node.arguments[0].type === 'Identifier' &&
+                node.arguments[0].name === '__dirname';
+
+              // handle case: require('node-gyp-build')(path.join(__dirname, '..'))
+              const withPathJoinDirname =
+                node.arguments.length === 1 &&
+                node.arguments[0].callee?.object?.name === 'path' &&
+                node.arguments[0].callee?.property?.name === 'join' &&
+                node.arguments[0].arguments.length === 2 &&
+                node.arguments[0].arguments[0].type === 'Identifier' &&
+                node.arguments[0].arguments[0].name === '__dirname' &&
+                node.arguments[0].arguments[1].type === 'Literal'
+
+              if (knownBindings.__dirname.shadowDepth === 0 && (withDirname || withPathJoinDirname)) {
+
+                const pathJoinedDir = withPathJoinDirname
+                  ? path.join(dir, node.arguments[0].arguments[1].value)
+                  : dir;
+
                 let resolved: string | undefined;
                 try {
+                  // the pkg could be 'node-gyp-build' or '@aminya/node-gyp-build'
+                  const pkgName = node.callee.arguments[0].value;
                   // use installed version of node-gyp-build since resolving
                   // binaries can differ among versions
-                  const nodeGypBuildPath = resolveFrom(dir, 'node-gyp-build')
-                  resolved = require(nodeGypBuildPath).path(dir)
-                }
-                catch (e) {
+                  const nodeGypBuildPath = resolveFrom(pathJoinedDir, pkgName)
+                  resolved = require(nodeGypBuildPath).path(pathJoinedDir)
+                } catch (e) {
                   try {
-                    resolved = nodeGypBuild.path(dir);
+                    resolved = nodeGypBuild.path(pathJoinedDir);
                   } catch (e) {}
                 }
                 if (resolved) {


Test patch:
diff --git a/test/integration/zeromq.js b/test/integration/zeromq.js
new file mode 100644
--- /dev/null
+++ b/test/integration/zeromq.js
@@ -0,0 +1,2 @@
+const zmq = require("zeromq");
+const sock = new zmq.Push;
\ No newline at end of file
diff --git a/test/unit.test.js b/test/unit.test.js
--- a/test/unit.test.js
+++ b/test/unit.test.js
@@ -203,10 +203,10 @@ for (const { testName, isRoot } of unitTests) {
       }
       let sortedFileList = [...fileList].sort()
 
-      if (testName === 'microtime-node-gyp') {
+      if (testName === 'microtime-node-gyp' || testName === 'zeromq-node-gyp') {
         let foundMatchingBinary = false
         sortedFileList = sortedFileList.filter(file => {
-          if (file.endsWith('node-napi.node') || file.endsWith('node.napi.node')) {
+          if (file.includes('prebuilds') && file.endsWith('.node')) {
             // remove from fileList for expected checking
             // as it will differ per platform
             foundMatchingBinary = true
diff --git a/test/unit/zeromq-node-gyp/input.js b/test/unit/zeromq-node-gyp/input.js
new file mode 100644
--- /dev/null
+++ b/test/unit/zeromq-node-gyp/input.js
@@ -0,0 +1,2 @@
+const zeromq = require('zeromq');
+const sock = new zmq.Push;
\ No newline at end of file
diff --git a/test/unit/zeromq-node-gyp/output.js b/test/unit/zeromq-node-gyp/output.js
new file mode 100644
--- /dev/null
+++ b/test/unit/zeromq-node-gyp/output.js
@@ -0,0 +1,11 @@
+[
+  "node_modules/@aminya/node-gyp-build/index.js",
+  "node_modules/@aminya/node-gyp-build/package.json",
+  "node_modules/zeromq/lib/draft.js",
+  "node_modules/zeromq/lib/index.js",
+  "node_modules/zeromq/lib/native.js",
+  "node_modules/zeromq/lib/util.js",
+  "node_modules/zeromq/package.json",
+  "package.json",
+  "test/unit/zeromq-node-gyp/input.js"
+]