Issue 1:
Is there an existing issue for this?

I have searched the existing issues

This issue exists in the latest npm version

I am using the latest npm

Current Behavior

We have a number of node14 projects that we are in the process of upgrading to 16 (and if we aren't careful, the LTS train will have moved on before we get there!).

Can anyone explain what has changed in npm 8(?), that could cause git+ssh dependencies to fail? I have a (fairly) minimal repro here, but almost no theory as to what could cause it. I don't know if it is related to docker, or npm, or some combination of the two.

The main branch builds fine:

$ make build
Building the image...
DOCKER_BUILDKIT=1 docker build --ssh default -t npm-chown .
[+] Building 2.2s (14/14) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 450B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:experimental 1.0s
=> CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5 0.0s
=> [internal] load metadata for docker.io/library/node:14-alpine 1.0s
=> [internal] load build context 0.0s
=> => transferring context: 69B 0.0s
=> [1/7] FROM docker.io/library/node:14-alpine@sha256:e22ee6a906e823dc592d6022a3f520676575b50320fe2c3916c88acb16f471ce 0.0s
=> CACHED [2/7] RUN apk --no-cache --virtual build-dependencies add make g++ git openssh 0.0s
=> CACHED [3/7] RUN mkdir -p /root/.ssh && chmod 0700 /root/.ssh && ssh-keyscan github.com >> /root/.ssh/known_hosts 0.0s
=> CACHED [4/7] WORKDIR /app/ 0.0s
=> CACHED [5/7] RUN chown -R node $PWD 0.0s
=> CACHED [6/7] ADD ./service/package.json . 0.0s
=> CACHED [7/7] RUN --mount=type=ssh npm i 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:fa984c52481af4a483017351da9902eb627eccdfc2f837ead12a73812b1bf14d 0.0s
=> => naming to docker.io/library/npm-chown

but the other two branches fail like this:

$ make build
Building the image...
DOCKER_BUILDKIT=1 docker build --ssh default -t npm-chown .
[+] Building 11.3s (14/14) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 467B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:experimental 0.8s
=> CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5 0.0s
=> [internal] load metadata for docker.io/library/node:14-alpine 0.7s
=> [1/8] FROM docker.io/library/node:14-alpine@sha256:e22ee6a906e823dc592d6022a3f520676575b50320fe2c3916c88acb16f471ce 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 69B 0.0s
=> CACHED [2/8] RUN apk --no-cache --virtual build-dependencies add make g++ git openssh 0.0s
=> CACHED [3/8] RUN mkdir -p /root/.ssh && chmod 0700 /root/.ssh && ssh-keyscan github.com >> /root/.ssh/known_hosts 0.0s
=> CACHED [4/8] WORKDIR /app/ 0.0s
=> CACHED [5/8] RUN chown -R node $PWD 0.0s
=> CACHED [6/8] ADD ./service/package.json . 0.0s
=> [7/8] RUN npm i -g npm 7.2s
=> ERROR [8/8] RUN --mount=type=ssh npm i 2.3s
------
> [8/8] RUN --mount=type=ssh npm i:
#14 2.263 npm ERR! code 128
#14 2.266 npm ERR! An unknown git error occurred
#14 2.267 npm ERR! command git --no-replace-objects ls-remote ssh://git@github.com/jashkenas/underscore.git
#14 2.267 npm ERR! Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
#14 2.267 npm ERR! git@github.com: Permission denied (publickey).
#14 2.267 npm ERR! fatal: Could not read from remote repository.
#14 2.267 npm ERR!
#14 2.267 npm ERR! Please make sure you have the correct access rights
#14 2.267 npm ERR! and the repository exists.
#14 2.269
#14 2.270 npm ERR! A complete log of this run can be found in:
#14 2.270 npm ERR! /root/.npm/_logs/2022-08-09T11_50_52_370Z-debug-0.log
------
executor failed running [/bin/sh -c npm i]: exit code: 128
make: *** [Makefile:7: build] Error 1

I have a workaround, to move the chown after npm i, but that takes my build time from ~1 min to ~12 mins. which isn't ideal.

Expected Behavior

I would expect it to work the same as npm6

Steps To Reproduce

see above

Environment

see above

Issue 2:
Is there an existing issue for this?

I have searched the existing issues

Current Behavior

After upgrading from npm 6 to npm 8.1.1, npm install started failing. This happens with npm 7.24.2 as well. I used --loglevel verbose and found the following:

npm timing command:install Completed in 29728ms
npm verb stack Error: git dep preparation failed
npm verb stack at ChildProcess.<anonymous> (C:\Users\RudeySH\AppData\Roaming\npm\node_modules\npm\node_modules\@npmcli\promise-spawn\index.js:64:27)
npm verb stack at ChildProcess.emit (events.js:400:28)
npm verb stack at maybeClose (internal/child_process.js:1058:16)
npm verb stack at Process.ChildProcess._handle.onexit (internal/child_process.js:293:5)
npm verb cwd E:\Sync\Projects\SteamHunters\SteamHunters\src\SteamHunters.Web.Mvc
npm verb Windows_NT 10.0.19042
npm verb argv "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Users\\RudeySH\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "install" "--loglevel" "verbose"
npm verb node v14.18.1
npm verb npm v8.1.1
npm ERR! code 1
npm ERR! git dep preparation failed
npm ERR! command C:\Program Files (x86)\nodejs\node.exe C:\Users\RudeySH\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js install --force --cache=C:\Users\RudeySH\AppData\Local\npm-cache --prefer-offline=false --prefer-online=false --offline=false --no-progress --no-save --no-audit --include=dev --include=peer --include=optional --no-package-lock-only --no-dry-run
npm ERR! npm verb cli [
npm ERR! npm verb cli 'C:\\Program Files (x86)\\nodejs\\node.exe',
npm ERR! npm verb cli 'C:\\Users\\RudeySH\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
npm ERR! npm verb cli 'install',
npm ERR! npm verb cli '--force',
npm ERR! npm verb cli '--cache=C:\\Users\\RudeySH\\AppData\\Local\\npm-cache',
npm ERR! npm verb cli '--prefer-offline=false',
npm ERR! npm verb cli '--prefer-online=false',
npm ERR! npm verb cli '--offline=false',
npm ERR! npm verb cli '--no-progress',
npm ERR! npm verb cli '--no-save',
npm ERR! npm verb cli '--no-audit',
npm ERR! npm verb cli '--include=dev',
npm ERR! npm verb cli '--include=peer',
npm ERR! npm verb cli '--include=optional',
npm ERR! npm verb cli '--no-package-lock-only',
npm ERR! npm verb cli '--no-dry-run'
npm ERR! npm verb cli ]

It get's more interesting a bit further down in the log:

npm ERR! npm timing build:link Completed in 126ms
npm ERR! npm info run iltorb@2.4.3 install node_modules/iltorb node ./scripts/install.js || node-gyp rebuild
npm ERR! npm info run iltorb@2.4.3 install { code: 1, signal: null }
npm ERR! npm timing reify:rollback:createSparse Completed in 1809ms
npm ERR! npm timing reify:rollback:retireShallow Completed in 0ms
npm ERR! npm timing command:install Completed in 13304ms
npm ERR! npm verb stack Error: command failed
npm ERR! npm verb stack at ChildProcess.<anonymous> (C:\Users\RudeySH\AppData\Roaming\npm\node_modules\npm\node_modules\@npmcli\promise-spawn\index.js:64:27)
npm ERR! npm verb stack at ChildProcess.emit (events.js:400:28)
npm ERR! npm verb stack at maybeClose (internal/child_process.js:1058:16)
npm ERR! npm verb stack at Process.ChildProcess._handle.onexit (internal/child_process.js:293:5)
npm ERR! npm verb pkgid iltorb@2.4.3
npm ERR! npm verb cwd C:\Users\RudeySH\AppData\Local\npm-cache\_cacache\tmp\git-cloneO1YrvZ
npm ERR! npm verb Windows_NT 10.0.19042
npm ERR! npm verb argv "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Users\\RudeySH\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "install" "--force" "--cache=C:\\Users\\RudeySH\\AppData\\Local\\npm-cache" "--prefer-offline=false" "--prefer-online=false" "--offline=false" "--no-progress" "--no-save" "--no-audit" "--include=dev" "--include=peer" "--include=optional" "--no-package-lock-only" "--no-dry-run"
npm ERR! npm verb node v14.18.1
npm ERR! npm verb npm v8.1.1
npm ERR! npm ERR! code 1
npm ERR! npm ERR! path C:\Users\RudeySH\AppData\Local\npm-cache\_cacache\tmp\git-cloneO1YrvZ\node_modules\iltorb
npm ERR! npm ERR! command failed
npm ERR! npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node ./scripts/install.js || node-gyp rebuild

I'm not sure why the cli is using the deprecated iltorb package, but it's readme states the following:

Prerequisites for Building

The following is required to build from source or when a pre-compiled binary does not exist.

Python 2.7

GCC 4.8+ (Unix) or windows-build-tools (Windows), see Node Building tools.

I'm guessing a pre-compiled binary doesn't exist, and that is why it's calling node-gyp rebuild.

I don't have Python nor windows-build-tools nor Visual C/C++ Build Tools installed, and I don't want to. I'm not trying to build native addons for for Node.js, after all.

Expected Behavior

Npm 7 and later versions should not require developers to install Python, nor should it require developers to install C/C++ development tools, if their project does not involve building native addons for Node.js. Just like npm 6 didn't have this requirement.

Steps To Reproduce

Install npm@8.1.1 on a machine that does not have Python or C/C++ development tools

In a new empty folder, run npm init -y

Install a popular npm package, for example npm i -D microbundle@0.11.0

Run npm install

See error...

Environment

OS: Windows_NT 10.0.19042

Node: v14.18.1

npm: v8.1.1