Issue 1:
Update: This issue explained in my comment seems to have been fixed in v7.11.0 (see my update comment), however, another user has described a similar error in v7.20.3, so I'm unsure whether I should close this issue or leave it open.

Using a git URL in the format github:geotiffjs/geotiff.js#commit-hash on NPM v7 fails with sha-512 integrity errors. However, using a branch-name/tag pointing to the same commit works fine.

I can't seem to find another Git repo where this is failing (except for forks of geotiff.js), so I think the issue might either with Github repos with periods in their name and/or with package.json prepare scripts.

This might be related to #2919 which has a similar title, but their logs and behaviour are different.

I'm fairly certain this is an issue with the CDN, since it doesn't happen in Node v6, and if I use a get+ssh URL on a new VM, the code fails before I get a pop-up with a The authenticity of host 'github.com (140.82.121.3)' can't be established.

Current Behavior:

For the geotiff/geotiff.js repo, using a git+https://...#commit-hash URL fails with a EINTEGRITY error, due to the sha512 error, e.g.

ubuntu@gittest:~/geotiff.js$ npm pack "git+https://github.com/geotiffjs/geotiff.js.git#c01b2dcf00d5a324cd27c38709cb6ca97adf01e2"
npm WARN tarball tarball data for git+ssh://git@github.com/geotiffjs/geotiff.js.git#c01b2dcf00d5a324cd27c38709cb6ca97adf01e2 (sha512-NdxkcAm6FPXBKyhAHuYp0yMyKcBboXzf00h9hqo+yAVhAPVk6d68GH11SV9n4v2OZ9Roe1zQfCzV4TuoeskVLw==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for git+ssh://git@github.com/geotiffjs/geotiff.js.git#c01b2dcf00d5a324cd27c38709cb6ca97adf01e2 (sha512-NdxkcAm6FPXBKyhAHuYp0yMyKcBboXzf00h9hqo+yAVhAPVk6d68GH11SV9n4v2OZ9Roe1zQfCzV4TuoeskVLw==) seems to be corrupted. Trying again.
npm ERR! code EINTEGRITY
npm ERR! sha512-NdxkcAm6FPXBKyhAHuYp0yMyKcBboXzf00h9hqo+yAVhAPVk6d68GH11SV9n4v2OZ9Roe1zQfCzV4TuoeskVLw== integrity checksum failed when using sha512: wanted sha512-NdxkcAm6FPXBKyhAHuYp0yMyKcBboXzf00h9hqo+yAVhAPVk6d68GH11SV9n4v2OZ9Roe1zQfCzV4TuoeskVLw== but got sha512-c2I+9X+wgzfOMS8/Fn09R8BjPrjEavxj0r39aO/loMvtWlA6BNxAlatfcnW8WHhOfgbEbIRi9k3WNpJh2yoxKQ==. (709406 bytes)

npm ERR! A complete log of this run can be found in:
npm ERR! /home/ubuntu/.npm/_logs/2021-03-26T14_59_04_649Z-debug.log

Attached logfile: 2021-03-26T14_59_04_649Z-debug.log

The same error occurs if you try to do npm install on the URL, or if you use a git+ssh url, or a short github: url: github:geotiffjs/geotiff.js#c01b2dcf00d5a324cd27c38709cb6ca97adf01e2

Expected Behavior:

Using NPM v6.14.11, the exact same command works fine.

# ON NPM v6
ubuntu@gittest:~/geotiff.js$ npm pack "git+https://github.com/geotiffjs/geotiff.js.git#c01b2dcf00d5a324cd27c38709cb6ca97adf01e2"
npm notice
npm notice 📦 geotiff@1.0.0
npm notice === Tarball Contents ===
npm notice 1.1kB LICENSE
# ...etc

Additionally, even on NPM v7.7.5, using a branch/tag name works fine: e.g.npm pack "github:github.com/geotiffjs/geotiff.js.git#v1.0.0" works on NPMv7, even though tag v1.0.0 points to the same commit.

Steps To Reproduce:

On NPM v7.7.5, run the following command:

npm pack "git+https://github.com/geotiffjs/geotiff.js.git#c01b2dcf00d5a324cd27c38709cb6ca97adf01e2"

You can also try spinning up an Ubuntu VM using minipool:

multipass launch --name npm-git-test && multipass shell npm-git-test

install NVM then restart shell curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash

Try working cmd on NPM v6.14.11: npm pack github:geotiffjs/geotiff.js#c01b2dcf00d5a324cd27c38709cb6ca97adf01e2

Update to NPM v7.7.5 and see the same command fail: npm install -g npm && npm pack github:geotiffjs/geotiff.js#c01b2dcf00d5a324cd27c38709cb6ca97adf01e2

Environment:

I've tested this on a VM running:

OS: Ubuntu 20.04.2 LTS

Node: v14.16.0

npm: 7.7.5

and a desktop running:

OS: Ubuntu 20.04.2 LTS

Node: v12.19.1

npm: 7.7.5

Issue 2:
Current Behavior:

The package-lock.json integrity value seems to depend on the OS/architecture. Take the following git dependency which specifies a commit hash:

npm pack "git+ssh://git@github.com/jhiesey/idb-kv-store.git#109ccad165fd6470e12fd66025da9e4743a46043"

The integrity value produced is different on these OSes/architectures:

Ubuntu 20.04, and macOS 11.2.3 (Intel): sha512-DnBTbDDxd9/9mwPehyraeuRTbNEqbWLcAdE3GC1trdBWWwKnkWsaU/X6mVLIKKB/IYWmG+cnL3ihg/Ql/rW5kg==

macOS 11.2.3 (Apple Silicon): sha512-T3ZWOM1TT+Ch/splApkEe1HwktWs+n/iHvDvtIGEI+4xuMGHite6mMujuNd8sen49ofLP/PxzprQMSPJK8APww==

Expected Behavior:

The integrity value should not be different on Apple Silicon (M1 chip) machines.

Steps To Reproduce:

Run npm pack "git+ssh://git@github.com/jhiesey/idb-kv-store.git#109ccad165fd6470e12fd66025da9e4743a46043" and inspect the integrity value from an M1 Mac. Node.js was installed from Homebrew using brew install node and the amd64 version was installed.

Also... @jhiesey and I dug into this a bit and found that the tarballs fetched from the GitHub CDN are exactly the same on M1 and other architectures, byte-for-byte. Same for the ungzipped tarballs – they are the same byte-for-byte. What differs, though, is the gzipped tarballs (.tar.gz) files. Those appear to have substantial differences when viewed in a hex editor.

Environment:

OS: Various, see above

Node: v15.11.0

npm: 7.6.2