Repository: netlify/cli
Pull request #5547 (2023-03-09 17:02:01)

fix!: publish shrinkwrap with prod dependencies only
#### Summary

Ref #5323 

This is a reattempt of #5359 which fixes the problems that occurred last time.

It removes the shrinkwrap from the repo and instead uses the normal package-lock.

On publish we remove devDependencies, scripts & ava properties from package.json and then create the shrinkwrap. This new way ensures that all optional dependencies are present in the shrinkwrap file. (Last time the optional dependencies were missing)

Also split up the release workflow into 2 jobs so we do not have to repeat the if: `${{ steps.release.outputs.release_created }}` for every single step.






github-actions[bot]: ## 📊 Benchmark results

*Comparing with c4b15f691f5250ee57f7de1cf5d399141f265d59*

### Package size: 263 MB
 (no change)
```
^                          271 MB                                                                         
│  264 MB  264 MB  264 MB   ┌──┐   264 MB  264 MB  264 MB  264 MB  264 MB  263 MB  263 MB  263 MB  263 MB 
│   ┌──┐    ┌──┐    ┌──┐    |  |    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T    
```
<details>
<summary>Legend</summary>

- T-30 (5fb20ff80a62e0103f5bfe2280f609890ef8ca80): 263 MB
- T-29 (881dff6cad0a3a85faee66d0075e2c663eaf51ca): 262 MB
- T-28 (ab792a9de6fad438910e50638aab7b0ff1980581): 262 MB
- T-27 (8fd947a481f549338aee8f5f89ca102bf3022e96): 262 MB
- T-26 (a78f6416b81e1f273329d4cd84f32c481aea0e50): 263 MB
- T-25 (b94711b5373fe0fd6eaab05a44907e4e97e46eaa): 263 MB
- T-24 (536db4e1d391ba77d22b5a1a54bd09936ae2d9b3): 263 MB
- T-23 (bbb80e9fd399bdbb25df5343c37175f25f7d502b): 263 MB
- T-22 (f667d278a38106353a69b432d7e0ae8f0aced330): 263 MB
- T-21 (48e94f713799df4491844d999e3e7bfb0e9cb604): 263 MB
- T-20 (5dd72695e90b3cee7260524546b4693049c71ae5): 263 MB
- T-19 (3566ad7c515848d854c6354457b186b417ac8c6f): 270 MB
- T-18 (3f71564e3ba4a299679462d7fbd4d6a7ea619e2f): 264 MB
- T-17 (43d185004e80c9ee67b39a663bda92e5c60c9929): 264 MB
- T-16 (c96c490cf336afed3b00d1841fdd79adc8d9602e): 264 MB
- T-15 (2cb229bc89aaeb9e2914eb1ce0f6d14bf9a16ba1): 264 MB
- T-14 (7c793d90cae5d4e6f55e58a6fe4784f4cf811bea): 264 MB
- T-13 (ab218a27f3c663e4049dc3b58cf59b135260dd38): 264 MB
- T-12 (e6cbcfc2f297aa67af793346751c8fdbb590511c): 264 MB
- T-11 (2488e87277735334247564e31bf9c22a6179a72d): 264 MB
- T-10 (5abd0316f7f812f131cdcaa34aed9fa2de11f932): 264 MB
- T-9 (78b8ed0b20a3d069988ef71f493ad65b46f99e96): 271 MB
- T-8 (af4fe0e0d56a488ff0ab8e1a13aed6a856f93a97): 264 MB
- T-7 (682f5319b85365013212147e25147e9eaebf815f): 264 MB
- T-6 (22345a822b53718a46e53801fdacd09ed81be44f): 264 MB
- T-5 (f0b54581a606841359cc039637c65fb396d811b5): 264 MB
- T-4 (f65b2a909a5be727c3d2315923136bd2472a2557): 264 MB
- T-3 (6b9441c298d544a7054dc423c173980595d8f00d): 263 MB
- T-2 (0f7ac190f9e1c7ed056d2feb1a1834c8305a048a): 263 MB
- T-1 (c4b15f691f5250ee57f7de1cf5d399141f265d59): 263 MB
- **T (current commit): 263 MB**</details>
<!--delta:post-package-size@{}-->
EverlastingBugstopper: Thanks for the work on this!

Patch:
diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml
index 6d591200..274ca19c 100644
--- a/.github/workflows/benchmark.yml
+++ b/.github/workflows/benchmark.yml
@@ -15,10 +15,8 @@ jobs:
       - name: Use Node.js
         uses: actions/setup-node@v3
         with:
-          node-version: '*'
-          cache: 'npm'
-          cache-dependency-path: 'npm-shrinkwrap.json'
-          check-latest: true
+          node-version: lts/*
+          cache: npm
       - name: Install dependencies
         run: npm ci --no-audit && npm prune --production
       - name: Get size
diff --git a/.github/workflows/pre-release.yml b/.github/workflows/pre-release.yml
index 5a887e26..de2a25c0 100644
--- a/.github/workflows/pre-release.yml
+++ b/.github/workflows/pre-release.yml
@@ -11,15 +11,11 @@ jobs:
       - uses: actions/checkout@v3
       - uses: actions/setup-node@v3
         with:
-          node-version: '*'
-          cache: 'npm'
-          check-latest: true
-          cache-dependency-path: 'npm-shrinkwrap.json'
+          node-version: lts/*
+          cache: npm
           registry-url: 'https://registry.npmjs.org'
       - name: Install core dependencies
         run: npm ci --no-audit
-      - name: Install site dependencies
-        run: npm run site:build:install
       - name: Extract tag and version
         id: extract
         run: |-
@@ -43,6 +39,6 @@ jobs:
       - name: Push changes
         run: git push --follow-tags
       - name: Run npm publish
-        run: npm publish --tag=${{ steps.extract.outputs.tag }} --ignore-scripts
+        run: npm publish --tag=${{ steps.extract.outputs.tag }}
         env:
           NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
diff --git a/.github/workflows/release-please.yml b/.github/workflows/release-please.yml
index 4ac9caff..3a650d41 100644
--- a/.github/workflows/release-please.yml
+++ b/.github/workflows/release-please.yml
@@ -4,41 +4,40 @@ on:
     branches:
       - main
 jobs:
-  release-please:
+  create-release:
     runs-on: ubuntu-latest
+    outputs:
+      release_created: ${{ steps.release.outputs.release_created }}
     steps:
       - uses: navikt/github-app-token-generator@a3831f44404199df32d8f39f7c0ad9bb8fa18b1c
         id: get-token
         with:
           private-key: ${{ secrets.TOKENS_PRIVATE_KEY }}
           app-id: ${{ secrets.TOKENS_APP_ID }}
+
       - uses: GoogleCloudPlatform/release-please-action@v3
         id: release
         with:
           token: ${{ steps.get-token.outputs.token }}
           release-type: node
-          package-name: 'netlify-cli'
+          package-name: netlify-cli
+
+  publish:
+    runs-on: ubuntu-latest
+    needs: create-release
+    if: ${{ needs.create-release.outputs.release_created }}
+    steps:
       - uses: actions/checkout@v3
-        if: ${{ steps.release.outputs.release_created }}
+
       - uses: actions/setup-node@v3
         with:
-          node-version: '*'
-          cache: 'npm'
-          cache-dependency-path: 'npm-shrinkwrap.json'
-          check-latest: true
+          node-version: lts/*
+          cache: npm
           registry-url: 'https://registry.npmjs.org'
-        if: ${{ steps.release.outputs.release_created }}
-      # required for linting to pass
-      - name: Install site dependencies
-        run: npm run site:build:install
-        if: ${{ steps.release.outputs.release_created }}
+
       - name: Install core dependencies
         run: npm ci --no-audit
-        if: ${{ steps.release.outputs.release_created }}
+
       - run: npm publish
-        if: ${{ steps.release.outputs.release_created }}
         env:
           NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
-          # required for tests to pass
-          NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
-          NETLIFY_TEST_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/verify-docs.yml b/.github/workflows/verify-docs.yml
index 01fc7af7..dca250bb 100644
--- a/.github/workflows/verify-docs.yml
+++ b/.github/workflows/verify-docs.yml
@@ -16,10 +16,8 @@ jobs:
       - name: Use Node.js
         uses: actions/setup-node@v3
         with:
-          node-version: '*'
-          cache: 'npm'
-          cache-dependency-path: 'npm-shrinkwrap.json'
-          check-latest: true
+          node-version: lts/*
+          cache: npm
       - name: Install core dependencies
         run: npm ci --no-audit
       - name: Install site dependencies
diff --git a/.npmrc b/.npmrc
deleted file mode 100644
index 3438c513..00000000
--- a/.npmrc
+++ /dev/null
@@ -1,1 +0,0 @@
-shrinkwrap=true
diff --git a/npm-shrinkwrap.json b/package-lock.json
similarity index 99%
rename from npm-shrinkwrap.json
rename to package-lock.json
index 9d6c874c..0da8749e 100644
--- a/npm-shrinkwrap.json
+++ b/package-lock.json
@@ -137,6 +137,7 @@
         "graphviz": "^0.0.9",
         "husky": "^8.0.0",
         "ini": "^2.0.0",
+        "is-ci": "^3.0.1",
         "mock-fs": "^5.1.2",
         "nock": "^13.2.4",
         "p-timeout": "^4.0.0",
diff --git a/package.json b/package.json
index 9495a631..bc821d5e 100644
--- a/package.json
+++ b/package.json
@@ -38,7 +38,7 @@
     "url": "https://github.com/netlify/cli/issues"
   },
   "scripts": {
-    "prepare": "husky install node_modules/@netlify/eslint-config-node/.husky/",
+    "prepare": "is-ci || husky install node_modules/@netlify/eslint-config-node/.husky/",
     "start": "node ./bin/run.mjs",
     "test": "run-s format test:dev",
     "format": "run-s format:check-fix:*",
@@ -68,11 +68,12 @@
     "site:build:install": "cd site && npm ci --no-audit",
     "site:build:assets": "cd site && npm run build",
     "postinstall": "node ./scripts/postinstall.mjs",
+    "prepublishOnly": "node ./scripts/prepare-for-publish.mjs",
     "certs": "openssl req -x509 -out localhost.crt -keyout localhost.key -newkey rsa:2048 -nodes -sha256 -subj \"/CN=localhost\" -extensions EXT -config certconf"
   },
   "config": {
     "eslint": "--cache --format=codeframe --max-warnings=0 \"{src,scripts,site,tests,.github}/**/*.{mjs,cjs,js,md,html}\" \"*.{mjs,cjs,js,md,html}\" \".*.{mjs,cjs,js,md,html}\"",
-    "prettier": "--ignore-path .eslintignore --loglevel=warn \"{src,tools,scripts,site,tests,.github}/**/*.{mjs,cjs,js,md,yml,json,html}\" \"*.{mjs,cjs,js,yml,json,html}\" \".*.{mjs,cjs,js,yml,json,html}\" \"!CHANGELOG.md\" \"!npm-shrinkwrap.json\" \"!**/*/package-lock.json\" \"!.github/**/*.md\""
+    "prettier": "--ignore-path .eslintignore --loglevel=warn \"{src,tools,scripts,site,tests,.github}/**/*.{mjs,cjs,js,md,yml,json,html}\" \"*.{mjs,cjs,js,yml,json,html}\" \".*.{mjs,cjs,js,yml,json,html}\" \"!CHANGELOG.md\" \"!**/*/package-lock.json\" \"!.github/**/*.md\""
   },
   "dependencies": {
     "@fastify/static": "^6.6.0",
@@ -198,6 +199,7 @@
     "graphviz": "^0.0.9",
     "husky": "^8.0.0",
     "ini": "^2.0.0",
+    "is-ci": "^3.0.1",
     "mock-fs": "^5.1.2",
     "nock": "^13.2.4",
     "p-timeout": "^4.0.0",
diff --git a/scripts/prepare-for-publish.mjs b/scripts/prepare-for-publish.mjs
new file mode 100644
index 00000000..a694655d
--- /dev/null
+++ b/scripts/prepare-for-publish.mjs
@@ -0,0 +1,46 @@
+import { readFile, writeFile } from 'fs/promises'
+import { dirname, join } from 'path'
+import { fileURLToPath } from 'url'
+
+import execa from 'execa'
+import ora from 'ora'
+
+// These scripts from package.json need to be preserved on publish
+const preserveScripts = new Set(['postinstall', 'postpack', 'preinstall', 'prepack', 'prepublish', 'prepublishOnly'])
+
+let spinner = ora({
+  spinner: 'star',
+  text: 'Patching package.json (removing devDependencies, scripts, etc)',
+}).start()
+
+const dir = dirname(fileURLToPath(import.meta.url))
+const packageJsonPath = join(dir, '../package.json')
+
+const pkgJson = JSON.parse(await readFile(packageJsonPath))
+
+delete pkgJson.devDependencies
+delete pkgJson.ava
+delete pkgJson.config
+
+// eslint-disable-next-line fp/no-loops
+for (const scriptName in pkgJson.scripts) {
+  if (preserveScripts.has(scriptName)) continue
+
+  delete pkgJson.scripts[scriptName]
+}
+await writeFile(packageJsonPath, JSON.stringify(pkgJson, null, 2))
+spinner.succeed()
+
+spinner = ora({
+  spinner: 'star',
+  text: 'Running `npm install --no-audit`',
+}).start()
+await execa('npm', ['install', '--no-audit'])
+spinner.succeed()
+
+spinner = ora({
+  spinner: 'star',
+  text: 'Running `npm shrinkwrap`',
+}).start()
+await execa('npm', ['shrinkwrap'])
+spinner.succeed()


Test patch:
diff --git a/.github/workflows/e2e-tests.yml b/.github/workflows/e2e-tests.yml
index a3091281..a9beaaf2 100644
--- a/.github/workflows/e2e-tests.yml
+++ b/.github/workflows/e2e-tests.yml
@@ -31,14 +31,13 @@ jobs:
         uses: actions/setup-node@v3
         with:
           node-version: ${{ matrix.node-version }}
-          cache: 'npm'
-          cache-dependency-path: 'npm-shrinkwrap.json'
+          cache: npm
           check-latest: true
       - name: Cache verdaccio storage
         uses: actions/cache@v3
         with:
           path: ./.verdaccio-storage
-          key: verdaccio-e2e-cli-${{ hashFiles('./npm-shrinkwrap.json') }}
+          key: verdaccio-e2e-cli-${{ hashFiles('./package-lock.json') }}
       - name: Install dependencies
         run: npm ci --no-audit
       - name: Install pnpm
diff --git a/.github/workflows/integration-tests.yml b/.github/workflows/integration-tests.yml
index 3b042dcc..c3f8fefc 100644
--- a/.github/workflows/integration-tests.yml
+++ b/.github/workflows/integration-tests.yml
@@ -47,8 +47,7 @@ jobs:
         uses: actions/setup-node@v3
         with:
           node-version: ${{ matrix.node-version }}
-          cache: 'npm'
-          cache-dependency-path: 'npm-shrinkwrap.json'
+          cache: npm
           check-latest: true
         if: '${{!steps.release-check.outputs.IS_RELEASE}}'
       - name: Setup Deno
diff --git a/.github/workflows/legacy-tests.yml b/.github/workflows/legacy-tests.yml
index 91d27a80..5e05641b 100644
--- a/.github/workflows/legacy-tests.yml
+++ b/.github/workflows/legacy-tests.yml
@@ -47,8 +47,7 @@ jobs:
         uses: actions/setup-node@v3
         with:
           node-version: ${{ matrix.node-version }}
-          cache: 'npm'
-          cache-dependency-path: 'npm-shrinkwrap.json'
+          cache: npm
           check-latest: true
         if: '${{!steps.release-check.outputs.IS_RELEASE}}'
       - name: Setup Deno
diff --git a/.github/workflows/unit-tests.yml b/.github/workflows/unit-tests.yml
index 43115bd3..4fcbb1eb 100644
--- a/.github/workflows/unit-tests.yml
+++ b/.github/workflows/unit-tests.yml
@@ -30,8 +30,7 @@ jobs:
         uses: actions/setup-node@v3
         with:
           node-version: ${{ matrix.node-version }}
-          cache: 'npm'
-          cache-dependency-path: 'npm-shrinkwrap.json'
+          cache: npm
           check-latest: true
         if: '${{!steps.release-check.outputs.IS_RELEASE}}'
       - name: Install core dependencies
diff --git a/tools/affected-test.mjs b/tools/affected-test.mjs
index b46e141d..6e92cd4c 100755
--- a/tools/affected-test.mjs
+++ b/tools/affected-test.mjs
@@ -33,7 +33,7 @@ export const getAffectedFiles = (changedFiles) => {
 
   // in this case all files are affected
   if (
-    changedFiles.includes('npm-shrinkwrap.json') ||
+    changedFiles.includes('package-lock.json') ||
     changedFiles.includes('package.json') ||
     changedFiles.includes(join('.github', 'workflows', 'main.yml'))
   ) {
diff --git a/tools/e2e/setup.mjs b/tools/e2e/setup.mjs
index cc3e4c75..8345a004 100644
--- a/tools/e2e/setup.mjs
+++ b/tools/e2e/setup.mjs
@@ -1,7 +1,7 @@
 import { appendFile, mkdtemp, readFile, rm, writeFile } from 'fs/promises'
 import { tmpdir } from 'os'
 import { dirname, join, normalize, sep } from 'path'
-import { cwd, env } from 'process'
+import { env } from 'process'
 import { fileURLToPath } from 'url'
 
 import execa from 'execa'
@@ -12,6 +12,7 @@ import { runServer } from 'verdaccio'
 import { fileExistsAsync } from '../../src/lib/fs.mjs'
 
 const dir = dirname(fileURLToPath(import.meta.url))
+const rootDir = normalize(join(dir, '../..'))
 
 const VERDACCIO_TIMEOUT_MILLISECONDS = 60 * 1000
 const START_PORT_RANGE = 5000
@@ -24,7 +25,7 @@ const getVerdaccioConfig = () => ({
   // workaround
   // on v5 the `self_path` still exists and will be removed in v6 of verdaccio
   self_path: dir,
-  storage: normalize(join(dir, '../../.verdaccio-storage')),
+  storage: normalize(join(rootDir, '.verdaccio-storage')),
   web: { title: 'Test Registry' },
   max_body_size: '128mb',
   // Disable creation of users this is only meant for integration testing
@@ -104,7 +105,7 @@ export const setup = async () => {
   const { storage, url } = await startRegistry()
   const workspace = await mkdtemp(`${tmpdir()}${sep}e2e-test-`)
 
-  const npmrc = fileURLToPath(new URL('../../.npmrc', import.meta.url))
+  const npmrc = join(rootDir, '.npmrc')
   const registryWithAuth = `//${url.hostname}:${url.port}/:_authToken=dummy`
   let backupNpmrc
 
@@ -125,10 +126,15 @@ export const setup = async () => {
     }
 
     // publish the CLI package to our registry
-    await execa('npm', ['publish', `--registry=${url}`, '--tag=testing', cwd()], {
+    await execa('npm', ['publish', `--registry=${url}`, '--tag=testing'], {
       stdio: env.DEBUG ? 'inherit' : 'ignore',
+      cwd: rootDir,
     })
 
+    // Reset the workspace, as npm publish does patch package.json etc
+    await execa('git', ['checkout', '.'], { cwd: rootDir })
+    await execa('npm', ['install', '--no-audit'], { cwd: rootDir })
+
     console.log(`------------------------------------------
   Published to ${url}
   Verdaccio: ${storage}
diff --git a/tools/tests/affected-files.test.mjs b/tools/tests/affected-files.test.mjs
index 3168a333..f6d9255b 100644
--- a/tools/tests/affected-files.test.mjs
+++ b/tools/tests/affected-files.test.mjs
@@ -44,9 +44,9 @@ test.only('should get all files marked as affected when the package.json is touc
   t.deepEqual(affectedFiles, mockedTestFiles)
 })
 
-test.serial('should get all files marked as affected when the npm-shrinkwrap.json is touched', async (t) => {
+test.serial('should get all files marked as affected when the package-lock.json is touched', async (t) => {
   const consoleStub = t.context.sandbox.stub(console, 'log').callsFake(() => {})
-  const { affectedFiles, mockedTestFiles } = await getAffectedFilesFromMock(['npm-shrinkwrap.json'])
+  const { affectedFiles, mockedTestFiles } = await getAffectedFilesFromMock(['package-lock.json'])
 
   t.truthy(consoleStub.firstCall.calledWith('All files are affected based on the changeset'))
   t.deepEqual(affectedFiles, mockedTestFiles)
diff --git a/tools/tests/utils/file-systems.mjs b/tools/tests/utils/file-systems.mjs
index 7ca5c9ec..9d3fb49f 100644
--- a/tools/tests/utils/file-systems.mjs
+++ b/tools/tests/utils/file-systems.mjs
@@ -1,7 +1,7 @@
 import { join } from 'path'
 
 const baseFiles = {
-  'npm-shrinkwrap.json': '',
+  'package-lock.json': '',
   'README.md': '',
 }