Repository: next.js
Issue: next build script not using npm token when downloading missing @next/swc-* optional dependencies
Labels: Upstream,bug,SWC

### Link to the code that reproduces this issue

https://github.com/robertwbradford/create-next-app-with-private-npm-registry

### To Reproduce

Our organization has a private npm registry that requires authentication to access it.  I have the necessary token in my npm configuration.

(Note that the repo I linked to for reproducing the error will not work as is because I have set the npm registry to a fake private registry. If you clone that repo and replace the `https://our-private-npm-registry.com` URLs in `.npmrc` and `package-lock.json` with a private registry you have access and that requires authentication, you can reproduce.)

1. Delete the project's `node_modules`
2. Run `npm ci --omit=dev --omit=optional`
3. Run `npm run build` (where the build script is `next build`)

### Current vs. Expected behavior

Next lists the various `@next/swc-*` packages as optional, so they are not installed as part of Step 2. Currently the `next build` script appears to use https://github.com/vercel/next.js/blob/9e33bf6ab929c81f694880460cce90156ecbfd6f/packages/next/src/lib/download-swc.ts#L30-L42 to manually download the needed `@next/swc-*` package when it is not detected. I would expect Next to install the needed package using the npm token from my npm configuration.

However, because Next is manually downloading/fetching this package (instead of using `npm i`), the token is not applied when requesting the download and we see this output:

```
my-app % npm ci --omit=dev --omit=optional
added 309 packages, and audited 310 packages in 6s

found 0 vulnerabilities

my-app % npm run build
> my-app@5.4.2 build
> next build

 ✓ Linting and checking validity of types    
   ▲ Next.js 14.0.3

   Downloading swc package @next/swc-darwin-arm64...
 ⨯ Failed to download swc package from https://our-private-npm-registry.com/@next/swc-darwin-arm64/-/swc-darwin-arm64-14.0.3.tgz
unhandledRejection Error: request failed with status 401
    at /Users/bradfordrw/dev/my-app/node_modules/next/dist/lib/download-swc.js:96:23
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async extractBinary (/Users/bradfordrw/dev/my-app/node_modules/next/dist/lib/download-swc.js:90:9)
    at async downloadNativeNextSwc (/Users/bradfordrw/dev/my-app/node_modules/next/dist/lib/download-swc.js:139:9)
    at async tryLoadNativeWithFallback (/Users/bradfordrw/dev/festack-bootcamp/node_modules/next/dist/build/swc/index.js:291:5)
    at async /Users/bradfordrw/dev/festack-bootcamp/node_modules/next/dist/build/swc/index.js:275:40
```

If we do not omit the optional dependencies in Step 2, it works fine. However, this is a problem because we end up with a huge application (900 MB+), and this is an issue when when trying to deploy through a pipeline. It appears that [the direction from Next is to omit optional dependencies](https://github.com/vercel/next.js/issues/32183#issuecomment-1233449967).

### Verify canary release

- [X] I verified that the issue exists in the latest Next.js canary release

### Provide environment information

```bash
Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 22.6.0: Thu Nov  2 07:43:57 PDT 2023; root:xnu-8796.141.3.701.17~6/RELEASE_ARM64_T6000
Binaries:
  Node: 20.9.0
  npm: 10.2.4
  Yarn: N/A
  pnpm: 8.6.7
Relevant Packages:
  next: 14.0.4-canary.25
  eslint-config-next: 14.0.3
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.3.2
Next.js Config:
  output: N/A
```


### Which area(s) are affected? (Select all that apply)

Operating System (Windows, MacOS, Linux), Package manager (npm, pnpm, Yarn), SWC transpilation

### Additional context

_No response_