Repository: promptfoo/promptfoo
Issue: Peer dependency considerations
Root cause: Architectural Constraint

Unfortunately, Node.js ecosystem has diverged from the [original peer dependency meaning](https://nodejs.org/en/blog/npm/peer-dependencies).

Both npm and pnpm install peer deps automatically, equating them to production deps.

- https://pnpm.io/next/npmrc#auto-install-peers
- https://github.blog/news-insights/product-news/presenting-v7-0-0-of-the-npm-cli/

However, this brings a problem, where _ALL_ peer deps are _ALWAYS_ installed. And promptfoo's own list isn't particularly small:

https://github.com/promptfoo/promptfoo/blob/c10489b884bb5ffaaf916772a727af4420ff255f/package.json#L73-L90

This adds significant weight to the package, and often installs unneeded vendor deps for services user will never use.

There's one notable dep that is heavy and, especially in CI, slows down things: `@playwright/browser-chromium`

This dep will try to download and install the entire Chrome browser. I know this can be turned off with `PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1` setting, but that's not an ideal solution.

I don't know what the correct solution is, just opening the issue for tracking, and perhaps an open discussion.

Hi @moltar, thanks for bringing this up! I’m currently looking into potential solutions to address this, including exploring ways to make these dependencies optional or restructuring how they’re managed. I've also started to pull more dependencies out of promptfoo. See https://github.com/promptfoo/promptfoo/pull/2833 and https://github.com/promptfoo/promptfoo/pull/2861 for examples.

I’ll update this issue as I make progress.

Apparently there's a way to mark peer deps optional:

https://docs.npmjs.com/cli/v11/configuring-npm/package-json#peerdependenciesmeta

Although this may constitute a breaking change and a major bump as everyone who got peers automatically before will no longer get them and will have a breakage. 

> Although this may constitute a breaking change and a major bump as everyone who got peers automatically before will no longer get them and will have a breakage.

`promptfoo` still has a `0.` at the start of the version number, and [NPM's default Caret range with `^` treats `0.X` version bumps as breaking](https://github.com/npm/node-semver#caret-ranges-123-025-004), so I don't think that would be an issue.

I would also love something similar! I was thinking of making a PR to add an `peerDependenciesMeta` field to the `package.json` to make them all as optional, but I couldn't think of a good way to automatically test in CI what happens if any of the `peerDependencies` don't exist.

[`npm install` has a `--omit` option](https://docs.npmjs.com/cli/v11/commands/npm-install#omit) that would allow us to exclude installing `optionalPeerDependencies` in a separate CI run, ~but I don't think `pnpm` sorts anything similar yet, and you can't convert a `pnpm-lock.yaml` file to `package-lock.json`~ _Edit_: Nevermind, for some reason I thought this repo was using `pnpm` not `npm` :laughing:.

I think it looks like pnpm is just not being nice!

https://github.com/pnpm/pnpm/issues/5928

Thanks for raising this @moltar, and to @aloisklink for the `peerDependenciesMeta` suggestion.

This has been addressed in #5948 - we moved the provider-specific dependencies from peer to optional. This means they won't be auto-installed anymore, so your install times (especially in CI) should be much better. No more surprise Chromium downloads.

If you're using a specific provider, you'll need to install its dependencies explicitly, but the docs for each provider list what's needed.

Closing this out, but feel free to reopen if you're still seeing issues with the dependency footprint.