Repository: promptfoo/promptfoo
Issue: Custom providers don't respect $PROMPTFOO_PYTHON
Root cause: Configuration Error

**Describe the bug**

When using `PROMPTFOO_PYTHON` to specify a Python virtual environment, providers are not executed using this environment. It executes in the system default environment instead. This is evident when the virtual environment contains packages not present in the system environment.

**To Reproduce**

1. Create a Python virtual environment.
2. Inside this environment, install a package you don't have in your system environment, e.g. `python -m pip install pypdfium2`
3. Verify this package is not in your default environment. Navigate away from the virtual environment and run `python -m pip show pypdfium2` and observe a warning that the package was not found.
4. Create a promptfoo config that loads a Python provider: `python:./my_provider.py`
5. In this provider, import the package: `import pypdfium2`
6. Export `PROMPTFOO_PYTHON` to point to your virtual environment.
7. Run `promptfoo eval`.

**Expected behavior**
GIVEN a Python virtual environment
AND the path to this environment is set in the `PROMPTFOO_PYTHON` environment variable
AND a custom provider is specified using `python:./my_provider.py`
WHEN I run `promptfoo eval`
THEN the provider should execute in the Python virtual environment.

**System information:**
```json
{
  "version": "0.120.8",
  "platform": {
    "os": "darwin",
    "release": "25.2.0",
    "arch": "arm64",
    "nodeVersion": "v24.12.0"
  },
  "env": {
    "telemetryDisabled": false,
    "telemetryDebug": false
  },
  "configInfo": {
    "configExists": false,
    "configContent": null
  }
}
```


<!-- linear-linkback -->
<p><a href="https://linear.app/promptfooo/issue/ENG-1353/custom-providers-dont-respect-dollarpromptfoo-python">ENG-1353</a></p>

<!-- This is an auto-generated issue plan by CodeRabbit -->


### 📝 CodeRabbit Plan Mode
Generate an implementation plan and prompts that you can use with your favorite coding agent.

- [ ] <!-- {"checkboxId": "8d4f2b9c-3e1a-4f7c-a9b2-d5e8f1c4a7b9"} --> Create Plan

<details>
<summary>Examples</summary>

- [Example 1](https://github.com/coderabbitai/git-worktree-runner/issues/29#issuecomment-3589134556)
- [Example 2](https://github.com/coderabbitai/git-worktree-runner/issues/12#issuecomment-3606665167)

</details>

---

<details>
<summary><b>🔗 Similar Issues</b></summary>

**Related Issues**
- https://github.com/promptfoo/promptfoo/issues/2402
- https://github.com/promptfoo/promptfoo/issues/5554
- https://github.com/promptfoo/promptfoo/issues/5534
- https://github.com/promptfoo/promptfoo/issues/1756
- https://github.com/promptfoo/promptfoo/issues/2534
- https://github.com/promptfoo/promptfoo/issues/3991
- https://github.com/promptfoo/promptfoo/issues/5519
- https://github.com/promptfoo/promptfoo/issues/3501
</details>
<details>
<summary><b>👤 Suggested Assignees</b></summary>

- [martinremy](https://github.com/martinremy)
- [dougli](https://github.com/dougli)
- [ladyofcode](https://github.com/ladyofcode)
- [bfdykstra](https://github.com/bfdykstra)
- [jammy23495](https://github.com/jammy23495)
</details>


---
<details>
<summary> 🧪 Issue enrichment is currently in open beta.</summary>


You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your `.coderabbit.yaml`:
```yaml
issue_enrichment:
  auto_enrich:
    enabled: false
```
</details>

💬 Have feedback or questions? Drop into our [discord](https://discord.gg/coderabbit) or [schedule a call](https://calendly.com/parth-coderabbit/issues-feedback)!