Repository: promptfoo/promptfoo
Issue: Error: spawn python ENOENT
Root cause: Environment Incompatibility

**Describe the bug**
A clear and concise description of what the bug is.
docker error log:
```
2024-08-07 20:05:17 Error: spawn python ENOENT
2024-08-07 20:05:17     at ChildProcess._handle.onexit (node:internal/child_process:286:19)
2024-08-07 20:05:17     at onErrorNT (node:internal/child_process:484:16)
2024-08-07 20:05:17     at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-08-07 20:05:17   errno: -2,
2024-08-07 20:05:17   code: 'ENOENT',
2024-08-07 20:05:17   syscall: 'spawn python',
2024-08-07 20:05:17   path: 'python',
2024-08-07 20:05:17   spawnargs: [
2024-08-07 20:05:17     '/app/src/web/nextui/.next/server/app/api/eval/job/wrapper.py',
2024-08-07 20:05:17     '/app/src/web/nextui/resp-gen-ip.py',
2024-08-07 20:05:17     'get_var',
2024-08-07 20:05:17     '/tmp/promptfoo-python-input-json-1723041317203-ff97d966e2159.json'
2024-08-07 20:05:17   ]
2024-08-07 20:05:17 }
```

**To Reproduce**
Steps to reproduce the behavior, including example Promptfoo configurations if applicable:
1. check out the latest code
2. build docker image
`docker build --build-arg NEXT_PUBLIC_PROMPTFOO_BASE_URL=http://localhost:3000 -t promptfoo-ui .`
3. run the container
`docker run -d --add-host host.docker.internal:host-gateway --name promptfoo_container -p 3000:3000  -p 11434:11434 -v /Users/deepakkumar.sahoo/add_bins/promptfoo_workspace:/root/.promptfoo -e OPENAI_API_KEY=sk-xxxx -e PROMPTFOO_CONFIG_DIR=~/.promptfoo promptfoo-ui`
4. can see the hosted instance in localhost:3000
5. Now create promptfooconfig.yaml file like below
```
description: 'Dynamic var example'

prompts:
  - "{{context}} \n\n {{userip}}"

providers:
  - id: ollama:llama3

tests:
  - vars:
      userip: file://resp-gen-ip.py
      context: file://resp-gen.py
```
6. copy resp-gen-ip.py and resp-gen-ip.py files to `/app/src/web/nextui/` since for some reason if the py files are not present here getting error.
```
No files found for variable userip at path /app/src/web/nextui/resp-gen-ip.py
2024-08-07 20:04:34     at generateVarCombinations (/app/src/web/nextui/.next/server/app/api/eval/job/route.js:10351:23)
2024-08-07 20:04:34     at Evaluator.evaluate (/app/src/web/nextui/.next/server/app/api/eval/job/route.js:10727:17)
2024-08-07 20:04:34     at evaluate (/app/src/web/nextui/.next/server/app/api/eval/job/route.js:11056:15)
2024-08-07 20:04:34     at Object.index_evaluate [as evaluate] (/app/src/web/nextui/.next/server/app/api/eval/job/route.js:11629:23)
2024-08-07 20:04:34     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
```
7. Run eval from the hosted instance

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**System information (please complete the following information):**

- Promptfoo version:

**Additional context**
Add any other context about the problem here.


![Uploading Screenshot 2024-08-07 at 8.14.18 PM.png…]()


Just now tried with the example 
https://github.com/promptfoo/promptfoo/blob/main/examples/python-assert-inline/promptfooconfig.yaml
Getting similar error
```
Error: spawn python ENOENT
2024-08-07 22:26:04     at ChildProcess._handle.onexit (node:internal/child_process:286:19)
2024-08-07 22:26:04     at onErrorNT (node:internal/child_process:484:16)
2024-08-07 22:26:04     at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-08-07 22:26:04   errno: -2,
2024-08-07 22:26:04   code: 'ENOENT',
2024-08-07 22:26:04   syscall: 'spawn python',
2024-08-07 22:26:04   path: 'python',
2024-08-07 22:26:04   spawnargs: [
2024-08-07 22:26:04     '/app/src/web/nextui/.next/server/app/api/eval/job/wrapper.py',
2024-08-07 22:26:04     '/tmp/temp-python-code-1723049764221-7895919c037ee.py',
2024-08-07 22:26:04     'main',
2024-08-07 22:26:04     '/tmp/promptfoo-python-input-json-1723049764222-9e2bbf7afd8ff.json'
2024-08-07 22:26:04   ]
2024-08-07 22:26:04 }


```

Has anyone tried python  tests or asserts directly in the UI by using hosted instance (using the docker image) ?

This happens because there is no python runtime in the Docker image.  The hosted instance doesn't support Python asserts at this time, because it is a less common use case to want to run the eval directly in the image, but you can probably get it working by installing Python in the Docker build - I'd happily accept a PR for this

Hi @dmockingbird551, thanks so much for bringing this up and for your patience! We fixed the issue by adding a Python runtime to the Docker image as part of [this PR](https://github.com/promptfoo/promptfoo/pull/1519). Apologies for not notifying you sooner! Please feel free to pull the latest image, and if you run into any further issues, don’t hesitate to re-open this or create a new issue.

I'm still seeing this problem after install via npm:
```
npm install -g promptfoo
promptfoo init
```

```
$ promptfoo --version
0.87.1
```

Any ideas?

You might need to set your PROMPTFOO_PYTHON environment variable:
https://www.promptfoo.dev/docs/providers/python/#setting-the-python-binary-path

On Sun, Sep 15, 2024 at 8:19 AM Chris DiGiano ***@***.***>
wrote:

> I'm still seeing this problem after install via npm:
>
> npm install -g promptfoo
> promptfoo init
>
> $ promptfoo --version
> 0.87.1
>
> Any ideas?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/promptfoo/promptfoo/issues/1389#issuecomment-2351635068>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACLYJRDDND5ORS3BSW7QVLZWWQO7AVCNFSM6AAAAABMESME7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJRGYZTKMBWHA>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>


Hi @justdigi, apologies for the confusion! I added Python support in #1519 but we removed it in #1599. I’ll re-add it and test it now. Thanks for your patience, and I’ll keep you posted once it's ready!

The fix is in https://github.com/promptfoo/promptfoo/pull/1677. I’ve also added checks in the CI to make sure Python stays in the image going forward. Thanks so much for your patience and for flagging this—please don’t hesitate to reach out if you run into any other issues.