Repository: promptfoo/promptfoo
Issue: ModuleNotFound error with custom python provider
Root cause: Environment Incompatibility

Hello,

I recently added a custom provider with the `call_api` function defined in the doc but when I try to run it, I am getting a `FAILError: ModuleNotFoundError`. It looks like promptfoo uses the packages from node to search for the langchain packages to run the chain in `call_api` but I  want to use the poetry packages instead

```
FAILError: ModuleNotFoundError: No module named 'langchain'

Error: ModuleNotFoundError: No module named 'langchain'
    at PythonShell.parseError (/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/node_modules/python-shell/index.js:303:21)
    at terminateIfNeeded (/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/node_modules/python-shell/index.js:193:32)
    at ChildProcess.<anonymous> (/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/node_modules/python-shell/index.js:185:13)
    at ChildProcess.emit (node:events:518:28)
    at ChildProcess._handle.onexit (node:internal/child_process:294:12)
-- Python Traceback -----
    File "/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/dist/src/providers/wrapper.py", line 25, in <module>
      result = call_api(script_path, prompt, options, context)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/dist/src/providers/wrapper.py", line 13, in call_api
      script_module = importlib.import_module(module_name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/Cellar/python@3.11/3.11.8/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
    File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
    File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 940, in exec_module
    File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
    File "/Users/my.name/Code/path/to/my/call_api_function.py", line 6, in <module>
      from langchain.prompts import PromptTemplate
---

```

<details>
<summary>Other example with another call_api function</summary>

```
FAILError: ModuleNotFoundError: No module named 'langchain_community'

Error: ModuleNotFoundError: No module named 'langchain_community'
    at PythonShell.parseError (/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/node_modules/python-shell/index.js:303:21)
    at terminateIfNeeded (/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/node_modules/python-shell/index.js:193:32)
    at ChildProcess.<anonymous> (/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/node_modules/python-shell/index.js:185:13)
    at ChildProcess.emit (node:events:518:28)
    at ChildProcess._handle.onexit (node:internal/child_process:294:12)
-- Python Traceback -----
    File "/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/dist/src/providers/wrapper.py", line 25, in <module>
      result = call_api(script_path, prompt, options, context)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/my.name/.nvm/versions/node/v20.11.1/lib/node_modules/promptfoo/dist/src/providers/wrapper.py", line 13, in call_api
      script_module = importlib.import_module(module_name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/Cellar/python@3.11/3.11.8/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
    File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
    File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 940, in exec_module
    File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
    File "/Users/my.name/Code/path/to/my/call_api_function.py", line 6, in <module>
      from api.src.internal.llm_chain import llm_chain
    File "/Users/my.name/Code/path/to/my/internal/chain.py", line 5, in <module>
      from langchain_community.retrievers import GoogleVertexAIMultiTurnSearchRetriever
---
```


</details>

I added the python binary from poetry `PROMPTFOO_PYTHON`, as described [here](https://promptfoo.dev/docs/providers/python/#troubleshooting) but I am getting the same error



Hey @romaintoub, can you try again with version 0.44.0, which was just released?  The docs were updated before the corresponding package was published.

It also looks like you may have to set `PYTHONPATH`, because it won't know where your python libraries are installed even with the pointer to the poetry binary.

I just tried with v0.44.0 and still getting the same error

just to clarify in the doc for [troubleshooting](https://promptfoo.dev/docs/providers/python/#troubleshooting):
- I need to set `PROMPTOO_PYTHON` as my python path for promptfoo? eg. If I use poetry this should be something like `path/to/.venv/bin/python`
- I need to set `PYTHONPATH` to the exact same path as well?

`PYTHONPATH` is an environment variable directly supported by the Python
interpreter.  I _think_ the poetry python binary will set it
automatically.

You're correct that `PROMPTFOO_PYTHON` should point to the binary.  You may
want to double check that you're pointing to the right binary by running
`poetry env info`.

On Mon, Mar 4, 2024 at 11:50 AM Romain ***@***.***> wrote:

> I just tried with v0.44.0 and still getting the same error
>
> just to clarify in the doc for troubleshooting
> <https://promptfoo.dev/docs/providers/python/#troubleshooting>:
>
>    - I need to set PROMPTOO_PYTHON as my python path for promptfoo? eg.
>    If I use poetry this should be something like path/to/.venv/bin/python
>    - I need to set PYTHONPATH to the exact same path as well?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/promptfoo/promptfoo/issues/524#issuecomment-1977341925>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACLYJUW52IGSGYW77EF3B3YWTF63AVCNFSM6AAAAABEFWEV2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGM2DCOJSGU>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>


I managed to make it work by reinstalling poetry, remove and reclone the repo, not sure how the PROMPTFOO_PYTHON was working though
thanks for your help!