Repository: promptfoo/promptfoo
Issue: version 0.45.0 breaks python providers
Root cause: Configuration Error

Using the provider: `providers: ['python:call_llm.py']`

returns the error:
`[FAIL] Error: ModuleNotFoundError: No module named 'call_llm'`

downgrading to version 0.44.0 solves the issue in my case.


This might have something to do with relative paths or python paths, because the example at https://github.com/promptfoo/promptfoo/tree/main/examples/python-provider works ok for me still.

Can you re-run your eval with `LOG_LEVEL=debug` and let me know what the output is?  For example, `LOG_LEVEL=debug promptfoo eval`

I have the same issue and here is the log
![Screenshot 2024-03-07 at 4 02 57 PM](https://github.com/promptfoo/promptfoo/assets/9929375/2c7134b2-826c-4de1-ac95-36fbf57911b2)
and the error show in the prompt web
![Screenshot 2024-03-07 at 4 05 39 PM](https://github.com/promptfoo/promptfoo/assets/9929375/ae90549d-19bf-468b-9ab5-5e7282775c8f)
and the config file is 

```
description: 'promptfoo templete'

prompts:
  prompt/shelving_prompt.txt: shelving
  prompt/prompt_1.txt: prompt_1
  prompt/prompt_2.txt: prompt_2
  prompt/prompt_few_1.txt: prompt_few_1
  prompt/know_map_prompt.txt: understand

defaultTest:
  assert:
    - type: python
      value: file://evaluation/is_answer_type.py

providers:
  - id: 'python:provider/my_script.py'
    config:
      additionalOption: 123
    prompts: [understand]

tests: [test/short_info.yml]
```

and also when I use 0.44.0. it worked fine. thanks for your contribution. This help me a lot.

version 44.0:
``` LOG_LEVEL=debug PROMPTFOO_PYTHON=/llmdata/training_scality_venv/bin/python npx promptfoo@0.44.0 eval --no-cache -j 1 --verbose

===================================================================================================================================================================================================================================
⚠️ The current version of promptfoo 0.44.0 is lower than the latest available version 0.45.1.

Please run npx promptfoo@latest or npm install -g promptfoo@latest to update.
===================================================================================================================================================================================================================================

Cache is disabled.
Running python script /home/patrick/promptfoo/call_llm.py with scriptPath call_llm.py and args ["/home/patrick/promptfoo/call_llm.py","What is the cube root of 389017?","{\"config\":{\"basePath\":\"\"}}","{\"vars\":{\"question\":\"What is the cube root of 389017?\"}}"]
Python script /home/patrick/promptfoo/call_llm.py returned: Importing module call_llm from /home/patrick/promptfoo...
Prompt: What is the cube root of 389017?
{"type": "final_result", "data": {"output": "\nTo find the cube root of a number, you need to find a number that, when multiplied by itself three times, gives you the original number. In this case, we are looking for the cube root of 389017.\n\nTo find the cube root of 389017, we can use the following formula:\n\ncube root of x = (x^(1/3))\n\nSo, we can plug in the number 389017 into the formula:\n\ncube root of 389017 = (389017^(1/3))\n\nUsing a calculator, we can simplify this expression to get:\n\ncube root of 389017 \u2248 62.4\n\nTherefore, the cube root of 389017 is approximately 62.4."}}
Eval #1 complete (1 of 1)
===================================================================================================================================================================================================================================
✔ Evaluation complete.

» Run promptfoo view to use the local web viewer
» Run promptfoo share to create a shareable URL
» This project needs your feedback. What's one thing we can improve? https://forms.gle/YFLgTe1dKJKNSCsU7
===================================================================================================================================================================================================================================
Successes: 0
Failures: 1
Token usage: Total 0, Prompt 0, Completion 0, Cached 0
Done.
```
version 45.0
```LOG_LEVEL=debug PROMPTFOO_PYTHON=/llmdata/training_scality_venv/bin/python npx promptfoo@0.45.0 eval --no-cache -j 1 --verbose
Need to install the following packages:
promptfoo@0.45.0
Ok to proceed? (y) y
npm WARN deprecated lodash-node@2.4.1: This package is discontinued. Use lodash@^4.0.0.

===================================================================================================================================================================================================================================
⚠️ The current version of promptfoo 0.45.0 is lower than the latest available version 0.45.1.

Please run npx promptfoo@latest or npm install -g promptfoo@latest to update.
===================================================================================================================================================================================================================================

Cache is disabled.
Running python script /home/patrick/.npm/_npx/4ffd68d229f0148d/node_modules/promptfoo/dist/src/providers/call_llm.py with scriptPath call_llm.py and args: What is the cube root of 389017?
{"config":{"basePath":""}}
{"vars":{"question":"What is the cube root of 389017?"}}
Error running Python script: Error: ModuleNotFoundError: No module named 'call_llm'
Eval #1 complete (1 of 1)
===================================================================================================================================================================================================================================
✔ Evaluation complete.

» Run promptfoo view to use the local web viewer
» Run promptfoo share to create a shareable URL
» This project needs your feedback. What's one thing we can improve? https://forms.gle/YFLgTe1dKJKNSCsU7
===================================================================================================================================================================================================================================
Successes: 0
Failures: 1
Token usage: Total 0, Prompt 0, Completion 0, Cached 0
Done.
```
it returns:
```
Error: ModuleNotFoundError: No module named 'call_llm'

Error: ModuleNotFoundError: No module named 'call_llm'
    at PythonShell.parseError (/home/patrick/.npm/_npx/4ffd68d229f0148d/node_modules/python-shell/index.js:303:21)
    at terminateIfNeeded (/home/patrick/.npm/_npx/4ffd68d229f0148d/node_modules/python-shell/index.js:193:32)
    at ChildProcess.<anonymous> (/home/patrick/.npm/_npx/4ffd68d229f0148d/node_modules/python-shell/index.js:185:13)
    at ChildProcess.emit (node:events:514:28)
    at ChildProcess._handle.onexit (node:internal/child_process:294:12)
-- Python Traceback -----
    File "/home/patrick/.npm/_npx/4ffd68d229f0148d/node_modules/promptfoo/dist/src/python/wrapper.py", line 25, in <module>
      result = call_method(script_path, method_name, *args)
    File "/home/patrick/.npm/_npx/4ffd68d229f0148d/node_modules/promptfoo/dist/src/python/wrapper.py", line 13, in call_method
      script_module = importlib.import_module(module_name)
    File "/opt/conda/lib/python3.10/importlib/__init__.py", line 126, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
    File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
    File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
```

version 45.1
```
 LOG_LEVEL=debug PROMPTFOO_PYTHON=/llmdata/training_scality_venv/bin/python npx promptfoo@0.45.1 eval --no-cache -j 1 --verbose
Need to install the following packages:
promptfoo@0.45.1
Ok to proceed? (y) y
npm WARN deprecated lodash-node@2.4.1: This package is discontinued. Use lodash@^4.0.0.
Cache is disabled.
Computing file hash for script /home/patrick/.npm/_npx/3d59446856b4bce3/node_modules/promptfoo/dist/src/providers/call_llm.py
Eval #1 complete (1 of 1)
===================================================================================================================================================================================================================================
✔ Evaluation complete.

» Run promptfoo view to use the local web viewer
» Run promptfoo share to create a shareable URL
» This project needs your feedback. What's one thing we can improve? https://forms.gle/YFLgTe1dKJKNSCsU7
===================================================================================================================================================================================================================================
Successes: 0
Failures: 1
Token usage: Total 0, Prompt 0, Completion 0, Cached 0
Done.
```
It returns
```Error: ENOENT: no such file or directory, open '/home/patrick/.npm/_npx/3d59446856b4bce3/node_modules/promptfoo/dist/src/providers/call_llm.py'

Error: ENOENT: no such file or directory, open '/home/patrick/.npm/_npx/3d59446856b4bce3/node_modules/promptfoo/dist/src/providers/call_llm.py'
    at Object.readFileUtf8 (node:internal/fs/sync:25:18)
    at Object.readFileSync (node:fs:441:19)
    at PythonProvider.callApi (/home/patrick/.npm/_npx/3d59446856b4bce3/node_modules/promptfoo/dist/src/providers/pythonCompletion.js:24:58)
    at Evaluator.runEval (/home/patrick/.npm/_npx/3d59446856b4bce3/node_modules/promptfoo/dist/src/evaluator.js:202:45)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /home/patrick/.npm/_npx/3d59446856b4bce3/node_modules/promptfoo/dist/src/evaluator.js:526:25```

To add more information/context.
I use a virtualenvironment using the venv module with python 3.10 (did not check with another Python version) with
`python3.10 -mvenv venv_path`

and the call_llm.py script is:
```python
"""
This is a Python script that makes a POST request to an API endpoint.
"""

import requests
import logging
logger = logging.getLogger('my_logger')
logger.setLevel(logging.INFO)
handler = logging.FileHandler('logfile.log')
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)


def call_api(prompt: str, options: dict, context: dict):
    """
    Call the API endpoint
    """

    print(f"\nPrompt: {prompt}")
    # The 'options' parameter contains additional configuration for the API call.
    additional_option = options.get('additionalOption', None)

    # The 'context' parameter provides info about which vars were used to create the final prompt.
    user_variable = context['vars'].get('userVariable', None)

    # Make a POST request to the API endpoint
    response = requests.post(
        'http://localhost:1081/',
        json={
            'user_prompt': prompt,
            'options': additional_option,
            'context': user_variable
        },
        headers={'Accept': 'application/json'},  # Add header accepting JSON
        timeout=60
    )

    # Check if the request was successful (status code 200)
    if response.status_code == 200:
        # Get the response data as a dictionary
        response_data = response.json()
        logger.info(f"Response data: {response_data}")

        # Extract the output from the response data
        output = response_data.get('output')

        # Handle any errors in the response
        if 'error' in response_data:
            error_message = response_data['error']
            # Handle the error condition here

        # Extract token usage information if available
        if 'tokenUsage' in response_data:
            token_usage = response_data['tokenUsage']
            token_count = token_usage.get('total')
            prompt_token_count = token_usage.get('prompt')
            completion_token_count = token_usage.get('completion')
            # Process token usage information here

    else:
        # Handle the case when the request was not successful
        error_message = f"Request failed with status code {response.status_code}"
        raise requests.exceptions.RequestException(error_message)

    # Construct the result dictionary
    result = {
        "output": output,
    }

    if 'error' in locals():
        result['error'] = error_message

    if 'token_count' in locals():
        result['tokenUsage'] = {"total": token_count, "prompt": prompt_token_count, "completion": completion_token_count}

    return result

```


Thanks, this is helpful. #535 should fix and I'll publish it in 0.45.2 asap

I confirm that version 0.45.2 fixes the issue.

Thank you for your help and for developing this software, you rock!