Repository: promptfoo/promptfoo
Issue: Is there a way to attach a Debugger to the call_api Python External Provider
Root cause: Documentation Deficiency

I was following this guide to implement a custom local provider of LLMs https://www.promptfoo.dev/docs/providers/python and seem to have some issues that I would love to debug.

My implementation :
```
    # context_instance contains both the system and user prompt for a task
    output = call_llm(context_instance, model=model, tokens=tokens, temperature=temperature)
    

    # The result should be a dictionary with at least an 'output' field.
    result = {
        "output": output,
    }
        }
    logger.error(f"DEBUG >>  {output}")
    logger.info(f"DEBUG >>  {result}")

    return result
```

When checking the logs the results is correctly populated however later in node_modules\promptfoo\dist\src\providers\pythonCompletion.js:52:36 the value is null.

**I was wandering how can I breakpoint promptfoo there and then attach a Python debugger ?**

### Expected Results
The outputed result is neatly displayed in the table and promtfoo works normally.
### Actual Results

| question                                                                                                             | [Development Phase ] Tests                                                                                 |
| ---------------------------------------------------------------------------------------------------------------------| -------------------------------------------------------------------------------------------------------------------------- |
| Give me list of Failed Transactions                         | [FAIL] TypeError: Cannot use 'in' operator to search for 'output' in null                                                  |
|                                                                                                                       | TypeError: Cannot use 'in' operator to search for 'output' in null                                                                                                                                                                                at PythonProvider.executePythonScript (npm\node_modules\promptfoo\dist\src\...            |


https://github.com/promptfoo/promptfoo/pull/957 improves the error message here

Can you run your eval with `LOG_LEVEL=debug` environment variable set?  That will help understand the root cause.

```
LOG_LEVEL=debug promptfoo eval
```

Hey @typpo  thanks for the quick response I managed to identify the issue without much debugging so now it's working normally and running LOG_LEVEL=debug would not yield something relevant.

However I still think it would be great for the application/developer community if there was a way to quickly attach a debugger to your custom code. Thank you !

Would be sweet to be able to hit a `import pdb; pdb.set_trace()` breakpoint. Currently you get an exception 

```
Error running Python script: bdb.BdbQuit
Stack Trace: Error: bdb.BdbQuit
    at PythonShell.parseError (/home/discord/.npm/_npx/81bbc6515d992ace/node_modules/python-shell/index.js:303:21)
    at terminateIfNeeded (/home/discord/.npm/_npx/81bbc6515d992ace/node_modules/python-shell/index.js:193:32)
    at ChildProcess.<anonymous> (/home/discord/.npm/_npx/81bbc6515d992ace/node_modules/python-shell/index.js:185:13)
    at ChildProcess.emit (node:events:518:28)
    at ChildProcess._handle.onexit (node:internal/child_process:294:12)
    --Python Traceback: --
    File "/home/discord/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/python/wrapper.py", line 34, in <module>
      result = call_method(script_path, method_name, *data)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/discord/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/python/wrapper.py", line 24, in call_method
      return method_to_call(*args)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/home/discord/github.com/discord/discord/discord_api/discord/lib/admin/cx_autopilot/promptfoo.py", line 36, in call_workflow
      print(prompt, options, context)
      ^^^^^
    File "/usr/lib/python3.11/bdb.py", line 90, in trace_dispatch
      return self.dispatch_line(frame)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.11/bdb.py", line 115, in dispatch_line
      if self.quitting: raise BdbQuit
                        ^^^^^^^^^^^^^
```
                        
However, [remote-pdb](https://pypi.org/project/remote-pdb/) works. Slap this in your code after pip installing it. 
```python
from remote_pdb import RemotePdb
RemotePdb('127.0.0.1', 4444).set_trace()
```  

Hey @SysOverdrive, thanks for the remote-pdb suggestion and for your patience! I just added some notes about debugging in https://github.com/promptfoo/promptfoo/commit/41cc82b2489efce4b167ebb25cc8cc6bcaf667b9, which should help clarify how to debug custom Python providers. We've also improved logging functionality - running with `LOG_LEVEL=debug` or using the `--verbose` flag should now provide more visibility into what’s happening.

I'm closing this issue, but if you're still encountering problems or have more suggestions, feel free to reflag or comment. Thanks again for your contributions and feedback!