Repository: promptfoo/promptfoo
Issue: Javascript object being passed to Python provider?
Root cause: Algorithmic Error

**Describe the bug**
I have a Python provider that stopped working.  I think there's some kind of serialization issue where the prompt is being serialized as a JS representation before being passed to the Python provider.

Here's a minimal example:

`promptfooconfig.yaml`

```
prompts: ['{{ question }}']
providers:
  - id: 'python:js_to_py_demo.py'
tests:
  - description: Example test
    vars:
      question: Why is the sky blue?
    assert:
      - type: contains
        value: scatter
```

`js_to_py_demo.py`

```
def call_api(prompt: str, options: dict[str, any], context: dict[str, any]):

    print(f"Prompt:>>>{prompt}<<<")

    output = {
        "text": "This is a response from the test provider",
    }

    return {"output": output}
```

**To Reproduce**

1. Run `LOG_LEVEL=debug npx promptfoo@latest eval`
2. Output contains:

<img width="214" alt="Screenshot 2024-12-13 at 17 48 20" src="https://github.com/user-attachments/assets/63feb4d5-84cb-43ad-be2e-a6f674817c2e" />

**Expected behavior**

Output should include `Prompt:>>>Why is the sky blue?<<<`

**Actual output**

```
% LOG_LEVEL=debug npx promptfoo@latest eval --no-cache
Running migrations from /Users/mremy/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/drizzle...
Running migrations from /Users/mremy/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/drizzle...
Cache is disabled.
Reading prompts from ["{{ question }}"]
Inserting prompt ef6fb466fa88d98eeffaf3867ef8b795f2a823ec6fb50efed675159da8dd4140
Inserting dataset 20066d831946f34047e62a28e55ff40a92877c1c4a0c2b9bb60467b8cfdc4d4e
Running 1 concurrent evaluations with 4 threads...
Computing file hash for script /Users/mremy/tmp/promptfoo_js_to_py_test/js_to_py_demo.py
Running python script /Users/mremy/tmp/promptfoo_js_to_py_test/js_to_py_demo.py with scriptPath js_to_py_demo.py and args: ["{\n  \"[object Object]\": null\n}",{"id":"python:js_to_py_demo.py","config":{"basePath":""}},{"vars":{"question":"Why is the sky blue?"},"prompt":{"raw":"{{ question }}","label":"{{ question }}"},"filters":{},"originalProvider":{"scriptPath":"js_to_py_demo.py","functionName":null,"label":"","delay":0}}]
Running Python wrapper with args: ["{\n  \"[object Object]\": null\n}",{"id":"python:js_to_py_demo.py","config":{"basePath":""}},{"vars":{"question":"Why is the sky blue?"},"prompt":{"raw":"{{ question }}","label":"{{ question }}"},"filters":{},"originalProvider":{"scriptPath":"js_to_py_demo.py","functionName":null,"label":"","delay":0}}]
Importing module js_to_py_demo from /Users/mremy/tmp/promptfoo_js_to_py_test ...
Prompt:>>>{
  "[object Object]": null
}<<<
Python script /Users/mremy/tmp/promptfoo_js_to_py_test/js_to_py_demo.py returned: {"type": "final_result", "data": {"output": {"text": "This is a response from the test provider"}}}
Eval #1 complete (1 of 1)
At least one evaluation failure occurred. This might be caused by the underlying call to the provider, or a test failure. Context:
[{"raw":"{{ question }}","label":"{{ question }}","id":"ef6fb466fa88d98eeffaf3867ef8b795f2a823ec6fb50efed675159da8dd4140","provider":"python:js_to_py_demo.py","metrics":{"score":0,"testPassCount":0,"testFailCount":1,"testErrorCount":0,"assertPassCount":0,"assertFailCount":1,"totalLatencyMs":420,"tokenUsage":{"total":0,"prompt":0,"completion":0,"cached":0,"numRequests":1},"namedScores":{},"namedScoresCount":{},"cost":0}}]
============================================================================================================================================================================================================================================
✔ Evaluation complete.

» Run promptfoo view to use the local web viewer
» Run promptfoo share to create a shareable URL
» This project needs your feedback. What's one thing we can improve? https://forms.gle/YFLgTe1dKJKNSCsU7
============================================================================================================================================================================================================================================
Successes: 0
Failures: 1
Errors: 0
Pass Rate: 0.00%
Done.
```


**Screenshots**


**System information:**

```
% npx promptfoo@latest debug
...
{
  "version": "0.101.1",
  "platform": {
    "os": "darwin",
    "release": "24.1.0",
    "arch": "arm64",
    "nodeVersion": "v20.17.0"
  },
  "env": {},
  "configInfo": {
    "defaultConfigPath": "/Users/mremy/tmp/promptfoo_js_to_py_test/promptfooconfig.yaml",
    "configExists": true,
    "configContent": {
      "config": {
        "tags": {},
        "prompts": [
          "{{ question }}"
        ],
        "providers": [
          {
            "id": "python:js_to_py_demo.py"
          }
        ],
        "tests": [
          {
            "description": "Example test",
            "vars": {
              "question": "Why is the sky blue?"
            },
            "assert": [
              {
                "type": "contains",
                "value": "scatter"
              }
            ]
          }
        ],
        "scenarios": [],
        "env": {},
        "sharing": true,
        "defaultTest": {
          "vars": {},
          "assert": [],
          "options": {},
          "metadata": {}
        },
        "outputPath": [],
        "extensions": [],
        "metadata": {}
      },
      "testSuite": {
        "tags": {},
        "prompts": [
          {
            "raw": "{{ question }}",
            "label": "{{ question }}"
          }
        ],
        "providers": [
          {
            "options": {
              "id": "python:js_to_py_demo.py",
              "config": {
                "basePath": "/Users/mremy/tmp/promptfoo_js_to_py_test"
              },
              "env": {}
            },
            "scriptPath": "js_to_py_demo.py",
            "functionName": null,
            "label": "",
            "config": {
              "basePath": "/Users/mremy/tmp/promptfoo_js_to_py_test"
            }
          }
        ],
        "providerPromptMap": {
          "python:js_to_py_demo.py": [
            "{{ question }}"
          ]
        },
        "tests": [
          {
            "description": "Example test",
            "vars": {
              "question": "Why is the sky blue?"
            },
            "assert": [
              {
                "type": "contains",
                "value": "scatter"
              }
            ]
          }
        ],
        "scenarios": [],
        "defaultTest": {
          "metadata": {},
          "options": {},
          "vars": {},
          "assert": []
        },
        "nunjucksFilters": {},
        "extensions": []
      },
      "basePath": "/Users/mremy/tmp/promptfoo_js_to_py_test"
    }
  }
}
```

**Additional context**


@martinremy can you hop on a quick call? If send me an email at michael@promptfoo.dev and I'll respond with a zoom link

Hi @martinremy, thanks for reporting this issue, for providing clear steps to reproduce, and helping me triage it on a call! The problem you’re encountering was due to https://github.com/promptfoo/promptfoo/pull/2386 and is resolved in [0.101.2](https://github.com/promptfoo/promptfoo/releases/tag/0.101.2). Please update to the latest version and let us know if you encounter any further issues. If you have any additional questions, feel free to reach out or open a new issue.

Thanks again for using Promptfoo!

@mldangelo confirmed fixed in latest, thank you for being so responsive!