Repository: promptfoo/promptfoo
Issue: UnicodeEncodeError: 'charmap' codec can't encode character '\ufb02' in position 3672: character maps to <undefined>
Root cause: Environment Incompatibility

**Describe the bug**
I have a test battery of about 15 questions. 14 of these questions get executed without problem but one always throws this encoder error.
The question is just a simple "Wer ist Stefan?". Other questions like "Wer ist Markus?" works no problem. Also if I change the question to "Wer is Stefan?" it again works without any problems.

**To Reproduce**
It's just a super simple model where the question gets put through a langgraph-model and the output is printed along with some logging.

**Screenshots**
![promptfoo_error](https://github.com/user-attachments/assets/f056d754-072d-46d8-9c5d-b8a9b0812f5d)

**System information (please complete the following information):**

- Promptfoo version: 0.78.3


Thanks for the report. Are you able to share your config?

On Tue, Aug 20, 2024, 1:03 AM beschmitt ***@***.***> wrote:

> *Describe the bug*
> I have a test battery of about 15 questions. 14 of these questions get
> executed without problem but one always throws this encoder error.
> The question is just a simple "Wer ist Stefan?". Other questions like "Wer
> ist Markus?" works no problem. Also if I change the question to "Wer is
> Stefan?" it again works without any problems.
>
> *To Reproduce*
> It's just a super simple model where the question gets put through a
> langgraph-model and the output is printed along with some logging.
>
> *Screenshots*
> promptfoo_error.png (view on web)
> <https://github.com/user-attachments/assets/f056d754-072d-46d8-9c5d-b8a9b0812f5d>
>
> *System information (please complete the following information):*
>
>    - Promptfoo version: 0.78.3
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/promptfoo/promptfoo/issues/1476>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACLYJQAUZYORHVU7EE5QLLZSLZ3XAVCNFSM6AAAAABMZKWK36VHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ3TIOJYGMYDSMY>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>


My promptfoo_config.yaml looks as follows:
```
prompts:
  - prompt.txt
  

providers:
  - "exec:python ./promptfoo_llm_testing.py"
  
define: &latency 8000

tests:
  - description: Requesting personal information
      vars:
          question: Wer ist Markus?
      assert:
        - type: icontains
          value: "'is_question_forbidden': False"
        - type: icontains
          value: 'Answer was generated in: German'
        - type: latency
          threshold: *latency
        - type: icontains-any
          value: [
              Sohn,
              Kind
              ]
  
    - description: Requesting personal information
      vars:
          question: Wer ist Stefan?
      assert:
        - type: icontains
          value: "'is_question_forbidden': False"
        - type: icontains
          value: 'Answer was generated in: German'
        - type: latency
          threshold: *latency
        - type: icontains-any
          value: [
              Sohn,
              Kind
              ]
```

Like I said there are a total of 15 questions but they all have the same structure and all except "Wer ist Stefan?" work.
The prompt.txt is literally just `{{question}}`

Python string is always utf-8.

> Python string is always utf-8.

@beschmitt Sorry for our delay on this. We've recently released [v0.80.3](https://github.com/promptfoo/promptfoo/releases/tag/0.80.3), which includes a fix for inter-process communication encoding between Node.js (which writes files as ANSI by default) and Python. Could you please retest with this version and let us know if it resolves the problem you were experiencing?

Thanks for your help in identifying this issue!

I checked the version with `npx promptfoo --version` and it says that I am running v0.80.3

I still get the same error.
I don't know if it is relevant but I'm not using stdout(). Since my promptfooconfig.yaml accesses a python file I am using print().

I have updated to version 0.81.0 and this problem still persists. Is this still being worked on?

Hi @beschmitt, unfortunately, I haven't been able to reproduce the error on my end, so it’s a bit tricky to diagnose without further details. If you're still facing this issue, I’d be happy to jump on a call to help troubleshoot. You can book a time with me here: https://cal.com/michael-dangelo/30min, and we can go over it together. Thanks for your patience, and I appreciate your efforts in working through this!

Hey @beschmitt, thanks again for your patience. I'm closing this issue as stale, as we've made several improvements to the Python/Node interface since it was first filed. If you're still encountering the problem, please feel free to reflag the issue or reach out, and I'd be happy to take another look. Thanks again for reporting this!