Repository: promptfoo/promptfoo
Issue: "cleaner" output separating out LLM responses from promptfoo and a minor nit
Root cause: Unimplemented Feature Gap

My use cases may be non-traditional but sharing as there may be others.

I often am using the eval outputs for further data analysis (e.g., in a spreadsheet). I'd prefer if the promptfoo generated text e.g., [Pass] and the LLM outputs were in separate fields (I output to .csv then import into a spreadsheet and do further steps to isolate the LLM outputs).

Minor nit / bug:
Also, the amount of text between [Pass] and the LLM output is not entirely consistent. E.g., In one set of evals for Llama 3 (I'm accessing via replicate) the LLM output starts at character 18, for the rest of the LLMs I'm testing the output starts at character 16.

Thanks.


Thank you for raising these issues (and many others), @aisonym. I appreciate your feedback and insights into how you're using promptfoo.

You've made some excellent points regarding the CSV output format. I agree that separating the promptfoo-generated text (like [PASS] and [FAIL]) from the LLM outputs into distinct fields is a better approach. Regarding the inconsistent whitespace you've noticed with Llama 3 outputs, I suspect that the extra whitespace is coming directly from the model itself (or the message parser) rather than being introduced by promptfoo. To my knowledge, we don't trim whitespace from model responses to preserve their exact output. You can check out our implementation [here](https://github.com/promptfoo/promptfoo/blob/main/src/util/index.ts#L48).

I'll search more in the code. In the meantime, I recommend continuing to split [PASS]/[FAIL] and trim in Excel or Google Sheets (or pandas, etc., if you do any other pre-processing).

Would you mind sharing a `promptfooconfig.yaml` file so I can try to replicate your whitespace issue?

Regarding other improvements to CSV export, I am thinking about:

1. Splitting pass/fail into new columns as we just discussed.
2. Reporting assertion results in distinct columns in the CSV.
3. Including individual scores for each type of assertion.

I would love to hear more about your use case and any other features related to CSV export that would improve your experience.

Depending on the scope of changes and telemetry around CSV export (you may be the only one using it), I may either implement these improvements in the current CSV export as a breaking migration or introduce them as a new version of the export format.

Give me a few days to think about this. Thank you!

Thank you for the continued responses.

Here's a promptfooconfig.yaml
```
prompts:
  - 'Are you happy? Please restrict your answer to only Yes or No'

providers:
  - id: "openai:gpt-4o"
  - id: "anthropic:messages:claude-3-5-sonnet-20240620"
  - id: replicate:meta/meta-llama-3-70b-instruct:fbfb20b472b2f3bdd101412a9f70a0ed4fc0ced78a77ff00970ee7a2383c575d
  - id: vertex:gemini-1.5-pro
```

This will reproduce the whitespace issue for me in a csv output

```
[openai:gpt-4o] Are you happy? Please restrict your answer to only Yes or No,[anthropic:messages:claude-3-5-sonnet-20240620] Are you happy? Please restrict your answer to only Yes or No,[replicate:meta/meta-llama-3-70b-instruct:fbfb20b472b2f3bdd101412a9f70a0ed4fc0ced78a77ff00970ee7a2383c575d] Are you happy? Please restrict your answer to only Yes or No,[vertex:gemini-1.5-pro] Are you happy? Please restrict your answer to only Yes or No
"[PASS] (1.00)

As an artificial intelligence, I don't have emotions, so I don't experience happiness or sadness.

Pass Reason: No assertions","[PASS] (1.00)

Yes

Pass Reason: No assertions","[PASS] (1.00)



Yes

Pass Reason: No assertions","[PASS] (1.00)

No. 


Pass Reason: No assertions"
```

FWIW - my use case is not measuring the happiness of AI models.

Thanks for the config! I am about to reproduce your issue. We have two different csv export mechanisms (one via the cli and one in the webui). I'll work on this for you.