Repository: promptfoo/promptfoo
Issue: evaluateOptions property gets deleted from .yaml file when it is edited through the web ui
Root cause: Validation Gap

At the moment I use evaluateOptions.repeat for retest reliability

In yaml these options are not displayed, but are still used during eval

**To Reproduce**
1. Click on 'Edit YAML'
2. Insert
```yaml
evaluateOptions:
  repeat: 3
```
3. Save and see no evaluateOptions


Also a question, maybe there are plans to specify the number of repetitions for a specific test?

Hi @mldangelo, thanks for the quick response — I'm looking forward to the new version!

As related, `defaultTest` is showing the same behavior — could you take a look at that as well?

Hey @welcomemax I did the fix for `defaultTest` not showing upon save in the YAML editor. 
When testing I see the order of config can be confusing and we can get to that in a bit as well. 

I will leave the issue open for a bit, in case you have some extra feedback, and if not, will close soon. Thanks a lot!

Thank you! I really appreciate your quick response!

I'm not sure about the sorting — this one seems quite logical to me:

- description
- env
- evaluateOptions
- extensions
- providers
- scenarios
- prompts
- defaultTest
- tests

Also, I don’t think it’s always necessary to output all fields like `env: {}` and `extensions: []` if they’re empty.

Also on the topic — I think `env` should either always be hidden entirely or have keys like OPENAI_API_KEY, etc., stripped from it.

In my specific case: there's an assistant from another project, and the embedded key doesn’t work for it. For this assistant, we input a different key in env, but after running evals, another user might discover the key in the config.

<img width="1175" alt="Image" src="https://github.com/user-attachments/assets/d7e8aa59-ec3c-418b-a2e1-d3beac1a4f9b" />

> Also on the topic — I think `env` should either always be hidden entirely or have keys like OPENAI_API_KEY, etc., stripped from it.
> 
> In my specific case: there's an assistant from another project, and the embedded key doesn’t work for it. For this assistant, we input a different key in env, but after running evals, another user might discover the key in the config.
> 
> <img alt="Image" width="1175" src="https://private-user-images.githubusercontent.com/5702867/448828522-d7e8aa59-ec3c-418b-a2e1-d3beac1a4f9b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg5MDM3ODEsIm5iZiI6MTc0ODkwMzQ4MSwicGF0aCI6Ii81NzAyODY3LzQ0ODgyODUyMi1kN2U4YWE1OS1lYzNjLTQxOGItYTJlMS1kM2JlYWMxYTRmOWIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDYwMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTA2MDJUMjIzMTIxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZTMyNWYzZTgxOGE4MGM0YzFjNGE2NmJiZGUyZGJjY2IyMzIwMDA3NmY0YmIzYjFjNjZiZDJkNmZiMDk1NjJhZCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.y9qsCXnJvkoXUPX5SvDK1K6X6vkwpxRTzWECXyBMBo8">

Thanks @welcomemax for sharing! Looked into hiding the environment variables. Currently promptfoo is a tool meant to be used by a team with trusted members. However, we would much appreciate if you would like to make a pull request with the changes and we would be happy to send you a shirt or other swag for your contributions! 😊