Repository: confident-ai/deepeval
Issue: [Question]: Should save_as(..., include_test_cases=True) extend or replace goldens when creating synthetic datasets?
Root cause: Documentation Deficiency

Hi team,

We are creating a golden dataset by using a [`ConversationSimulator`](https://github.com/confident-ai/deepeval/blob/main/deepeval/simulator/conversation_simulator.py) (see code below), where we generate synthetic conversational test cases and want to save them as a new golden dataset. When calling [EvaluationDataset.save_as(file_type="json", ..., include_test_cases=True)](https://github.com/confident-ai/deepeval/blob/main/deepeval/dataset/dataset.py#L932C5-L938C14), I noticed that the method extends the goldens list with the converted test cases, rather than replacing the turns in goldens with the simulated test cases.

This results in the output file containing both the original goldens and the simulated test cases. For synthetic data workflows, it is often desirable to save only the simulated test cases as goldens.

Is this the intended behavior? If yes, could you clarify the reasoning or recommended workflow for creating a purely synthetic golden dataset?

### Example

Here’s what we expect (only the simulated conversation):
```python
[
    {
        "scenario": "A user is in a hurry and wants a quote as quickly as possible.",
        "turns": "[{\"role\": \"user\", \"content\": \"Need print quote fast. Small print, urgent.\", ...}]",
        "expected_outcome": "Provide a fast, concise quote with minimal questions.",
        "user_description": "The user is impatient and wants immediate results.",
        "context": null,
        "name": null,
        "comments": null
    }
]
```
But what we actually get is both the original golden (with turns: null) and the simulated test case (with the actual conversation):

```python
[
    {
        "scenario": "A user is in a hurry and wants a quote as quickly as possible.",
        "turns": null,
        "expected_outcome": "Provide a fast, concise quote with minimal questions.",
        "user_description": "The user is impatient and wants immediate results.",
        "context": null,
        "name": null,
        "comments": null
    },
    {
        "scenario": "A user is in a hurry and wants a quote as quickly as possible.",
        "turns": "[{\"role\": \"user\", \"content\": \"Need print quote fast. Small object, urgent.\", ...}]",
        "expected_outcome": "Provide a fast, concise quote with minimal questions.",
        "user_description": "The user is impatient and wants immediate results. ",
        "context": null,
        "name": null,
        "comments": null
    }
]
```
### Questions:

- Is it intended that both original goldens and simulated test cases are included in the output when using `include_test_cases=True`?
- If not, could you provide a way to save only the simulated test cases (not extend, but replace the goldens)?

Thanks!


```python
from deepeval.dataset import ConversationalGolden
from deepeval.simulator import ConversationSimulator
from deepeval.dataset import EvaluationDataset


# Create a conversation goldens and dataset
conversation_goldens = [
  ConversationalGolden(
          scenario="A user is in a hurry and wants a quote as quickly as possible.",
          expected_outcome="Provide a fast, concise quote with minimal questions.",
          user_description=(
              "The user is impatient and wants immediate results. "
          ),
      ),
]
dataset = EvaluationDataset(goldens=conversation_goldens)

# Generate synthetic test cases using a simulator
conversational_test_cases = simulator.simulate(conversational_goldens=conversation_goldens, max_user_simulations=3)
for test_case in conversational_test_cases:
    dataset.add_test_case(test_case)

# Save as golden dataset
dataset.save_as(
    file_type="json",
    directory="./synthetic_data",
    file_name="synthetic_dataset",
    include_test_cases=True,
)
```

Hey @RakeshReddyKondeti,
Yes, it is the intended behavior to extend the current goldens with the test cases when you save the dataset.
If you would like to replace these goldens one simple solution I can suggest you is add a `dataset.goldens = []` right before you save your dataset. This empties out the goldens in the dataset and store only the test cases.

Another way is to create a new dataset and update the test cases there and save that dataset. Since you want to replace your goldens, I think the first option would suffice here:

```python
from deepeval.dataset import ConversationalGolden
from deepeval.simulator import ConversationSimulator
from deepeval.dataset import EvaluationDataset


# Create a conversation goldens and dataset
conversation_goldens = [
  ConversationalGolden(
          scenario="A user is in a hurry and wants a quote as quickly as possible.",
          expected_outcome="Provide a fast, concise quote with minimal questions.",
          user_description=(
              "The user is impatient and wants immediate results. "
          ),
      ),
]
dataset = EvaluationDataset(goldens=conversation_goldens)

# Generate synthetic test cases using a simulator
conversational_test_cases = simulator.simulate(conversational_goldens=conversation_goldens, max_user_simulations=3)
for test_case in conversational_test_cases:
    dataset.add_test_case(test_case)

dataset.goldens = [] # <-- This line here

# Save as golden dataset
dataset.save_as(
    file_type="json",
    directory="./synthetic_data",
    file_name="synthetic_dataset",
    include_test_cases=True,
)
```

Lemme know if there's any other doubts you have