Repository: confident-ai/deepeval
Issue: EvaluationDataset save_as method loses critical fields (tools_called, metadata, etc.)
Root cause: Algorithmic Error

**❗BEFORE YOU BEGIN❗**
Are you on discord? 🤗 We'd love to have you asking questions on discord instead: https://discord.com/invite/a3K9c8GRGt

**Describe the bug**
The `save_as` method in EvaluationDataset loses important information when saving datasets. This affects both single-turn and multi-turn datasets:

1. **Single-turn datasets**: Missing `tools_called`, `expected_tools`, `additional_metadata`, and `custom_column_key_values` fields from Golden objects
2. **Multi-turn datasets**: The `format_turns` method only preserves `role`, `content`, and `retrieval_context` from Turn objects, losing critical fields like `tools_called`, `user_id`, `additional_metadata`, and MCP-related fields

**To Reproduce**
Steps to reproduce the behavior:

**For single-turn datasets:**
1. Create a Golden object with `tools_called` and `expected_tools` information
2. Add it to an EvaluationDataset
3. Call `dataset.save_as("json", "./output")`
4. Check the saved file - `tools_called` and `expected_tools` information is missing

**For multi-turn datasets:**
1. Create Turn objects with `tools_called`, `user_id`, `additional_metadata`, or MCP fields
2. Create a ConversationalGolden with these turns
3. Add to EvaluationDataset and call `save_as`
4. Check the saved file - only `role`, `content`, and `retrieval_context` are preserved

**Expected behavior**
The saved file should contain complete object information, preserving all fields from Golden and Turn objects.

**Code Analysis**

**Issue 1: Golden objects missing fields in save_as method**
In `dataset.py` (lines 1542-1570), when creating new Golden objects, only partial fields are included:

```python
goldens = [
    Golden(
        input=golden.input,
        expected_output=golden.expected_output,
        actual_output=golden.actual_output,
        retrieval_context=golden.retrieval_context,
        context=golden.context,
        name=golden.name,
        comments=golden.comments,
        source_file=golden.source_file,
    )
    for golden in self.goldens
]
```

Missing fields:
- `tools_called`
- `expected_tools` 
- `additional_metadata`
- `custom_column_key_values`

**Issue 2: format_turns method loses Turn fields**
In `utils.py` (lines 85-96), the `format_turns` method only preserves limited fields:

```python
def format_turns(turns: List[Turn]) -> str:
    res = []
    for turn in turns:
        cur_turn = {
            "role": turn.role,
            "content": turn.content,
            "retrieval_context": (
                turn.retrieval_context if turn.retrieval_context else None
            ),
        }
        res.append(cur_turn)
    return json.dumps(res)
```

Missing Turn fields:
- `user_id`
- `tools_called`
- `mcp_tools_called`
- `mcp_resources_called`
- `mcp_prompts_called`
- `additional_metadata`

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
 - OS: All platforms
 - Browser: N/A
 - Version: 3.6.6

**Smartphone (please complete the following information):**
 - Device: N/A
 - OS: N/A
 - Browser: N/A
 - Version: N/A

**Additional context**
This bug significantly affects users who use deepeval for Agent evaluation, as tool call information and metadata are critical for:
- Agent performance evaluation
- Multi-turn conversation analysis
- MCP (Model Context Protocol) interactions
- User identification in conversations
- Custom metadata tracking

The issue impacts:
- All users using the `save_as` method
- Particularly affects Agent evaluation scenarios
- Multi-turn conversation datasets lose significant context
- Data integrity issues may lead to inaccurate analysis

**Suggested Fix**
1. **For Golden objects**: Include all fields when creating Golden objects in the `save_as` method
2. **For Turn objects**: Update `format_turns` method to preserve all Turn fields
3. **For output formats**: Add support for these fields in JSON, CSV, and JSONL outputs
4. **For parsing**: Update `parse_turns` method to handle the additional fields
5. Ensure consistency between serialization and deserialization across all data types


Hi, I looked through this and will be happy to take this on!

#2227 

@wjunwei2001 Thank you for the fix.

Hey @wjunwei2001 i just commented on the PR, thank you!

thank you @wjunwei2001 for the fix, it will be released in a few hours with the latest release!

@danerlt released :)