Repository: explodinggradients/ragas
Issue: TypeError(unsupported operand type(s) for +=: 'dict' and 'dict')
Root cause: Validation Gap

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
KeyboardInterrupt: Exception raised in Job[1]: TypeError(unsupported operand type(s) for +=: 'dict' and 'dict')
I'm assuming there is a bug in evaluation mertic

Ragas version: 0.1.20
Python version: 3.11.3 | packaged by conda-forge

**Code to Reproduce**
Share code to reproduce the issue
https://gist.github.com/jinnoh47/f58dae6aabb007bd4b5a356807ec5695



**Error trace**

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
Cell In[25], line 54
     52 data_point = gpt_llm_dataset[i]
     53 try:
---> 54     result = evaluate(Dataset.from_dict({
     55         "question": [data_point["question"]],
     56         "contexts": [data_point["contexts"]],
     57         "answer": [data_point["answer"]],
     58         "ground_truth": [data_point["ground_truth"]]
     59     }), samoo_llm_metrics)
     60     results.append(result)
     61 except Exception as e:

File /opt/conda/lib/python3.11/site-packages/ragas/_analytics.py:129, in track_was_completed.<locals>.wrapper(*args, **kwargs)
    126 @wraps(func)
    127 def wrapper(*args: P.args, **kwargs: P.kwargs) -> t.Any:
    128     track(IsCompleteEvent(event_type=func.__name__, is_completed=False))
--> 129     result = func(*args, **kwargs)
    130     track(IsCompleteEvent(event_type=func.__name__, is_completed=True))
    132     return result

File /opt/conda/lib/python3.11/site-packages/ragas/evaluation.py:295, in evaluate(dataset, metrics, llm, embeddings, callbacks, in_ci, run_config, token_usage_parser, raise_exceptions, column_map, show_progress)
    292 scores = []
    293 try:
    294     # get the results
--> 295     results = executor.results()
    296     if results == []:
    297         raise ExceptionInRunner()

File /opt/conda/lib/python3.11/site-packages/ragas/executor.py:118, in Executor.results(self)
    114         results.append(r)
    116     return results
--> 118 results = asyncio.run(_aresults())
    119 sorted_results = sorted(results, key=lambda x: x[0])
    120 return [r[1] for r in sorted_results]

File /opt/conda/lib/python3.11/site-packages/nest_asyncio.py:35, in _patch_asyncio.<locals>.run(main, debug)
     33 task = asyncio.ensure_future(main)
     34 try:
---> 35     return loop.run_until_complete(task)
     36 finally:
     37     if not task.done():

File /opt/conda/lib/python3.11/site-packages/nest_asyncio.py:84, in _patch_loop.<locals>.run_until_complete(self, future)
     82     f._log_destroy_pending = False
     83 while not f.done():
---> 84     self._run_once()
     85     if self._stopping:
     86         break

File /opt/conda/lib/python3.11/site-packages/nest_asyncio.py:107, in _patch_loop.<locals>._run_once(self)
    100     heappop(scheduled)
    102 timeout = (
    103     0 if ready or self._stopping
    104     else min(max(
    105         scheduled[0]._when - self.time(), 0), 86400) if scheduled
    106     else None)
--> 107 event_list = self._selector.select(timeout)
    108 self._process_events(event_list)
    110 end_time = self.time() + self._clock_resolution

File /opt/conda/lib/python3.11/selectors.py:468, in EpollSelector.select(self, timeout)
    466 ready = []
    467 try:
--> 468     fd_event_list = self._selector.poll(timeout, max_ev)
    469 except InterruptedError:
    470     return ready

KeyboardInterrupt: 
Exception raised in Job[1]: TypeError(unsupported operand type(s) for +=: 'dict' and 'dict')

**Expected behavior**

This code is expected to evaluate the dataset using the specified metrics (faithfulness, answer_relevancy, answer_correctness) and handle any errors that occur during the evaluation process. The results should be appended to the results list.

**Additional context**
Add any other context about the problem here.

<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


<!-- Answer -->
The `TypeError` you're encountering, "unsupported operand type(s) for +=: 'dict' and 'dict'," typically occurs when there is an attempt to use the `+=` operator on two dictionaries, which is not supported in Python. This issue might be related to how results are being aggregated or returned within the `evaluate` function or the metrics being used.

A similar issue was discussed in the Ragas repository, where a `TypeError` involving dictionary operations was caused by inconsistent value types for the same key in the dictionaries being merged <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/src/experimental/ragas_experimental/testset/utils.py)</sup>. In your case, it seems like the error might be occurring during the evaluation process when trying to append results to a list.

To resolve this, you should check the part of your code where dictionaries are being combined or aggregated. Ensure that you are not using the `+=` operator with dictionaries. Instead, you can use the `update()` method to merge dictionaries, like so:

```python
dict1.update(dict2)
```

This will add the key-value pairs from `dict2` into `dict1`.

Additionally, ensure that the data types for each key in the dataset dictionary passed to the `evaluate` function are as expected:

- `question`: `list[str]`
- `contexts`: `list[list[str]]`
- `answer`: `list[str]`
- `ground_truth`: `list[list[str]]` <sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/evaluation.py)</sup>.

If the issue persists, you might want to review the specific metrics being used and how they handle the results.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


<!-- Answer -->
Could you provide more details about how the `evaluate` function is being used in your code? Specifically, it would be helpful to see how the results from the `evaluate` function are being handled or aggregated. Additionally, if you have any custom metrics or modifications to the `evaluate` function, please share those as well. This information will help in identifying the source of the issue.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


<!-- Answer -->
Could you describe the specific part of the code where the `evaluate` function is called and how its results are being processed? Additionally, if there are any custom metrics or configurations being used with the `evaluate` function, please provide details on those as well. This information will help in diagnosing the issue more effectively.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


<!-- Answer -->
The `TypeError` you're encountering might be due to how the results from the `evaluate` function are being aggregated. Since you're using the `evaluate` function from the Ragas library, which returns a `Result` object containing scores in a dataset format, ensure that you're handling these results correctly.

Instead of using `+=` to combine results, you should append each `Result` object to a list or convert it to a dictionary using the `to_pandas` method and then handle it accordingly. Here's a general approach:

1. **Append Results**: If you're storing results in a list, simply append each `Result` object:
   ```python
   results.append(result)
   ```

2. **Convert to DataFrame**: If you need to manipulate the results further, convert them to a pandas DataFrame:
   ```python
   df = result.to_pandas()
   ```

3. **Ensure Correct Data Types**: Make sure the data types for each key in the dataset dictionary passed to the `evaluate` function are as expected:
   - `question`: `list[str]`
   - `contexts`: `list[list[str]]`
   - `answer`: `list[str]`
   - `ground_truth`: `list[list[str]]`

By following these steps, you should be able to handle the results without encountering the `TypeError`. If the issue persists, double-check the logic where results are being processed to ensure no unsupported operations are being performed on dictionaries.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*