Repository: confident-ai/deepeval
Issue: Make evaluate() return a dictionary with scores and reasons
Root cause: Unimplemented Feature Gap

**❗BEFORE YOU BEGIN❗**
We discussed this on Discord and I got asked to open this issue.

**Is your feature request related to a problem? Please describe.**
NO

**Describe the solution you'd like**
I'd like evaluate() to return the results from metrics applied to a list of test cases. Something like this:

```
results = {
        "contextual_precision_score": [],
        "contextual_precision_reason": [],
        "contextual_recall_score": [],
        "contextual_recall_reason": [],
        "answer_relevancy_score": [],
        "answer_relevancy_reason": [],
        "faithfulness_score": [],
        "faithfulness_reason": [],
    }

```

**Describe alternatives you've considered**
I have implemented that by running the `measure` method per every metric per every test case. Here an example:

```
def run_measurements(qa_set_df, nrows=5):
    """
    Run the measurement of contextual_precision, contextual_recall,
    answer_relevancy, and faithfulness metrics implemented by
    DeepEval. 
    :param qa_set_df: A dataframe containing the Q/A pairs,
    ground truth and context required by DeepEval to run a
    test case. 
    :param nrows: Number of the first n rows to be extracted from
    the qa_set_df to run the test cases.
    :return: A dictionary with the scores and reasoning used by the
    evaluator LLM to provide the score.
    """

    results = {
        "contextual_precision_score": [],
        "contextual_precision_reason": [],
        "contextual_recall_score": [],
        "contextual_recall_reason": [],
        "answer_relevancy_score": [],
        "answer_relevancy_reason": [],
        "faithfulness_score": [],
        "faithfulness_reason": [],
    }
    
    for _, row in qa_set_df.head(nrows).iterrows():
        
        test_case = LLMTestCase(
            input=row['query'],
            actual_output=row['answer'],
            expected_output=row['ground_truth'],
            retrieval_context=[row['contexts']],
        )
    
        contextual_precision.measure(test_case)
        contextual_recall.measure(test_case)
        answer_relevancy.measure(test_case)
        faithfulness.measure(test_case)
    
        results["contextual_precision_score"].append(contextual_precision.score)
        results["contextual_precision_reason"].append(contextual_precision.reason)
        results["contextual_recall_score"].append(contextual_recall.score)
        results["contextual_recall_reason"].append(contextual_recall.reason)
        results["answer_relevancy_score"].append(answer_relevancy.score)
        results["answer_relevancy_reason"].append(answer_relevancy.reason)
        results["faithfulness_score"].append(faithfulness.score)
        results["faithfulness_reason"].append(faithfulness.reason)
        
    return results
```


**Additional context**

Here a view of a Pandas df stats extracted from the dictionary that holds the measurements. 

```
       contextual_precision_score  contextual_recall_score  \
count                         5.0                 5.000000   
mean                          1.0                 0.950000   
std                           0.0                 0.111803   
min                           1.0                 0.750000   
50%                           1.0                 1.000000   
max                           1.0                 1.000000   

       answer_relevancy_score  faithfulness_score  
count                5.000000            5.000000  
mean                 0.914286            0.443571  
std                  0.191663            0.322007  
min                  0.571429            0.125000  
50%                  1.000000            0.400000  
max                  1.000000            0.800000  

```

@vecorro Addressed on discord