Repository: explodinggradients/ragas
Issue: Common Problem: KeyError: 0 for Metrics during "Run ragas metrics for evaluating RAG"
Root cause: Validation Gap

[v] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
I have the same problem as trish11953 from the issue [#1770](https://github.com/explodinggradients/ragas/issues/1770) if I only use the FactualCorrectness or LLMContextPrecisionWithReference (1 out of 5 times) metric. Maybe it's also somehow similar to the issue with Faithfulness?

I also checked metrics LLMContextRecall and SemanticSimilarity - they work perfect each time. 
**UPD**: I checked metric LLMContextRecall again and received the same error, that was shown in error trace. I think it's a common problem for all metrics.

Ragas version: 0.2.8
Python version: 3.12.2

**Code to Reproduce**
...
``` python
eval_dataset = EvaluationDataset.from_pandas(df)
metrics = [
    #LLMContextRecall(llm=generator_llm), 
    FactualCorrectness(llm=generator_llm), 
    #Faithfulness(llm=generator_llm),
    #LLMContextPrecisionWithReference(llm=generator_llm),
    #SemanticSimilarity(embeddings=generator_embeddings)
]

results = evaluate(dataset=eval_dataset, metrics=metrics)
```

**Error trace**
```
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt claim_decomposition_prompt failed to parse output: The output parser failed to parse the output including retries.
Exception raised in Job[1]: RagasOutputParserException(The output parser failed to parse the output including retries.)
Evaluating: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [01:12<00:00, 36.36s/it] 
Traceback (most recent call last):
  File "my-project\backend\ragas\ragas_evaluation.py", line 66, in <module>
    results = evaluate(dataset=eval_dataset, metrics=metrics)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "my-project\venv\Lib\site-packages\ragas\_analytics.py", line 205, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "my-project\venv\Lib\site-packages\ragas\evaluation.py", line 333, in evaluate
    result = EvaluationResult(
             ^^^^^^^^^^^^^^^^^
  File "<string>", line 10, in __init__
  File "my-project\venv\Lib\site-packages\ragas\dataset_schema.py", line 410, in __post_init__
    self.traces = parse_run_traces(self.ragas_traces, run_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "my-project\venv\Lib\site-packages\ragas\callbacks.py", line 167, in parse_run_traces
    "output": prompt_trace.outputs.get("output", {})[0],
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
KeyError: 0
```

**Expected behavior**
Output the top evaluations results

Hi, there~ I also met the same question as you. I failed in "Faithfulness", so I comment this metric and use the others. Then I successed! Here is the code snippet: 
metrics = [
    LLMContextRecall(llm=evaluator_llm), 
    FactualCorrectness(llm=evaluator_llm), 
    # Faithfulness(llm=evaluator_llm),
    SemanticSimilarity(embeddings=evaluator_embeddings)
]
results = evaluate(dataset=dataset, metrics=metrics)

Hi @Austin-QW, thanks for your response! 
I also met the same problem with the metric "Faithfulness", but even when I commented it out, the metric "FactualCorrectness" doesn't work at all, and the metric "LLMContextPrecisionWithReference" works in about 80% of all attempts

I got Nan on the “Faithfulness” , anyone else experiencing the same problem

@neverlatetolearn0 following shahules786's explanation from [#1773](https://github.com/explodinggradients/ragas/pull/1773) NaN means undetermined.
I would suggest just to reevaluate your RAG for the questions for which NaN was returned, or if necessary, divide the test set into smaller pieces and evaluate them independently.

Problem was fixed since 0.2.9