Repository: explodinggradients/ragas
Issue: Is there any way to get the reason for the ragas score?
Root cause: Documentation Deficiency

[ X ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
Not able to see the reason for ragas score.

Ragas version: 0.3.2
Python version: 3.11.6

**Code to Reproduce**
for trace in predictions:
        # --- build reference
        reference = infer_reference(trace["overall_input"])
        print("============reference===============")
        print(reference)
        reference_tool_calls = sorted(
            [ToolCall(name=tool, args={}) for tool in reference.get("expected_tools", [])],
            key=lambda x: x.name
        )
        actual_tool_called = sorted(
            [ToolCall(name=tool, args={}) for tool in set(trace["tool_calls"])],
            key=lambda x: x.name
        )

        # --- multi-turn sample
        sample = MultiTurnSample(
            user_input=[
                RHMessage(content=trace["overall_input"]),
                RAMessage(content=trace["overall_output"], tool_calls=actual_tool_called),
            ],
            reference_tool_calls=reference_tool_calls,
            reference=reference["reference"],
            reference_topics=[reference["reference_topic"]]
        )

        # --- single-turn sample
        test_data = SingleTurnSample(
            user_input=trace["overall_input"],
            response=trace["overall_output"]
        )

        # --- run evaluations
        dataset_multi = EvaluationDataset(samples=[sample])
        # dataset_single = EvaluationDataset(samples=[test_data])

        multi_results = evaluate(dataset=dataset_multi, metrics=multi_turn_metrics)

**Error trace**
Not getting reason for ragas score.

**Expected behavior**
Need to see both score and reason




<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


Hi @divyamuralidharan-ai,

Great question! Yes, you can access the reasoning for Ragas scores. The way to access it depends on which metrics you're using and how you're accessing the results.
You can access via `EvaluationResult.scores` or convert to DataFrame and inspect.
If you're using custom metrics, you can return [`MetricResult`](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/result.py) objects that include reasoning.
Ref: [here](https://docs.ragas.io/en/latest/getstarted/experiments_quickstart/) for MetricResult.

LLM-based metrics (like AspectCritic, AgentGoalAccuracy) automatically generate reasoning.
Non-LLM metrics (like ToolCallAccuracy, ToolCallF1) return numeric scores without reasoning.

```py
df = multi_results.to_pandas()
print(df.columns.tolist())

for col in df.columns:
    if 'reason' in col:
        print(f"\n{col}:")
        print(df[col].tolist())
```

@Rahul2512Chauhan had some thoughts on this as well.

cc: @jjmachan 

Thanks for clarifying, @anistark  Got it now — LLM metrics already generate reasoning, and non-LLM ones are numeric-only.  

I was curious — would it make sense to add some form of lightweight explanation for non-LLM metrics too or is it better to keep them purely numeric?


hi @anistark ,
I am using AgentGoalAccuracy. ToolCallAccuracy and TopicAdherence. How can i get the reason for this.

> hi [@anistark](https://github.com/anistark) , I am using AgentGoalAccuracy. ToolCallAccuracy and TopicAdherence. How can i get the reason for this.

`ToolCallAccuracy` don't suppose will return a reasoning. For `AgentGoalAccuracy` and `TopicAdherence`, you can do something like this:

```py
for i, trace in enumerate(results.traces):
    row = {'sample_index': i}

    row.update(trace.scores)

    if 'agent_goal_accuracy' in trace:
        aga_trace = trace['agent_goal_accuracy']
        if 'CompareOutcomePrompt' in aga_trace:
            output = aga_trace['CompareOutcomePrompt']['output']
            row['aga_reason'] = output.get('reason', '')
        if 'InferGoalOutcomePrompt' in aga_trace:
            output = aga_trace['InferGoalOutcomePrompt']['output']
            row['aga_user_goal'] = output.get('user_goal', '')
            row['aga_end_state'] = output.get('end_state', '')

df_reasoning = pd.DataFrame(reasoning_data)
df_reasoning.to_string()
```

For topic adherence, You can get for topics and classifications in it similarly create the dataframe and get output.

It seems the issue was answered, closing this now.

I am trying to pull the explanation for faithfulness using the evaluate() call. I run the following code from above:
df = multi_results.to_pandas()
print(df.columns.tolist())

for col in df.columns:
    if 'reason' in col:
        print(f"\n{col}:")
        print(df[col].tolist())

I get a blank output, meaning, reason is not available. Am I missing anything? Any help will be greatly appreciated.