Repository: explodinggradients/ragas
Issue: Answer Relevance doesnt need context
Root cause: Validation Gap

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
the answer relevance metric, does not need context as input, it doesnt use it anywhere
so why is it when i try to run it without contexts list it gives me error?

even in the documentation you mentioned that it needs contexts, but you did not used it anywhere


<!-- Greeting -->
@mohamad-tohidi Hello there! I'm Dosu, a bot here to assist you with bug fixes, questions, and becoming a contributor. I'm here to help while you wait for a human maintainer.

<!-- Answer -->
The `AnswerRelevance` metric does indeed require a `contexts` list because it uses the contexts to extract relevant sentences necessary to answer the question. The contexts are joined into a single string and then tokenized into sentences for further processing. Without providing a `contexts` list, the metric cannot perform its intended function, leading to an error.

To resolve the error, ensure that you provide a `contexts` list when running the metric. The `contexts` list should be included in the `row` dictionary with the key "contexts". Here is an example of how to structure the `row` dictionary:

```python
row = {
    "question": "What is the capital of France?",
    "contexts": ["Paris is the capital of France.", "It is known for its cafe culture and landmarks like the Eiffel Tower."]
}
```

This will allow the `AnswerRelevancy` class to properly process the contexts and compute the score <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/docs/concepts/metrics/answer_relevance.md)</sup><sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_answer_relevance.py)</sup>.

The exact error message you receive when the `contexts` list is not provided is a `KeyError`, triggered by the following code:

```python
question, contexts = row["question"], row["contexts"]
```

If `row["contexts"]` is not provided, it will raise a `KeyError` <sup>[[3]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_context_relevancy.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


@dosu I'd like to add to this that, nowhere in the prompt it's instructing to go through the contexts. It's just taking care of non-committals and for contexts it's just few shot prompting. Maybe due to this I've noticed it's giving better scores for some of the wrong answers as compared to the correct ones. I'm no expert to say this but I think the prompt needs a little more fine tuning.

Cheers and thanks!

Looking at the prompt, I also feel like the contexts shouldn't be required for answer relevance.

I understand the motivation for adding it in, exposed by @shahules786 on issue https://github.com/explodinggradients/ragas/issues/440, but for many other cases, the context is not critical.

  @jjmachan, if that's ok, I could prepare a PR that makes the `context` optional: if it's provided, fine, maintain the current behavior; otherwise, generate the questions based on the answer alone. What do you think?

@gustavocidornelas your help would be highly appreciated - thanks a lot for offering! @shahules786 what do you think should be the next step?

You're right. From the past experience, it seems to me that adding contexts just to fit edge cases in the relevancy metric is not very useful. Just raised a PR to fix this issue.