Repository: explodinggradients/ragas
Issue: How can I customize data generation response of Data Generator class for Azure Open AI?
Root cause: Documentation Deficiency

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.

**Your Question**
How can I customize data generation response of Data Generator class for Azure Open AI? I only want question and ground truth generated, I will use contexts from the retrieval query data because that is what we are passing in the context right.

**Code Examples**
generator = TestsetGenerator.from_langchain(
    generator_llm=azure_model, critic_llm=azure_model, embeddings=azure_embeddings
)

testset = generator.generate_with_langchain_docs(
    documents,
    test_size=10,
    raise_exceptions=False,
    with_debugging_logs=False,
    distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25},
)
**Additional context**
We are using contexts data from retrieved text while generating answer right? 


<!-- Answer -->
To customize the data generation response of the `Data Generator` class for Azure Open AI to only generate questions and ground truth using contexts from the retrieval query data, you need to modify the `generate_datarow` method in the `Evolution` class. This method already generates a `DataRow` object that includes the question, contexts, and ground truth.

Here is the relevant part of the code that you need to focus on:

```python
async def generate_datarow(
    self,
    question: str,
    current_nodes: CurrentNodes,
    evolution_type: str,
):
    assert self.generator_llm is not None, "generator_llm cannot be None"

    node_content = [
        f"{i+1}\t{n.page_content}" for i, n in enumerate(current_nodes.nodes)
    ]
    results = await self.generator_llm.generate(
        prompt=self.find_relevant_context_prompt.format(
            question=question, contexts=node_content
        )
    )
    relevant_contexts_result = await json_loader.safe_load(
        results.generations[0][0].text.strip(), llm=self.generator_llm
    )
    relevant_context_indices = (
        relevant_contexts_result.get("relevant_contexts", None)
        if isinstance(relevant_contexts_result, dict)
        else None
    )
    if relevant_context_indices is not None:
        relevant_context_indices = [
            idx for idx in relevant_context_indices if isinstance(idx, int)
        ]

    if relevant_context_indices is None or not relevant_context_indices:
        relevant_context = CurrentNodes(
            root_node=current_nodes.root_node, nodes=current_nodes.nodes
        )
    else:
        selected_nodes = [
            current_nodes.nodes[i - 1]
            for i in relevant_context_indices
            if i - 1 < len(current_nodes.nodes)
        ]
        relevant_context = (
            CurrentNodes(root_node=selected_nodes[0], nodes=selected_nodes)
            if selected_nodes
            else current_nodes
        )

    merged_nodes = self.merge_nodes(relevant_context)
    results = await self.generator_llm.generate(
        prompt=self.question_answer_prompt.format(
            question=question, context=merged_nodes.page_content
        )
    )
    answer = await json_loader.safe_load(
        results.generations[0][0].text.strip(), self.generator_llm
    )
    answer = answer if isinstance(answer, dict) else {}
    logger.debug("answer generated: %s", answer)
    answer = (
        np.nan if answer.get("verdict") == "-1" else answer.get("answer", np.nan)
    )

    return DataRow(
        question=question.strip('"'),
        contexts=[n.page_content for n in relevant_context.nodes],
        ground_truth=answer,
        evolution_type=evolution_type,
        metadata=[n.metadata for n in relevant_context.nodes],
    )
```

To ensure that only questions and ground truth are generated, you can focus on the `question` and `ground_truth` fields of the `DataRow` object. The `contexts` field is already derived from the retrieval query data, so it will be included as part of the context for generating the ground truth.

You can use this method as is, as it already aligns with your requirement to generate questions and ground truth using contexts from the retrieval query data <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/evolutions.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


@dosu Aren't we using answer's retrieval contexts while calculating scores?

<!-- Answer -->
Yes, we are using the answer's retrieval contexts while calculating scores.

For example, in the `AnswerRelevancy` class, the `retrieved_contexts` key is used to provide the retrieval contexts for scoring:

```python
@dataclass
class AnswerRelevancy(MetricWithLLM, MetricWithEmbeddings, SingleTurnMetric):
    ...
    _required_columns: t.Dict[MetricType, t.Set[str]] = field(
        default_factory=lambda: {
            MetricType.SINGLE_TURN: {
                "user_input",
                "response",
                "retrieved_contexts",
            }
        }
    )
    ...
    def _create_question_gen_prompt(self, row: t.Dict) -> PromptValue:
        ans, ctx = row["response"], row["retrieved_contexts"]
        return self.question_generation.format(answer=ans, context="\n".join(ctx))
```

Additionally, context recall is a metric that measures the extent to which the retrieved context aligns with the annotated answer, treated as the ground truth. It is computed using the `question`, `ground truth`, and the retrieved `context` <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/docs/howtos/integrations/opik.ipynb)</sup><sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_answer_relevance.py)</sup><sup>[[3]](https://github.com/explodinggradients/ragas/blob/main/docs/concepts/metrics/context_recall.md)</sup><sup>[[4]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_noise_sensitivity.py)</sup><sup>[[5]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_faithfulness.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*