Repository: explodinggradients/ragas
Issue: support OpenAI's new batch API interface
Root cause: Unimplemented Feature Gap

OpenAI's new batch API avoids rate limiting (making the process more reliable), and it cuts costs in half; however, it can add delays, and it would definitely add complexity.  But, it would provide a cheaper, more reliable experience for the end user.

For more details, please see:

- https://cookbook.openai.com/examples/batch_processing
- https://platform.openai.com/docs/guides/batch
- https://platform.openai.com/docs/api-reference/batch

Even if ragas doesn't support the batch API directly, it would be really nice to make some convenience methods to the metrics that make it easier to use the _ragas_ API to render all of the samples into a batch format.  This would have the added benefit of helping the end user understand exactly what is being sent to the model for evaluation.

Concretely, it would be really nice for every `Metric` to have an associated `.render(input: SingleTurnSample)` method.

Assuming I have a list of samples with all of the requisite information, I still have to jump through a _ton_ of hoops to get the prompts rendering cleanly.  Here's some really messy sample code that demonstrates:

```py
samples = [
    SingleTurnSample(
        user_input=q,
        retrieved_contexts=rc,
        reference_contexts=sc # ground-truth context
        reference=a # ground-truth response
        response=r,
    )
    for q, rc, sc, a, r in zip(queries, retrieved, src_chunks, answers, responses)
]

context_precision_wr = LLMContextPrecisionWithReference()
context_precision_wor = LLMContextPrecisionWithoutReference()
context_recall = LLMContextRecall()
response_relevance = ResponseRelevancy() 
faithfulness = Faithfulness() 
```

Ragas relabels RAGAs relabels SingleTurnSample keys when rendering prompts, so create relabeller

```py
singleturnsample_to_prompttemplate = {
    "context_precision_wr": lambda sts: QAC(
        question=sts.user_input,
        context="\n".join(sts.retrieved_contexts),
        answer=sts.reference,  # with reference
    ),
    "context_precision_wor": lambda sts: QAC(
        question=sts.user_input,
        context="\n".join(sts.retrieved_contexts),
        answer=sts.response,  # no reference
    ),
    "context_recall": lambda sts: QCA(
        question=sts.user_input,
        context="\n".join(sts.retrieved_contexts),
        answer=sts.reference,
    ),
    "response_relevance": lambda sts: ResponseRelevanceInput(response=sts.response),
    "faithfulness_statements": lambda sts: FaithfulnessStatements(
        question=sts.user_input,
        answer=sts.response,
        sentences=dict(enumerate(faithfulness.sentence_segmenter.segment(sts.response))),
    ),
    # "faithfulness": lambda sts: NLIStatementInput(
    #     context="\n".join(sts.retrieved_contexts),
    #     statements=[],  # inject statements
    # ),
}
```

Now we can create the batch jsonl file:

```py
# all prompts contain this string; use it to split system and user roles
SPLIT_ROLES_ON = "Now perform the same with the following input\n"

for metric in metrics:
    with (outfile_dir / f"ragas_{metric['name']}.jsonl").open("w") as f:
        for i, sample in enumerate(samples):

            # remap SingleTurnSample keys to match examples in prompt template
            remap = singleturnsample_to_prompttemplate[metric["name"]](sample)

            # render messages content
            system_prompt, user_prompt = metric["prompt_template"].to_string(remap).split(SPLIT_ROLES_ON)

            # construct API call
            line = {
                "custom_id": i,
                "method": "POST",
                "url": "/v1/chat/completions",
                "body": {
                    "model": model,
                    "messages": [
                        {"role": "system", "content": system_prompt},
                        {"role": "user", "content": user_prompt},
                    ],
                    "response_format": {"type": "json_object"},
                    "temperature": 0,
                },
            }
            f.write(json.dumps(line) + "\n")
```

> **Note** - faithfulness and response relevance are multi-step metrics; so the above would provide the intermediary responses that need to then be constructed into another batch file for processing before evaluation.