Repository: explodinggradients/ragas
Issue: It takes two hours to evaluate two indicators with 500 data records. Is there any method to accelerate the evaluation?
Root cause: Algorithmic Error

It takes two hours to evaluate two indicators with 500 data records. Is there any method to accelerate the evaluation?


<!-- Greeting -->
Hey @wangsir-cn! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!

<!-- Answer -->
Yes, there is a method to accelerate the evaluation of two indicators with 500 data records. The provided code already uses asynchronous evaluation with the `is_async=True` parameter in the `evaluate` function. This should help in speeding up the evaluation process.

Here is the relevant part of the code:

```python
if __name__ == "__main__":
    # asyncio
    print("Starting [Asyncio]")
    start = time.time()
    _ = evaluate(
        eval_dataset,
        metrics=metrics,
        is_async=True,
    )
    print(f"Time taken [Asyncio]: {time.time() - start:.2f}s")
```

Additionally, you can leverage the Athina platform to further optimize and monitor your evaluations. Athina allows you to run Ragas evaluations programmatically and view the results on its UI. This can provide more granular insights and potentially improve the efficiency of your evaluation process.

To use Athina, follow these steps:

1. Install Athina's Python SDK:
    ```bash
    pip install athina
    ```

2. Create an account at [app.athina.ai](https://app.athina.ai) and obtain an API key.

3. Use the following sample code to run your evaluations:

    ```python
    import os
    from athina.evals import (
        RagasAnswerCorrectness,
        RagasAnswerRelevancy,
        RagasContextRelevancy,
        RagasFaithfulness,
    )
    from athina.loaders import RagasLoader
    from athina.keys import AthinaApiKey, OpenAiApiKey
    from athina.runner.run import EvalRunner
    import pandas as pd

    # Set your API keys
    OpenAiApiKey.set_key(os.getenv("OPENAI_API_KEY"))
    AthinaApiKey.set_key(os.getenv("ATHINA_API_KEY"))

    # Load your dataset
    dataset = RagasLoader().load_json("raw_data.json")

    # Configure the eval suite
    eval_model = "gpt-3.5-turbo"
    eval_suite = [
        RagasAnswerCorrectness(),
        RagasFaithfulness(),
        RagasContextRelevancy(),
        RagasAnswerRelevancy(),
    ]

    # Run the evaluation suite
    batch_eval_result = EvalRunner.run_suite(
        evals=eval_suite,
        data=dataset,
        max_parallel_evals=1,  # Adjust this based on your rate limits
    )

    pd.DataFrame(batch_eval_result)
    ```

By using Athina, you can also configure Ragas to run automatically on your production logs, providing continuous evaluation and monitoring of your model's performance. This can significantly reduce the time required for manual evaluations and provide more timely insights <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/tests/benchmarks/benchmark_eval.py)</sup><sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/docs/howtos/integrations/athina.ipynb)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


@wangsir-cn were you able to figure it out? we do use async to make it as fast as possible but the biggest time sink is the LLM. Which one are you using?