Repository: explodinggradients/ragas
Issue: [R-257] [Feature Request] Llamaindex support
Root cause: Unimplemented Feature Gap

the title says it all
but this is for ragas v0.1 😉 

<sub>[R-257](https://linear.app/exploding-gradients/issue/R-257/[feature-request]-llamaindex-support)</sub>

Hi @jjmachan,
Thanks for creating this issue ticket.
To give more context:
Python version: 3.11.4
Ragas Version: Latest on Pypi(0.1.0rc1)

According to [this article](https://docs.llamaindex.ai/en/stable/examples/finetuning/openai_fine_tuning.html#initial-eval-with-gpt-3-5-turbo-query-engine) from `llama-index` they are using Ragas to evaluate the model.
I am trying to replicate it using `Azure Openai` instead of `Openai` directly.
Here is the snippet of code:

```
from llama_index import VectorStoreIndex, ServiceContext, SimpleDirectoryReader
from llama_index.embeddings import AzureOpenAIEmbedding
from datasets import Dataset
from ragas import evaluate
from ragas.metrics import answer_relevancy, faithfulness
import random


llm = AzureOpenAI(
    engine="gpt-35-turbo-16k",
    model="gpt-35-turbo-16k",
    temperature=0.3,
    azure_endpoint="xxxxx",
    api_key="xxxxx",
    api_version="xxxx",
)

embed_model = AzureOpenAIEmbedding(model="text-embedding-ada-002",
                                   deployment_name="text-embedding-ada-002",
                                   azure_endpoint="xxxxx",
                                   api_key="xxxxx",
                                   api_version="xxxxx")

documents = SimpleDirectoryReader(
    input_files=["sample-pdf-file.pdf"]
).load_data()

questions = []
with open("eval_questions.txt", "r") as f:
    for line in f:
        questions.append(line.strip())


gpt_35_context = ServiceContext.from_defaults(
    embed_model=embed_model,
    llm=llm, context_window=2048
)

index = VectorStoreIndex.from_documents(
    documents, service_context=gpt_35_context
)

query_engine = index.as_query_engine(similarity_top_k=2)

contexts = []
answers = []

for question in questions:
    response = query_engine.query(question)
    contexts.append([x.node.get_content() for x in response.source_nodes])
    answers.append(str(response))

ds = Dataset.from_dict(
    {
        "question": questions,
        "answer": answers,
        "contexts": contexts,
    }
)

result = evaluate(ds, metrics=[answer_relevancy, faithfulness], llm=llm, embeddings=embed_model)
print(result)
```

Here is the error that I'm receiving:
`AttributeError: 'AzureOpenAI' object has no attribute 'generate_text'`

I am not sure if this issue is coming from support for `llama-index` or `Azure Openai`

form where are you importing `llm = AzureOpenAI`? if its from llamaIndex that is the problem

@jjmachan Yes you are right! It's being imported from llama-index.
So I should wait for this feature to be implemented!
Thanks!

Hi: 
Is this feature being actively worked on? The downgrade of Llamaindex isn't an option for my project, but I would like to leverage Ragas if possible. 

> Hi: Is this feature being actively worked on? The downgrade of Llamaindex isn't an option for my project, but I would like to leverage Ragas if possible.

I am in the same boat and would love to use Ragas in my llamaindex project.

> > Hi: Is this feature being actively worked on? The downgrade of Llamaindex isn't an option for my project, but I would like to leverage Ragas if possible.
> 
> I am in the same boat and would love to use Ragas in my llamaindex project.

Same here! :-) 

> > > Hi: Is this feature being actively worked on? The downgrade of Llamaindex isn't an option for my project, but I would like to leverage Ragas if possible.
> > 
> > 
> > I am in the same boat and would love to use Ragas in my llamaindex project.
> 
> Same here! :-)

FWIW I found that the built-in evaluation module is working perfectly fine. I am using the [correctness evaluator](https://docs.llamaindex.ai/en/stable/examples/evaluation/correctness_eval/)

> > > > Hi: Is this feature being actively worked on? The downgrade of Llamaindex isn't an option for my project, but I would like to leverage Ragas if possible.
> > > 
> > > 
> > > I am in the same boat and would love to use Ragas in my llamaindex project.
> > 
> > 
> > Same here! :-)
> 
> FWIW I found that the built-in evaluation module is working perfectly fine. I am using the [correctness evaluator](https://docs.llamaindex.ai/en/stable/examples/evaluation/correctness_eval/)

谢谢你，我已经成功在我的llama-index项目中实现了


this will be out in the next release `v0.1.9`

My sincere apologies for not taking finishing this up earlier