Repository: explodinggradients/ragas
Issue: Metrics answer_similarity and answer_correctness not working with AzureOpenAIEmbeddings
Root cause: Interface Contract Mismatch

**Describe the bug**
The issue appears when using any of the above mentioned metrics with AzureOpenAIEmbeddings. Apparently, the langchain_openai AzureOpenAIEmbeddings does not implement embed_texts and RAGAS code attempts to call to the embedding model in the stated way.

Ragas version: Latest (built from source)
Python version: 3.9.16

**Code to Reproduce**

```python
  from ragas.metrics import (
    answer_similarity,
    answer_correctness
)

from ragas import evaluate, adapt


chat_model = AzureChatOpenAI(
    openai_api_base=xxxxxx,
    openai_api_version=xxxxxx,
    deployment_name=xxxxxx,
    openai_api_key=xxxxxx,
    max_tokens=5000,
    temperature=0.0
    )

embedding_model = AzureOpenAIEmbeddings(
    azure_deployment=xxxxxx,
    azure_endpoint=xxxxxx,
    api_version=xxxxxx,
    api_key=xxxxxx
)

metrics = [
    answer_similarity,
    answer_correctness
]

result = evaluate(
    dataset_evaluation_1,
    metrics=metrics,
    llm=chat_model,
    embeddings=embedding_model
)

df = result.to_pandas()
df.head()
```

**Error trace**
```
  File ".venv/lib/python3.9/site-packages/ragas/metrics/_answer_similarity.py", line 69, in _ascore
    embeddings_1 = np.array(await self.embeddings.embed_texts(ground_truth))
AttributeError: 'AzureOpenAIEmbeddings' object has no attribute 'embed_texts'
```


Faced the same issue.

fixed this with #553 
once testing is done, I'll merge it in - thanks a lot for bringing this to our attention 🙂 

@jjmachan This is not only for langchain right? I am using llama-index but this problem is mostly related to Azure OpenAI rather than langchain or llama-index as far as I know.

oh @nariman-zk the support for llamaIndex is not present right now, basically because we haven't implemented the wrappers for it. 

I've just created an issue for your so that you can track the progress https://github.com/explodinggradients/ragas/issues/557. will get to this by this week itself Nariman :)

@alexgg278 could confirm if your issue is fixed?

@jjmachan tested and working! Thanks so much for the quick fix, really appreciate it :)

It seems that #553 made it wrapped automatically, but the type annontations are not updated, so IDEs still consider it a problem if not wrapping manually.  
And the same situation on the param `llm`

Ragas metrics faithfulness not working for true or false type questions

Hey @soundaryabala123 can you please raise this as a separate issue? I can help with it. 

Hi ,

I have opened the new issue can you please help me with this .
RAGAS evaluation metrics faithfulness is not working well for true or false
type of questions. #609


On Thu, Feb 15, 2024 at 1:42 AM Shahul ES ***@***.***> wrote:

> Hey @soundaryabala123 <https://github.com/soundaryabala123> can you
> please raise this as a separate issue? I can help with it.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/explodinggradients/ragas/issues/545#issuecomment-1945516952>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AXRBO2G7TINWTZI5SWN23GDYTW35LAVCNFSM6AAAAABCXCOSJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBVGUYTMOJVGI>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


@stydxm that is true - do you want to make a quick fix for that? If not I'll get to it. We should type annotate that feature

> @stydxm that is true - do you want to make a quick fix for that? If not I'll get to it. We should type annotate that feature

Just fix it please, I'm afraid that I'll doing that the inappropriate way.

Is it possible to use AzureOpenai instance to create with the testsetGenerator? I tried a workaround but I get this error
[210](file:///C:/Users/User/anaconda3/envs/autodeskenv/lib/site-packages/ragas/testset/docstore.py:210) # split documents with self.splitter into smaller nodes
    [211](file:///C:/Users/User/anaconda3/envs/autodeskenv/lib/site-packages/ragas/testset/docstore.py:211) nodes = [
...
    [240](file:///C:/Users/User/anaconda3/envs/autodeskenv/lib/site-packages/ragas/testset/docstore.py:240)     )
    [241](file:///C:/Users/User/anaconda3/envs/autodeskenv/lib/site-packages/ragas/testset/docstore.py:241)     result_idx += 1
    [243](file:///C:/Users/User/anaconda3/envs/autodeskenv/lib/site-packages/ragas/testset/docstore.py:243) if n.keyphrases == []:

AttributeError: 'AzureOpenAIEmbeddings' object has no attribute 'embed_text'