Repository: explodinggradients/ragas
Issue: Ragas with own LLMs - ImportError: cannot import name 'AzureOpenAIEmbeddings' from 'langchain.embeddings'
Root cause: External Dependency Breakage

Hello,

I want to use Ragas to evaluate my RAG model. I am not using (and don't want to use) OpenAI but models that are running locally on my laptop (Llama 2 variations). So I got to this page of the ragas documentation: https://docs.ragas.io/en/latest/howtos/customisations/llms.html

I tried to change the code for my use case: 
```
from langchain.llms import LlamaCpp
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

def build_llm():
        callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])

        n_gpu_layers = 1  # Metal set to 1 is enough.
        n_batch = 512  # Should be between 1 and n_ctx, consider the amount of RAM of your Apple Silicon Chip.

        llm = LlamaCpp(
        max_tokens =cfg.MAX_TOKENS,
        model_path=cfg.MODEL_BIN_PATH,
        temperature=cfg.TEMPERATURE,
        top_p=1,
        f16_kv=True,
        n_ctx=8000, # davor 6000, dann 2048
        n_gpu_layers=n_gpu_layers,
        n_batch=n_batch,
        callback_manager=callback_manager, 
        verbose=False, # Verbose is required to pass to the callback manager
        )
        
        return llm
```
```
from ragas.llms import LangchainLLM

llm = build_llm() # this is loading the Llama 2 model with LlamaCpp from Langchain
llm = LangchainLLM(llm=llm)
from ragas.metrics import (
    context_precision,
    answer_relevancy,
    faithfulness,
    context_recall,
)
from ragas.metrics.critique import harmfulness

# change the LLM

faithfulness.llm = llm
answer_relevancy.llm = llm
context_precision.llm = llm
context_recall.llm = llm
harmfulness.llm = llm
```

However my code fails when importing the different metrics. I get the following error: 
```
File [~/anaconda3/lib/python3.11/site-packages/ragas/embeddings/base.py:9](https://file+.vscode-resource.vscode-cdn.net/Users/mweissenba001/Documents/rag_example/All%20Models%20one%20Script/~/anaconda3/lib/python3.11/site-packages/ragas/embeddings/base.py:9)
      6 from typing import List
      8 import numpy as np
----> 9 from langchain.embeddings import AzureOpenAIEmbeddings as BaseAzureOpenAIEmbeddings
     10 from langchain.embeddings import OpenAIEmbeddings as BaseOpenAIEmbeddings
     11 from langchain.schema.embeddings import Embeddings

ImportError: cannot import name 'AzureOpenAIEmbeddings' from 'langchain.embeddings' (/Users/mweissenba001/anaconda3/lib/python3.11/site-packages/langchain/embeddings/__init__.py)
```

The weird thing is that I also don't want something to do with Azure, so I don't want "AzurOpenAIEmbeddings" to be imported. I want to run the evaluation locally on my device. 
So what do I have to do to evaluate with Ragas with my own models?

Can you please print the langchain and ragas versions in your environment?
```
pip show langchain ragas
```

@weissenbacherpwc :

Try this 
`from langchain.embeddings.azure_openai import AzureOpenAIEmbeddings`


@younes-io 
`from langchain.embeddings import AzureOpenAIEmbeddings as BaseAzureOpenAIEmbeddings` should work fine, at least for recent versions of langchain.

> Can you please print the langchain and ragas versions in your environment?
> 
> ```
> pip show langchain ragas
> ```

- Ragas Version: 0.0.20
- Langchain Version 0.0.326

Can I run Ragas locally and not on Azure or OpenAI?
(I have also added the build_llm function, just in case)

Unless you have a reason to use earlier versions, this should fix your problem:
```
pip install --upgrade langchain ragas
```
It looks like pip reqs are too flexible, which can lead to problems like this.

Thanks @ceferisbarov upgrading works. I wasn't aware that I am not using the most recent version of Langchain. 

For people with the same error: Restarting the kernel after upgrading was required.

Thanks a lot for pitching in and helping us  folks :) @ceferisbarov @younes-io @weissenbacherpwc 

One more question. With the code above, evaluating faithfulness works with this:
```
# data
from datasets import load_dataset

fiqa_eval = load_dataset("explodinggradients/fiqa", "ragas_eval")

# evaluate
from ragas import evaluate

result = evaluate(
    fiqa_eval["baseline"].select(range(5)),  # showing only 5 for demonstration
    metrics=[faithfulness],
)
result
# {'faithfulness': 0.9500}
```

When I try to use more metrics like here:
```
# evaluate
from ragas import evaluate

result = evaluate(
    fiqa_eval["baseline"].select(range(3)), # selecting only 3
    metrics=[
        context_precision,
        faithfulness,
        answer_relevancy,
        context_recall,
    ],
)
result
```
I get following error:
`OpenAIKeyNotFound: OpenAI API key not found! Seems like your trying to use Ragas metrics with OpenAI endpoints. Please set 'OPENAI_API_KEY' environment variable`

For my understanding, I already updated the LLM to my specific one, so what is the issue here?


This can be from `answer_relevancy` as this metric uses openai embeddings by default. You can actually swap it with [azure embeddings ](https://github.com/explodinggradients/ragas/blob/1b02b773d1bbb0ee183eb80e829903c11253f2f8/src/ragas/embeddings/base.py#L50) by passing it to [here](https://github.com/explodinggradients/ragas/blob/1b02b773d1bbb0ee183eb80e829903c11253f2f8/src/ragas/metrics/_answer_relevance.py#L59)

```
embeddings = #initialise azure embeddings
answer_relevancy = AnswerRelevancy(embeddings=embeddings)
```

Refer docs for full [code](https://docs.ragas.io/en/latest/howtos/customisations/azure-openai.html#configuring-them-for-azure-openai-endpoints) @weissenbacherpwc 

so can I also use my own embeddings or HuggingfaceEmbeddings (saw it in full docs)?

Yes, absolutely @weissenbacherpwc 

Ok, when changing `answer_relevancy`, the same error with the OpenAiKeyNotFound occurs:
```
from langchain.embeddings import HuggingFaceBgeEmbeddings, HuggingFaceEmbeddings
    
model_name = "intfloat/multilingual-e5-large"
encode_kwargs = {'normalize_embeddings': True} # set True to compute cosine similarity

embeddings = HuggingFaceEmbeddings(
    model_name=model_name,
    model_kwargs={'device': 'mps'},
    encode_kwargs=encode_kwargs
)

from ragas.metrics import AnswerRelevancy
answer_relevancy = AnswerRelevancy(embeddings=embeddings)

# evaluate
from ragas import evaluate
from ragas.metrics import AnswerRelevancy

result = evaluate(
    fiqa_eval["baseline"].select(range(3)), # selecting only 3
    metrics=[
        context_precision,
        faithfulness,
        answer_relevancy, #das nutzt OpenAi als Default, das müsste man im SourceCode ändern
        context_recall,
    ],
)

result
```