Repository: explodinggradients/ragas
Issue: AnswerRelevancy errored with OpenAI key when it is set to use different LLM
Root cause: Configuration Error

[ X] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

When using the evaluate function for AnswerRelevancy metric with llm=evaluator_llm and the llm is set to non-openai ones, it errored with message about OpenAI key not set.

Ragas version:
Python version:

**Code to Reproduce**
evaluator_llm = LangchainLLMWrapper(ChatCohere(cohere_api_key=COHERE_API_KEY, model="command-r-plus-08-2024"))

def generate_ragas_results(dataset):
    results = evaluate(
        dataset = dataset,
        metrics=[
            Faithfulness(llm=evaluator_llm),
            AnswerRelevancy(llm=evaluator_llm),
            LLMContextRecall(llm=evaluator_llm),
            LLMContextPrecisionWithReference(llm=evaluator_llm),
            FactualCorrectness(llm=evaluator_llm)
        ],
        llm = evaluator_llm
    )

    df = results.to_pandas()

    return df

**Error trace**
---------------------------------------------------------------------------
OpenAIError                               Traceback (most recent call last)
[<ipython-input-67-4ecc8405fb7f>](https://localhost:8080/#) in <cell line: 5>()
      3 
      4 # generate evaluation metrics using RAGAS
----> 5 research_result_df = generate_ragas_results(research_dataset)

5 frames
    [... skipping hidden 1 frame]

[/usr/local/lib/python3.10/dist-packages/openai/_client.py](https://localhost:8080/#) in __init__(self, api_key, organization, project, base_url, timeout, max_retries, default_headers, default_query, http_client, _strict_response_validation)
    103             api_key = os.environ.get("OPENAI_API_KEY")
    104         if api_key is None:
--> 105             raise OpenAIError(
    106                 "The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable"
    107             )

OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable


**Expected behavior**
It produces the AnswerRelevancy


**Additional context**
Add any other context about the problem here.

<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


Encountering similar issue


Hi @WillMusing, @azraimahadan,

This error might be caused by the embedding model not being set, as it seems the model wasn't passed. You can refer to the code below for reference:

```py
from ragas import SingleTurnSample 
from ragas.metrics import ResponseRelevancy
from ragas.llms import LangchainLLMWrapper
from langchain_openai.embeddings import OpenAIEmbeddings
from langchain_openai import ChatOpenAI

evaluator_llm = LangchainLLMWrapper(ChatOpenAI(model="gpt-4o-mini"))
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

sample = SingleTurnSample(
        user_input="When was the first super bowl?",
        response="The first superbowl was held on Jan 15, 1967",
        retrieved_contexts=[
            "The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles."
        ]
    )


scorer = ResponseRelevancy(llm=evaluator_llm, embeddings=embeddings)
await scorer.single_turn_ascore(sample)
```
Output
```
0.9165088378587264
```

It seems the issue was answered, closing this now.