Repository: explodinggradients/ragas
Issue: Issue with AzureOpenAI gpt-4o within Langchain framework
Root cause: Documentation Deficiency

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
Using AzureOpenAI from langchain (`from langchain_openai import AzureOpenAI `) getting an error when trying to use `gpt-4o `deployment. 

```
Exception raised in Job[0]: BadRequestError(Error code: 400 - {'error': {'code': 'OperationNotSupported', 'message': 'The completion operation does not work with the specified model, gpt-4o. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993.'}})
```

I think this originates from OpenAI Completetions API is legacy 

https://platform.openai.com/docs/guides/completions

Ragas version: `0.1.16`
Python version: `3.10.14`

**Code to Reproduce**

```
from langchain_openai import AzureOpenAI 
from datasets import Dataset 
from ragas.metrics import context_recall
from ragas import evaluate
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

gpt_4o = AzureOpenAI(model_name="gpt-4o",
                         deployment_name="XXXXXXXXX",
                         openai_api_version="2024-06-01",
                         azure_endpoint="XXXXXXXXXXXX",
                         azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(),
                                                                           "https://cognitiveservices.azure.com/.default")
                         )

data_samples = {
    'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
    'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
    'contexts' : [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], 
    ['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']],
    'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
}
dataset = Dataset.from_dict(data_samples)
score = evaluate(dataset, metrics=[context_recall], llm=gpt_4o)
score.to_pandas()
```

![image](https://github.com/user-attachments/assets/f851aa79-389e-4965-92d0-65e1c07d57db)


**Error trace**

`Exception raised in Job[0]: BadRequestError(Error code: 400 - {'error': {'code': 'OperationNotSupported', 'message': 'The completion operation does not work with the specified model, gpt-4o. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993.'}})`

**Expected behavior**
Using a `gpt-35-turbo` deployment works fine 😄 
![image](https://github.com/user-attachments/assets/fcd482aa-0f0a-467f-a926-07b80750db62)


**Additional context**


<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


<!-- Greeting -->
Hey there, @sam-h-long! I'm here to help you with any bugs, questions, or contributions you have. Let's dive in and tackle this together!

<!-- Answer -->
I found a similar closed issue that might help you resolve your problem. The issue was related to the `AzureChatOpenAI` object and had a solution that involved correcting typos and updating configurations. Here are the steps you can follow:

1. **Correct the Typo in Your Code:**
   Ensure that you are correctly assigning the `AzureOpenAI` instance to the `llm` parameter in the `evaluate` function. For example:
   ```python
   gpt_4o = AzureOpenAI(
       model_name="gpt-4o",
       deployment_name="XXXXXXXXX",
       openai_api_version="2024-06-01",
       azure_endpoint="XXXXXXXXXXXX",
       azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
   )
   ```

2. **Use the Correct Configuration Parameters:**
   Make sure you are using the correct parameters for the `AzureOpenAI` model. The `gpt-4o` model might not support the `completion` operation. Instead, you should use the `chat` operation. Here is an example configuration:
   ```python
   from langchain_openai.chat_models import AzureChatOpenAI

   azure_model = AzureChatOpenAI(
       openai_api_version="2023-05-15",
       azure_endpoint="https://<your-endpoint>.openai.azure.com/",
       azure_deployment="your-deployment-name",
       model="gpt-4o",
       validate_base_url=False,
   )
   ```

3. **Check Supported Operations:**
   The error message indicates that the `completion` operation is not supported for the `gpt-4o` model. You should verify the supported operations for the `gpt-4o` model in the [Azure OpenAI documentation](https://go.microsoft.com/fwlink/?linkid=2197993).

4. **Example Usage:**
   Here is an example of how to use the `AzureChatOpenAI` model with the `evaluate` function:
   ```python
   from langchain_openai.chat_models import AzureChatOpenAI
   from datasets import Dataset
   from ragas.metrics import context_recall
   from ragas import evaluate
   from azure.identity import DefaultAzureCredential, get_bearer_token_provider

   azure_model = AzureChatOpenAI(
       openai_api_version="2023-05-15",
       azure_endpoint="https://<your-endpoint>.openai.azure.com/",
       azure_deployment="your-deployment-name",
       model="gpt-4o",
       validate_base_url=False,
   )

   data_samples = {
       'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
       'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
       'contexts': [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], 
                    ['The Green Bay Packers...Green Bay, Wisconsin.', 'The Packers compete...Football Conference']],
       'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
   }
   dataset = Dataset.from_dict(data_samples)
   score = evaluate(dataset, metrics=[context_recall], llm=azure_model)
   score.to_pandas()
   ```

By following these steps, you should be able to resolve the `BadRequestError` and correctly use the `gpt-4o` model with the Langchain framework <sup>[[1]](https://github.com/explodinggradients/ragas/issues/221)</sup><sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/docs/howtos/customisations/azure-openai.ipynb)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


Thank you @dosu 

Switching from `AzureOpenAI` to `AzureChatOpenAI` worked 😄 

Please close this issue 

<!-- Answer -->
You're welcome! I'm glad to hear that switching to `AzureChatOpenAI` resolved the issue. If you have any more questions or need further assistance, feel free to ask. Have a great day!


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*