Repository: explodinggradients/ragas
Issue: IndexError: Invalid key: 0 is out of bounds for size 0 
Root cause: External Dependency Breakage

**Describe the bug**
IndexError: Invalid key: 0 is out of bounds for size 0. This happens in the latest version installed from source.

Ragas version: git+https://github.com/explodinggradients/ragas.git@5f105c08b7579188aea1113334dae6b6a8a15660#egg=ragas
Python version: 3.9

**Code to Reproduce**
```
from datasets import load_dataset
import os
from ragas.metrics import (
    context_precision,
    answer_relevancy,
    faithfulness,
    context_recall,
)
from ragas.metrics.critique import harmfulness
from langchain_community.chat_models import AzureChatOpenAI
from langchain_community.embeddings import AzureOpenAIEmbeddings
from ragas.llms import LangchainLLMWrapper
from ragas import evaluate

# list of metrics we're going to use
metrics = [
    faithfulness,
    answer_relevancy,
    context_recall,
    context_precision,
    harmfulness,
]
fiqa_eval = load_dataset("explodinggradients/fiqa", "ragas_eval")
print(fiqa_eval)

# Set the OpenAI API key
os.environ["AZURE_OPENAI_API_KEY"] = "******"
os.environ["OPENAI_API_KEY"] = "******"
# os.environ["OPENAI_API_TYPE"] = "azure"
os.environ["OPENAI_API_VERSION"] ="******"



azure_model = AzureChatOpenAI(
    deployment_name="******",
    model="gpt-35-turbo",
    openai_api_base="******",
    openai_api_type="azure",
)
# wrapper around azure_model
ragas_azure_model = LangchainLLMWrapper(azure_model)
answer_relevancy.llm = ragas_azure_model

# init and change the embeddings
# only for answer_relevancy
azure_embeddings = AzureOpenAIEmbeddings(
    deployment_name="******",
    model="gpt-35-turbo",
    openai_api_base="******",
    openai_api_type="azure",
)

# embeddings can be used as it is
answer_relevancy.embeddings = azure_embeddings

for m in metrics:
    m.__setattr__("llm", ragas_azure_model)

fiqa_eval = load_dataset("explodinggradients/fiqa", "ragas_eval")
result = evaluate(
    fiqa_eval["baseline"],
    metrics=metrics,
)

print(result)
```

**Error trace**
```
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/chat_models/__init__.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.chat_models import ChatOpenAI`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/chat_models/__init__.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.chat_models import AzureChatOpenAI`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/chat_models/__init__.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.chat_models import ChatVertexAI`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/llms/__init__.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.llms import AzureOpenAI`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/llms/__init__.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.llms import OpenAI`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/llms/__init__.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.llms import VertexAI`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/embeddings/__init__.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.embeddings import AzureOpenAIEmbeddings`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/embeddings/__init__.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.embeddings import FastEmbedEmbeddings`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain/embeddings/__init__.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.embeddings import OpenAIEmbeddings`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
DatasetDict({
    baseline: Dataset({
        features: ['question', 'ground_truths', 'answer', 'contexts'],
        num_rows: 30
    })
})
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class `langchain_community.chat_models.azure_openai.AzureChatOpenAI` was deprecated in langchain-community 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run `pip install -U langchain-openai` and import as `from langchain_openai import AzureChatOpenAI`.
  warn_deprecated(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_community/chat_models/azure_openai.py:174: UserWarning: As of openai>=1.0.0, if `deployment_name` (or alias `azure_deployment`) is specified then `openai_api_base` (or alias `base_url`) should not be. Instead use `deployment_name` (or alias `azure_deployment`) and `azure_endpoint`.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_community/chat_models/azure_openai.py:182: UserWarning: As of openai>=1.0.0, if `openai_api_base` (or alias `base_url`) is specified it is expected to be of the form https://example-resource.azure.openai.com/openai/deployments/example-deployment. Updating https://mvpgomi.openai.azure.com/openai to https://mvpgomi.openai.azure.com/openai.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class `langchain_community.embeddings.azure_openai.AzureOpenAIEmbeddings` was deprecated in langchain-community 0.1.0 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run `pip install -U langchain-openai` and import as `from langchain_openai import AzureOpenAIEmbeddings`.
  warn_deprecated(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_community/embeddings/openai.py:268: UserWarning: WARNING! deployment_name is not default parameter.
                    deployment_name was transferred to model_kwargs.
                    Please confirm that deployment_name is what you intended.
  warnings.warn(
/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class `langchain_community.chat_models.openai.ChatOpenAI` was deprecated in langchain-community 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run `pip install -U langchain-openai` and import as `from langchain_openai import ChatOpenAI`.
  warn_deprecated(
Evaluating:   5%|█▍                             | 7/150 [00:04<01:24,  1.69it/s]
Traceback (most recent call last):
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/evaluation.py", line 176, in evaluate
    raise e
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/evaluation.py", line 159, in evaluate
    results = executor.results()
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/executor.py", line 118, in results
    raise e
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/executor.py", line 114, in results
    r = future.result()
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/executor.py", line 36, in wrapped_callable
    return counter, callable(*args, **kwargs)
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/metrics/base.py", line 75, in score
    raise e
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/metrics/base.py", line 71, in score
    score = self._score(row=row, callbacks=group_cm)
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/metrics/_answer_relevance.py", line 136, in _score
    return self._calculate_score(response, row)
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/metrics/_answer_relevance.py", line 114, in _calculate_score
    cosine_sim = self.calculate_similarity(question, gen_questions)
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/metrics/_answer_relevance.py", line 88, in calculate_similarity
    question_vec = np.asarray(self.embeddings.embed_query(question)).reshape(1, -1)
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_community/embeddings/openai.py", line 697, in embed_query
    return self.embed_documents([text])[0]
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_community/embeddings/openai.py", line 668, in embed_documents
    return self._get_len_safe_embeddings(texts, engine=engine)
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_community/embeddings/openai.py", line 494, in _get_len_safe_embeddings
    response = embed_with_retry(
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/langchain_community/embeddings/openai.py", line 116, in embed_with_retry
    return embeddings.client.create(**kwargs)
TypeError: create() got an unexpected keyword argument 'deployment_name'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/abhijithneilabraham/Desktop/ragas_test.py", line 62, in <module>
    result = evaluate(
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/evaluation.py", line 178, in evaluate
    result = Result(
  File "<string>", line 6, in __init__
  File "/Users/abhijithneilabraham/Documents/GitHub/ragas/src/ragas/evaluation.py", line 207, in __post_init__
    for cn in self.scores[0].keys():
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 2803, in __getitem__
    return self._getitem(key)
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 2787, in _getitem
    pa_subtable = query_table(self._data, key, indices=self._indices if self._indices is not None else None)
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/datasets/formatting/formatting.py", line 583, in query_table
    _check_valid_index_key(key, size)
  File "/Users/abhijithneilabraham/mambaforge/envs/py/lib/python3.9/site-packages/datasets/formatting/formatting.py", line 526, in _check_valid_index_key
    raise IndexError(f"Invalid key: {key} is out of bounds for size {size}")
IndexError: Invalid key: 0 is out of bounds for size 0
```


I encountered the same issue as you;have you managed to resolve it?

@herman159 Yes, the issue is resolved with the latest release of ragas and refer the docs. I also removed answer relevancy from the metrics before the release and it worked, maybe the new release has more fixes. Check out latest docs here: https://docs.ragas.io/en/latest/