Repository: explodinggradients/ragas
Issue: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead.
Root cause: Environment Incompatibility

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
A clear and concise description of what the bug is.

Ragas version:
Python version:

**Code to Reproduce**
Install latest version of ragas using git

attempt
from datasets import Dataset 
from ragas.metrics import context_recall
from ragas import evaluate

data_samples = {
    'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
    'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
    'contexts' : [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], 
    ['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']],
    'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
}
dataset = Dataset.from_dict(data_samples)
score = evaluate(dataset,metrics=[context_recall])
score.to_pandas()

**Error trace**

**Expected behavior**
A clear and concise description of what you expected to happen.


**Additional context**
Evaluating:   0%
 0/2 [00:00<?, ?it/s]
Exception in thread Thread-14:
Traceback (most recent call last):
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/executor.py", line 95, in run
    results = self.loop.run_until_complete(self._aresults())
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/executor.py", line 83, in _aresults
    raise e
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/executor.py", line 78, in _aresults
    r = await future
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/asyncio/tasks.py", line 571, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/executor.py", line 37, in sema_coro
    return await coro
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/executor.py", line 111, in wrapped_callable_async
    return counter, await callable(*args, **kwargs)
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/metrics/base.py", line 132, in ascore
    raise e
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/metrics/base.py", line 125, in ascore
    score = await asyncio.wait_for(
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
    return fut.result()
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/metrics/_context_recall.py", line 169, in _ascore
    results = await self.llm.generate(
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/llms/base.py", line 93, in generate
    return await agenerate_text_with_retry(
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/tenacity/_asyncio.py", line 88, in async_wrapped
    return await fn(*args, **kwargs)
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/tenacity/_asyncio.py", line 47, in __call__
    do = self.iter(retry_state=retry_state)
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/tenacity/__init__.py", line 314, in iter
    return fut.result()
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/tenacity/_asyncio.py", line 50, in __call__
    result = await fn(*args, **kwargs)
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/llms/base.py", line 170, in agenerate_text
    return await self.langchain_llm.agenerate_prompt(
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 687, in agenerate_prompt
    return await self.agenerate(
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 647, in agenerate
    raise exceptions[0]
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 832, in _agenerate_with_cache
    result = await self._agenerate(
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/langchain_openai/chat_models/base.py", line 623, in _agenerate
    response = await self.async_client.create(messages=message_dicts, **params)
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 1181, in create
    return await self._post(
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/openai/_base_client.py", line 1790, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/openai/_base_client.py", line 1493, in request
    return await self._request(
  File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/openai/_base_client.py", line 1584, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-NkLIY***************************************lyRC. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}
---------------------------------------------------------------------------
ExceptionInRunner                         Traceback (most recent call last)
Cell In[153], line 13
      5 data_samples = {
      6     'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
      7     'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
   (...)
     10     'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
     11 }
     12 dataset = Dataset.from_dict(data_samples)
---> 13 score = evaluate(dataset,metrics=[context_recall])
     14 score.to_pandas()

File /home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/evaluation.py:255, in evaluate(dataset, metrics, llm, embeddings, callbacks, in_ci, is_async, run_config, raise_exceptions, column_map)
    252     if not evaluation_group_cm.ended:
    253         evaluation_rm.on_chain_error(e)
--> 255     raise e
    256 else:
    257     result = Result(
    258         scores=Dataset.from_list(scores),
    259         dataset=dataset,
    260         binary_columns=binary_metrics,
    261     )

File /home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/ragas/evaluation.py:237, in evaluate(dataset, metrics, llm, embeddings, callbacks, in_ci, is_async, run_config, raise_exceptions, column_map)
    235 results = executor.results()
    236 if results == []:
--> 237     raise ExceptionInRunner()
    239 # convert results to dataset_like
    240 for i, _ in enumerate(dataset):

ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead.

<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


<!-- Greeting -->
Hello, @thistleknot! I'm here to help you with your bug. I'm an bot that can help you solve bugs, answer questions, and become a contributor. Let's get to the bottom of this together!

<!-- Answer -->
To resolve the `openai.AuthenticationError` with error code 401, ensure that the `OPENAI_API_KEY` environment variable is set correctly with a valid OpenAI API key. Here is the relevant code snippet to set the API key:

```python
import os
from getpass import getpass
import openai

# Set your OpenAI API key if it is not already set as an environment variable
if not (openai_api_key := os.getenv("OPENAI_API_KEY")):
    openai_api_key = getpass("🔑 Enter your OpenAI API key: ")
openai.api_key = openai_api_key
os.environ["OPENAI_API_KEY"] = openai_api_key
```

Make sure to replace `"your-openai-key"` with your actual OpenAI API key. If the key is incorrect or missing, you will encounter the `openai.AuthenticationError`.

Additionally, you can pass `raise_exceptions=False` in the `evaluate` function to show only a warning message instead of raising an exception:

```python
from datasets import Dataset 
from ragas import evaluate
from ragas.metrics import context_recall

data_samples = {
    'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
    'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
    'contexts' : [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], 
    ['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']],
    'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
}

dataset = Dataset.from_dict(data_samples)

# Ensure the API key is correct
os.environ["OPENAI_API_KEY"] = "your-valid-openai-key"

score = evaluate(dataset, metrics=[context_recall], raise_exceptions=False)
score.to_pandas()
```

This will prevent the `ExceptionInRunner` from stopping your execution and will only show a warning message <sup>[[1]](https://github.com/explodinggradients/ragas/issues/934)</sup><sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/README.md)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


thank you!
from datasets import Dataset 
from ragas.metrics import context_recall
from ragas import evaluate

data_samples = {
    'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
    'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
    'contexts' : [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], 
    ['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']],
    'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
}
dataset = Dataset.from_dict(data_samples)
score = evaluate(dataset,metrics=[context_recall],llm=chat)
score.to_pandas()
fixed it