Repository: explodinggradients/ragas
Issue: RuntimeError: ('Fatal error occurred while running async tasks.', APIConnectionError('Connection error.')) when using answer_correctness with Bedrock LLM
Root cause: Documentation Deficiency

**Describe the bug**

I am trying to run an answer_correctness evaluation on a simple dataset using AWS Bedrock as the LLM (anthropic's claude v2).

I have succeeded with following the bedrock example for faithfulness, answer_relevancy and context_precision. But answer_correctness is giving me the error:

`RuntimeError: ('Fatal error occurred while running async tasks.', APIConnectionError('Connection error.'))`

Ragas version: 0.0.21
Python version: 3.10.6

**Code to Reproduce**

```python
from ragas.metrics import answer_correctness
from ragas.llms import LangchainLLM
from ragas import evaluate

from langchain.chat_models import BedrockChat
import nest_asyncio  # CHECK NOTES
from datasets import Dataset

metrics = [answer_correctness]

config = {
    "region_name": "us-east-1", 
    "model_id": "anthropic.claude-v2",
    "model_kwargs": {"temperature": 0},
}

bedrock_model = BedrockChat(
    region_name=config["region_name"],
    model_id=config["model_id"],
    model_kwargs=config["model_kwargs"],
)
# wrapper around bedrock_model
ragas_bedrock_model = LangchainLLM(bedrock_model)

# patch the new RagasLLM instance
answer_correctness.llm = ragas_bedrock_model

# for m in metrics:
#     m.__setattr__("llm", ragas_bedrock_model)

# Sample data
data = {
    "question": ["What is the capital of France?"],
    "contexts": [["Paris is the capital of France.", "France is in Europe"]],
    "answer":         ["Paris"],
    "ground_truths": [["Paris"]]
}

# Create a Dataset
dataset = Dataset.from_dict(data)

# NOTES: Only used when running on a jupyter notebook, otherwise comment or remove this function.
nest_asyncio.apply()

result = evaluate(
    dataset,
    metrics=[answer_correctness],
)

result
```

**Error trace**

The trace is big...

```
TypeError                                 Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1307, in AsyncAPIClient._request(self, cast_to, options, stream, stream_cls, remaining_retries)
   1306 try:
-> 1307     response = await self._client.send(request, auth=self.custom_auth, stream=stream)
   1308     log.debug(
   1309         'HTTP Request: %s %s "%i %s"', request.method, request.url, response.status_code, response.reason_phrase
   1310     )

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1617, in AsyncClient.send(self, request, stream, auth, follow_redirects)
   1615 auth = self._build_request_auth(request, auth)
-> 1617 response = await self._send_handling_auth(
   1618     request,
   1619     auth=auth,
   1620     follow_redirects=follow_redirects,
   1621     history=[],
   1622 )
   1623 try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1645, in AsyncClient._send_handling_auth(self, request, auth, follow_redirects, history)
   1644 while True:
-> 1645     response = await self._send_handling_redirects(
   1646         request,
   1647         follow_redirects=follow_redirects,
   1648         history=history,
   1649     )
   1650     try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1682, in AsyncClient._send_handling_redirects(self, request, follow_redirects, history)
   1680     await hook(request)
-> 1682 response = await self._send_single_request(request)
   1683 try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1719, in AsyncClient._send_single_request(self, request)
   1718 with request_context(request=request):
-> 1719     response = await transport.handle_async_request(request)
   1721 assert isinstance(response.stream, AsyncByteStream)

File /opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py:366, in AsyncHTTPTransport.handle_async_request(self, request)
    365 with map_httpcore_exceptions():
--> 366     resp = await self._pool.handle_async_request(req)
    368 assert isinstance(resp.stream, typing.AsyncIterable)

File /opt/conda/lib/python3.10/site-packages/httpcore/_async/connection_pool.py:232, in AsyncConnectionPool.handle_async_request(self, request)
    230     deadline = float("inf")
--> 232 async with self._pool_lock:
    233     self._requests.append(status)

File /opt/conda/lib/python3.10/site-packages/httpcore/_synchronization.py:69, in AsyncLock.__aenter__(self)
     68 elif self._backend == "asyncio":
---> 69     await self._anyio_lock.acquire()
     71 return self

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_synchronization.py:119, in Lock.acquire(self)
    118 try:
--> 119     self.acquire_nowait()
    120 except WouldBlock:

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_synchronization.py:150, in Lock.acquire_nowait(self)
    144 """
    145 Acquire the lock, without blocking.
    146 
    147 :raises ~WouldBlock: if the operation would block
    148 
    149 """
--> 150 task = get_current_task()
    151 if self._owner_task == task:

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_testing.py:59, in get_current_task()
     53 """
     54 Return the current task.
     55 
     56 :return: a representation of the current task
     57 
     58 """
---> 59 return get_asynclib().get_current_task()

File /opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:1850, in get_current_task()
   1849 def get_current_task() -> TaskInfo:
-> 1850     return _create_task_info(current_task())

File /opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:1846, in _create_task_info(task)
   1844     parent_id = task_state.parent_id
-> 1846 return TaskInfo(id(task), parent_id, name, get_coro(task))

TypeError: descriptor 'get_coro' for '_asyncio.Task' objects doesn't apply to a 'Task' object

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1307, in AsyncAPIClient._request(self, cast_to, options, stream, stream_cls, remaining_retries)
   1306 try:
-> 1307     response = await self._client.send(request, auth=self.custom_auth, stream=stream)
   1308     log.debug(
   1309         'HTTP Request: %s %s "%i %s"', request.method, request.url, response.status_code, response.reason_phrase
   1310     )

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1617, in AsyncClient.send(self, request, stream, auth, follow_redirects)
   1615 auth = self._build_request_auth(request, auth)
-> 1617 response = await self._send_handling_auth(
   1618     request,
   1619     auth=auth,
   1620     follow_redirects=follow_redirects,
   1621     history=[],
   1622 )
   1623 try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1645, in AsyncClient._send_handling_auth(self, request, auth, follow_redirects, history)
   1644 while True:
-> 1645     response = await self._send_handling_redirects(
   1646         request,
   1647         follow_redirects=follow_redirects,
   1648         history=history,
   1649     )
   1650     try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1682, in AsyncClient._send_handling_redirects(self, request, follow_redirects, history)
   1680     await hook(request)
-> 1682 response = await self._send_single_request(request)
   1683 try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1719, in AsyncClient._send_single_request(self, request)
   1718 with request_context(request=request):
-> 1719     response = await transport.handle_async_request(request)
   1721 assert isinstance(response.stream, AsyncByteStream)

File /opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py:366, in AsyncHTTPTransport.handle_async_request(self, request)
    365 with map_httpcore_exceptions():
--> 366     resp = await self._pool.handle_async_request(req)
    368 assert isinstance(resp.stream, typing.AsyncIterable)

File /opt/conda/lib/python3.10/site-packages/httpcore/_async/connection_pool.py:232, in AsyncConnectionPool.handle_async_request(self, request)
    230     deadline = float("inf")
--> 232 async with self._pool_lock:
    233     self._requests.append(status)

File /opt/conda/lib/python3.10/site-packages/httpcore/_synchronization.py:69, in AsyncLock.__aenter__(self)
     68 elif self._backend == "asyncio":
---> 69     await self._anyio_lock.acquire()
     71 return self

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_synchronization.py:119, in Lock.acquire(self)
    118 try:
--> 119     self.acquire_nowait()
    120 except WouldBlock:

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_synchronization.py:150, in Lock.acquire_nowait(self)
    144 """
    145 Acquire the lock, without blocking.
    146 
    147 :raises ~WouldBlock: if the operation would block
    148 
    149 """
--> 150 task = get_current_task()
    151 if self._owner_task == task:

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_testing.py:59, in get_current_task()
     53 """
     54 Return the current task.
     55 
     56 :return: a representation of the current task
     57 
     58 """
---> 59 return get_asynclib().get_current_task()

File /opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:1850, in get_current_task()
   1849 def get_current_task() -> TaskInfo:
-> 1850     return _create_task_info(current_task())

File /opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:1846, in _create_task_info(task)
   1844     parent_id = task_state.parent_id
-> 1846 return TaskInfo(id(task), parent_id, name, get_coro(task))

TypeError: descriptor 'get_coro' for '_asyncio.Task' objects doesn't apply to a 'Task' object

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1307, in AsyncAPIClient._request(self, cast_to, options, stream, stream_cls, remaining_retries)
   1306 try:
-> 1307     response = await self._client.send(request, auth=self.custom_auth, stream=stream)
   1308     log.debug(
   1309         'HTTP Request: %s %s "%i %s"', request.method, request.url, response.status_code, response.reason_phrase
   1310     )

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1617, in AsyncClient.send(self, request, stream, auth, follow_redirects)
   1615 auth = self._build_request_auth(request, auth)
-> 1617 response = await self._send_handling_auth(
   1618     request,
   1619     auth=auth,
   1620     follow_redirects=follow_redirects,
   1621     history=[],
   1622 )
   1623 try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1645, in AsyncClient._send_handling_auth(self, request, auth, follow_redirects, history)
   1644 while True:
-> 1645     response = await self._send_handling_redirects(
   1646         request,
   1647         follow_redirects=follow_redirects,
   1648         history=history,
   1649     )
   1650     try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1682, in AsyncClient._send_handling_redirects(self, request, follow_redirects, history)
   1680     await hook(request)
-> 1682 response = await self._send_single_request(request)
   1683 try:

File /opt/conda/lib/python3.10/site-packages/httpx/_client.py:1719, in AsyncClient._send_single_request(self, request)
   1718 with request_context(request=request):
-> 1719     response = await transport.handle_async_request(request)
   1721 assert isinstance(response.stream, AsyncByteStream)

File /opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py:366, in AsyncHTTPTransport.handle_async_request(self, request)
    365 with map_httpcore_exceptions():
--> 366     resp = await self._pool.handle_async_request(req)
    368 assert isinstance(resp.stream, typing.AsyncIterable)

File /opt/conda/lib/python3.10/site-packages/httpcore/_async/connection_pool.py:232, in AsyncConnectionPool.handle_async_request(self, request)
    230     deadline = float("inf")
--> 232 async with self._pool_lock:
    233     self._requests.append(status)

File /opt/conda/lib/python3.10/site-packages/httpcore/_synchronization.py:69, in AsyncLock.__aenter__(self)
     68 elif self._backend == "asyncio":
---> 69     await self._anyio_lock.acquire()
     71 return self

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_synchronization.py:119, in Lock.acquire(self)
    118 try:
--> 119     self.acquire_nowait()
    120 except WouldBlock:

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_synchronization.py:150, in Lock.acquire_nowait(self)
    144 """
    145 Acquire the lock, without blocking.
    146 
    147 :raises ~WouldBlock: if the operation would block
    148 
    149 """
--> 150 task = get_current_task()
    151 if self._owner_task == task:

File /opt/conda/lib/python3.10/site-packages/anyio/_core/_testing.py:59, in get_current_task()
     53 """
     54 Return the current task.
     55 
     56 :return: a representation of the current task
     57 
     58 """
---> 59 return get_asynclib().get_current_task()

File /opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:1850, in get_current_task()
   1849 def get_current_task() -> TaskInfo:
-> 1850     return _create_task_info(current_task())

File /opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:1846, in _create_task_info(task)
   1844     parent_id = task_state.parent_id
-> 1846 return TaskInfo(id(task), parent_id, name, get_coro(task))

TypeError: descriptor 'get_coro' for '_asyncio.Task' objects doesn't apply to a 'Task' object

The above exception was the direct cause of the following exception:

APIConnectionError                        Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/ragas/async_utils.py:42, in run_async_tasks(tasks, show_progress, progress_bar_desc)
     41 try:
---> 42     outputs: List[Any] = loop.run_until_complete(_gather())
     43 except Exception as e:
     44     # run the operation w/o tqdm on hitting a fatal
     45     # may occur in some environments where tqdm.asyncio
     46     # is not supported

File /opt/conda/lib/python3.10/site-packages/nest_asyncio.py:89, in _patch_loop.<locals>.run_until_complete(self, future)
     87     raise RuntimeError(
     88         'Event loop stopped before Future completed.')
---> 89 return f.result()

File /opt/conda/lib/python3.10/asyncio/futures.py:201, in Future.result(self)
    200 if self._exception is not None:
--> 201     raise self._exception.with_traceback(self._exception_tb)
    202 return self._result

File /opt/conda/lib/python3.10/asyncio/tasks.py:234, in Task.__step(***failed resolving arguments***)
    233     else:
--> 234         result = coro.throw(exc)
    235 except StopIteration as exc:

File /opt/conda/lib/python3.10/site-packages/ragas/async_utils.py:39, in run_async_tasks.<locals>._gather()
     38 async def _gather() -> List[Any]:
---> 39     return await asyncio.gather(*tasks_to_execute)

File /opt/conda/lib/python3.10/asyncio/tasks.py:304, in Task.__wakeup(self, future)
    303 try:
--> 304     future.result()
    305 except BaseException as exc:
    306     # This may also be a cancellation.

File /opt/conda/lib/python3.10/asyncio/tasks.py:232, in Task.__step(***failed resolving arguments***)
    229 if exc is None:
    230     # We use the `send` method directly, because coroutines
    231     # don't have `__iter__` and `__next__` methods.
--> 232     result = coro.send(None)
    233 else:

File /opt/conda/lib/python3.10/site-packages/tenacity/_asyncio.py:88, in AsyncRetrying.wraps.<locals>.async_wrapped(*args, **kwargs)
     86 @functools.wraps(fn)
     87 async def async_wrapped(*args: t.Any, **kwargs: t.Any) -> t.Any:
---> 88     return await fn(*args, **kwargs)

File /opt/conda/lib/python3.10/site-packages/tenacity/_asyncio.py:47, in AsyncRetrying.__call__(self, fn, *args, **kwargs)
     46 while True:
---> 47     do = self.iter(retry_state=retry_state)
     48     if isinstance(do, DoAttempt):

File /opt/conda/lib/python3.10/site-packages/tenacity/__init__.py:325, in BaseRetrying.iter(self, retry_state)
    324 if self.reraise:
--> 325     raise retry_exc.reraise()
    326 raise retry_exc from fut.exception()

File /opt/conda/lib/python3.10/site-packages/tenacity/__init__.py:158, in RetryError.reraise(self)
    157 if self.last_attempt.failed:
--> 158     raise self.last_attempt.result()
    159 raise self

File /opt/conda/lib/python3.10/concurrent/futures/_base.py:451, in Future.result(self, timeout)
    450 elif self._state == FINISHED:
--> 451     return self.__get_result()
    453 self._condition.wait(timeout)

File /opt/conda/lib/python3.10/concurrent/futures/_base.py:403, in Future.__get_result(self)
    402 try:
--> 403     raise self._exception
    404 finally:
    405     # Break a reference cycle with the exception in self._exception

File /opt/conda/lib/python3.10/site-packages/tenacity/_asyncio.py:50, in AsyncRetrying.__call__(self, fn, *args, **kwargs)
     49 try:
---> 50     result = await fn(*args, **kwargs)
     51 except BaseException:  # noqa: B902

File /opt/conda/lib/python3.10/site-packages/ragas/llms/openai.py:170, in OpenAIBase.agenerate(self, prompt, n, temperature, callbacks)
    161 @retry_decorator
    162 async def agenerate(
    163     self,
   (...)
    168 ) -> LLMResult:
    169     # TODO: use callbacks for llm generate
--> 170     completion = await self._client.chat.completions.create(
    171         model=self.model,
    172         messages=[convert_message_to_dict(m) for m in prompt.format_messages()],  # type: ignore
    173         temperature=temperature,
    174         n=n,
    175     )
    177     return self.create_llm_result(completion)

File /opt/conda/lib/python3.10/site-packages/openai/resources/chat/completions.py:1199, in AsyncCompletions.create(self, messages, model, frequency_penalty, function_call, functions, logit_bias, max_tokens, n, presence_penalty, response_format, seed, stop, stream, temperature, tool_choice, tools, top_p, user, extra_headers, extra_query, extra_body, timeout)
   1152 @required_args(["messages", "model"], ["messages", "model", "stream"])
   1153 async def create(
   1154     self,
   (...)
   1197     timeout: float | httpx.Timeout | None | NotGiven = NOT_GIVEN,
   1198 ) -> ChatCompletion | AsyncStream[ChatCompletionChunk]:
-> 1199     return await self._post(
   1200         "/chat/completions",
   1201         body=maybe_transform(
   1202             {
   1203                 "messages": messages,
   1204                 "model": model,
   1205                 "frequency_penalty": frequency_penalty,
   1206                 "function_call": function_call,
   1207                 "functions": functions,
   1208                 "logit_bias": logit_bias,
   1209                 "max_tokens": max_tokens,
   1210                 "n": n,
   1211                 "presence_penalty": presence_penalty,
   1212                 "response_format": response_format,
   1213                 "seed": seed,
   1214                 "stop": stop,
   1215                 "stream": stream,
   1216                 "temperature": temperature,
   1217                 "tool_choice": tool_choice,
   1218                 "tools": tools,
   1219                 "top_p": top_p,
   1220                 "user": user,
   1221             },
   1222             completion_create_params.CompletionCreateParams,
   1223         ),
   1224         options=make_request_options(
   1225             extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout
   1226         ),
   1227         cast_to=ChatCompletion,
   1228         stream=stream or False,
   1229         stream_cls=AsyncStream[ChatCompletionChunk],
   1230     )

File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1482, in AsyncAPIClient.post(self, path, cast_to, body, files, options, stream, stream_cls)
   1479 opts = FinalRequestOptions.construct(
   1480     method="post", url=path, json_data=body, files=await async_to_httpx_files(files), **options
   1481 )
-> 1482 return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)

File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1283, in AsyncAPIClient.request(self, cast_to, options, stream, stream_cls, remaining_retries)
   1274 async def request(
   1275     self,
   1276     cast_to: Type[ResponseT],
   (...)
   1281     remaining_retries: Optional[int] = None,
   1282 ) -> ResponseT | _AsyncStreamT:
-> 1283     return await self._request(
   1284         cast_to=cast_to,
   1285         options=options,
   1286         stream=stream,
   1287         stream_cls=stream_cls,
   1288         remaining_retries=remaining_retries,
   1289     )

File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1337, in AsyncAPIClient._request(self, cast_to, options, stream, stream_cls, remaining_retries)
   1336 if retries > 0:
-> 1337     return await self._retry_request(options, cast_to, retries, stream=stream, stream_cls=stream_cls)
   1338 raise APIConnectionError(request=request) from err

File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1364, in AsyncAPIClient._retry_request(self, options, cast_to, remaining_retries, response_headers, stream, stream_cls)
   1362 await anyio.sleep(timeout)
-> 1364 return await self._request(
   1365     options=options,
   1366     cast_to=cast_to,
   1367     remaining_retries=remaining,
   1368     stream=stream,
   1369     stream_cls=stream_cls,
   1370 )

File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1337, in AsyncAPIClient._request(self, cast_to, options, stream, stream_cls, remaining_retries)
   1336 if retries > 0:
-> 1337     return await self._retry_request(options, cast_to, retries, stream=stream, stream_cls=stream_cls)
   1338 raise APIConnectionError(request=request) from err

File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1364, in AsyncAPIClient._retry_request(self, options, cast_to, remaining_retries, response_headers, stream, stream_cls)
   1362 await anyio.sleep(timeout)
-> 1364 return await self._request(
   1365     options=options,
   1366     cast_to=cast_to,
   1367     remaining_retries=remaining,
   1368     stream=stream,
   1369     stream_cls=stream_cls,
   1370 )

File /opt/conda/lib/python3.10/site-packages/openai/_base_client.py:1338, in AsyncAPIClient._request(self, cast_to, options, stream, stream_cls, remaining_retries)
   1337         return await self._retry_request(options, cast_to, retries, stream=stream, stream_cls=stream_cls)
-> 1338     raise APIConnectionError(request=request) from err
   1340 return self._process_response(
   1341     cast_to=cast_to,
   1342     options=options,
   (...)
   1345     stream_cls=stream_cls,
   1346 )

APIConnectionError: Connection error.

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
Cell In[58], line 7
      4 # NOTES: Only used when running on a jupyter notebook, otherwise comment or remove this function.
      5 nest_asyncio.apply()
----> 7 result = evaluate(
      8     dataset,
      9     metrics=[answer_correctness],
     10 )
     12 result

File /opt/conda/lib/python3.10/site-packages/ragas/evaluation.py:105, in evaluate(dataset, metrics, column_map)
    103         binary_metrics.append(metric.name)
    104     print(f"evaluating with [{metric.name}]")
--> 105     scores.append(metric.score(dataset).select_columns(metric.name))
    107 # log the evaluation event
    108 metrics_names = [m.name for m in metrics]

File /opt/conda/lib/python3.10/site-packages/ragas/metrics/base.py:76, in Metric.score(self, dataset, callbacks)
     74 with trace_as_chain_group(f"ragas_{self.name}", callback_manager=cm) as group:
     75     for batch in tqdm(self.get_batches(len(dataset))):
---> 76         score = self._score_batch(dataset.select(batch), callbacks=group)
     77         scores.extend(score)
     79 return dataset.add_column(f"{self.name}", scores)

File /opt/conda/lib/python3.10/site-packages/ragas/metrics/_answer_correctness.py:66, in AnswerCorrectness._score_batch(self, dataset, callbacks, callback_group_name)
     63     ds_faithfulness = dataset
     65 ds_faithfulness = ds_faithfulness.rename_columns({"ground_truths": "contexts"})
---> 66 faith_scores = self.faithfulness._score_batch(ds_faithfulness)  # type: ignore
     67 similarity_scores = self.answer_similarity._score_batch(dataset)  # type: ignore
     69 scores_stacked = np.vstack([faith_scores, similarity_scores])

File /opt/conda/lib/python3.10/site-packages/ragas/metrics/_faithfulness.py:88, in Faithfulness._score_batch(self, ds, callbacks, callback_group_name)
     84     human_prompt = LONG_FORM_ANSWER_PROMPT.format(question=q, answer=a)
     85     prompts.append(ChatPromptTemplate.from_messages([human_prompt]))
---> 88 result = self.llm.generate(prompts, callbacks=batch_group)
     89 list_statements: list[list[str]] = []
     90 for output in result.generations:
     91     # use only the first generation for each prompt

File /opt/conda/lib/python3.10/site-packages/ragas/llms/openai.py:153, in OpenAIBase.generate(self, prompts, n, temperature, callbacks)
    146 def generate(
    147     self,
    148     prompts: list[ChatPromptTemplate],
   (...)
    151     callbacks: t.Optional[Callbacks] = None,
    152 ) -> t.Any:  # TODO: LLMResult
--> 153     llm_results = run_async_tasks(
    154         [self.agenerate(p, n, temperature, callbacks) for p in prompts]
    155     )
    157     generations = [r.generations[0] for r in llm_results]
    158     llm_output = _compute_token_usage_langchain(llm_results)

File /opt/conda/lib/python3.10/site-packages/ragas/async_utils.py:47, in run_async_tasks(tasks, show_progress, progress_bar_desc)
     42     outputs: List[Any] = loop.run_until_complete(_gather())
     43 except Exception as e:
     44     # run the operation w/o tqdm on hitting a fatal
     45     # may occur in some environments where tqdm.asyncio
     46     # is not supported
---> 47     raise RuntimeError("Fatal error occurred while running async tasks.", e)
     48 return outputs

RuntimeError: ('Fatal error occurred while running async tasks.', APIConnectionError('Connection error.'))
```

**Expected behavior**

I expected the call to succeed.


**Additional context**

Running in a Jupyter notebook. Other metrics (faithfulness, answer_relevancy, context_precision) work fine.


I was able to pinpoint the error and work around it.

AnswerCorrectness has an `answer_similarity` attr that must be initialized to use a Bedrock embeddings model.

This is the working code for the example above, so I am closing the ticket.

```python
from ragas.metrics import AnswerCorrectness, AnswerSimilarity
from ragas.llms import LangchainLLM
from ragas import evaluate

from langchain.chat_models import BedrockChat
from langchain.embeddings import BedrockEmbeddings

import nest_asyncio  # CHECK NOTES
from datasets import Dataset

config = {
    "region_name": "us-east-1", 
    "model_id": "anthropic.claude-v2",
    "model_kwargs": {"temperature": 0},
}

bedrock_model = BedrockChat(
    region_name=config["region_name"],
    model_id=config["model_id"],
    model_kwargs=config["model_kwargs"],
)
# wrapper around bedrock_model
ragas_bedrock_model = LangchainLLM(bedrock_model)

bedrock_embeddings = BedrockEmbeddings(
    region_name=config["region_name"],
)

# patch the new RagasLLM instance
answer_similarity = AnswerSimilarity(llm=ragas_bedrock_model, embeddings=bedrock_embeddings)
answer_correctness.llm = AnswerCorrectness(llm=ragas_bedrock_model, answer_similarity=answer_similarity)

# Sample data
data = {
    "question": ["What is the capital of France?"],
    "contexts": [["Paris is the capital of France.", "France is in Europe"]],
    "answer":         ["Paris"],
    "ground_truths": [["Paris"]]
}

# Create a Dataset
dataset = Dataset.from_dict(data)

# NOTES: Only used when running on a jupyter notebook, otherwise comment or remove this function.
nest_asyncio.apply()

result = evaluate(
    dataset,
    metrics=[answer_correctness],
)

result
```

Has the problem been solved？