Repository: explodinggradients/ragas
Issue: Hello，calculate faithfulness Error: No statements were generated from the answer
Root cause: Validation Gap

[ ]Hello I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
Hello, when I run a sample like this with chinese:
from datasets import Dataset
import os
from ragas import evaluate
from ragas.metrics import faithfulness, answer_similarity, answer_correctness

os.environ["OPENAI_API_KEY"] ="xxxxxx"

data_samples = {
    'question': [''],
    'answer': ['当站台一侧或一组灯带不亮时，处理步骤如下：\n\n1. **安全措施**：首先确保在处理故障时断开电源，以避免电击风险。\n\n2. **检查LED日光灯**：如果是个别灯不亮，检查这些LED日光灯是否故障。拆下不亮的灯泡，检查是否需要更换新的灯泡或其驱动器。\n\n3. **检查电压**：如果部分灯不亮，可能是电压问题。使用电压表检测电源电压是否在正常范围内。如果电压不稳定，可能需要调整电源或查找其他故障源。\n\n4. **检查供电箱和控制空开**：如果整侧灯带不亮，检查供电箱的灯带切换和控制空开是否正常工作。确认空开没有跳闸，如果发现异常，进行调整或更换。\n\n5. **检查卡位和安装**：确认灯带的卡位是否牢固，是否存在损坏导致灯带无法正常工作。如有必要，修复或更换卡位以确保灯带固定。\n\n6. **检查接线**：检查连接线是否松动或老化。紧固接线，确保所有连接处无松动现象，必要时更换损坏的电线。\n\n7. **重新接通电源并测试**：在完成上述步骤后，重新接通电源，并测试站台灯带是否恢复正常。如果问题仍未解决，可能需要进一步的专业电气检查和维修，并通知相关人员配合。\n\n请务必在执行这些步骤时遵守所有安全规定，并在必要时寻求专业帮助。'],
    'contexts' : [['']],
    'ground_truth': ['当站台一侧或一组灯带不亮时，应按照以下步骤进行处理：\n1.检查空气开关：\n到达现场后，首先通过车站电话报告环调维修人员已到现场，并向车站工作人员申请C2类临修点。\n检查相应灯带箱的空气开关是否跳闸或烧坏。如果空气开关跳闸，尝试复位或更换空气开关。\n2.检查线路电压：\n使用万用表或电笔测量线路是否有电。如果发现线路没有电，进一步检查电源是否正常供电。\n3.检查线路完整性：\n检查线路是否短路或烧坏。可以通过目视检查或使用测电工具来确定线路的状况。如果发现短路或烧坏情况，进行相应的维修或更换线路。\n4.处理异常情况：\n清理线路中的异物或修复损坏的部分。如果发现灯带本身损坏，需要更换灯带。\n5.报告处理结果：\n处理完成后，向环调和车站报告处理结果，确保记录维修日志以备后续参考。\n联系人信息：\n如果在处理过程中需要进一步的支持或确认解决方案，可以联系以下负责人：\n电力调度员：张三，电话号码：12345678，负责供电日常组织、指挥工作，保证整个地铁供电系统安全运行和连续供电。\n故障报告：\n根据上述步骤生成的故障报告应包括以下内容：\n1.故障描述：站台一侧或一组灯带不亮。\n2.故障原因：可能是空气开关跳闸、线路断电或线路短路等原因引起。\n3.处理步骤：检查空气开关、线路电压和线路完整性，清理异物或修复损坏部分。\n4.联系人信息：电力调度员张三，电话号码：12345678。\n5.处理结果：处理完成后报告环调和车站，并记录维修日志。\n按照以上步骤进行操作，确保灯带故障能够及时、准确地得到处理和解决。']
}

dataset = Dataset.from_dict(data_samples)

score = evaluate(dataset,metrics=[answer_similarity,faithfulness,answer_correctness])
print (score["answer_correctness"])
print(score.to_pandas())

Ragas version: 0.1.9
Python version: 3.10

**Code to Reproduce**
Share code to reproduce the issue

**Error trace**
Evaluating:   0%|          | 0/3 [00:00<?, ?it/s]No statements were generated from the answer.
Evaluating:  67%|██████▋   | 2/3 [01:15<00:37, 37.71s/it]
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 67, in map_httpcore_exceptions
    yield
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 371, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection_pool.py", line 268, in handle_async_request
    raise exc
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection_pool.py", line 251, in handle_async_request
    response = await connection.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection.py", line 103, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 133, in handle_async_request
    raise exc
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 111, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 176, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 226, in _receive_event
    raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1522, in _request
    response = await self._client.send(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1646, in send
    response = await self._send_handling_auth(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1674, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1711, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1748, in _send_single_request
    response = await transport.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 370, in handle_async_request
    with map_httpcore_exceptions():
  File "D:\ProgramData\anaconda3\lib\contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 84, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 67, in map_httpcore_exceptions
    yield
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 371, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection_pool.py", line 268, in handle_async_request
    raise exc
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection_pool.py", line 251, in handle_async_request
    response = await connection.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection.py", line 103, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 133, in handle_async_request
    raise exc
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 111, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 176, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 226, in _receive_event
    raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1522, in _request
    response = await self._client.send(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1646, in send
    response = await self._send_handling_auth(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1674, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1711, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1748, in _send_single_request
    response = await transport.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 370, in handle_async_request
    with map_httpcore_exceptions():
  File "D:\ProgramData\anaconda3\lib\contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 84, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 67, in map_httpcore_exceptions
    yield
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 371, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection_pool.py", line 268, in handle_async_request
    raise exc
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection_pool.py", line 251, in handle_async_request
    response = await connection.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\connection.py", line 103, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 133, in handle_async_request
    raise exc
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 111, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 176, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpcore\_async\http11.py", line 226, in _receive_event
    raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1522, in _request
    response = await self._client.send(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1646, in send
    response = await self._send_handling_auth(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1674, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1711, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_client.py", line 1748, in _send_single_request
    response = await transport.handle_async_request(request)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 370, in handle_async_request
    with map_httpcore_exceptions():
  File "D:\ProgramData\anaconda3\lib\contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\httpx\_transports\default.py", line 84, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\ProgramData\anaconda3\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 95, in run
    results = self.loop.run_until_complete(self._aresults())
  File "D:\ProgramData\anaconda3\lib\asyncio\base_events.py", line 649, in run_until_complete
    return future.result()
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 83, in _aresults
    raise e
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 78, in _aresults
    r = await future
  File "D:\ProgramData\anaconda3\lib\asyncio\tasks.py", line 571, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 37, in sema_coro
    return await coro
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\executor.py", line 111, in wrapped_callable_async
    return counter, await callable(*args, **kwargs)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\metrics\base.py", line 125, in ascore
    raise e
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\metrics\base.py", line 121, in ascore
    score = await self._ascore(row=row, callbacks=group_cm, is_async=is_async)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\metrics\_answer_correctness.py", line 253, in _ascore
    is_statement_present = await self.llm.generate(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\llms\base.py", line 93, in generate
    return await agenerate_text_with_retry(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\asyncio\__init__.py", line 189, in async_wrapped
    return await copy(fn, *args, **kwargs)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\asyncio\__init__.py", line 111, in __call__
    do = await self.iter(retry_state=retry_state)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\asyncio\__init__.py", line 153, in iter
    result = await action(retry_state)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\_utils.py", line 99, in inner
    return call(*args, **kwargs)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
  File "D:\ProgramData\anaconda3\lib\concurrent\futures\_base.py", line 451, in result
    return self.__get_result()
  File "D:\ProgramData\anaconda3\lib\concurrent\futures\_base.py", line 403, in __get_result
    raise self._exception
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\tenacity\asyncio\__init__.py", line 114, in __call__
    result = await fn(*args, **kwargs)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\llms\base.py", line 170, in agenerate_text
    return await self.langchain_llm.agenerate_prompt(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\langchain_core\language_models\chat_models.py", line 609, in agenerate_prompt
    return await self.agenerate(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\langchain_core\language_models\chat_models.py", line 569, in agenerate
    raise exceptions[0]
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\langchain_core\language_models\chat_models.py", line 754, in _agenerate_with_cache
    result = await self._agenerate(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\langchain_openai\chat_models\base.py", line 666, in _agenerate
    response = await self.async_client.create(messages=message_dicts, **params)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\resources\chat\completions.py", line 1214, in create
    return await self._post(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1790, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1493, in request
    return await self._request(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1546, in _request
    return await self._retry_request(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1615, in _retry_request
    return await self._request(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1546, in _request
    return await self._retry_request(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1615, in _retry_request
    return await self._request(
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\openai\_base_client.py", line 1556, in _request
    raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
Traceback (most recent call last):
  File "E:\PCI-Agent\ragas_examples.py", line 25, in <module>
    score = evaluate(dataset,metrics=[answer_similarity,faithfulness,answer_correctness])
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\evaluation.py", line 250, in evaluate
    raise e
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\evaluation.py", line 232, in evaluate
    raise ExceptionInRunner()
ragas.exceptions.ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead.


**Expected behavior**
A clear and concise description of what you expected to happen.


**Additional context**
Add any other context about the problem here.

<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


<!-- Greeting -->
Hello, @13416157913! I'm here to help you with any bugs, questions, or contributions.

<!-- Answer -->
To resolve the "No statements were generated from the answer" error and the OpenAI API connection error when calculating the faithfulness metric using the Ragas library with Chinese text, you need to adapt the prompts to the Chinese language. The `Faithfulness` class has an `adapt` method that can be used to change the language of the prompts.

Here is how you can adapt the `Faithfulness` metric to Chinese:

```python
# Assuming `faithfulness` is an instance of the Faithfulness class
faithfulness.adapt(language="chinese")
```

This will adapt the `nli_statements_message` and `statement_prompt` to Chinese, which should help in generating statements and connecting to the OpenAI API correctly for Chinese text <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_faithfulness.py)</sup>.

For the `RemoteProtocolError: Server disconnected without sending a response` error, common causes include network issues, API rate limits, server-side issues, or incorrect API keys. Implementing a retry mechanism can help handle transient errors. Here is an example of how you might implement a retry mechanism:

```python
import os
import time
from datasets import Dataset
from ragas import evaluate
from ragas.metrics import faithfulness, answer_similarity, answer_correctness
from openai.error import RateLimitError, APIError

os.environ["OPENAI_API_KEY"] = "your-openai-key"

data_samples = {
    'question': [''],
    'answer': ['当站台一侧或一组灯带不亮时，处理步骤如下：\n\n1. **安全措施**：首先确保在处理故障时断开电源，以避免电击风险。\n\n2. **检查LED日光灯**：如果是个别灯不亮，检查这些LED日光灯是否故障。拆下不亮的灯泡，检查是否需要更换新的灯泡或其驱动器。\n\n3. **检查电压**：如果部分灯不亮，可能是电压问题。使用电压表检测电源电压是否在正常范围内。如果电压不稳定，可能需要调整电源或查找其他故障源。\n\n4. **检查供电箱和控制空开**：如果整侧灯带不亮，检查供电箱的灯带切换和控制空开是否正常工作。确认空开没有跳闸，如果发现异常，进行调整或更换。\n\n5. **检查卡位和安装**：确认灯带的卡位是否牢固，是否存在损坏导致灯带无法正常工作。如有必要，修复或更换卡位以确保灯带固定。\n\n6. **检查接线**：检查连接线是否松动或老化。紧固接线，确保所有连接处无松动现象，必要时更换损坏的电线。\n\n7. **重新接通电源并测试**：在完成上述步骤后，重新接通电源，并测试站台灯带是否恢复正常。如果问题仍未解决，可能需要进一步的专业电气检查和维修，并通知相关人员配合。\n\n请务必在执行这些步骤时遵守所有安全规定，并在必要时寻求专业帮助。'],
    'contexts': [['']],
    'ground_truth': ['当站台一侧或一组灯带不亮时，应按照以下步骤进行处理：\n1.检查空气开关：\n到达现场后，首先通过车站电话报告环调维修人员已到现场，并向车站工作人员申请C2类临修点。\n检查相应灯带箱的空气开关是否跳闸或烧坏。如果空气开关跳闸，尝试复位或更换空气开关。\n2.检查线路电压：\n使用万用表或电笔测量线路是否有电。如果发现线路没有电，进一步检查电源是否正常供电。\n3.检查线路完整性：\n检查线路是否短路或烧坏。可以通过目视检查或使用测电工具来确定线路的状况。如果发现短路或烧坏情况，进行相应的维修或更换线路。\n4.处理异常情况：\n清理线路中的异物或修复损坏的部分。如果发现灯带本身损坏，需要更换灯带。\n5.报告处理结果：\n处理完成后，向环调和车站报告处理结果，确保记录维修日志以备后续参考。\n联系人信息：\n如果在处理过程中需要进一步的支持或确认解决方案，可以联系以下负责人：\n电力调度员：张三，电话号码：12345678，负责供电日常组织、指挥工作，保证整个地铁供电系统安全运行和连续供电。\n故障报告：\n根据上述步骤生成的故障报告应包括以下内容：\n1.故障描述：站台一侧或一组灯带不亮。\n2.故障原因：可能是空气开关跳闸、线路断电或线路短路等原因引起。\n3.处理步骤：检查空气开关、线路电压和线路完整性，清理异物或修复损坏部分。\n4.联系人信息：电力调度员张三，电话号码：12345678。\n5.处理结果：处理完成后报告环调和车站，并记录维修日志。\n按照以上步骤进行操作，确保灯带故障能够及时、准确地得到处理和解决。']
}

dataset = Dataset.from_dict(data_samples)

def evaluate_with_retries(dataset, metrics, retries=3, delay=5):
    for attempt in range(retries):
        try:
            score = evaluate(dataset, metrics=metrics)
            return score.to_pandas()
        except (RateLimitError, APIError) as e:
            if attempt < retries - 1:
                time.sleep(delay)
                delay *= 2  # Exponential backoff
            else:
                raise e

score_df = evaluate_with_retries(dataset, metrics=[answer_similarity, faithfulness, answer_correctness])
print(score_df)
```

This code will retry the evaluation up to three times with an exponential backoff delay if a `RateLimitError` or `APIError` occurs <sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/README.md)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


> `RemoteProtocolError: Server disconnected without sending a response` error, common causes include network issues, API rate limits, server-side issues, or incorrect API keys. Implementing a retry mechanism can help handle transient errors. Here is an example of how you might implement a retry mechanism:

hello,when I set faithfulness.adapt(language="chinese") in my code, I meet another error,
score = evaluate(dataset,metrics=[answer_similarity,faithfulness.adapt(language="chinese"),answer_correctness])
error:
Traceback (most recent call last):
  File "E:\PCI-Agent\ragas_examples.py", line 25, in <module>
    score = evaluate(dataset,metrics=[answer_similarity,faithfulness.adapt(language="chinese"),answer_correctness])
  File "C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\metrics\_faithfulness.py", line 302, in adapt
    assert self.llm is not None, "LLM is not set"
AssertionError: LLM is not set

<!-- Answer -->
To resolve the `AssertionError: LLM is not set` when trying to adapt the `Faithfulness` metric to Chinese, you need to ensure that the `llm` attribute of the `Faithfulness` class is set before calling the `adapt` method. Here is how you can do it:

1. Initialize the language model.
2. Set the `llm` attribute of the `Faithfulness` instance.
3. Call the `adapt` method.

Here is an example:

```python
from ragas.llms import LangchainLLMWrapper
from langchain_openai.chat_models import ChatOpenAI
from ragas.metrics import faithfulness, answer_similarity, answer_correctness
from datasets import Dataset
import os

os.environ["OPENAI_API_KEY"] = "your-openai-key"

# Initialize your LLM
llm_instance = LangchainLLMWrapper(ChatOpenAI())

# Set the LLM for the faithfulness metric
faithfulness.llm = llm_instance

# Adapt the faithfulness metric to Chinese
faithfulness.adapt(language="chinese")

data_samples = {
    'question': [''],
    'answer': ['当站台一侧或一组灯带不亮时，处理步骤如下：\n\n1. **安全措施**：首先确保在处理故障时断开电源，以避免电击风险。\n\n2. **检查LED日光灯**：如果是个别灯不亮，检查这些LED日光灯是否故障。拆下不亮的灯泡，检查是否需要更换新的灯泡或其驱动器。\n\n3. **检查电压**：如果部分灯不亮，可能是电压问题。使用电压表检测电源电压是否在正常范围内。如果电压不稳定，可能需要调整电源或查找其他故障源。\n\n4. **检查供电箱和控制空开**：如果整侧灯带不亮，检查供电箱的灯带切换和控制空开是否正常工作。确认空开没有跳闸，如果发现异常，进行调整或更换。\n\n5. **检查卡位和安装**：确认灯带的卡位是否牢固，是否存在损坏导致灯带无法正常工作。如有必要，修复或更换卡位以确保灯带固定。\n\n6. **检查接线**：检查连接线是否松动或老化。紧固接线，确保所有连接处无松动现象，必要时更换损坏的电线。\n\n7. **重新接通电源并测试**：在完成上述步骤后，重新接通电源，并测试站台灯带是否恢复正常。如果问题仍未解决，可能需要进一步的专业电气检查和维修，并通知相关人员配合。\n\n请务必在执行这些步骤时遵守所有安全规定，并在必要时寻求专业帮助。'],
    'contexts': [['']],
    'ground_truth': ['当站台一侧或一组灯带不亮时，应按照以下步骤进行处理：\n1.检查空气开关：\n到达现场后，首先通过车站电话报告环调维修人员已到现场，并向车站工作人员申请C2类临修点。\n检查相应灯带箱的空气开关是否跳闸或烧坏。如果空气开关跳闸，尝试复位或更换空气开关。\n2.检查线路电压：\n使用万用表或电笔测量线路是否有电。如果发现线路没有电，进一步检查电源是否正常供电。\n3.检查线路完整性：\n检查线路是否短路或烧坏。可以通过目视检查或使用测电工具来确定线路的状况。如果发现短路或烧坏情况，进行相应的维修或更换线路。\n4.处理异常情况：\n清理线路中的异物或修复损坏的部分。如果发现灯带本身损坏，需要更换灯带。\n5.报告处理结果：\n处理完成后，向环调和车站报告处理结果，确保记录维修日志以备后续参考。\n联系人信息：\n如果在处理过程中需要进一步的支持或确认解决方案，可以联系以下负责人：\n电力调度员：张三，电话号码：12345678，负责供电日常组织、指挥工作，保证整个地铁供电系统安全运行和连续供电。\n故障报告：\n根据上述步骤生成的故障报告应包括以下内容：\n1.故障描述：站台一侧或一组灯带不亮。\n2.故障原因：可能是空气开关跳闸、线路断电或线路短路等原因引起。\n3.处理步骤：检查空气开关、线路电压和线路完整性，清理异物或修复损坏部分。\n4.联系人信息：电力调度员张三，电话号码：12345678。\n5.处理结果：处理完成后报告环调和车站，并记录维修日志。\n按照以上步骤进行操作，确保灯带故障能够及时、准确地得到处理和解决。']
}

dataset = Dataset.from_dict(data_samples)

# Evaluate the dataset
score = evaluate(dataset, metrics=[answer_similarity, faithfulness, answer_correctness])
print(score.to_pandas())
```

This ensures that the `llm` attribute is set before calling the `adapt` method, resolving the `AssertionError` <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_faithfulness.py)</sup><sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_summarization.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


> To resolve the `AssertionError: LLM is not set` when trying to adapt the `Faithfulness` metric to Chinese, you need to ensure that the `llm` attribute of the `Faithfulness` class is set before calling the `adapt` method. Here is how you can do it:
> 
> 1. Initialize the language model.
> 2. Set the `llm` attribute of the `Faithfulness` instance.
> 3. Call the `adapt` method.
> 
> Here is an example:
> 
> ```python
> from ragas.llms import LangchainLLMWrapper
> from langchain_openai.chat_models import ChatOpenAI
> from ragas.metrics import faithfulness, answer_similarity, answer_correctness
> from datasets import Dataset
> import os
> 
> os.environ["OPENAI_API_KEY"] = "your-openai-key"
> 
> # Initialize your LLM
> llm_instance = LangchainLLMWrapper(ChatOpenAI())
> 
> # Set the LLM for the faithfulness metric
> faithfulness.llm = llm_instance
> 
> # Adapt the faithfulness metric to Chinese
> faithfulness.adapt(language="chinese")
> 
> data_samples = {
>     'question': [''],
>     'answer': ['当站台一侧或一组灯带不亮时，处理步骤如下：\n\n1. **安全措施**：首先确保在处理故障时断开电源，以避免电击风险。\n\n2. **检查LED日光灯**：如果是个别灯不亮，检查这些LED日光灯是否故障。拆下不亮的灯泡，检查是否需要更换新的灯泡或其驱动器。\n\n3. **检查电压**：如果部分灯不亮，可能是电压问题。使用电压表检测电源电压是否在正常范围内。如果电压不稳定，可能需要调整电源或查找其他故障源。\n\n4. **检查供电箱和控制空开**：如果整侧灯带不亮，检查供电箱的灯带切换和控制空开是否正常工作。确认空开没有跳闸，如果发现异常，进行调整或更换。\n\n5. **检查卡位和安装**：确认灯带的卡位是否牢固，是否存在损坏导致灯带无法正常工作。如有必要，修复或更换卡位以确保灯带固定。\n\n6. **检查接线**：检查连接线是否松动或老化。紧固接线，确保所有连接处无松动现象，必要时更换损坏的电线。\n\n7. **重新接通电源并测试**：在完成上述步骤后，重新接通电源，并测试站台灯带是否恢复正常。如果问题仍未解决，可能需要进一步的专业电气检查和维修，并通知相关人员配合。\n\n请务必在执行这些步骤时遵守所有安全规定，并在必要时寻求专业帮助。'],
>     'contexts': [['']],
>     'ground_truth': ['当站台一侧或一组灯带不亮时，应按照以下步骤进行处理：\n1.检查空气开关：\n到达现场后，首先通过车站电话报告环调维修人员已到现场，并向车站工作人员申请C2类临修点。\n检查相应灯带箱的空气开关是否跳闸或烧坏。如果空气开关跳闸，尝试复位或更换空气开关。\n2.检查线路电压：\n使用万用表或电笔测量线路是否有电。如果发现线路没有电，进一步检查电源是否正常供电。\n3.检查线路完整性：\n检查线路是否短路或烧坏。可以通过目视检查或使用测电工具来确定线路的状况。如果发现短路或烧坏情况，进行相应的维修或更换线路。\n4.处理异常情况：\n清理线路中的异物或修复损坏的部分。如果发现灯带本身损坏，需要更换灯带。\n5.报告处理结果：\n处理完成后，向环调和车站报告处理结果，确保记录维修日志以备后续参考。\n联系人信息：\n如果在处理过程中需要进一步的支持或确认解决方案，可以联系以下负责人：\n电力调度员：张三，电话号码：12345678，负责供电日常组织、指挥工作，保证整个地铁供电系统安全运行和连续供电。\n故障报告：\n根据上述步骤生成的故障报告应包括以下内容：\n1.故障描述：站台一侧或一组灯带不亮。\n2.故障原因：可能是空气开关跳闸、线路断电或线路短路等原因引起。\n3.处理步骤：检查空气开关、线路电压和线路完整性，清理异物或修复损坏部分。\n4.联系人信息：电力调度员张三，电话号码：12345678。\n5.处理结果：处理完成后报告环调和车站，并记录维修日志。\n按照以上步骤进行操作，确保灯带故障能够及时、准确地得到处理和解决。']
> }
> 
> dataset = Dataset.from_dict(data_samples)
> 
> # Evaluate the dataset
> score = evaluate(dataset, metrics=[answer_similarity, faithfulness, answer_correctness])
> print(score.to_pandas())
> ```
> 
> This ensures that the `llm` attribute is set before calling the `adapt` method, resolving the `AssertionError` [[1]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_faithfulness.py)[[2]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_summarization.py).
> 
> _To continue talking to [Dosu](https://dosu.dev), mention @dosu._

Hello, I use your solution, add code like this:
from datasets import Dataset
import os
from ragas import evaluate
from ragas.metrics import faithfulness, answer_similarity, answer_correctness
from ragas.llms import LangchainLLMWrapper
from langchain_openai.chat_models import ChatOpenAI

os.environ["OPENAI_API_KEY"] = "xxxxxx"

# Initialize your LLM
llm_instance = LangchainLLMWrapper(ChatOpenAI())

# Set the LLM for the faithfulness metric
faithfulness.llm = llm_instance

# Adapt the faithfulness metric to Chinese
faithfulness.adapt(language="chinese")

data_samples = {
    'question': [''],
    'answer': ['当站台一侧或一组灯带不亮时，处理步骤如下：\n\n1. **安全措施**：首先确保在处理故障时断开电源，以避免电击风险。\n\n2. **检查LED日光灯**：如果是个别灯不亮，检查这些LED日光灯是否故障。拆下不亮的灯泡，检查是否需要更换新的灯泡或其驱动器。\n\n3. **检查电压**：如果部分灯不亮，可能是电压问题。使用电压表检测电源电压是否在正常范围内。如果电压不稳定，可能需要调整电源或查找其他故障源。\n\n4. **检查供电箱和控制空开**：如果整侧灯带不亮，检查供电箱的灯带切换和控制空开是否正常工作。确认空开没有跳闸，如果发现异常，进行调整或更换。\n\n5. **检查卡位和安装**：确认灯带的卡位是否牢固，是否存在损坏导致灯带无法正常工作。如有必要，修复或更换卡位以确保灯带固定。\n\n6. **检查接线**：检查连接线是否松动或老化。紧固接线，确保所有连接处无松动现象，必要时更换损坏的电线。\n\n7. **重新接通电源并测试**：在完成上述步骤后，重新接通电源，并测试站台灯带是否恢复正常。如果问题仍未解决，可能需要进一步的专业电气检查和维修，并通知相关人员配合。\n\n请务必在执行这些步骤时遵守所有安全规定，并在必要时寻求专业帮助。'],
    'contexts' : [['']],
    'ground_truth': ['当站台一侧或一组灯带不亮时，应按照以下步骤进行处理：\n1.检查空气开关：\n到达现场后，首先通过车站电话报告环调维修人员已到现场，并向车站工作人员申请C2类临修点。\n检查相应灯带箱的空气开关是否跳闸或烧坏。如果空气开关跳闸，尝试复位或更换空气开关。\n2.检查线路电压：\n使用万用表或电笔测量线路是否有电。如果发现线路没有电，进一步检查电源是否正常供电。\n3.检查线路完整性：\n检查线路是否短路或烧坏。可以通过目视检查或使用测电工具来确定线路的状况。如果发现短路或烧坏情况，进行相应的维修或更换线路。\n4.处理异常情况：\n清理线路中的异物或修复损坏的部分。如果发现灯带本身损坏，需要更换灯带。\n5.报告处理结果：\n处理完成后，向环调和车站报告处理结果，确保记录维修日志以备后续参考。\n联系人信息：\n如果在处理过程中需要进一步的支持或确认解决方案，可以联系以下负责人：\n电力调度员：张三，电话号码：12345678，负责供电日常组织、指挥工作，保证整个地铁供电系统安全运行和连续供电。\n故障报告：\n根据上述步骤生成的故障报告应包括以下内容：\n1.故障描述：站台一侧或一组灯带不亮。\n2.故障原因：可能是空气开关跳闸、线路断电或线路短路等原因引起。\n3.处理步骤：检查空气开关、线路电压和线路完整性，清理异物或修复损坏部分。\n4.联系人信息：电力调度员张三，电话号码：12345678。\n5.处理结果：处理完成后报告环调和车站，并记录维修日志。\n按照以上步骤进行操作，确保灯带故障能够及时、准确地得到处理和解决。']
}

dataset = Dataset.from_dict(data_samples)

score = evaluate(dataset,metrics=[answer_similarity,faithfulness,answer_correctness])
print (score["faithfulness"])
print(score.to_pandas())

also meet the error: No statements were generated from the answer.

Evaluating:  33%|███▎      | 1/3 [00:01<00:03,  1.81s/it]No statements were generated from the answer.
Evaluating:  67%|██████▋   | 2/3 [00:02<00:01,  1.12s/it]#########################输出tp的值：0
#########################输出fp的值：0
#########################输出fn的值：15
Evaluating: 100%|██████████| 3/3 [00:40<00:00, 13.34s/it]
C:\Users\Lenovo\AppData\Roaming\Python\Python310\site-packages\ragas\evaluation.py:299: RuntimeWarning: Mean of empty slice
  value = np.nanmean(self.scores[cn])
nan
  question  ... answer_correctness
0           ...           0.235555

[1 rows x 7 columns]

Process finished with exit code 0




<!-- Answer -->
To resolve the issue of "No statements were generated from the answer" when calculating the faithfulness metric using the Ragas library with Chinese text, you need to ensure that the `sentence_segmenter` is properly set up to handle Chinese text. The `sentence_segmenter` is responsible for breaking down the text into sentences, which are then used to generate simpler statements.

Here is the relevant part of the code where the `sentence_segmenter` is set up:

```python
def __post_init__(self):
    if self.sentence_segmenter is None:
        language = self.nli_statements_message.language
        self.sentence_segmenter = get_segmenter(language=language, clean=False)
```

To handle Chinese text, you need to make sure that the `get_segmenter` function can return a segmenter that works with Chinese. If `get_segmenter` does not support Chinese, you may need to implement or integrate a segmenter that can handle Chinese text.

Here is an example of how you might modify the `__post_init__` method to ensure the segmenter is set up correctly for Chinese:

```python
def __post_init__(self):
    if self.sentence_segmenter is None:
        language = self.nli_statements_message.language
        if language == "chinese":
            self.sentence_segmenter = ChineseSegmenter()  # Replace with actual Chinese segmenter
        else:
            self.sentence_segmenter = get_segmenter(language=language, clean=False)
```

Make sure to replace `ChineseSegmenter` with the actual implementation or library that can segment Chinese text. This should help in generating statements from the answer, thus resolving the issue <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_faithfulness.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


Closing as resolved in latest versions.