Repository: xorbitsai/inference

Issue:
RuntimeError: Cluster is not available after multiple attempts
### Describe the bug
A clear and concise description of what the bug is.

### To Reproduce
To help us to reproduce this bug, please provide information below:

1. Your Python version.
2. The version of xinference you use.
3. Versions of crucial packages.
4. Full stack of the error.
5. Minimized code to reproduce the error.

### Expected behavior
A clear and concise description of what you expected to happen.

### Additional context
Add any other context about the problem here.

以使用pip install "xinference[all]"配置环境
运行xinference-local报错如下：
2023-12-22 16:03:51,162 xinference.core.supervisor 25692 INFO     Xinference supervisor 127.0.0.1:36225 started
2023-12-22 16:03:51,166 xinference.core.worker 25692 INFO     Xinference worker 127.0.0.1:36225 started
2023-12-22 16:03:51,167 xinference.core.worker 25692 INFO     Purge cache directory: C:\Users\weipeng\.xinference\cache
Process Process-1:
Traceback (most recent call last):
  File "D:\Anaconda\envs\xinf\lib\multiprocessing\process.py", line 314, in _bootstrap
    self.run()
  File "D:\Anaconda\envs\xinf\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "D:\Anaconda\envs\xinf\lib\site-packages\xinference\deploy\local.py", line 71, in run
    loop.run_until_complete(task)
  File "D:\Anaconda\envs\xinf\lib\asyncio\base_events.py", line 649, in run_until_complete
    return future.result()
  File "D:\Anaconda\envs\xinf\lib\site-packages\xinference\deploy\local.py", line 52, in _start_local_cluster
    await start_worker_components(
  File "D:\Anaconda\envs\xinf\lib\site-packages\xinference\deploy\worker.py", line 39, in start_worker_components
    await xo.create_actor(
  File "D:\Anaconda\envs\xinf\lib\site-packages\xoscar\api.py", line 59, in create_actor
    return await ctx.create_actor(actor_cls, *args, uid=uid, address=address, **kwargs)
  File "D:\Anaconda\envs\xinf\lib\site-packages\xoscar\backends\context.py", line 143, in create_actor
    return self._process_result_message(result)
  File "D:\Anaconda\envs\xinf\lib\site-packages\xoscar\backends\context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "D:\Anaconda\envs\xinf\lib\site-packages\xoscar\backends\pool.py", line 596, in create_actor
    await self._run_coro(message.message_id, actor.__post_create__())
  File "D:\Anaconda\envs\xinf\lib\site-packages\xoscar\backends\pool.py", line 368, in _run_coro
    return await coro
  File "D:\Anaconda\envs\xinf\lib\site-packages\xinference\core\worker.py", line 114, in __post_create__
    loop.add_signal_handler(
  File "D:\Anaconda\envs\xinf\lib\asyncio\events.py", line 553, in add_signal_handler
    raise NotImplementedError
NotImplementedError: [address=127.0.0.1:36225, pid=25692]
Traceback (most recent call last):
  File "D:\Anaconda\envs\xinf\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "D:\Anaconda\envs\xinf\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "D:\Anaconda\envs\xinf\Scripts\xinference-local.exe\__main__.py", line 7, in <module>
  File "D:\Anaconda\envs\xinf\lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "D:\Anaconda\envs\xinf\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "D:\Anaconda\envs\xinf\lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "D:\Anaconda\envs\xinf\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "D:\Anaconda\envs\xinf\lib\site-packages\xinference\deploy\cmdline.py", line 167, in local
    start_local_cluster(log_level=log_level, host=host, port=port)
  File "D:\Anaconda\envs\xinf\lib\site-packages\xinference\deploy\cmdline.py", line 80, in start_local_cluster
    main(
  File "D:\Anaconda\envs\xinf\lib\site-packages\xinference\deploy\local.py", line 91, in main
    raise RuntimeError("Cluster is not available after multiple attempts")
RuntimeError: Cluster is not available after multiple attempts





Discussion:
@weipeng008005 更改这两个环境变量，默认都是3，可以调大一点，例如这样启动：
```
XINFERENCE_HEALTH_CHECK_INTERVAL=10 XINFERENCE_HEALTH_CHECK_ATTEMPTS=6 xinference-local
```
> @weipeng008005 更改这两个环境变量，默认都是3，可以调大一点，例如这样启动：
> 
> ```
> XINFERENCE_HEALTH_CHECK_INTERVAL=10 XINFERENCE_HEALTH_CHECK_ATTEMPTS=6 xinference-local
> ```

'XINFERENCE_HEALTH_CHECK_INTERVAL' 不是内部或外部命令，也不是可运行的程序
或批处理文件。
> @weipeng008005 更改这两个环境变量，默认都是3，可以调大一点，例如这样启动：
> 
> ```
> XINFERENCE_HEALTH_CHECK_INTERVAL=10 XINFERENCE_HEALTH_CHECK_ATTEMPTS=6 xinference-local
> ```

windows10
python版本3.10
xinference版本0.7.3
> > @weipeng008005 更改这两个环境变量，默认都是3，可以调大一点，例如这样启动：
> > ```
> > XINFERENCE_HEALTH_CHECK_INTERVAL=10 XINFERENCE_HEALTH_CHECK_ATTEMPTS=6 xinference-local
> > ```
> 
> 'XINFERENCE_HEALTH_CHECK_INTERVAL' 不是内部或外部命令，也不是可运行的程序 或批处理文件。

Windows环境设置这两个环境变量，我发的这个是Linux的使用方式，Windows如下：
```
set XINFERENCE_HEALTH_CHECK_INTERVAL=10
set XINFERENCE_HEALTH_CHECK_ATTEMPTS=6
xinference-local
```
> xinference-local

嗯，我意识到了。按照这个方式启动依然报一样的错
> > xinference-local
> 
> 嗯，我意识到了。按照这个方式启动依然报一样的错

是我们的一个bug，我们稍后发个hotfix 版本，多谢反馈。
> > > xinference-local
> > 
> > 
> > 嗯，我意识到了。按照这个方式启动依然报一样的错
> 
> 是我们的一个bug，我们稍后发个hotfix 版本，多谢反馈。

好的，谢谢
> > > xinference-local
> > 
> > 
> > 嗯，我意识到了。按照这个方式启动依然报一样的错
> 
> 是我们的一个bug，我们稍后发个hotfix 版本，多谢反馈。

请问我可以怎么使用这个版本？

Resolving patch:
diff --git a/xinference/core/worker.py b/xinference/core/worker.py
--- a/xinference/core/worker.py
+++ b/xinference/core/worker.py
@@ -106,14 +106,17 @@ async def __post_create__(self):
             ),
         }
 
-        async def singal_handler():
-            await self._supervisor_ref.remove_worker(self.address)
-            os._exit(0)
+        # Windows does not have signal handler
+        if os.name != "nt":
 
-        loop = asyncio.get_running_loop()
-        loop.add_signal_handler(
-            signal.SIGINT, lambda: asyncio.create_task(singal_handler())
-        )
+            async def signal_handler():
+                await self._supervisor_ref.remove_worker(self.address)
+                os._exit(0)
+
+            loop = asyncio.get_running_loop()
+            loop.add_signal_handler(
+                signal.SIGINT, lambda: asyncio.create_task(signal_handler())
+            )
 
     async def __pre_destroy__(self):
         self._upload_task.cancel()


Test patch:
diff --git a/xinference/model/llm/tests/test_llm_family.py b/xinference/model/llm/tests/test_llm_family.py
--- a/xinference/model/llm/tests/test_llm_family.py
+++ b/xinference/model/llm/tests/test_llm_family.py
@@ -11,7 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-
+import codecs
 import json
 import os
 import shutil
@@ -168,7 +168,7 @@ def test_builtin_llm_families():
     json_path = os.path.join(
         os.path.dirname(os.path.abspath(__file__)), "..", "llm_family.json"
     )
-    for json_obj in json.load(open(json_path)):
+    for json_obj in json.load(codecs.open(json_path, "r", encoding="utf-8")):
         LLMFamilyV1.parse_obj(json_obj)