Repository: xorbitsai/inference

Issue:
BUG：v0.9.2 modelscope源启动bge-reranker-base和bge-reranker-large模型失败
### Describe the bug
显卡 RTX3090，驱动530.30.2，cuda V12.1.105，bge-reranker-base和bge-reranker-large模型不能被正常启动，且无报错

### To Reproduce

1. conda 4.5.11，python 3.10
2. pip install "xinference[all]"
3. XINFERENCE_MODEL_SRC=modelscope XINFERENCE_HOME=/home/rkty/models xinference-local --host 0.0.0.0 --port 9997
4. xinference launch --model-name bge-reranker-large --model-type rerank --model-uid 12
5.  xinference list

### Expected behavior
没有模型被加载，且无报错

### Attachment
![屏幕截图 2024-03-13 084614](https://github.com/xorbitsai/inference/assets/45476210/2ef0e302-8ded-44fc-b13a-eb7a159e9bd3)
![屏幕截图 2024-03-13 084541](https://github.com/xorbitsai/inference/assets/45476210/457a1b56-668e-4265-a422-5289190b18e0)
![屏幕截图 2024-03-13 084550](https://github.com/xorbitsai/inference/assets/45476210/5cdd192f-fa9d-4403-ad49-9f4a0ddf50ca)




Discussion:
在RTX 4090上复现了这个问题，驱动版本 535.129.03，cuda版本 11.8.89，conda 22.11.1，python 3.10
1、启动bge-reranker-large uuid=12，xinference list无启动模型
2、启动bge-large-zh-v1.5 uuid=12，报错uuid重复
3、启动bge-large-zh-v1.5 uuid=13，xinference list显示模型正常启动，显存被正常占用
![屏幕截图 2024-03-13 100034](https://github.com/xorbitsai/inference/assets/45476210/37bb89ce-045b-4fc5-8acd-1b63029379e6)

![屏幕截图 2024-03-13 100051](https://github.com/xorbitsai/inference/assets/45476210/0c2f1df5-9e93-4c3b-9e73-1141f4855fe4)

![屏幕截图 2024-03-13 100359](https://github.com/xorbitsai/inference/assets/45476210/6bc255c7-0aaa-43b0-9f8e-b0e10d27da74)


@bigzws 收到。我这边可以复现，本周发布新版时将修复。原因是xinference list命令目前还没有加入rerank模型的显示。
另外，launch rerank模型后看不到显存占用是正常的，embedding和rerank模型要在真正使用时才会看到显存占用。
另外，基于我目前的情况，还有一个问题，dify调用bge-reranker报错，但是在xinference服务中，没有任何报错提示，仅bge-reranker报错，LLM和embedding模型可正常添加到dify
![屏幕截图 2024-03-13 112742](https://github.com/xorbitsai/inference/assets/45476210/b42d6275-df3f-49ba-b52f-53ebe340b134)
![屏幕截图 2024-03-13 112736](https://github.com/xorbitsai/inference/assets/45476210/6f05312c-373a-40e3-82b1-3a763c8d95f8)

dify版本0.5.8
@bigzws dify问题请到dify社区咨询

Resolving patch:
diff --git a/xinference/deploy/cmdline.py b/xinference/deploy/cmdline.py
--- a/xinference/deploy/cmdline.py
+++ b/xinference/deploy/cmdline.py
@@ -488,6 +488,22 @@ def list_model_registrations(
             ),
             file=sys.stderr,
         )
+    elif model_type == "rerank":
+        for registration in registrations:
+            model_name = registration["model_name"]
+            model_family = client.get_model_registration(model_type, model_name)
+            table.append(
+                [
+                    model_type,
+                    model_family["model_name"],
+                    model_family["language"],
+                    registration["is_builtin"],
+                ]
+            )
+        print(
+            tabulate(table, headers=["Type", "Name", "Language", "Is-built-in"]),
+            file=sys.stderr,
+        )
     elif model_type == "image":
         for registration in registrations:
             model_name = registration["model_name"]
@@ -711,6 +727,9 @@ def model_list(endpoint: Optional[str]):
 
     llm_table = []
     embedding_table = []
+    rerank_table = []
+    image_table = []
+    audio_table = []
     models = client.list_models()
     for model_uid, model_spec in models.items():
         if model_spec["model_type"] == "LLM":
@@ -733,6 +752,23 @@ def model_list(endpoint: Optional[str]):
                     model_spec["dimensions"],
                 ]
             )
+        elif model_spec["model_type"] == "rerank":
+            rerank_table.append(
+                [model_uid, model_spec["model_type"], model_spec["model_name"]]
+            )
+        elif model_spec["model_type"] == "image":
+            image_table.append(
+                [
+                    model_uid,
+                    model_spec["model_type"],
+                    model_spec["model_name"],
+                    str(model_spec["controlnet"]),
+                ]
+            )
+        elif model_spec["model_type"] == "audio":
+            audio_table.append(
+                [model_uid, model_spec["model_type"], model_spec["model_name"]]
+            )
     if llm_table:
         print(
             tabulate(
@@ -748,6 +784,7 @@ def model_list(endpoint: Optional[str]):
             ),
             file=sys.stderr,
         )
+        print()  # add a blank line for better visual experience
     if embedding_table:
         print(
             tabulate(
@@ -761,6 +798,34 @@ def model_list(endpoint: Optional[str]):
             ),
             file=sys.stderr,
         )
+        print()
+    if rerank_table:
+        print(
+            tabulate(
+                rerank_table,
+                headers=["UID", "Type", "Name"],
+            ),
+            file=sys.stderr,
+        )
+        print()
+    if image_table:
+        print(
+            tabulate(
+                image_table,
+                headers=["UID", "Type", "Name", "Controlnet"],
+            ),
+            file=sys.stderr,
+        )
+        print()
+    if audio_table:
+        print(
+            tabulate(
+                audio_table,
+                headers=["UID", "Type", "Name"],
+            ),
+            file=sys.stderr,
+        )
+        print()
 
 
 @cli.command(


Test patch: