Repository: xorbitsai/inference

Issue:
BUG: RESTful API not compatible with OpenAI API
### Describe the bug
you said your RESTful API(compatible with OpenAI API)，but i post a openai json ，it response a errror.
![image](https://github.com/xorbitsai/inference/assets/16047426/cc029855-fb5b-4358-a779-33d79912a1dd)

![image](https://github.com/xorbitsai/inference/assets/16047426/4b7dbde6-d481-4cde-a5bc-040ed2122b23)

my json payload is from openai  and change model name to model id:
```
{
     "model": "c0021658-3289-11ee-89d1-00155d722c2a",
     "messages": [{"role": "user", "content": "Say this is a test!"}],
     "temperature": 0.7
   }
```
![image](https://github.com/xorbitsai/inference/assets/16047426/36be3c3e-b0a3-467e-9a17-e2e6e3be0472)

the error info:
```
getattr(self._model, "chat")(prompt, *args, **kwargs)
TypeError: [address=127.0.0.1:32829, pid=116] ChatglmCppChatModel.chat() takes from 2 to 4 positional arguments but 5 were given
```


Discussion:
Thanks for reporting this issue! @jiayini1119 Could you pls have a look at this issue?

Resolving patch:
diff --git a/xinference/core/restful_api.py b/xinference/core/restful_api.py
--- a/xinference/core/restful_api.py
+++ b/xinference/core/restful_api.py
@@ -480,7 +480,7 @@ async def create_chat_completion(
             (msg["content"] for msg in body.messages if msg["role"] == "system"), None
         )
 
-        chat_history = body.messages
+        chat_history = body.messages[:-1]  # exclude the prompt
 
         model_uid = body.model
 
@@ -494,6 +494,26 @@ async def create_chat_completion(
             logger.error(e, exc_info=True)
             raise HTTPException(status_code=500, detail=str(e))
 
+        try:
+            desc = await self._supervisor_ref.describe_model(model_uid)
+
+        except ValueError as ve:
+            logger.error(str(ve), exc_info=True)
+            raise HTTPException(status_code=400, detail=str(ve))
+
+        except Exception as e:
+            logger.error(e, exc_info=True)
+            raise HTTPException(status_code=500, detail=str(e))
+
+        is_chatglm_ggml = desc.get(
+            "model_format"
+        ) == "ggmlv3" and "chatglm" in desc.get("model_name", "")
+
+        if is_chatglm_ggml and system_prompt is not None:
+            raise HTTPException(
+                status_code=400, detail="ChatGLM ggml does not have system prompt"
+            )
+
         if body.stream:
             # create a pair of memory object streams
             send_chan, recv_chan = anyio.create_memory_object_stream(10)
@@ -501,9 +521,12 @@ async def create_chat_completion(
             async def event_publisher(inner_send_chan: MemoryObjectSendStream):
                 async with inner_send_chan:
                     try:
-                        iterator = await model.chat(
-                            prompt, system_prompt, chat_history, kwargs
-                        )
+                        if is_chatglm_ggml:
+                            iterator = await model.chat(prompt, chat_history, kwargs)
+                        else:
+                            iterator = await model.chat(
+                                prompt, system_prompt, chat_history, kwargs
+                            )
                         async for chunk in iterator:
                             await inner_send_chan.send(dict(data=json.dumps(chunk)))
                             if await request.is_disconnected():
@@ -525,7 +548,10 @@ async def event_publisher(inner_send_chan: MemoryObjectSendStream):
 
         else:
             try:
-                return await model.chat(prompt, system_prompt, chat_history, kwargs)
+                if is_chatglm_ggml:
+                    return await model.chat(prompt, chat_history, kwargs)
+                else:
+                    return await model.chat(prompt, system_prompt, chat_history, kwargs)
             except Exception as e:
                 logger.error(e, exc_info=True)
                 raise HTTPException(status_code=500, detail=str(e))


Test patch: