Repository: xorbitsai/inference
Pull request: FEAT: llama.cpp backend support multimodal
- llama.cpp backend support mutimodal projectors
- gemma3 gguf support multimodal
- makes the inference error clear

Fixes: https://github.com/xorbitsai/inference/issues/3416

Problem statement:
使用llama运行gemma3，识别图片时报错
### System Info / 系統信息

官方docker镜像v1.5.1

### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

- [ ] docker / docker
- [ ] pip install / 通过 pip install 安装
- [ ] installation from source / 从源码安装

### Version info / 版本信息

官方docker镜像，版本1.5.1

### The command used to start Xinference / 用以启动 xinference 的命令

xinference-local -H 0.0.0.0

### Reproduction / 复现过程

1. 使用dify接入xinferecne模型
2. 聊天的时候输入图片

出现错误
```
supervisor-1  | Traceback (most recent call last):
supervisor-1  |   File "/usr/local/lib/python3.10/dist-packages/xinference/model/llm/llama_cpp/core.py", line 308, in _handle_chat_completion
supervisor-1  |     self._llm.handle_chat_completions(
supervisor-1  |   File "xllamacpp.pyx", line 2073, in xllamacpp.xllamacpp.Server.handle_chat_completions
supervisor-1  | RuntimeError: Failed to parse messages: Unsupported content part type: "image_url"; messages = [
supervisor-1  |   {
supervisor-1  |     "role": "user",
supervisor-1  |     "content": "你好"
supervisor-1  |   },
supervisor-1  |   {
supervisor-1  |     "role": "assistant",
supervisor-1  |     "content": "你好！很高兴认识你。有什么我可以帮助你的吗？ 😊"
supervisor-1  |   },
supervisor-1  |   {
supervisor-1  |     "role": "user",
supervisor-1  |     "content": "你是谁"
supervisor-1  |   },
supervisor-1  |   {
supervisor-1  |     "role": "assistant",
supervisor-1  |     "content": "我是一个大型语言模型，由 Google 训练。 \n\n简单来说，我是一个人工智能程序，可以理解和生成人类语言。 我可以：\n\n*   回答你的问题\n*   写不同类型的文本格式，例如诗歌、代码、脚本、音乐作品、电子邮件、信件等。\n*   翻译语言\n*   总结文本\n*   进行对话\n\n我还在不断学习和进步！\n\n你有什么想问我的吗？"
supervisor-1  |   },
supervisor-1  |   {
supervisor-1  |     "role": "user",
supervisor-1  |     "content": [
supervisor-1  |       {
supervisor-1  |         "type": "text",
supervisor-1  |         "text": "图里有什么"
supervisor-1  |       },
supervisor-1  |       {
supervisor-1  |         "type": "image_url",
supervisor-1  |         "image_url": {
supervisor-1  |           "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADkAAAAbCAIAAABJHrDvAAACwUlEQVRYCdWX3Y6aQBTHeSnvfQXvvSg3TXeTte1eaNImbbKT9JJ3MLEXXa/aRzAm+xQqg4yi8g2CMOC0MwjChhU2TSo1kzhzOMP5zeE/X5zt+v9L4TJQy/EaVTKwrEJZLcdbb1QI0Xwhzebw6mW+kCBE641qOV4Gars+pxk2faBsD0EQxzFpwC+O40MQrJUthEgz7AyXo23daABhCYKq6RAi094nuBxaKSVejTGt1puVskvEwPn+oTFgJSCe54tQTlLLRVEjNFqCyUwYR/OFZFiu5XjcS06p/QnwY5Q2CHkCbWF6bhI0vGu1Oy+Ugmeu0+uqsznUzcusE6FAwAuALzB1h0tCKGtSofHhuAueUpDliP9nrCzmFHRa/B1gWAlZq905wzWHlYICAbSFEUveFHS6w3HSbKX5q6+B+Hh03H2adfrv7vfx8Zi3lNYrNbAc8R0wIVSgTK/pt071OhFaBTuLclEDox8/+ZuBfzgtO3vPf/Ou//3xVylf3ljJypyfSTY3hzIZoOEdG1I1a4hxr//wtvc5CMIgCPnbQa//EEVRHqu0Xo+1tGvRWF8DhJAQ4w8D8H4Abu+/3n/6FmJcfFl5q5r1IgRbEJgGpiCRCgtzUQMJSIjxzccvvf5DTVBCSDXr8zFOhGw+5R4VF6YarISQwyGoD/oKVrYUsCUzYYXjbn7NSmfYib4ea26otarVeU00QOdNOsPy86nVpp++IIBsL2DjoftIYcOrhVXqVM1a2u0qxjNrw88uURTPF9LpPOB5/lWyVTMoPROK8ol1td7U7HYVN7RS5NXmdCYURVnV9KtwVAZVNZ3i6fbprL3dGaIoS0tk2w7G1TteZYC/dwhDbNuOtESiKG93RiIAeo/VTVfVLRkpoijPmnHnni0kUZRlpKi6pZvu+W5o2nuGa+80c6saDSk7zVR1Ow/6J6+/AVY5lnn59SJwAAAAAElFTkSuQmCC",
supervisor-1  |           "detail": "high"
supervisor-1  |         }
supervisor-1  |       }
supervisor-1  |     ]
supervisor-1  |   }
supervisor-1  | ]
```

### Expected behavior / 期待表现

正常识别图片内容

Hints from issue discussion:
@codingl2k1 llama.cpp 支持 gemma-3 图片输入了吗？
> [@codingl2k1](https://github.com/codingl2k1) llama.cpp 支持 gemma-3 图片输入了吗？

目前 llama server 还不支持 multimodal（wip）：https://github.com/ggml-org/llama.cpp/tree/master/tools/server
有个 libmtmd 提供了 multimodal 的功能，但不是个完整的 server（上面那个 server 的 multimodal 还在开发中）
好吧（捂脸），谢谢解答

Commit message: feat: [UI] add the multimodal_projector parameter

Patch to review:
diff --git a/.github/workflows/python.yaml b/.github/workflows/python.yaml
index 60b181f074..9bdf77d6fd 100644
--- a/.github/workflows/python.yaml
+++ b/.github/workflows/python.yaml
@@ -125,7 +125,7 @@ jobs:
             sudo rm -rf "$AGENT_TOOLSDIRECTORY"
           fi
           pip install -e ".[dev]"
-          pip install xllamacpp
+          pip install "xllamacpp>=0.1.16"
           if [ "$MODULE" == "metal" ]; then
             conda install -c conda-forge "ffmpeg<7"
             pip install "mlx>=0.22.0"
diff --git a/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/backends.po b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/backends.po
index 34f8c57d80..825a26246b 100644
--- a/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/backends.po
+++ b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/backends.po
@@ -8,7 +8,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Xinference \n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2025-04-19 01:03+0800\n"
+"POT-Creation-Date: 2025-05-14 10:03+0200\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language: zh_CN\n"
@@ -17,7 +17,7 @@ msgstr ""
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=utf-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"Generated-By: Babel 2.14.0\n"
+"Generated-By: Babel 2.16.0\n"
 
 #: ../../source/user_guide/backends.rst:5
 msgid "Backends"
@@ -56,8 +56,9 @@ msgid ""
 "llama.cpp, and ``llama-cpp-python`` will be deprecated. For Xinference "
 "v1.6.0, ``llama-cpp-python`` will be removed."
 msgstr ""
-"自 Xinference v1.5.0 起，``xllamacpp`` 成为 llama.cpp 的默认选项，"
-"``llama-cpp-python`` 将被弃用；在 Xinference v1.6.0 中，``llama-cpp-python`` 将被移除。"
+"自 Xinference v1.5.0 起，``xllamacpp`` 成为 llama.cpp 的默认选项，``llama"
+"-cpp-python`` 将被弃用；在 Xinference v1.6.0 中，``llama-cpp-python`` 将"
+"被移除。"
 
 #: ../../source/user_guide/backends.rst:24
 msgid ""
@@ -72,73 +73,130 @@ msgstr ""
 "https://github.com/abetlen/llama-cpp-python#installation-with-openblas--"
 "cublas--clblast--metal>`_。"
 
-#: ../../source/user_guide/backends.rst:30
+#: ../../source/user_guide/backends.rst:29
+msgid "Common Issues"
+msgstr "常见问题"
+
+#: ../../source/user_guide/backends.rst:31
+msgid ""
+"**Server error: {'code': 500, 'message': 'failed to process image', "
+"'type': 'server_error'}**"
+msgstr ""
+
+#: ../../source/user_guide/backends.rst:33
+#: ../../source/user_guide/backends.rst:57
+msgid "The error logs from server:"
+msgstr "服务端日志："
+
+#: ../../source/user_guide/backends.rst:48
+msgid ""
+"This could be caused by running out of memory. You can try reducing "
+"memory usage by decreasing ``n_ctx``."
+msgstr "可能由于内存不足导致。你可以尝试减小 ``n_ctx`` 解决。"
+
+#: ../../source/user_guide/backends.rst:50
+msgid ""
+"**Server error: {'code': 400, 'message': 'the request exceeds the "
+"available context size. try increasing the context size or enable context"
+" shift', 'type': 'invalid_request_error'}**"
+msgstr ""
+
+#: ../../source/user_guide/backends.rst:52
+msgid ""
+"If you are using the multimodal feature, the ``ctx_shift`` is disabled by"
+" default. Please increase the context size by either increasing ``n_ctx``"
+" or reducing ``n_parallel``."
+msgstr ""
+"如果你正在使用 multimodal 功能，``ctx_shift`` 会被默认关闭。请尝试增加 ``"
+"n_ctx`` 或者减小 ``n_parallel`` 以增加每个 slot 的 context 大小。"
+
+#: ../../source/user_guide/backends.rst:55
+msgid ""
+"**Server error: {'code': 500, 'message': 'Input prompt is too big "
+"compared to KV size. Please try increasing KV size.', 'type': "
+"'server_error'}**"
+msgstr ""
+
+#: ../../source/user_guide/backends.rst:67
+msgid ""
+"This could be caused by the KV cache allocation failure. You can try to "
+"reduce the context size by either reducing ``n_ctx`` or increasing "
+"``n_parallel``, or loading a partial model onto the GPU by adjusting "
+"``n_gpu_layers``. Be aware that if you are handling inference requests "
+"serially, increasing ``n_parallel`` can't improve the latency or "
+"throughput."
+msgstr ""
+"可能由于 KV cache 创建失败导致。你可以通过减小 ``n_ctx`` 或者增加 ``n_"
+"parallel`` 或者调节 ``n_gpu_layers`` 参数加载部分模型到 GPU 来解决。请"
+"注意，如果你只处理串行推理请求，增加 ``n_parallel`` 并不会带来性能提升。"
+
+#: ../../source/user_guide/backends.rst:72
 msgid "transformers"
 msgstr "transformers"
 
-#: ../../source/user_guide/backends.rst:31
+#: ../../source/user_guide/backends.rst:73
 msgid ""
 "Transformers supports the inference of most state-of-art models. It is "
 "the default backend for models in PyTorch format."
 msgstr "Transformers 支持绝大部分新出的模型。是 Pytorch 格式模型默认使用的引擎。"
 
-#: ../../source/user_guide/backends.rst:36
+#: ../../source/user_guide/backends.rst:78
 msgid "vLLM"
 msgstr "vLLM"
 
-#: ../../source/user_guide/backends.rst:37
+#: ../../source/user_guide/backends.rst:79
 msgid "vLLM is a fast and easy-to-use library for LLM inference and serving."
 msgstr "vLLM 是一个非常高效并且易用的大语言模型推理引擎。"
 
-#: ../../source/user_guide/backends.rst:39
+#: ../../source/user_guide/backends.rst:81
 msgid "vLLM is fast with:"
 msgstr "vLLM 具有以下特点："
 
-#: ../../source/user_guide/backends.rst:41
+#: ../../source/user_guide/backends.rst:83
 msgid "State-of-the-art serving throughput"
 msgstr "领先的推理吞吐量"
 
-#: ../../source/user_guide/backends.rst:42
+#: ../../source/user_guide/backends.rst:84
 msgid "Efficient management of attention key and value memory with PagedAttention"
 msgstr "使用 PagedAttention 高效管理注意力键和值记忆"
 
-#: ../../source/user_guide/backends.rst:43
+#: ../../source/user_guide/backends.rst:85
 msgid "Continuous batching of incoming requests"
 msgstr "对传入请求进行连续批处理"
 
-#: ../../source/user_guide/backends.rst:44
+#: ../../source/user_guide/backends.rst:86
 msgid "Optimized CUDA kernels"
 msgstr "优化的 CUDA 内核"
 
-#: ../../source/user_guide/backends.rst:46
+#: ../../source/user_guide/backends.rst:88
 msgid ""
 "When the following conditions are met, Xinference will choose vLLM as the"
 " inference engine:"
 msgstr "当满足以下条件时，Xinference 会自动选择 vLLM 作为推理引擎："
 
-#: ../../source/user_guide/backends.rst:48
+#: ../../source/user_guide/backends.rst:90
 msgid "The model format is ``pytorch``, ``gptq`` or ``awq``."
 msgstr "模型格式为 ``pytorch`` ， ``gptq`` 或者 ``awq`` 。"
 
-#: ../../source/user_guide/backends.rst:49
+#: ../../source/user_guide/backends.rst:91
 msgid "When the model format is ``pytorch``, the quantization is ``none``."
 msgstr "当模型格式为 ``pytorch`` 时，量化选项需为 ``none`` 。"
 
-#: ../../source/user_guide/backends.rst:50
+#: ../../source/user_guide/backends.rst:92
 msgid "When the model format is ``awq``, the quantization is ``Int4``."
 msgstr "当模型格式为 ``awq`` 时，量化选项需为 ``Int4`` 。"
 
-#: ../../source/user_guide/backends.rst:51
+#: ../../source/user_guide/backends.rst:93
 msgid ""
 "When the model format is ``gptq``, the quantization is ``Int3``, ``Int4``"
 " or ``Int8``."
 msgstr "当模型格式为 ``gptq`` 时，量化选项需为 ``Int3``, ``Int4`` 或 ``Int8`` 。"
 
-#: ../../source/user_guide/backends.rst:52
+#: ../../source/user_guide/backends.rst:94
 msgid "The system is Linux and has at least one CUDA device"
 msgstr "操作系统为 Linux 并且至少有一个支持 CUDA 的设备"
 
-#: ../../source/user_guide/backends.rst:53
+#: ../../source/user_guide/backends.rst:95
 msgid ""
 "The model family (for custom models) / model name (for builtin models) is"
 " within the list of models supported by vLLM"
@@ -146,37 +204,37 @@ msgstr ""
 "自定义模型的 ``model_family`` 字段和内置模型的 ``model_name`` 字段在 vLLM"
 " 的支持列表中。"
 
-#: ../../source/user_guide/backends.rst:55
+#: ../../source/user_guide/backends.rst:97
 msgid "Currently, supported model includes:"
 msgstr "目前，支持的模型包括："
 
-#: ../../source/user_guide/backends.rst:59
+#: ../../source/user_guide/backends.rst:101
 msgid ""
 "``llama-2``, ``llama-3``, ``llama-3.1``, ``llama-3.2-vision``, "
 "``llama-2-chat``, ``llama-3-instruct``, ``llama-3.1-instruct``, "
 "``llama-3.3-instruct``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:60
+#: ../../source/user_guide/backends.rst:102
 msgid ""
 "``mistral-v0.1``, ``mistral-instruct-v0.1``, ``mistral-instruct-v0.2``, "
 "``mistral-instruct-v0.3``, ``mistral-nemo-instruct``, ``mistral-large-"
 "instruct``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:61
+#: ../../source/user_guide/backends.rst:103
 msgid "``codestral-v0.1``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:62
+#: ../../source/user_guide/backends.rst:104
 msgid "``Yi``, ``Yi-1.5``, ``Yi-chat``, ``Yi-1.5-chat``, ``Yi-1.5-chat-16k``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:63
+#: ../../source/user_guide/backends.rst:105
 msgid "``code-llama``, ``code-llama-python``, ``code-llama-instruct``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:64
+#: ../../source/user_guide/backends.rst:106
 msgid ""
 "``deepseek``, ``deepseek-coder``, ``deepseek-chat``, ``deepseek-coder-"
 "instruct``, ``deepseek-r1-distill-qwen``, ``deepseek-v2-chat``, "
@@ -184,101 +242,113 @@ msgid ""
 "``deepseek-r1``, ``deepseek-r1-distill-llama``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:65
+#: ../../source/user_guide/backends.rst:107
 msgid "``yi-coder``, ``yi-coder-chat``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:66
+#: ../../source/user_guide/backends.rst:108
 msgid "``codeqwen1.5``, ``codeqwen1.5-chat``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:67
+#: ../../source/user_guide/backends.rst:109
 msgid ""
 "``qwen2.5``, ``qwen2.5-coder``, ``qwen2.5-instruct``, ``qwen2.5-coder-"
 "instruct``, ``qwen2.5-instruct-1m``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:68
+#: ../../source/user_guide/backends.rst:110
 msgid "``baichuan-2-chat``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:69
+#: ../../source/user_guide/backends.rst:111
 msgid "``internlm2-chat``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:70
+#: ../../source/user_guide/backends.rst:112
 msgid "``internlm2.5-chat``, ``internlm2.5-chat-1m``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:71
+#: ../../source/user_guide/backends.rst:113
 msgid "``qwen-chat``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:72
+#: ../../source/user_guide/backends.rst:114
 msgid "``mixtral-instruct-v0.1``, ``mixtral-8x22B-instruct-v0.1``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:73
+#: ../../source/user_guide/backends.rst:115
 msgid "``chatglm3``, ``chatglm3-32k``, ``chatglm3-128k``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:74
-msgid "``glm4-chat``, ``glm4-chat-1m``"
+#: ../../source/user_guide/backends.rst:116
+msgid "``glm4-chat``, ``glm4-chat-1m``, ``glm4-0414``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:75
+#: ../../source/user_guide/backends.rst:117
 msgid "``codegeex4``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:76
+#: ../../source/user_guide/backends.rst:118
 msgid "``qwen1.5-chat``, ``qwen1.5-moe-chat``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:77
+#: ../../source/user_guide/backends.rst:119
 msgid "``qwen2-instruct``, ``qwen2-moe-instruct``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:78
+#: ../../source/user_guide/backends.rst:120
 msgid "``QwQ-32B-Preview``, ``QwQ-32B``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:79
+#: ../../source/user_guide/backends.rst:121
 msgid "``marco-o1``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:80
+#: ../../source/user_guide/backends.rst:122
 msgid "``fin-r1``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:81
+#: ../../source/user_guide/backends.rst:123
+msgid "``seallms-v3``"
+msgstr ""
+
+#: ../../source/user_guide/backends.rst:124
+msgid "``skywork-or1-preview``"
+msgstr ""
+
+#: ../../source/user_guide/backends.rst:125
 msgid "``gemma-it``, ``gemma-2-it``, ``gemma-3-1b-it``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:82
+#: ../../source/user_guide/backends.rst:126
 msgid "``orion-chat``, ``orion-chat-rag``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:83
+#: ../../source/user_guide/backends.rst:127
 msgid "``c4ai-command-r-v01``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:84
+#: ../../source/user_guide/backends.rst:128
 msgid "``minicpm3-4b``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:85
+#: ../../source/user_guide/backends.rst:129
 msgid "``internlm3-instruct``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:86
+#: ../../source/user_guide/backends.rst:130
 msgid "``moonlight-16b-a3b-instruct``"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:92
+#: ../../source/user_guide/backends.rst:131
+msgid "``qwen3``"
+msgstr ""
+
+#: ../../source/user_guide/backends.rst:138
 msgid "SGLang"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:93
+#: ../../source/user_guide/backends.rst:139
 msgid ""
 "`SGLang <https://github.com/sgl-project/sglang>`_ has a high-performance "
 "inference runtime with RadixAttention. It significantly accelerates the "
@@ -290,11 +360,11 @@ msgstr ""
 " 的高性能推理运行时。它通过在多个调用之间自动重用KV缓存，显著加速了复杂 "
 "LLM 程序的执行。它还支持其他常见推理技术，如连续批处理和张量并行处理。"
 
-#: ../../source/user_guide/backends.rst:100
+#: ../../source/user_guide/backends.rst:146
 msgid "MLX"
 msgstr ""
 
-#: ../../source/user_guide/backends.rst:101
+#: ../../source/user_guide/backends.rst:147
 msgid ""
 "`MLX <https://github.com/ml-explore/mlx-examples/tree/main/llms>`_ "
 "provides efficient runtime to run LLM on Apple silicon. It's recommended "
@@ -323,3 +393,6 @@ msgstr ""
 #~ msgid "``gemma-it``, ``gemma-2-it``"
 #~ msgstr ""
 
+#~ msgid "``glm4-chat``, ``glm4-chat-1m``"
+#~ msgstr ""
+
diff --git a/doc/source/user_guide/backends.rst b/doc/source/user_guide/backends.rst
index bf325da850..dbbeca445f 100644
--- a/doc/source/user_guide/backends.rst
+++ b/doc/source/user_guide/backends.rst
@@ -8,7 +8,7 @@ Xinference supports multiple backends for different models. After the user speci
 xinference will automatically select the appropriate backend.
 
 llama.cpp
-~~~~~~~~~
+=========
 
 Xinference now supports `xllamacpp <https://github.com/xorbitsai/xllamacpp>`_ which developed by Xinference team,
 and `llama-cpp-python <https://github.com/abetlen/llama-cpp-python>`_ to run llama.cpp backend.
@@ -25,15 +25,57 @@ For `llama-cpp-python`, we recommend that users install  on the worker themselve
 parameters according to the hardware to achieve the best inference efficiency. Please refer to the
 `llama-cpp-python installation guide <https://github.com/abetlen/llama-cpp-python#installation-with-openblas--cublas--clblast--metal>`_.
 
+Common Issues
+-------------
+
+- **Server error: {'code': 500, 'message': 'failed to process image', 'type': 'server_error'}**
+
+  The error logs from server:
+
+  .. code-block::
+
+    encoding image or slice...
+    slot update_slots: id  0 | task 0 | kv cache rm [10, end)
+    srv  process_chun: processing image...
+    ggml_metal_graph_compute: command buffer 0 failed with status 5
+    error: Internal Error (0000000e:Internal Error)
+    clip_image_batch_encode: ggml_backend_sched_graph_compute failed with error -1
+    failed to encode image
+    srv  process_chun: image processed in 2288 ms
+    mtmd_helper_eval failed with status 1
+    slot update_slots: id  0 | task 0 | failed to process image, res = 1
+
+  This could be caused by running out of memory. You can try reducing memory usage by decreasing ``n_ctx``.
+
+- **Server error: {'code': 400, 'message': 'the request exceeds the available context size. try increasing the context size or enable context shift', 'type': 'invalid_request_error'}**
+
+  If you are using the multimodal feature, the ``ctx_shift`` is disabled by default. Please increase the context size by
+  either increasing ``n_ctx`` or reducing ``n_parallel``.
+
+- **Server error: {'code': 500, 'message': 'Input prompt is too big compared to KV size. Please try increasing KV size.', 'type': 'server_error'}**
+
+  The error logs from server:
+
+  .. code-block::
+
+    ggml_metal_graph_compute: command buffer 1 failed with status 5
+    error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
+    graph_compute: ggml_backend_sched_graph_compute_async failed with error -1
+    llama_decode: failed to decode, ret = -3
+    srv  update_slots: failed to decode the batch: KV cache is full - try increasing it via the context size, i = 0, n_batch = 2048, ret = -3
+
+  This could be caused by the KV cache allocation failure. You can try to reduce the context size by either reducing
+  ``n_ctx`` or increasing ``n_parallel``, or loading a partial model onto the GPU by adjusting ``n_gpu_layers``. Be aware
+  that if you are handling inference requests serially, increasing ``n_parallel`` can't improve the latency or throughput.
 
 transformers
-~~~~~~~~~~~~
+============
 Transformers supports the inference of most state-of-art models. It is the default backend for models in PyTorch format.
 
 .. _vllm_backend:
 
 vLLM
-~~~~
+====
 vLLM is a fast and easy-to-use library for LLM inference and serving.
 
 vLLM is fast with:
@@ -87,12 +129,13 @@ Currently, supported model includes:
 - ``internlm3-instruct``
 - ``moonlight-16b-a3b-instruct``
 - ``qwen3``
+
 .. vllm_end
 
 .. _sglang_backend:
 
 SGLang
-~~~~~~
+======
 `SGLang <https://github.com/sgl-project/sglang>`_ has a high-performance inference runtime with RadixAttention.
 It significantly accelerates the execution of complex LLM programs by automatic KV cache reuse across multiple calls.
 And it also supports other common techniques like continuous batching and tensor parallelism.
@@ -100,7 +143,7 @@ And it also supports other common techniques like continuous batching and tensor
 .. _mlx_backend:
 
 MLX
-~~~
+===
 `MLX <https://github.com/ml-explore/mlx-examples/tree/main/llms>`_ provides efficient runtime
 to run LLM on Apple silicon. It's recommended to use for Mac users when running on Apple silicon
 if the model has MLX format support.
diff --git a/setup.cfg b/setup.cfg
index 3e6dea9432..87454de5b0 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -94,7 +94,7 @@ intel =
     torch==2.1.0a0
     intel_extension_for_pytorch==2.1.10+xpu
 llama_cpp =
-    xllamacpp
+    xllamacpp>=0.1.16
 transformers =
     transformers>=4.46.0
     torch
diff --git a/xinference/api/restful_api.py b/xinference/api/restful_api.py
index 7fe8c4146e..c2c6b88bef 100644
--- a/xinference/api/restful_api.py
+++ b/xinference/api/restful_api.py
@@ -997,6 +997,7 @@ async def launch_model(
         replica = payload.get("replica", 1)
         n_gpu = payload.get("n_gpu", "auto")
         request_limits = payload.get("request_limits", None)
+        multimodal_projector = payload.get("multimodal_projector", None)
         peft_model_config = payload.get("peft_model_config", None)
         worker_ip = payload.get("worker_ip", None)
         gpu_idx = payload.get("gpu_idx", None)
@@ -1014,6 +1015,7 @@ async def launch_model(
             "replica",
             "n_gpu",
             "request_limits",
+            "multimodal_projector",
             "peft_model_config",
             "worker_ip",
             "gpu_idx",
@@ -1063,6 +1065,7 @@ async def launch_model(
                 n_gpu=n_gpu,
                 request_limits=request_limits,
                 wait_ready=wait_ready,
+                multimodal_projector=multimodal_projector,
                 peft_model_config=peft_model_config,
                 worker_ip=worker_ip,
                 gpu_idx=gpu_idx,
diff --git a/xinference/client/restful/restful_client.py b/xinference/client/restful/restful_client.py
index 6161722869..130c822fe4 100644
--- a/xinference/client/restful/restful_client.py
+++ b/xinference/client/restful/restful_client.py
@@ -970,6 +970,7 @@ def launch_model(
         replica: int = 1,
         n_worker: int = 1,
         n_gpu: Optional[Union[int, str]] = "auto",
+        multimodal_projector: Optional[str] = None,
         peft_model_config: Optional[Dict] = None,
         request_limits: Optional[int] = None,
         worker_ip: Optional[str] = None,
@@ -1003,6 +1004,8 @@ def launch_model(
         n_gpu: Optional[Union[int, str]],
             The number of GPUs used by the model, default is "auto". If n_worker>1, means number of GPUs per worker.
             ``n_gpu=None`` means cpu only, ``n_gpu=auto`` lets the system automatically determine the best number of GPUs to use.
+        multimodal_projector:  Optional[str]
+            The projector for multimodal inference with the llama.cpp backend.
         peft_model_config: Optional[Dict]
             - "lora_list": A List of PEFT (Parameter-Efficient Fine-Tuning) model and path.
             - "image_lora_load_kwargs": A Dict of lora load parameters for image model
@@ -1041,6 +1044,7 @@ def launch_model(
             "model_size_in_billions": model_size_in_billions,
             "model_format": model_format,
             "quantization": quantization,
+            "multimodal_projector": multimodal_projector,
             "replica": replica,
             "n_worker": n_worker,
             "n_gpu": n_gpu,
diff --git a/xinference/client/tests/test_client.py b/xinference/client/tests/test_client.py
index 5f69124661..60239becac 100644
--- a/xinference/client/tests/test_client.py
+++ b/xinference/client/tests/test_client.py
@@ -165,10 +165,6 @@ def _check(stream=False):
     assert len(client.list_models()) == 0
 
 
-def test_RESTful_client_xllamacpp(set_use_xllamacpp, setup):
-    test_RESTful_client(setup)
-
-
 @pytest.mark.skipif(os.name == "nt", reason="Skip windows")
 def test_list_cached_models(setup):
     endpoint, _ = setup
diff --git a/xinference/conftest.py b/xinference/conftest.py
index 8ec9199b52..675253e5d1 100644
--- a/xinference/conftest.py
+++ b/xinference/conftest.py
@@ -304,10 +304,3 @@ def setup_with_auth():
             os.remove(auth_file)
         except:
             pass
-
-
-@pytest.fixture
-def set_use_xllamacpp():
-    os.environ["USE_XLLAMACPP"] = "1"
-    yield
-    del os.environ["USE_XLLAMACPP"]
diff --git a/xinference/core/supervisor.py b/xinference/core/supervisor.py
index 4807cc4d92..fbb112c832 100644
--- a/xinference/core/supervisor.py
+++ b/xinference/core/supervisor.py
@@ -927,6 +927,7 @@ async def launch_builtin_model(
         request_limits: Optional[int] = None,
         wait_ready: bool = True,
         model_version: Optional[str] = None,
+        multimodal_projector: Optional[str] = None,
         peft_model_config: Optional[PeftModelConfig] = None,
         worker_ip: Optional[str] = None,
         gpu_idx: Optional[Union[int, List[int]]] = None,
@@ -955,6 +956,7 @@ async def launch_builtin_model(
                 request_limits=request_limits,
                 wait_ready=wait_ready,
                 model_version=model_version,
+                multimodal_projector=multimodal_projector,
                 peft_model_config=peft_model_config,
                 worker_ip=worker_ip,
                 gpu_idx=gpu_idx,
@@ -1095,6 +1097,7 @@ async def _launch_one_model(worker_ref, _replica_model_uid, rank: int):
                 model_type=model_type,
                 n_gpu=n_gpu,
                 request_limits=request_limits,
+                multimodal_projector=multimodal_projector,
                 peft_model_config=peft_model_config,
                 gpu_idx=replica_gpu_idx,
                 download_hub=download_hub,
@@ -1218,6 +1221,7 @@ async def _launch_builtin_sharded_model(
         request_limits: Optional[int] = None,
         wait_ready: bool = True,
         model_version: Optional[str] = None,
+        multimodal_projector: Optional[str] = None,
         peft_model_config: Optional[PeftModelConfig] = None,
         worker_ip: Optional[str] = None,
         gpu_idx: Optional[Union[int, List[int]]] = None,
@@ -1283,6 +1287,7 @@ async def _launch_model():
                             model_size_in_billions=model_size_in_billions,
                             model_format=model_format,
                             quantization=quantization,
+                            multimodal_projector=multimodal_projector,
                             model_engine=model_engine,
                             model_type=model_type,
                             n_gpu=n_gpu,
diff --git a/xinference/core/tests/test_restful_api.py b/xinference/core/tests/test_restful_api.py
index 4669716acd..c940bd39e4 100644
--- a/xinference/core/tests/test_restful_api.py
+++ b/xinference/core/tests/test_restful_api.py
@@ -317,11 +317,6 @@ async def test_restful_api(setup):
     assert custom_model_reg is None
 
 
-@pytest.mark.asyncio
-async def test_restful_api_xllamacpp(set_use_xllamacpp, setup):
-    await test_restful_api(setup)
-
-
 def test_restful_api_for_embedding(setup):
     model_name = "gte-base"
     model_spec = BUILTIN_EMBEDDING_MODELS[model_name]
@@ -1116,6 +1111,9 @@ async def test_openai(setup):
         "model_name": "qwen1.5-chat",
         "model_size_in_billions": "0_5",
         "quantization": "q4_0",
+        "n_ctx": 128,
+        "n_parallel": 1,
+        "use_mmap": True,
     }
 
     response = requests.post(url, json=payload)
@@ -1244,6 +1242,9 @@ def test_launch_model_async(setup):
         "model_name": "qwen1.5-chat",
         "model_size_in_billions": "0_5",
         "quantization": "q4_0",
+        "n_ctx": 128,
+        "n_parallel": 1,
+        "use_mmap": True,
     }
 
     response = requests.post(url, json=payload)
@@ -1283,6 +1284,9 @@ def test_cancel_launch_model(setup):
         "model_name": "qwen2.5-instruct",
         "model_size_in_billions": "0_5",
         "quantization": "q4_0",
+        "n_ctx": 128,
+        "n_parallel": 1,
+        "use_mmap": True,
     }
 
     response = requests.post(url, json=payload)
@@ -1319,6 +1323,9 @@ def test_events(setup):
         "model_name": "qwen1.5-chat",
         "model_size_in_billions": "0_5",
         "quantization": "q4_0",
+        "n_ctx": 128,
+        "n_parallel": 1,
+        "use_mmap": True,
     }
 
     response = requests.post(url, json=payload)
@@ -1335,7 +1342,8 @@ def test_events(setup):
 
     # delete again
     url = f"{endpoint}/v1/models/test_qwen_15"
-    requests.delete(url)
+    response = requests.delete(url)
+    response.raise_for_status()
 
     response = requests.get(events_url)
     response_data = response.json()
diff --git a/xinference/core/worker.py b/xinference/core/worker.py
index 0aacd1dbe7..57395a7d21 100644
--- a/xinference/core/worker.py
+++ b/xinference/core/worker.py
@@ -914,6 +914,7 @@ async def launch_builtin_model(
         n_worker: Optional[int] = 1,
         shard: Optional[int] = 0,
         driver_info: Optional[dict] = None,
+        multimodal_projector: Optional[str] = None,
         peft_model_config: Optional[PeftModelConfig] = None,
         request_limits: Optional[int] = None,
         gpu_idx: Optional[Union[int, List[int]]] = None,
@@ -1060,6 +1061,7 @@ async def launch_builtin_model(
                             model_format,
                             model_size_in_billions,
                             quantization,
+                            multimodal_projector,
                             peft_model_config,
                             download_hub,
                             model_path,
diff --git a/xinference/deploy/docker/Dockerfile b/xinference/deploy/docker/Dockerfile
index fe99067473..5b3cefb388 100644
--- a/xinference/deploy/docker/Dockerfile
+++ b/xinference/deploy/docker/Dockerfile
@@ -40,7 +40,7 @@ RUN pip install --upgrade -i "$PIP_INDEX" pip setuptools wheel&& \
     git restore . && \
     pip install -i "$PIP_INDEX" --no-deps "." && \
     pip uninstall xllamacpp -y && \
-    pip install xllamacpp --index-url https://xorbitsai.github.io/xllamacpp/whl/cu124 && \
+    pip install "xllamacpp>=0.1.16" --index-url https://xorbitsai.github.io/xllamacpp/whl/cu124 && \
     # clean packages
     pip cache purge
 
diff --git a/xinference/deploy/docker/cpu.Dockerfile b/xinference/deploy/docker/cpu.Dockerfile
index a74c168796..c3743b8663 100644
--- a/xinference/deploy/docker/cpu.Dockerfile
+++ b/xinference/deploy/docker/cpu.Dockerfile
@@ -27,7 +27,7 @@ RUN python -m pip install --upgrade -i "$PIP_INDEX" pip && \
     python setup.py build_web && \
     git restore . && \
     pip install -i "$PIP_INDEX" --no-deps "." && \
-    pip install -i "$PIP_INDEX" xllamacpp && \
+    pip install -i "$PIP_INDEX" "xllamacpp>=0.1.16" && \
     # clean packages
     pip cache purge
 
diff --git a/xinference/model/core.py b/xinference/model/core.py
index d381788d39..588cddd1aa 100644
--- a/xinference/model/core.py
+++ b/xinference/model/core.py
@@ -59,6 +59,7 @@ def create_model_instance(
     model_format: Optional[str] = None,
     model_size_in_billions: Optional[Union[int, str]] = None,
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
     peft_model_config: Optional[PeftModelConfig] = None,
     download_hub: Optional[
         Literal["huggingface", "modelscope", "openmind_hub", "csghub"]
@@ -84,6 +85,7 @@ def create_model_instance(
             model_format,
             model_size_in_billions,
             quantization,
+            multimodal_projector,
             peft_model_config,
             download_hub,
             model_path,
diff --git a/xinference/model/llm/__init__.py b/xinference/model/llm/__init__.py
index e046e52a1b..141efbd11b 100644
--- a/xinference/model/llm/__init__.py
+++ b/xinference/model/llm/__init__.py
@@ -73,7 +73,7 @@ def generate_engine_config_by_model_family(model_family):
         model_size_in_billions = spec.model_size_in_billions
         quantizations = spec.quantizations
         for quantization in quantizations:
-            # traverse all supported engines to match the name, format, size in billions and quatization of model
+            # traverse all supported engines to match the name, format, size in billions and quantization of model
             for engine in SUPPORTED_ENGINES:
                 if not check_format_with_engine(
                     model_format, engine
@@ -107,6 +107,10 @@ def generate_engine_config_by_model_family(model_family):
                                     "llm_class": cls,
                                 }
                             )
+                            if hasattr(spec, "multimodal_projectors"):
+                                engine_params[-1][
+                                    "multimodal_projectors"
+                                ] = spec.multimodal_projectors
                         engines[engine] = engine_params
                         break
     LLM_ENGINES[model_name] = engines
diff --git a/xinference/model/llm/core.py b/xinference/model/llm/core.py
index 0d70a535b3..4dd311b7c0 100644
--- a/xinference/model/llm/core.py
+++ b/xinference/model/llm/core.py
@@ -160,12 +160,14 @@ def __init__(
         llm_family: "LLMFamilyV1",
         llm_spec: "LLMSpecV1",
         quantization: Optional[str],
+        multimodal_projector: Optional[str] = None,
         model_path: Optional[str] = None,
     ):
         super().__init__(address, devices, model_path=model_path)
         self._llm_family = llm_family
         self._llm_spec = llm_spec
         self._quantization = quantization
+        self._multimodal_projector = multimodal_projector
 
     @property
     def spec(self):
@@ -185,6 +187,7 @@ def to_dict(self):
             "model_family": self._llm_family.model_family
             or self._llm_family.model_name,
             "quantization": self._quantization,
+            "multimodal_projector": self._multimodal_projector,
             "model_hub": self._llm_spec.model_hub,
             "revision": self._llm_spec.model_revision,
             "context_length": self._llm_family.context_length,
@@ -204,6 +207,7 @@ def to_version_info(self):
             "model_file_location": model_file_location,
             "cache_status": cache_status,
             "quantization": self._quantization,
+            "multimodal_projector": self._multimodal_projector,
             "model_format": self._llm_spec.model_format,
             "model_size_in_billions": self._llm_spec.model_size_in_billions,
         }
@@ -212,10 +216,19 @@ def to_version_info(self):
 def generate_llm_description(llm_family: "LLMFamilyV1") -> Dict[str, List[Dict]]:
     res = defaultdict(list)
     for spec in llm_family.model_specs:
+        multimodal_projectors = getattr(spec, "multimodal_projectors", None)
         for q in spec.quantizations:
-            res[llm_family.model_name].append(
-                LLMDescription(None, None, llm_family, spec, q).to_version_info()
-            )
+            if multimodal_projectors:
+                for mmproj in multimodal_projectors:
+                    res[llm_family.model_name].append(
+                        LLMDescription(
+                            None, None, llm_family, spec, q, mmproj
+                        ).to_version_info()
+                    )
+            else:
+                res[llm_family.model_name].append(
+                    LLMDescription(None, None, llm_family, spec, q).to_version_info()
+                )
     return res
 
 
@@ -228,6 +241,7 @@ def create_llm_model_instance(
     model_format: Optional[str] = None,
     model_size_in_billions: Optional[Union[int, str]] = None,
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
     peft_model_config: Optional[PeftModelConfig] = None,
     download_hub: Optional[
         Literal["huggingface", "modelscope", "openmind_hub", "csghub"]
@@ -261,7 +275,7 @@ def create_llm_model_instance(
     logger.debug(f"Launching {model_uid} with {llm_cls.__name__}")
 
     if not model_path:
-        model_path = cache(llm_family, llm_spec, quantization)
+        model_path = cache(llm_family, llm_spec, quantization, multimodal_projector)
 
     peft_model = peft_model_config.peft_model if peft_model_config else None
     if peft_model is not None:
@@ -284,9 +298,10 @@ def create_llm_model_instance(
                 model_uid, llm_family, llm_spec, quantization, model_path, kwargs
             )
     else:
+        kwargs["multimodal_projector"] = multimodal_projector
         model = llm_cls(
             model_uid, llm_family, llm_spec, quantization, model_path, kwargs
         )
     return model, LLMDescription(
-        subpool_addr, devices, llm_family, llm_spec, quantization
+        subpool_addr, devices, llm_family, llm_spec, quantization, multimodal_projector
     )
diff --git a/xinference/model/llm/llama_cpp/core.py b/xinference/model/llm/llama_cpp/core.py
index b5a9fc14ac..2694228759 100644
--- a/xinference/model/llm/llama_cpp/core.py
+++ b/xinference/model/llm/llama_cpp/core.py
@@ -135,6 +135,15 @@ def load(self):
                 if os.path.exists(legacy_model_file_path):
                     model_path = legacy_model_file_path
 
+        multimodal_projector = self._llamacpp_model_config.get(
+            "multimodal_projector", ""
+        )
+        mmproj = (
+            os.path.join(self.model_path, multimodal_projector)
+            if multimodal_projector
+            else ""
+        )
+
         try:
             params = CommonParams()
             # Compatible with xllamacpp changes
@@ -142,6 +151,7 @@ def load(self):
                 params.model = model_path
             except Exception:
                 params.model.path = model_path
+            params.mmproj.path = mmproj
             if self.model_family.chat_template:
                 params.chat_template = self.model_family.chat_template
             # This is the default value, could be overwritten by _llamacpp_model_config
@@ -207,11 +217,13 @@ def _ok_callback(ok):
                     q.put(res)
                 except Exception as e:
                     logger.exception("handle_completions callback failed: %s", e)
+                    q.put(_Error(str(e)))
 
             try:
                 self._llm.handle_completions(prompt_json, _error_callback, _ok_callback)
             except Exception as ex:
                 logger.exception("handle_completions failed: %s", ex)
+                q.put(_Error(str(ex)))
             q.put(_Done)
 
         assert self._executor
@@ -271,6 +283,7 @@ def _ok_callback(ok):
                     q.put(res)
                 except Exception as e:
                     logger.exception("handle_chat_completions callback failed: %s", e)
+                    q.put(_Error(str(e)))
 
             try:
                 self._llm.handle_chat_completions(
@@ -278,6 +291,7 @@ def _ok_callback(ok):
                 )
             except Exception as ex:
                 logger.exception("handle_chat_completions failed: %s", ex)
+                q.put(_Error(str(ex)))
             q.put(_Done)
 
         assert self._executor
@@ -288,7 +302,7 @@ def _ok_callback(ok):
             def _to_iterator():
                 while (r := q.get()) is not _Done:
                     if type(r) is _Error:
-                        raise Exception("Got error in chat stream: %s", r.msg)
+                        raise Exception(f"Got error in chat stream: {r.msg}")
                     # Get valid keys (O(1) lookup)
                     chunk_keys = ChatCompletionChunk.__annotations__
                     # The chunk may contain additional keys (e.g., system_fingerprint),
@@ -302,5 +316,5 @@ def _to_iterator():
         else:
             r = q.get()
             if type(r) is _Error:
-                raise Exception("Got error in chat: %s", r.msg)
+                raise Exception(f"Got error in chat: {r.msg}")
             return self._to_chat_completion(r, self.reasoning_parser)
diff --git a/xinference/model/llm/llama_cpp/tests/test_gguf.py b/xinference/model/llm/llama_cpp/tests/test_gguf.py
index cb7c7f0a8a..55c4253c2a 100644
--- a/xinference/model/llm/llama_cpp/tests/test_gguf.py
+++ b/xinference/model/llm/llama_cpp/tests/test_gguf.py
@@ -12,6 +12,10 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import base64
+
+import requests
+
 from .....client import Client
 
 
@@ -32,3 +36,58 @@ def test_gguf(setup):
     assert "id" in completion
     assert "text" in completion["choices"][0]
     assert len(completion["choices"][0]["text"]) > 0
+
+
+def test_gguf_multimodal(setup):
+    IMG_URL_0 = "https://github.com/bebechien/gemma/blob/main/surprise.png?raw=true"
+
+    response = requests.get(IMG_URL_0)
+    response.raise_for_status()  # Raise an exception for bad status codes
+    IMG_BASE64_0 = "data:image/png;base64," + base64.b64encode(response.content).decode(
+        "utf-8"
+    )
+
+    endpoint, _ = setup
+    client = Client(endpoint)
+
+    r = client.query_engine_by_model_name("gemma-3-it")
+    assert (
+        "mmproj-google_gemma-3-4b-it-f16.gguf"
+        in r["llama.cpp"][0]["multimodal_projectors"]
+    )
+
+    model_uid = client.launch_model(
+        model_name="gemma-3-it",
+        model_engine="llama.cpp",
+        model_size_in_billions=4,
+        model_format="ggufv2",
+        quantization="IQ4_XS",
+        multimodal_projector="mmproj-google_gemma-3-4b-it-f16.gguf",
+        n_ctx=512,
+        n_parallel=1,
+    )
+    assert len(client.list_models()) == 1
+    model = client.get_model(model_uid)
+
+    completion = model.chat(
+        [
+            {
+                "role": "user",
+                "content": [
+                    {"type": "text", "text": "What is this:\n"},
+                    {
+                        "type": "image_url",
+                        "image_url": {
+                            "url": IMG_BASE64_0,
+                        },
+                    },
+                ],
+            }
+        ],
+        generate_config={"max_tokens": 128},
+    )
+    content = completion["choices"][0]["message"]["content"]
+    assert "id" in completion
+    assert "black" in content
+    assert "white" in content
+    assert "cat" in content
diff --git a/xinference/model/llm/llm_family.json b/xinference/model/llm/llm_family.json
index c415a65fd1..f25c4537f9 100644
--- a/xinference/model/llm/llm_family.json
+++ b/xinference/model/llm/llm_family.json
@@ -5362,6 +5362,11 @@
           "Q8_0",
           "bf16"
         ],
+        "multimodal_projectors": [
+          "mmproj-google_gemma-3-4b-it-f16.gguf",
+          "mmproj-google_gemma-3-4b-it-f32.gguf",
+          "mmproj-google_gemma-3-4b-it-bf16.gguf"
+        ],
         "model_id": "bartowski/google_gemma-3-4b-it-GGUF",
         "model_file_name_template": "google_gemma-3-4b-it-{quantization}.gguf"
       },
@@ -5393,6 +5398,11 @@
           "Q8_0",
           "bf16"
         ],
+        "multimodal_projectors": [
+          "mmproj-google_gemma-3-12b-it-f16.gguf",
+          "mmproj-google_gemma-3-12b-it-f32.gguf",
+          "mmproj-google_gemma-3-12b-it-bf16.gguf"
+        ],
         "model_id": "bartowski/google_gemma-3-12b-it-GGUF",
         "model_file_name_template": "google_gemma-3-12b-it-{quantization}.gguf"
       },
@@ -5424,6 +5434,11 @@
           "Q8_0",
           "bf16"
         ],
+        "multimodal_projectors": [
+          "mmproj-google_gemma-3-27b-it-f16.gguf",
+          "mmproj-google_gemma-3-27b-it-f32.gguf",
+          "mmproj-google_gemma-3-27b-it-bf16.gguf"
+        ],
         "model_id": "bartowski/google_gemma-3-27b-it-GGUF",
         "model_file_name_template": "google_gemma-3-27b-it-{quantization}.gguf"
       },
diff --git a/xinference/model/llm/llm_family.py b/xinference/model/llm/llm_family.py
index 5b266c3b1c..41e271ee92 100644
--- a/xinference/model/llm/llm_family.py
+++ b/xinference/model/llm/llm_family.py
@@ -65,6 +65,7 @@ class LlamaCppLLMSpecV1(BaseModel):
     # Must in order that `str` first, then `int`
     model_size_in_billions: Union[str, int]
     quantizations: List[str]
+    multimodal_projectors: Optional[List[str]]
     model_id: Optional[str]
     model_file_name_template: str
     model_file_name_split_template: Optional[str]
@@ -321,6 +322,7 @@ def cache(
     llm_family: LLMFamilyV1,
     llm_spec: "LLMSpecV1",
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ) -> str:
     legacy_cache_path = get_legacy_cache_path(
         llm_family.model_name,
@@ -338,16 +340,24 @@ def cache(
         else:
             if llm_spec.model_hub == "huggingface":
                 logger.info(f"Caching from Hugging Face: {llm_spec.model_id}")
-                return cache_from_huggingface(llm_family, llm_spec, quantization)
+                return cache_from_huggingface(
+                    llm_family, llm_spec, quantization, multimodal_projector
+                )
             elif llm_spec.model_hub == "modelscope":
                 logger.info(f"Caching from Modelscope: {llm_spec.model_id}")
-                return cache_from_modelscope(llm_family, llm_spec, quantization)
+                return cache_from_modelscope(
+                    llm_family, llm_spec, quantization, multimodal_projector
+                )
             elif llm_spec.model_hub == "openmind_hub":
                 logger.info(f"Caching from openmind_hub: {llm_spec.model_id}")
-                return cache_from_openmind_hub(llm_family, llm_spec, quantization)
+                return cache_from_openmind_hub(
+                    llm_family, llm_spec, quantization, multimodal_projector
+                )
             elif llm_spec.model_hub == "csghub":
                 logger.info(f"Caching from CSGHub: {llm_spec.model_id}")
-                return cache_from_csghub(llm_family, llm_spec, quantization)
+                return cache_from_csghub(
+                    llm_family, llm_spec, quantization, multimodal_projector
+                )
             else:
                 raise ValueError(f"Unknown model hub: {llm_spec.model_hub}")
 
@@ -543,13 +553,34 @@ def _get_meta_path(
     model_format: str,
     model_hub: str,
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ):
     if model_format == "pytorch":
         if model_hub == "huggingface":
             return os.path.join(cache_dir, "__valid_download")
         else:
             return os.path.join(cache_dir, f"__valid_download_{model_hub}")
-    elif model_format in ["ggufv2", "gptq", "awq", "fp8", "mlx"]:
+    elif model_format == "ggufv2":
+        assert quantization is not None
+        if multimodal_projector is None:
+            # Compatible with old cache file to avoid re-download model.
+            if model_hub == "huggingface":
+                return os.path.join(cache_dir, f"__valid_download_{quantization}")
+            else:
+                return os.path.join(
+                    cache_dir, f"__valid_download_{model_hub}_{quantization}"
+                )
+        else:
+            if model_hub == "huggingface":
+                return os.path.join(
+                    cache_dir, f"__valid_download_{quantization}_{multimodal_projector}"
+                )
+            else:
+                return os.path.join(
+                    cache_dir,
+                    f"__valid_download_{model_hub}_{quantization}_{multimodal_projector}",
+                )
+    elif model_format in ["gptq", "awq", "fp8", "mlx"]:
         assert quantization is not None
         if model_hub == "huggingface":
             return os.path.join(cache_dir, f"__valid_download_{quantization}")
@@ -567,6 +598,7 @@ def _skip_download(
     model_hub: str,
     model_revision: Optional[str],
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ) -> bool:
     if model_format in ["pytorch", "mindspore"]:
         model_hub_to_meta_path = {
@@ -591,7 +623,14 @@ def _skip_download(
                     logger.warning(f"Cache {cache_dir} exists, but it was from {hub}")
                     return True
             return False
-    elif model_format in ["ggufv2", "gptq", "awq", "fp8", "mlx"]:
+    elif model_format == "ggufv2":
+        assert quantization is not None
+        return os.path.exists(
+            _get_meta_path(
+                cache_dir, model_format, model_hub, quantization, multimodal_projector
+            )
+        )
+    elif model_format in ["gptq", "awq", "fp8", "mlx"]:
         assert quantization is not None
         return os.path.exists(
             _get_meta_path(cache_dir, model_format, model_hub, quantization)
@@ -605,6 +644,7 @@ def _generate_meta_file(
     llm_family: "LLMFamilyV1",
     llm_spec: "LLMSpecV1",
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ):
     assert not valid_model_revision(
         meta_path, llm_spec.model_revision
@@ -614,12 +654,16 @@ def _generate_meta_file(
 
         from .core import LLMDescription
 
-        desc = LLMDescription(None, None, llm_family, llm_spec, quantization)
+        desc = LLMDescription(
+            None, None, llm_family, llm_spec, quantization, multimodal_projector
+        )
         json.dump(desc.to_dict(), f)
 
 
 def _generate_model_file_names(
-    llm_spec: "LLMSpecV1", quantization: Optional[str] = None
+    llm_spec: "LLMSpecV1",
+    quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ) -> Tuple[List[str], str, bool]:
     file_names = []
     final_file_name = llm_spec.model_file_name_template.format(
@@ -650,6 +694,8 @@ def _generate_model_file_names(
                 quantization=quantization, part=part
             )
             file_names.append(file_name)
+    if multimodal_projector:
+        file_names.append(multimodal_projector)
 
     return file_names, final_file_name, need_merge
 
@@ -671,6 +717,7 @@ def cache_from_csghub(
     llm_family: LLMFamilyV1,
     llm_spec: "LLMSpecV1",
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ) -> str:
     """
     Cache model from CSGHub. Return the cache directory.
@@ -686,6 +733,7 @@ def cache_from_csghub(
         llm_spec.model_hub,
         llm_spec.model_revision,
         quantization,
+        multimodal_projector,
     ):
         return cache_dir
 
@@ -705,7 +753,7 @@ def cache_from_csghub(
 
     elif llm_spec.model_format in ["ggufv2"]:
         file_names, final_file_name, need_merge = _generate_model_file_names(
-            llm_spec, quantization
+            llm_spec, quantization, multimodal_projector
         )
 
         for filename in file_names:
@@ -729,9 +777,15 @@ def cache_from_csghub(
         raise ValueError(f"Unsupported format: {llm_spec.model_format}")
 
     meta_path = _get_meta_path(
-        cache_dir, llm_spec.model_format, llm_spec.model_hub, quantization
+        cache_dir,
+        llm_spec.model_format,
+        llm_spec.model_hub,
+        quantization,
+        multimodal_projector,
+    )
+    _generate_meta_file(
+        meta_path, llm_family, llm_spec, quantization, multimodal_projector
     )
-    _generate_meta_file(meta_path, llm_family, llm_spec, quantization)
 
     return cache_dir
 
@@ -740,6 +794,7 @@ def cache_from_modelscope(
     llm_family: LLMFamilyV1,
     llm_spec: "LLMSpecV1",
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ) -> str:
     """
     Cache model from Modelscope. Return the cache directory.
@@ -754,6 +809,7 @@ def cache_from_modelscope(
         llm_spec.model_hub,
         llm_spec.model_revision,
         quantization,
+        multimodal_projector,
     ):
         return cache_dir
 
@@ -772,7 +828,7 @@ def cache_from_modelscope(
 
     elif llm_spec.model_format in ["ggufv2"]:
         file_names, final_file_name, need_merge = _generate_model_file_names(
-            llm_spec, quantization
+            llm_spec, quantization, multimodal_projector
         )
 
         for filename in file_names:
@@ -795,7 +851,11 @@ def cache_from_modelscope(
         raise ValueError(f"Unsupported format: {llm_spec.model_format}")
 
     meta_path = _get_meta_path(
-        cache_dir, llm_spec.model_format, llm_spec.model_hub, quantization
+        cache_dir,
+        llm_spec.model_format,
+        llm_spec.model_hub,
+        quantization,
+        multimodal_projector,
     )
     _generate_meta_file(meta_path, llm_family, llm_spec, quantization)
 
@@ -806,6 +866,7 @@ def cache_from_openmind_hub(
     llm_family: LLMFamilyV1,
     llm_spec: "LLMSpecV1",
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ) -> str:
     """
     Cache model from openmind_hub. Return the cache directory.
@@ -819,6 +880,7 @@ def cache_from_openmind_hub(
         llm_spec.model_hub,
         llm_spec.model_revision,
         quantization,
+        multimodal_projector,
     ):
         return cache_dir
 
@@ -839,7 +901,11 @@ def cache_from_openmind_hub(
         raise ValueError(f"Unsupported format: {llm_spec.model_format}")
 
     meta_path = _get_meta_path(
-        cache_dir, llm_spec.model_format, llm_spec.model_hub, quantization
+        cache_dir,
+        llm_spec.model_format,
+        llm_spec.model_hub,
+        quantization,
+        multimodal_projector,
     )
     _generate_meta_file(meta_path, llm_family, llm_spec, quantization)
 
@@ -850,6 +916,7 @@ def cache_from_huggingface(
     llm_family: LLMFamilyV1,
     llm_spec: "LLMSpecV1",
     quantization: Optional[str] = None,
+    multimodal_projector: Optional[str] = None,
 ) -> str:
     """
     Cache model from Hugging Face. Return the cache directory.
@@ -863,6 +930,7 @@ def cache_from_huggingface(
         llm_spec.model_hub,
         llm_spec.model_revision,
         quantization,
+        multimodal_projector,
     ):
         return cache_dir
 
@@ -889,7 +957,7 @@ def cache_from_huggingface(
     elif llm_spec.model_format in ["ggufv2"]:
         assert isinstance(llm_spec, LlamaCppLLMSpecV1)
         file_names, final_file_name, need_merge = _generate_model_file_names(
-            llm_spec, quantization
+            llm_spec, quantization, multimodal_projector
         )
 
         for file_name in file_names:
@@ -914,7 +982,11 @@ def cache_from_huggingface(
         raise ValueError(f"Unsupported model format: {llm_spec.model_format}")
 
     meta_path = _get_meta_path(
-        cache_dir, llm_spec.model_format, llm_spec.model_hub, quantization
+        cache_dir,
+        llm_spec.model_format,
+        llm_spec.model_hub,
+        quantization,
+        multimodal_projector,
     )
     _generate_meta_file(meta_path, llm_family, llm_spec, quantization)
 
diff --git a/xinference/model/llm/llm_family_modelscope.json b/xinference/model/llm/llm_family_modelscope.json
index d69017a19b..8f5f138087 100644
--- a/xinference/model/llm/llm_family_modelscope.json
+++ b/xinference/model/llm/llm_family_modelscope.json
@@ -3330,6 +3330,11 @@
           "Q8_0",
           "bf16"
         ],
+        "multimodal_projectors": [
+          "mmproj-google_gemma-3-4b-it-f16.gguf",
+          "mmproj-google_gemma-3-4b-it-f32.gguf",
+          "mmproj-google_gemma-3-4b-it-bf16.gguf"
+        ],
         "model_id": "bartowski/google_gemma-3-4b-it-GGUF",
         "model_file_name_template": "google_gemma-3-4b-it-{quantization}.gguf",
         "model_hub": "modelscope"
@@ -3353,6 +3358,11 @@
           "Q8_0",
           "bf16"
         ],
+        "multimodal_projectors": [
+          "mmproj-google_gemma-3-12b-it-f16.gguf",
+          "mmproj-google_gemma-3-12b-it-f32.gguf",
+          "mmproj-google_gemma-3-12b-it-bf16.gguf"
+        ],
         "model_id": "bartowski/google_gemma-3-12b-it-GGUF",
         "model_file_name_template": "google_gemma-3-12b-it-{quantization}.gguf",
         "model_hub": "modelscope"
@@ -3376,6 +3386,11 @@
           "Q8_0",
           "bf16"
         ],
+        "multimodal_projectors": [
+          "mmproj-google_gemma-3-27b-it-f16.gguf",
+          "mmproj-google_gemma-3-27b-it-f32.gguf",
+          "mmproj-google_gemma-3-27b-it-bf16.gguf"
+        ],
         "model_id": "bartowski/google_gemma-3-27b-it-GGUF",
         "model_file_name_template": "google_gemma-3-27b-it-{quantization}.gguf",
         "model_hub": "modelscope"
diff --git a/xinference/model/llm/tests/test_llm_family.py b/xinference/model/llm/tests/test_llm_family.py
index f62645a3e6..41c794edce 100644
--- a/xinference/model/llm/tests/test_llm_family.py
+++ b/xinference/model/llm/tests/test_llm_family.py
@@ -136,7 +136,7 @@ def test_serialize_llm_family_v1():
         stop=["hello", "world"],
     )
 
-    expected = """{"version": 1, "context_length": 2048, "model_name": "TestModel", "model_lang": ["en"], "model_ability": ["embed", "generate"], "model_description": null, "model_family": null, "model_specs": [{"model_format": "ggufv2", "model_hub": "huggingface", "model_size_in_billions": 2, "activated_size_in_billions": null, "quantizations": ["q4_0", "q4_1"], "quantization_parts": {"q4_2": ["a", "b"]}, "model_id": "example/TestModel", "model_revision": "123", "model_file_name_template": "TestModel.{quantization}.bin", "model_file_name_split_template": "TestModel.{quantization}.bin.{part}", "model_uri": null}, {"model_format": "pytorch", "model_hub": "huggingface", "model_size_in_billions": 3, "activated_size_in_billions": null, "quantizations": ["int8", "int4", "none"], "model_id": "example/TestModel", "model_revision": "456", "model_uri": null}], "chat_template": "xyz", "stop_token_ids": [1, 2, 3], "stop": ["hello", "world"], "reasoning_start_tag":null, "reasoning_end_tag":null, "virtualenv":null}"""
+    expected = """{"version": 1, "context_length": 2048, "model_name": "TestModel", "model_lang": ["en"], "model_ability": ["embed", "generate"], "model_description": null, "model_family": null, "model_specs": [{"model_format": "ggufv2", "model_hub": "huggingface", "model_size_in_billions": 2, "activated_size_in_billions": null, "quantizations": ["q4_0", "q4_1"], "quantization_parts": {"q4_2": ["a", "b"]}, "model_id": "example/TestModel", "model_revision": "123", "model_file_name_template": "TestModel.{quantization}.bin", "model_file_name_split_template": "TestModel.{quantization}.bin.{part}", "model_uri": null, "multimodal_projectors": null}, {"model_format": "pytorch", "model_hub": "huggingface", "model_size_in_billions": 3, "activated_size_in_billions": null, "quantizations": ["int8", "int4", "none"], "model_id": "example/TestModel", "model_revision": "456", "model_uri": null}], "chat_template": "xyz", "stop_token_ids": [1, 2, 3], "stop": ["hello", "world"], "reasoning_start_tag":null, "reasoning_end_tag":null, "virtualenv":null}"""
     assert json.loads(llm_family.json()) == json.loads(expected)
 
     llm_family_context_length = LLMFamilyV1(
diff --git a/xinference/web/ui/src/locales/en.json b/xinference/web/ui/src/locales/en.json
index 00bb476862..cbdd9853d2 100644
--- a/xinference/web/ui/src/locales/en.json
+++ b/xinference/web/ui/src/locales/en.json
@@ -53,6 +53,7 @@
     "modelFormat": "Model Format",
     "modelSize": "Model Size",
     "quantization": "Quantization",
+    "multimodelProjector": "Multimodel Projector",
     "nGPU": "GPU Count per Replica",
     "nGPUPerWorker": "GPU Count per Worker",
     "nGpuLayers": "N GPU Layers",
diff --git a/xinference/web/ui/src/locales/zh.json b/xinference/web/ui/src/locales/zh.json
index 621d2e7eda..0d7f358eb2 100644
--- a/xinference/web/ui/src/locales/zh.json
+++ b/xinference/web/ui/src/locales/zh.json
@@ -53,6 +53,7 @@
     "modelFormat": "模型格式",
     "modelSize": "模型大小",
     "quantization": "量化",
+    "multimodelProjector": "多模型投影器",
     "nGPU": "GPU 数量 (每个副本)",
     "nGPUPerWorker": "每个 Worker 上的 GPU 数量",
     "nGpuLayers": "GPU 层数",
diff --git a/xinference/web/ui/src/scenes/launch_model/data/data.js b/xinference/web/ui/src/scenes/launch_model/data/data.js
index 6101e66632..324f7804e1 100644
--- a/xinference/web/ui/src/scenes/launch_model/data/data.js
+++ b/xinference/web/ui/src/scenes/launch_model/data/data.js
@@ -22,6 +22,7 @@ export const llmAllDataKey = [
   'peft_model_config',
   'quantization_config',
   'enable_thinking',
+  'multimodal_projectors',
 ]
 
 export const additionalParameterTipList = {
diff --git a/xinference/web/ui/src/scenes/launch_model/modelCard.js b/xinference/web/ui/src/scenes/launch_model/modelCard.js
index c61f04383c..7e5d51a0ce 100644
--- a/xinference/web/ui/src/scenes/launch_model/modelCard.js
+++ b/xinference/web/ui/src/scenes/launch_model/modelCard.js
@@ -101,6 +101,7 @@ const ModelCard = ({
   const [modelFormat, setModelFormat] = useState('')
   const [modelSize, setModelSize] = useState('')
   const [quantization, setQuantization] = useState('')
+  const [multimodalProjector, setMultimodalProjector] = useState('')
   const [nWorker, setNWorker] = useState(1)
   const [nGPU, setNGPU] = useState('auto')
   const [nGpu, setNGpu] = useState(gpuAvailable === 0 ? 'CPU' : 'GPU')
@@ -122,6 +123,9 @@ const ModelCard = ({
   const [formatOptions, setFormatOptions] = useState([])
   const [sizeOptions, setSizeOptions] = useState([])
   const [quantizationOptions, setQuantizationOptions] = useState([])
+  const [multimodalProjectorOptions, setMultimodalProjectorOptions] = useState(
+    []
+  )
   const [customDeleted, setCustomDeleted] = useState(false)
   const [customParametersArr, setCustomParametersArr] = useState([])
   const [quantizationParametersArr, setQuantizationParametersArr] = useState([])
@@ -232,13 +236,31 @@ const ModelCard = ({
             .flatMap((item) => item.quantizations)
         ),
       ]
+      const multimodal_projectors = [
+        ...new Set(
+          enginesObj[modelEngine]
+            .filter(
+              (item) =>
+                item.model_format === modelFormat &&
+                item.model_size_in_billions === convertModelSize(modelSize)
+            )
+            .flatMap((item) => item.multimodal_projectors || [])
+        ),
+      ]
       setQuantizationOptions(quants)
+      setMultimodalProjectorOptions(multimodal_projectors || [])
       if (!quants.includes(quantization)) {
         setQuantization('')
       }
       if (quants.length === 1) {
         setQuantization(quants[0])
       }
+      if (!multimodal_projectors.includes(multimodalProjector)) {
+        setMultimodalProjector('')
+      }
+      if (multimodal_projectors.length > 0 && !multimodalProjector) {
+        setMultimodalProjector(multimodal_projectors[0])
+      }
     }
   }, [modelEngine, modelFormat, modelSize])
 
@@ -352,6 +374,8 @@ const ModelCard = ({
       model_path: modelPath?.trim() === '' ? null : modelPath?.trim(),
     }
 
+    if (multimodalProjector)
+      modelDataWithID_LLM.multimodal_projector = multimodalProjector
     if (nGPULayers >= 0) modelDataWithID_LLM.n_gpu_layers = nGPULayers
     if (modelData.model_ability?.includes('hybrid'))
       modelDataWithID_LLM.enable_thinking = enableThinking
@@ -647,6 +671,7 @@ const ModelCard = ({
       model_format,
       model_size_in_billions,
       quantization,
+      multimodal_projector,
       n_worker,
       n_gpu,
       n_gpu_layers,
@@ -671,6 +696,7 @@ const ModelCard = ({
     setModelFormat(model_format || '')
     setModelSize(String(model_size_in_billions) || '')
     setQuantization(quantization || '')
+    setMultimodalProjector(multimodal_projector || '')
     setNWorker(Number(n_worker) || 1)
     setNGPU(n_gpu || 'auto')
     if (n_gpu_layers >= 0) {
@@ -842,6 +868,7 @@ const ModelCard = ({
       setModelFormat('')
       setModelSize('')
       setQuantization('')
+      setMultimodalProjector('')
       setNWorker(1)
       setNGPU('auto')
       setReplica(1)
@@ -1551,6 +1578,56 @@ const ModelCard = ({
                     </Select>
                   </FormControl>
                 </Grid>
+                {multimodalProjectorOptions.length > 0 && (
+                  <Grid item xs={12}>
+                    <FormControl
+                      variant="outlined"
+                      margin="normal"
+                      fullWidth
+                      disabled={!modelFormat || !modelSize}
+                    >
+                      <InputLabel id="multimodelProjector-label">
+                        {t('launchModel.multimodelProjector')}
+                      </InputLabel>
+                      <Select
+                        className="textHighlight"
+                        labelId="multimodelProjector-label"
+                        value={multimodalProjector}
+                        onChange={(e) => setMultimodalProjector(e.target.value)}
+                        label={t('launchModel.multimodelProjector')}
+                      >
+                        {multimodalProjectorOptions.map((projector) => {
+                          const specs = modelData.model_specs
+                            .filter((spec) => spec.model_format === modelFormat)
+                            .filter(
+                              (spec) =>
+                                spec.model_size_in_billions ===
+                                convertModelSize(modelSize)
+                            )
+
+                          const spec = specs.find((s) => {
+                            return s.multimodal_projectors.includes(projector)
+                          })
+                          const cached = Array.isArray(spec?.cache_status)
+                            ? spec?.cache_status[
+                                spec?.multimodal_projectors.indexOf(projector)
+                              ]
+                            : spec?.cache_status
+
+                          const displayedProjector = cached
+                            ? projector + ' ' + t('launchModel.cached')
+                            : projector
+
+                          return (
+                            <MenuItem key={projector} value={projector}>
+                              {displayedProjector}
+                            </MenuItem>
+                          )
+                        })}
+                      </Select>
+                    </FormControl>
+                  </Grid>
+                )}
                 <Grid item xs={12}>
                   <FormControl
                     variant="outlined"


Code review comments:
diff_hunk: @@ -1014,6 +1015,7 @@ async def launch_model(
             "replica",
             "n_gpu",
             "request_limits",
+            "multimodal_projector",
path: xinference/api/restful_api.py
text: @user1:
This param is too specific, can we just put it into the kwargs? and let the llama.cpp engine handle it.

@author:
Fixed. Thanks.