Repository: xorbitsai/inference
Pull request: FEAT: Simple OAuth2 system
Fixes #698 

<img width="1849" alt="image" src="https://github.com/xorbitsai/inference/assets/109656400/69156198-e6d0-4094-8c1a-3b9251d4175c">

<img width="1288" alt="image" src="https://github.com/xorbitsai/inference/assets/109656400/663fd072-fb37-46a4-9131-deb0009d7603">


# Detail
1. add `--auth-config` option for `xinference-local` and `xinference-supervisor` command for startup xinference with auth system.
2. Nothing changed when using xinference without auth.
3. For web users:
Firstly, a login web page for sign in when the user tries to open the web UI. Then use the web as before. If user does not have the right permission, UI would show error.
4. For command line users, user should login first:
```
xinference login --username <name> --password <pass>
``` 
Then use the command as before.

5. For SDK users, also login first:
```
from xinference.client import Client
client = Client('<endpoint>')
client.login('<name>', '<pass>')
```



Problem statement:
Provide some authentication mechanism for UI and API
### Is your feature request related to a problem? Please describe
Currently anyone with the xinference URL can make requests and add/remove models. I need a little more control. It does not have to be necessarily very sophisticated initially, some sort of shared secret would do, although if you guys are willing to consider more full fledged authentication mechanisms that would certainly be welcome.

### Describe the solution you'd like
The simplest viable solution for me here would be a single shared secret, like a "root"/password type of credentials, with "password" being configurable

Hints from issue discussion:
Yes, it's in our roadmap, and we plan to implement it within next two to three releases. Suggestions are welcome.
I'll provide some suggestions but first I want to repeat that I could use a very simple mechanism initially like I described above.

But if we are talking more full fledged mechanism then here are some thoughts.

1. The API and the UI need to rely on separate mechanisms for authentication. My suggestion here is that the API relies on tokens that are generated by the users from the UI, like many other websites with a rest API do (e.g. github). The API should use the same protocol mechanism as openAI itself (ie an Authorization HTTP header with a Bearer string) for maximum compatibility with the openai-python SDK

2. There are several options for the UI here, but what would be most useful to me is SSO integration with either Microsoft Active Directory or LDAP (that way my users could rely on the same credentials that let them access other apps throughout the company) along with some way to assign users to roles like "read-only", "token-creator", "model-deployment" that would help me control who gets to spin models up and down, and who gets to make requests to the models.

Commit message: Add chinese doc for Oauth2

Patch to review:
diff --git a/.github/workflows/python.yaml b/.github/workflows/python.yaml
index b5f969f797..70fa563fed 100644
--- a/.github/workflows/python.yaml
+++ b/.github/workflows/python.yaml
@@ -128,6 +128,8 @@ jobs:
             ${{ env.SELF_HOST_PYTHON }} -m pip install -U modelscope
             ${{ env.SELF_HOST_PYTHON }} -m pip install -U sse_starlette
             ${{ env.SELF_HOST_PYTHON }} -m pip install -U xoscar
+            ${{ env.SELF_HOST_PYTHON }} -m pip install -U "python-jose[cryptography]"
+            ${{ env.SELF_HOST_PYTHON }} -m pip install -U "passlib[bcrypt]"
             ${{ env.SELF_HOST_PYTHON }} -m pytest --timeout=1500 \
               -W ignore::PendingDeprecationWarning \
               --cov-config=setup.cfg --cov-report=xml --cov=xinference xinference/model/image/tests/test_stable_diffusion.py
diff --git a/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/auth_system.po b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/auth_system.po
new file mode 100644
index 0000000000..9df5ac6d60
--- /dev/null
+++ b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/auth_system.po
@@ -0,0 +1,225 @@
+# SOME DESCRIPTIVE TITLE.
+# Copyright (C) 2023, Xorbits Inc.
+# This file is distributed under the same license as the Xinference package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, 2024.
+#
+#, fuzzy
+msgid ""
+msgstr ""
+"Project-Id-Version: Xinference \n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2024-01-10 11:33+0800\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language: zh_CN\n"
+"Language-Team: zh_CN <LL@li.org>\n"
+"Plural-Forms: nplurals=1; plural=0;\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=utf-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Generated-By: Babel 2.12.1\n"
+
+#: ../../source/user_guide/auth_system.rst:5
+msgid "Simple OAuth2 System (experimental)"
+msgstr "OAuth2 系统（实验性质）"
+
+#: ../../source/user_guide/auth_system.rst:7
+msgid ""
+"Xinference builds an In-memory OAuth2 authentication and authorization "
+"system using the account-password mode."
+msgstr ""
+"Xinference 使用了账号密码的模式构建了一个基于内存的 OAuth2 的身份验证和授权系统。"
+
+#: ../../source/user_guide/auth_system.rst:10
+msgid ""
+"If you don't have authentication and authorization requirements, you can "
+"use Xinference as before, without any changes."
+msgstr ""
+"如果没有身份验证和授权的要求，可以像之前一样使用 Xinference，无需任何改动。"
+
+#: ../../source/user_guide/auth_system.rst:14
+msgid "Permissions"
+msgstr "权限"
+
+#: ../../source/user_guide/auth_system.rst:15
+msgid ""
+"Currently, Xinference system internally defines some interface "
+"permissions:"
+msgstr ""
+"目前，Xinference 内部定义了以下几个接口权限："
+
+#: ../../source/user_guide/auth_system.rst:17
+msgid "``models:list``: Permission to list models and get models' information."
+msgstr "``models:list``: 获取模型列表和信息的权限。"
+
+#: ../../source/user_guide/auth_system.rst:18
+msgid "``models:read``: Permission to use models."
+msgstr "``models:read``: 使用模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:19
+msgid "``models:register``: Permission to register custom models."
+msgstr "``models:register``: 注册模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:20
+msgid "``models:unregister``: Permission to unregister custom models."
+msgstr "``models:unregister``: 取消注册模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:21
+msgid "``models:start``: Permission to launch models."
+msgstr "``models:start``: 启动模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:22
+msgid "``models:stop``: Permission to stop running models."
+msgstr "``models:stop``: 停止模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:23
+msgid "``admin``: Administrators have permissions for all interfaces."
+msgstr "``admin``: 管理员拥有所有接口的权限。"
+
+#: ../../source/user_guide/auth_system.rst:27
+msgid "Startup"
+msgstr "开始使用"
+
+#: ../../source/user_guide/auth_system.rst:28
+msgid ""
+"All authentication and authorization information needs to be specified "
+"and loaded into memory when Xinference is started. Xinference requires a "
+"JSON-formatted file with the following specific fields:"
+msgstr ""
+"在启动 Xinference 时，需要指定所有的验证和授权信息。当前，Xinference 需要一个"
+" JSON 文件，其中包含以下特定字段："
+
+#: ../../source/user_guide/auth_system.rst:59
+msgid ""
+"``auth_config``: This field is used to configure security-related "
+"information."
+msgstr ""
+"``auth_config``: 这个字段配置与安全相关的信息。"
+
+#: ../../source/user_guide/auth_system.rst:61
+msgid ""
+"``algorithm``: The algorithm used for token generation and parsing. "
+"``HS256`` or ``RS256`` is recommended."
+msgstr "``algorithm``: 用于令牌生成与解析的算法。推荐使用 `HS256`` 或者 ``RS256`` 。"
+
+#: ../../source/user_guide/auth_system.rst:63
+msgid ""
+"``secret_key``: The secret_key used for token generation and parsing. Use"
+" this command to generate: ``openssl rand -hex 32``."
+msgstr "``secret_key``: 用于令牌生成和解析的密钥。可以使用该命令生成： ``openssl rand -hex 32`` 。"
+
+#: ../../source/user_guide/auth_system.rst:65
+msgid ""
+"``token_expire_in_minutes``: Reserved field indicating the expiration "
+"time of the token. The current open-source version of Xinference does not"
+" check the expiration time of tokens."
+msgstr ""
+"``token_expire_in_minutes``: 保留字段，表示令牌失效时间。目前 Xinference 开源版本"
+"不会检查令牌过期时间。"
+
+#: ../../source/user_guide/auth_system.rst:67
+msgid ""
+"``user_config``: This field is used to configure user and permission "
+"information. Each user information is composed of these fields:"
+msgstr ""
+"``user_config``: 这个字段用来配置用户和权限信息。每个用户信息由以下字段组成："
+
+#: ../../source/user_guide/auth_system.rst:69
+msgid "``username``: string field for username."
+msgstr "``username``: 字符串，表示用户名"
+
+#: ../../source/user_guide/auth_system.rst:71
+msgid "``password``: string field for password."
+msgstr "``password``: 字符串，表示密码"
+
+#: ../../source/user_guide/auth_system.rst:73
+msgid ""
+"``permissions``: A list containing strings representing the permissions "
+"that this user has. The permissions are described as above."
+msgstr ""
+"``permissions``: 字符串列表，表示该用户拥有的权限。权限描述如上权限部分文档所述。"
+
+#: ../../source/user_guide/auth_system.rst:76
+msgid ""
+"Once you have configured such a JSON file, use the ``--auth-config`` "
+"option to enable Xinference with the authentication and authorization "
+"system. For example, for local startup:"
+msgstr ""
+"配置好这样一个 JSON 文件后，可以使用 ``--auth-config`` 选项启用具有身份验证和"
+"授权系统的 Xinference。例如，本地启动的命令如下所示："
+
+#: ../../source/user_guide/auth_system.rst:83
+msgid ""
+"For distributed startup, just specify this option when starting the "
+"supervisor:"
+msgstr ""
+"在分布式环境下，只需要在启动 supervisor 的是指定这个选项："
+
+#: ../../source/user_guide/auth_system.rst:91
+msgid "Usage"
+msgstr "使用"
+
+#: ../../source/user_guide/auth_system.rst:92
+msgid ""
+"For Xinference with the authentication and authorization system enabled, "
+"all usage remains the same, except for the addition of a login step at "
+"the beginning."
+msgstr ""
+"使用带有权限管理的 Xinference 服务与正常的版本保持一致，只是在开始阶段添加了登录步骤。"
+
+#: ../../source/user_guide/auth_system.rst:94
+msgid "Signin for command line users:"
+msgstr "使用命令行登录："
+
+#: ../../source/user_guide/auth_system.rst:101
+msgid "For python SDK users:"
+msgstr "使用 Python SDK 登录："
+
+#: ../../source/user_guide/auth_system.rst:110
+msgid ""
+"For web UI users, when opening the web UI, you will first be directed to "
+"the login page. After logging in, you can use the web UI normally."
+msgstr ""
+"对于 Web UI 的用户，在打开 Web UI 时，将首先跳转到登录页面。登录后，就可以正常使用"
+"Web UI 的功能。"
+
+#: ../../source/user_guide/auth_system.rst:114
+msgid "Http Status Code"
+msgstr "Http 状态码"
+
+#: ../../source/user_guide/auth_system.rst:115
+msgid "Add the following two HTTP status codes:"
+msgstr "添加了以下两种 HTTP 状态码："
+
+#: ../../source/user_guide/auth_system.rst:117
+msgid "``401 Unauthorized``: login information or token verifies failed."
+msgstr "``401 Unauthorized``: 登录信息或者令牌验证失效。"
+
+#: ../../source/user_guide/auth_system.rst:118
+msgid "``403 Forbidden``: No enough permissions when accessing interfaces."
+msgstr "``403 Forbidden``: 没有足够的权限访问接口。"
+
+#: ../../source/user_guide/auth_system.rst:120
+msgid ""
+"For the command line, SDK, or web UI users, there will be clear "
+"information prompts when encountering authorization and permissions "
+"issues."
+msgstr "对于命令行、SDK 或 Web UI 用户，在遇到授权和权限问题时，会有明确的信息提示。"
+
+#: ../../source/user_guide/auth_system.rst:124
+msgid "Note"
+msgstr "注意"
+
+#: ../../source/user_guide/auth_system.rst:125
+msgid ""
+"This feature is still in an experimental stage. Feel free to provide "
+"feedback on usage issues or improvement suggestions through `GitHub "
+"issues <https://github.com/xorbitsai/inference/issues>`_ or `our Slack "
+"<https://join.slack.com/t/xorbitsio/shared_invite/zt-1o3z9ucdh-"
+"RbfhbPVpx7prOVdM1CAuxg>`_."
+msgstr ""
+"该功能处于实验阶段。欢迎通过 `GitHub "
+"issues <https://github.com/xorbitsai/inference/issues>`_ 或者"
+" `Slack <https://join.slack.com/t/xorbitsio/shared_invite/zt-1o3z9ucdh-"
+"RbfhbPVpx7prOVdM1CAuxg>`_ 提供反馈和建议。"
+
diff --git a/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/client_api.po b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/client_api.po
index c4080f0eab..134def615b 100644
--- a/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/client_api.po
+++ b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/client_api.po
@@ -8,7 +8,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Xinference \n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2023-12-25 17:11+0800\n"
+"POT-Creation-Date: 2024-01-10 11:33+0800\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language: zh_CN\n"
@@ -17,7 +17,7 @@ msgstr ""
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=utf-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"Generated-By: Babel 2.11.0\n"
+"Generated-By: Babel 2.12.1\n"
 
 #: ../../source/user_guide/client_api.rst:5
 msgid "Client API"
@@ -39,8 +39,8 @@ msgid ""
 "can connect to the xinference server through this endpoint using the "
 "Client."
 msgstr ""
-"在命令日志里会打印服务地址，上述日志中为 `http://127.0.0.1:9997`。用户可以通过 "
-"Client 连接 Xinference 服务。"
+"在命令日志里会打印服务地址，上述日志中为 `http://127.0.0.1:9997`。用户可以通过 Client 连接 Xinference "
+"服务。"
 
 #: ../../source/user_guide/client_api.rst:20
 msgid ""
@@ -60,41 +60,89 @@ msgstr "列出所有内置支持的 LLM 模型："
 msgid "To initialize an LLM and chat:"
 msgstr "初始化一个大语言模型并且与之对话："
 
-#: ../../source/user_guide/client_api.rst:63
+#: ../../source/user_guide/client_api.rst:41
+#: ../../source/user_guide/client_api.rst:162
+#: ../../source/user_guide/client_api.rst:233
+msgid "Xinference Client"
+msgstr "Xinference Client"
+
+#: ../../source/user_guide/client_api.rst:66
+#: ../../source/user_guide/client_api.rst:194
+#: ../../source/user_guide/client_api.rst:257
+msgid "OpenAI Client"
+msgstr "OpenAI Client"
+
+#: ../../source/user_guide/client_api.rst:68
+msgid ""
+"Openai client request with the same function as before, excluding launch "
+"model. More details refer to: https://platform.openai.com/docs/api-"
+"reference/chat?lang=python"
+msgstr ""
+"使用 Openai 发送请求时，除了创建模型，其余的请求都保持与 Openai 的接口兼容。"
+"Openai 使用方式可以参考 https://platform.openai.com/docs/api-reference/chat?lang=python"
+
+#: ../../source/user_guide/client_api.rst:90
+msgid "OpenAI Client Tool Calls"
+msgstr "OpenAI 工具调用"
+
+#: ../../source/user_guide/client_api.rst:135
+#: ../../source/user_guide/client_api.rst:176
+#: ../../source/user_guide/client_api.rst:208
+#: ../../source/user_guide/client_api.rst:248
+#: ../../source/user_guide/client_api.rst:272
+#: ../../source/user_guide/client_api.rst:300
+msgid "Output:"
+msgstr "输出："
+
+#: ../../source/user_guide/client_api.rst:144
 msgid "Embedding"
 msgstr "Embedding"
 
-#: ../../source/user_guide/client_api.rst:65
+#: ../../source/user_guide/client_api.rst:146
 msgid "To list the available built-in embedding models:"
 msgstr "列出所有内置支持的 embedding 模型："
 
-#: ../../source/user_guide/client_api.rst:78
+#: ../../source/user_guide/client_api.rst:159
 msgid "To launch an embedding model and embed text:"
 msgstr "拉起 embedding 模型并使用文本向量化："
 
-#: ../../source/user_guide/client_api.rst:92
-#: ../../source/user_guide/client_api.rst:138
-#: ../../source/user_guide/client_api.rst:168
-msgid "Output:"
-msgstr "输出："
+#: ../../source/user_guide/client_api.rst:196
+msgid ""
+"Openai client request with the same function as before, excluding launch "
+"model. More details refer to: https://platform.openai.com/docs/api-"
+"reference/embeddings?lang=python"
+msgstr ""
+"使用 Openai 发送请求时，除了创建模型，其余的请求都保持与 Openai 的接口兼容。"
+"Openai 使用方式可以参考 https://platform.openai.com/docs/api-reference/embeddings?lang=python"
+
 
-#: ../../source/user_guide/client_api.rst:110
+#: ../../source/user_guide/client_api.rst:215
 msgid "Image"
 msgstr "图片"
 
-#: ../../source/user_guide/client_api.rst:112
+#: ../../source/user_guide/client_api.rst:217
 msgid "To list the available built-in image models:"
 msgstr "列出所有内置的文生图模型："
 
-#: ../../source/user_guide/client_api.rst:123
+#: ../../source/user_guide/client_api.rst:230
 msgid "To initiate an image model and generate an image using a text prompt:"
 msgstr "初始化一个文生图模型并通过提示词生成图片："
 
-#: ../../source/user_guide/client_api.rst:147
+#: ../../source/user_guide/client_api.rst:259
+msgid ""
+"Openai client request with the same function as before, excluding launch "
+"model. More details refer to: https://platform.openai.com/docs/api-"
+"reference/images/create?lang=python"
+msgstr ""
+"使用 Openai 发送请求时，除了创建模型，其余的请求都保持与 Openai 的接口兼容。"
+"Openai 使用方式可以参考 https://platform.openai.com/docs/api-reference/images/create?lang=python"
+
+
+#: ../../source/user_guide/client_api.rst:279
 msgid "Rerank"
 msgstr "Rerank"
 
-#: ../../source/user_guide/client_api.rst:148
+#: ../../source/user_guide/client_api.rst:280
 msgid "To launch a rerank model and compute the similarity scores:"
 msgstr "拉起 rerank 模型并计算文本相似度："
 
diff --git a/doc/source/user_guide/auth_system.rst b/doc/source/user_guide/auth_system.rst
new file mode 100644
index 0000000000..eeeed7181c
--- /dev/null
+++ b/doc/source/user_guide/auth_system.rst
@@ -0,0 +1,127 @@
+.. _user_guide_auth_system:
+
+===================================
+Simple OAuth2 System (experimental)
+===================================
+
+Xinference builds an In-memory OAuth2 authentication and authorization system using the account-password mode.
+
+.. note::
+   If you don't have authentication and authorization requirements, you can use Xinference as before, without any changes.
+
+
+Permissions
+===========
+Currently, Xinference system internally defines some interface permissions:
+
+* ``models:list``: Permission to list models and get models' information.
+* ``models:read``: Permission to use models.
+* ``models:register``: Permission to register custom models.
+* ``models:unregister``: Permission to unregister custom models.
+* ``models:start``: Permission to launch models.
+* ``models:stop``: Permission to stop running models.
+* ``admin``: Administrators have permissions for all interfaces.
+
+
+Startup
+=======
+All authentication and authorization information needs to be specified and loaded into memory when Xinference is started.
+Xinference requires a JSON-formatted file with the following specific fields:
+
+.. code-block:: json
+
+    {
+        "auth_config": {
+            "algorithm": "HS256",
+            "secret_key": "09d25e094faa6ca2556c818166b7a9563b93f7099f6f0f4caa6cf63b88e8d3e7",
+            "token_expire_in_minutes": 30
+        },
+        "user_config": [
+            {
+                "username": "user1",
+                "password": "secret1",
+                "permissions": [
+                    "admin"
+                ]
+            },
+            {
+                "username": "user2",
+                "password": "secret2",
+                "permissions": [
+                    "models:list",
+                    "models:read"
+                ]
+            }
+        ]
+    }
+
+
+* ``auth_config``: This field is used to configure security-related information.
+
+   * ``algorithm``: The algorithm used for token generation and parsing. ``HS256`` or ``RS256`` is recommended.
+
+   * ``secret_key``: The secret_key used for token generation and parsing. Use this command to generate: ``openssl rand -hex 32``.
+
+   * ``token_expire_in_minutes``: Reserved field indicating the expiration time of the token. The current open-source version of Xinference does not check the expiration time of tokens.
+
+* ``user_config``: This field is used to configure user and permission information. Each user information is composed of these fields:
+
+   * ``username``: string field for username.
+
+   * ``password``: string field for password.
+
+   * ``permissions``: A list containing strings representing the permissions that this user has. The permissions are described as above.
+
+
+Once you have configured such a JSON file, use the ``--auth-config`` option to enable Xinference with the authentication and authorization system. For example, for local startup:
+
+.. code-block:: bash
+
+   xinference-local -H 0.0.0.0 --auth-config /path/to/your_json_config_file
+
+
+For distributed startup, just specify this option when starting the supervisor:
+
+.. code-block:: bash
+
+   xinference-supervisor -H <supervisor_ip> --auth-config /path/to/your_json_config_file
+
+
+Usage
+=====
+For Xinference with the authentication and authorization system enabled, all usage remains the same, except for the addition of a login step at the beginning.
+
+Signin for command line users:
+
+.. code-block:: bash
+
+   xinference login -e <endpoint> --username <username> --password <password>
+
+
+For python SDK users:
+
+.. code-block:: python
+
+   from xinference.client import Client
+   client = Client('<endpoint>')
+   client.login('<name>', '<pass>')
+
+
+For web UI users, when opening the web UI, you will first be directed to the login page. After logging in, you can use the web UI normally.
+
+
+Http Status Code
+================
+Add the following two HTTP status codes:
+
+* ``401 Unauthorized``: login information or token verifies failed.
+* ``403 Forbidden``: No enough permissions when accessing interfaces.
+
+For the command line, SDK, or web UI users, there will be clear information prompts when encountering authorization and permissions issues.
+
+
+Note
+====
+This feature is still in an experimental stage.
+Feel free to provide feedback on usage issues or improvement suggestions through `GitHub issues <https://github.com/xorbitsai/inference/issues>`_ or
+`our Slack <https://join.slack.com/t/xorbitsio/shared_invite/zt-1o3z9ucdh-RbfhbPVpx7prOVdM1CAuxg>`_.
diff --git a/doc/source/user_guide/index.rst b/doc/source/user_guide/index.rst
index bca65cec4f..5cdf96e95e 100644
--- a/doc/source/user_guide/index.rst
+++ b/doc/source/user_guide/index.rst
@@ -11,3 +11,4 @@ User Guide
    backends
    client_api
    spec_decoding
+   auth_system
diff --git a/setup.cfg b/setup.cfg
index 893f9c334b..aa037fa46e 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -41,6 +41,8 @@ install_requires =
     modelscope>=1.10.0
     sse_starlette>=1.6.5  # ensure_bytes API break change: https://github.com/sysid/sse-starlette/issues/65
     openai>1  # For typing
+    python-jose[cryptography]
+    passlib[bcrypt]
 
 [options.packages.find]
 exclude =
diff --git a/xinference/api/oauth2/__init__.py b/xinference/api/oauth2/__init__.py
new file mode 100644
index 0000000000..37f6558d95
--- /dev/null
+++ b/xinference/api/oauth2/__init__.py
@@ -0,0 +1,13 @@
+# Copyright 2022-2023 XProbe Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
diff --git a/xinference/api/oauth2/common.py b/xinference/api/oauth2/common.py
new file mode 100644
index 0000000000..3d74b66482
--- /dev/null
+++ b/xinference/api/oauth2/common.py
@@ -0,0 +1,14 @@
+# Copyright 2022-2023 XProbe Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+XINFERENCE_OAUTH2_CONFIG = None
diff --git a/xinference/api/oauth2/core.py b/xinference/api/oauth2/core.py
new file mode 100644
index 0000000000..e1a6724de0
--- /dev/null
+++ b/xinference/api/oauth2/core.py
@@ -0,0 +1,93 @@
+# Copyright 2022-2023 XProbe Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+from typing import List, Optional, Union
+
+from fastapi import Depends, HTTPException, status
+from fastapi.security import OAuth2PasswordBearer, SecurityScopes
+from jose import JWTError, jwt
+from pydantic import BaseModel, ValidationError
+from typing_extensions import Annotated
+
+from .types import AuthStartupConfig, User
+
+logger = logging.getLogger(__name__)
+
+
+oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
+
+
+def get_db():
+    from .common import XINFERENCE_OAUTH2_CONFIG
+
+    # In a real enterprise-level environment, this should be the database
+    yield XINFERENCE_OAUTH2_CONFIG
+
+
+def get_user(db_users: List[User], username: str) -> Optional[User]:
+    for user in db_users:
+        if user.username == username:
+            return user
+    return None
+
+
+class TokenData(BaseModel):
+    username: Union[str, None] = None
+    scopes: List[str] = []
+
+
+def verify_token(
+    security_scopes: SecurityScopes,
+    token: Annotated[str, Depends(oauth2_scheme)],
+    config: Optional[AuthStartupConfig] = Depends(get_db),
+):
+    if security_scopes.scopes:
+        authenticate_value = f'Bearer scope="{security_scopes.scope_str}"'
+    else:
+        authenticate_value = "Bearer"
+    credentials_exception = HTTPException(
+        status_code=status.HTTP_401_UNAUTHORIZED,
+        detail="Could not validate credentials",
+        headers={"WWW-Authenticate": authenticate_value},
+    )
+
+    try:
+        assert config is not None
+        payload = jwt.decode(
+            token,
+            config.auth_config.secret_key,
+            algorithms=[config.auth_config.algorithm],
+            options={"verify_exp": False},  # TODO: supports token expiration
+        )
+        username: str = payload.get("sub")
+        if username is None:
+            raise credentials_exception
+        token_scopes = payload.get("scopes", [])
+        # TODO: check expire
+        token_data = TokenData(scopes=token_scopes, username=username)
+    except (JWTError, ValidationError):
+        raise credentials_exception
+    user = get_user(config.user_config, username=token_data.username)  # type: ignore
+    if user is None:
+        raise credentials_exception
+    if "admin" in token_data.scopes:
+        return user
+    for scope in security_scopes.scopes:
+        if scope not in token_data.scopes:
+            raise HTTPException(
+                status_code=status.HTTP_403_FORBIDDEN,
+                detail="Not enough permissions",
+                headers={"WWW-Authenticate": authenticate_value},
+            )
+    return user
diff --git a/xinference/api/oauth2/types.py b/xinference/api/oauth2/types.py
new file mode 100644
index 0000000000..b0a86a5314
--- /dev/null
+++ b/xinference/api/oauth2/types.py
@@ -0,0 +1,36 @@
+# Copyright 2022-2023 XProbe Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from typing import List
+
+from pydantic import BaseModel
+
+
+class LoginUserForm(BaseModel):
+    username: str
+    password: str
+
+
+class User(LoginUserForm):
+    permissions: List[str]
+
+
+class AuthConfig(BaseModel):
+    algorithm: str = "HS256"
+    secret_key: str
+    token_expire_in_minutes: int
+
+
+class AuthStartupConfig(BaseModel):
+    auth_config: AuthConfig
+    user_config: List[User]
diff --git a/xinference/api/oauth2/utils.py b/xinference/api/oauth2/utils.py
new file mode 100644
index 0000000000..9980b7722a
--- /dev/null
+++ b/xinference/api/oauth2/utils.py
@@ -0,0 +1,44 @@
+# Copyright 2022-2023 XProbe Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from datetime import datetime, timedelta
+from typing import Union
+
+from jose import jwt
+from passlib.context import CryptContext
+
+pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
+
+
+def create_access_token(
+    data: dict,
+    secret_key: str,
+    algorithm: str,
+    expires_delta: Union[timedelta, None] = None,
+):
+    to_encode = data.copy()
+    if expires_delta:
+        expire = datetime.utcnow() + expires_delta
+    else:
+        expire = datetime.utcnow() + timedelta(minutes=15)
+    to_encode.update({"exp": expire})
+    encoded_jwt = jwt.encode(to_encode, secret_key, algorithm=algorithm)
+    return encoded_jwt
+
+
+def verify_password(plain_password, hashed_password):
+    return pwd_context.verify(plain_password, hashed_password)
+
+
+def get_password_hash(password):
+    return pwd_context.hash(password)
diff --git a/xinference/api/restful_api.py b/xinference/api/restful_api.py
index dd628a08d9..3bdee59210 100644
--- a/xinference/api/restful_api.py
+++ b/xinference/api/restful_api.py
@@ -21,9 +21,11 @@
 import pprint
 import sys
 import warnings
+from datetime import timedelta
 from typing import Any, List, Optional, Union
 
 import gradio as gr
+import pydantic
 import xoscar as xo
 from fastapi import (
     APIRouter,
@@ -34,9 +36,12 @@
     Query,
     Request,
     Response,
+    Security,
     UploadFile,
+    status,
 )
 from fastapi.middleware.cors import CORSMiddleware
+from fastapi.responses import JSONResponse
 from fastapi.staticfiles import StaticFiles
 from PIL import Image
 from pydantic import BaseModel, Field
@@ -57,11 +62,14 @@
     CreateCompletion,
     ImageList,
 )
+from .oauth2.core import get_user, verify_token
+from .oauth2.types import AuthStartupConfig, LoginUserForm, User
+from .oauth2.utils import create_access_token, get_password_hash, verify_password
 
 logger = logging.getLogger(__name__)
 
 
-class JSONResponse(StarletteJSONResponse):
+class JSONResponse(StarletteJSONResponse):  # type: ignore # noqa: F811
     def render(self, content: Any) -> bytes:
         return json_dumps(content)
 
@@ -125,16 +133,48 @@ class BuildGradioInterfaceRequest(BaseModel):
     model_lang: List[str]
 
 
+def authenticate_user(db_users: List[User], username: str, password: str):
+    user = get_user(db_users, username)
+    if not user:
+        return False
+    if not verify_password(password, user.password):
+        return False
+    return user
+
+
 class RESTfulAPI:
-    def __init__(self, supervisor_address: str, host: str, port: int):
+    def __init__(
+        self,
+        supervisor_address: str,
+        host: str,
+        port: int,
+        auth_config_file: Optional[str] = None,
+    ):
         super().__init__()
         self._supervisor_address = supervisor_address
         self._host = host
         self._port = port
         self._supervisor_ref = None
+        self._auth_config: AuthStartupConfig = self.init_auth_config(auth_config_file)
         self._router = APIRouter()
         self._app = FastAPI()
 
+    @staticmethod
+    def init_auth_config(auth_config_file: Optional[str]):
+        from .oauth2 import common
+
+        if auth_config_file:
+            config: AuthStartupConfig = pydantic.parse_file_as(
+                path=auth_config_file, type_=AuthStartupConfig
+            )
+            for user in config.user_config:
+                user.password = get_password_hash(user.password)
+            common.XINFERENCE_OAUTH2_CONFIG = config  # type: ignore
+            return config
+
+    def is_authenticated(self):
+        return False if self._auth_config is None else True
+
     @staticmethod
     def handle_request_limit_error(e: Exception):
         if "Rate limit reached" in str(e):
@@ -147,6 +187,33 @@ async def _get_supervisor_ref(self) -> xo.ActorRefType[SupervisorActor]:
             )
         return self._supervisor_ref
 
+    async def login_for_access_token(self, form_data: LoginUserForm) -> JSONResponse:
+        user = authenticate_user(
+            self._auth_config.user_config, form_data.username, form_data.password
+        )
+        if not user:
+            raise HTTPException(
+                status_code=status.HTTP_401_UNAUTHORIZED,
+                detail="Incorrect username or password",
+                headers={"WWW-Authenticate": "Bearer"},
+            )
+        assert user is not None and isinstance(user, User)
+        access_token_expires = timedelta(
+            minutes=self._auth_config.auth_config.token_expire_in_minutes
+        )
+        access_token = create_access_token(
+            data={"sub": user.username, "scopes": user.permissions},
+            secret_key=self._auth_config.auth_config.secret_key,
+            algorithm=self._auth_config.auth_config.algorithm,
+            expires_delta=access_token_expires,
+        )
+        return JSONResponse(
+            content={"access_token": access_token, "token_type": "bearer"}
+        )
+
+    async def is_cluster_authenticated(self) -> JSONResponse:
+        return JSONResponse(content={"auth": self.is_authenticated()})
+
     def serve(self, logging_conf: Optional[dict] = None):
         self._app.add_middleware(
             CORSMiddleware,
@@ -155,8 +222,10 @@ def serve(self, logging_conf: Optional[dict] = None):
             allow_methods=["*"],
             allow_headers=["*"],
         )
+
+        # internal interface
         self._router.add_api_route("/status", self.get_status, methods=["GET"])
-        self._router.add_api_route("/v1/models", self.list_models, methods=["GET"])
+        # conflict with /v1/models/{model_uid} below, so register this first
         self._router.add_api_route(
             "/v1/models/prompts", self._get_builtin_prompts, methods=["GET"]
         )
@@ -166,52 +235,115 @@ def serve(self, logging_conf: Optional[dict] = None):
         self._router.add_api_route(
             "/v1/cluster/devices", self._get_devices_count, methods=["GET"]
         )
+        self._router.add_api_route("/v1/address", self.get_address, methods=["GET"])
+
+        # user interface
+        self._router.add_api_route(
+            "/v1/ui/{model_uid}",
+            self.build_gradio_interface,
+            methods=["POST"],
+            dependencies=[Security(verify_token, scopes=["models:read"])]
+            if self.is_authenticated()
+            else None,
+        )
+        self._router.add_api_route(
+            "/token", self.login_for_access_token, methods=["POST"]
+        )
+        self._router.add_api_route(
+            "/v1/cluster/auth", self.is_cluster_authenticated, methods=["GET"]
+        )
+        self._router.add_api_route(
+            "/v1/models",
+            self.list_models,
+            methods=["GET"],
+            dependencies=[Security(verify_token, scopes=["models:list"])]
+            if self.is_authenticated()
+            else None,
+        )
+
+        self._router.add_api_route(
+            "/v1/models/{model_uid}",
+            self.describe_model,
+            methods=["GET"],
+            dependencies=[Security(verify_token, scopes=["models:list"])]
+            if self.is_authenticated()
+            else None,
+        )
         self._router.add_api_route(
-            "/v1/models/{model_uid}", self.describe_model, methods=["GET"]
+            "/v1/models",
+            self.launch_model,
+            methods=["POST"],
+            dependencies=[Security(verify_token, scopes=["models:start"])]
+            if self.is_authenticated()
+            else None,
         )
-        self._router.add_api_route("/v1/models", self.launch_model, methods=["POST"])
         self._router.add_api_route(
             "/experimental/speculative_llms",
             self.launch_speculative_llm,
             methods=["POST"],
+            dependencies=[Security(verify_token, scopes=["models:start"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
-            "/v1/models/{model_uid}", self.terminate_model, methods=["DELETE"]
+            "/v1/models/{model_uid}",
+            self.terminate_model,
+            methods=["DELETE"],
+            dependencies=[Security(verify_token, scopes=["models:stop"])]
+            if self.is_authenticated()
+            else None,
         )
-        self._router.add_api_route("/v1/address", self.get_address, methods=["GET"])
         self._router.add_api_route(
             "/v1/completions",
             self.create_completion,
             methods=["POST"],
             response_model=Completion,
+            dependencies=[Security(verify_token, scopes=["models:read"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
             "/v1/embeddings",
             self.create_embedding,
             methods=["POST"],
+            dependencies=[Security(verify_token, scopes=["models:read"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
             "/v1/rerank",
             self.rerank,
             methods=["POST"],
+            dependencies=[Security(verify_token, scopes=["models:read"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
             "/v1/images/generations",
             self.create_images,
             methods=["POST"],
             response_model=ImageList,
+            dependencies=[Security(verify_token, scopes=["models:read"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
             "/v1/images/variations",
             self.create_variations,
             methods=["POST"],
             response_model=ImageList,
+            dependencies=[Security(verify_token, scopes=["models:read"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
             "/v1/chat/completions",
             self.create_chat_completion,
             methods=["POST"],
             response_model=ChatCompletion,
+            dependencies=[Security(verify_token, scopes=["models:read"])]
+            if self.is_authenticated()
+            else None,
         )
 
         # for custom models
@@ -219,25 +351,33 @@ def serve(self, logging_conf: Optional[dict] = None):
             "/v1/model_registrations/{model_type}",
             self.register_model,
             methods=["POST"],
+            dependencies=[Security(verify_token, scopes=["models:register"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
             "/v1/model_registrations/{model_type}/{model_name}",
             self.unregister_model,
             methods=["DELETE"],
+            dependencies=[Security(verify_token, scopes=["models:unregister"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
             "/v1/model_registrations/{model_type}",
             self.list_model_registrations,
             methods=["GET"],
+            dependencies=[Security(verify_token, scopes=["models:list"])]
+            if self.is_authenticated()
+            else None,
         )
         self._router.add_api_route(
             "/v1/model_registrations/{model_type}/{model_name}",
             self.get_model_registrations,
             methods=["GET"],
-        )
-
-        self._router.add_api_route(
-            "/v1/ui/{model_uid}", self.build_gradio_interface, methods=["POST"]
+            dependencies=[Security(verify_token, scopes=["models:list"])]
+            if self.is_authenticated()
+            else None,
         )
 
         self._app.include_router(self._router)
@@ -467,7 +607,7 @@ async def launch_model(self, request: Request) -> JSONResponse:
         return JSONResponse(content={"model_uid": model_uid})
 
     async def build_gradio_interface(
-        self, model_uid: str, body: BuildGradioInterfaceRequest
+        self, model_uid: str, body: BuildGradioInterfaceRequest, request: Request
     ) -> JSONResponse:
         """
         Separate build_interface with launch_model
@@ -492,6 +632,7 @@ async def build_gradio_interface(
         from ..core.chat_interface import LLMInterface
 
         try:
+            access_token = request.headers.get("Authorization")
             internal_host = "localhost" if self._host == "0.0.0.0" else self._host
             interface = LLMInterface(
                 endpoint=f"http://{internal_host}:{self._port}",
@@ -504,6 +645,7 @@ async def build_gradio_interface(
                 model_ability=body.model_ability,
                 model_description=body.model_description,
                 model_lang=body.model_lang,
+                access_token=access_token,
             ).build()
             gr.mount_gradio_app(self._app, interface, f"/{model_uid}")
         except ValueError as ve:
@@ -921,11 +1063,20 @@ async def get_model_registrations(
 
 
 def run(
-    supervisor_address: str, host: str, port: int, logging_conf: Optional[dict] = None
+    supervisor_address: str,
+    host: str,
+    port: int,
+    logging_conf: Optional[dict] = None,
+    auth_config_file: Optional[str] = None,
 ):
     logger.info(f"Starting Xinference at endpoint: http://{host}:{port}")
     try:
-        api = RESTfulAPI(supervisor_address=supervisor_address, host=host, port=port)
+        api = RESTfulAPI(
+            supervisor_address=supervisor_address,
+            host=host,
+            port=port,
+            auth_config_file=auth_config_file,
+        )
         api.serve(logging_conf=logging_conf)
     except SystemExit:
         logger.warning("Failed to create socket with port %d", port)
@@ -936,7 +1087,10 @@ def run(
             logger.info(f"Found available port: {port}")
             logger.info(f"Starting Xinference at endpoint: http://{host}:{port}")
             api = RESTfulAPI(
-                supervisor_address=supervisor_address, host=host, port=port
+                supervisor_address=supervisor_address,
+                host=host,
+                port=port,
+                auth_config_file=auth_config_file,
             )
             api.serve(logging_conf=logging_conf)
         else:
@@ -944,10 +1098,15 @@ def run(
 
 
 def run_in_subprocess(
-    supervisor_address: str, host: str, port: int, logging_conf: Optional[dict] = None
+    supervisor_address: str,
+    host: str,
+    port: int,
+    logging_conf: Optional[dict] = None,
+    auth_config_file: Optional[str] = None,
 ) -> multiprocessing.Process:
     p = multiprocessing.Process(
-        target=run, args=(supervisor_address, host, port, logging_conf)
+        target=run,
+        args=(supervisor_address, host, port, logging_conf, auth_config_file),
     )
     p.daemon = True
     p.start()
diff --git a/xinference/client/restful/restful_client.py b/xinference/client/restful/restful_client.py
index 6a8c918c50..c081ede84c 100644
--- a/xinference/client/restful/restful_client.py
+++ b/xinference/client/restful/restful_client.py
@@ -53,9 +53,10 @@ class RESTfulModelHandle:
     programmatically.
     """
 
-    def __init__(self, model_uid: str, base_url: str):
+    def __init__(self, model_uid: str, base_url: str, auth_headers: Dict):
         self._model_uid = model_uid
         self._base_url = base_url
+        self.auth_headers = auth_headers
 
 
 class RESTfulEmbeddingModelHandle(RESTfulModelHandle):
@@ -82,7 +83,7 @@ def create_embedding(self, input: Union[str, List[str]]) -> "Embedding":
         """
         url = f"{self._base_url}/v1/embeddings"
         request_body = {"model": self._model_uid, "input": input}
-        response = requests.post(url, json=request_body)
+        response = requests.post(url, json=request_body, headers=self.auth_headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to create the embeddings, detail: {_get_error_string(response)}"
@@ -135,7 +136,7 @@ def rerank(
             "max_chunks_per_doc": max_chunks_per_doc,
             "return_documents": return_documents,
         }
-        response = requests.post(url, json=request_body)
+        response = requests.post(url, json=request_body, headers=self.auth_headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to rerank documents, detail: {response.json()['detail']}"
@@ -182,7 +183,7 @@ def text_to_image(
             "response_format": response_format,
             "kwargs": json.dumps(kwargs),
         }
-        response = requests.post(url, json=request_body)
+        response = requests.post(url, json=request_body, headers=self.auth_headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to create the images, detail: {_get_error_string(response)}"
@@ -246,10 +247,7 @@ def image_to_image(
         for key, value in params.items():
             files.append((key, (None, value)))
         files.append(("image", ("image", image, "application/octet-stream")))
-        response = requests.post(
-            url,
-            files=files,
-        )
+        response = requests.post(url, files=files, headers=self.auth_headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to variants the images, detail: {_get_error_string(response)}"
@@ -302,7 +300,9 @@ def generate(
 
         stream = bool(generate_config and generate_config.get("stream"))
 
-        response = requests.post(url, json=request_body, stream=stream)
+        response = requests.post(
+            url, json=request_body, stream=stream, headers=self.auth_headers
+        )
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to generate completion, detail: {_get_error_string(response)}"
@@ -384,7 +384,9 @@ def chat(
                 request_body[key] = value
 
         stream = bool(generate_config and generate_config.get("stream"))
-        response = requests.post(url, json=request_body, stream=stream)
+        response = requests.post(
+            url, json=request_body, stream=stream, headers=self.auth_headers
+        )
 
         if response.status_code != 200:
             raise RuntimeError(
@@ -468,7 +470,9 @@ def chat(
                 request_body[key] = value
 
         stream = bool(generate_config and generate_config.get("stream"))
-        response = requests.post(url, json=request_body, stream=stream)
+        response = requests.post(
+            url, json=request_body, stream=stream, headers=self.auth_headers
+        )
 
         if response.status_code != 200:
             raise RuntimeError(
@@ -536,7 +540,9 @@ def chat(
                 request_body[key] = value
 
         stream = bool(generate_config and generate_config.get("stream"))
-        response = requests.post(url, json=request_body, stream=stream)
+        response = requests.post(
+            url, json=request_body, stream=stream, headers=self.auth_headers
+        )
 
         if response.status_code != 200:
             raise RuntimeError(
@@ -589,7 +595,9 @@ def generate(
 
         stream = bool(generate_config and generate_config.get("stream"))
 
-        response = requests.post(url, json=request_body, stream=stream)
+        response = requests.post(
+            url, json=request_body, stream=stream, headers=self.auth_headers
+        )
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to generate completion, detail: {response.json()['detail']}"
@@ -605,6 +613,47 @@ def generate(
 class Client:
     def __init__(self, base_url):
         self.base_url = base_url
+        self._headers = {}
+        self._cluster_authed = False
+        self._check_cluster_authenticated()
+
+    def _set_token(self, token: Optional[str]):
+        if not self._cluster_authed or token is None:
+            return
+        self._headers["Authorization"] = f"Bearer {token}"
+
+    def _get_token(self) -> Optional[str]:
+        return (
+            str(self._headers["Authorization"]).replace("Bearer ", "")
+            if "Authorization" in self._headers
+            else None
+        )
+
+    def _check_cluster_authenticated(self):
+        url = f"{self.base_url}/v1/cluster/auth"
+        response = requests.get(url)
+        if response.status_code != 200:
+            raise RuntimeError(
+                f"Failed to get cluster information, detail: {response.json()['detail']}"
+            )
+        response_data = response.json()
+        self._cluster_authed = bool(response_data["auth"])
+
+    def login(self, username: str, password: str):
+        if not self._cluster_authed:
+            return
+        url = f"{self.base_url}/token"
+
+        payload = {"username": username, "password": password}
+
+        response = requests.post(url, json=payload)
+        if response.status_code != 200:
+            raise RuntimeError(f"Failed to login, detail: {response.json()['detail']}")
+
+        response_data = response.json()
+        # Only bearer token for now
+        access_token = response_data["access_token"]
+        self._headers["Authorization"] = f"Bearer {access_token}"
 
     def list_models(self) -> Dict[str, Dict[str, Any]]:
         """
@@ -619,7 +668,7 @@ def list_models(self) -> Dict[str, Dict[str, Any]]:
 
         url = f"{self.base_url}/v1/models"
 
-        response = requests.get(url)
+        response = requests.get(url, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to list model, detail: {_get_error_string(response)}"
@@ -664,7 +713,7 @@ def launch_speculative_llm(
         }
 
         url = f"{self.base_url}/experimental/speculative_llms"
-        response = requests.post(url, json=payload)
+        response = requests.post(url, json=payload, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to launch model, detail: {_get_error_string(response)}"
@@ -739,7 +788,7 @@ def launch_model(
         for key, value in kwargs.items():
             payload[str(key)] = value
 
-        response = requests.post(url, json=payload)
+        response = requests.post(url, json=payload, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to launch model, detail: {_get_error_string(response)}"
@@ -766,7 +815,7 @@ def terminate_model(self, model_uid: str):
 
         url = f"{self.base_url}/v1/models/{model_uid}"
 
-        response = requests.delete(url)
+        response = requests.delete(url, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to terminate model, detail: {_get_error_string(response)}"
@@ -774,7 +823,7 @@ def terminate_model(self, model_uid: str):
 
     def _get_supervisor_internal_address(self):
         url = f"{self.base_url}/v1/address"
-        response = requests.get(url)
+        response = requests.get(url, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(f"Failed to get supervisor internal address")
         response_data = response.json()
@@ -806,7 +855,7 @@ def get_model(self, model_uid: str) -> RESTfulModelHandle:
         """
 
         url = f"{self.base_url}/v1/models/{model_uid}"
-        response = requests.get(url)
+        response = requests.get(url, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to get the model description, detail: {_get_error_string(response)}"
@@ -815,21 +864,35 @@ def get_model(self, model_uid: str) -> RESTfulModelHandle:
 
         if desc["model_type"] == "LLM":
             if desc["model_format"] == "ggmlv3" and "chatglm" in desc["model_name"]:
-                return RESTfulChatglmCppGenerateModelHandle(model_uid, self.base_url)
+                return RESTfulChatglmCppGenerateModelHandle(
+                    model_uid, self.base_url, auth_headers=self._headers
+                )
             elif "chat" in desc["model_ability"]:
-                return RESTfulChatModelHandle(model_uid, self.base_url)
+                return RESTfulChatModelHandle(
+                    model_uid, self.base_url, auth_headers=self._headers
+                )
             elif "generate" in desc["model_ability"]:
-                return RESTfulGenerateModelHandle(model_uid, self.base_url)
+                return RESTfulGenerateModelHandle(
+                    model_uid, self.base_url, auth_headers=self._headers
+                )
             else:
                 raise ValueError(f"Unrecognized model ability: {desc['model_ability']}")
         elif desc["model_type"] == "embedding":
-            return RESTfulEmbeddingModelHandle(model_uid, self.base_url)
+            return RESTfulEmbeddingModelHandle(
+                model_uid, self.base_url, auth_headers=self._headers
+            )
         elif desc["model_type"] == "image":
-            return RESTfulImageModelHandle(model_uid, self.base_url)
+            return RESTfulImageModelHandle(
+                model_uid, self.base_url, auth_headers=self._headers
+            )
         elif desc["model_type"] == "rerank":
-            return RESTfulRerankModelHandle(model_uid, self.base_url)
+            return RESTfulRerankModelHandle(
+                model_uid, self.base_url, auth_headers=self._headers
+            )
         elif desc["model_type"] == "multimodal":
-            return RESTfulMultimodalModelHandle(model_uid, self.base_url)
+            return RESTfulMultimodalModelHandle(
+                model_uid, self.base_url, auth_headers=self._headers
+            )
         else:
             raise ValueError(f"Unknown model type:{desc['model_type']}")
 
@@ -876,7 +939,7 @@ def describe_model(self, model_uid: str):
         """
 
         url = f"{self.base_url}/v1/models/{model_uid}"
-        response = requests.get(url)
+        response = requests.get(url, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to get the model description, detail: {_get_error_string(response)}"
@@ -903,7 +966,7 @@ def register_model(self, model_type: str, model: str, persist: bool):
         """
         url = f"{self.base_url}/v1/model_registrations/{model_type}"
         request_body = {"model": model, "persist": persist}
-        response = requests.post(url, json=request_body)
+        response = requests.post(url, json=request_body, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to register model, detail: {_get_error_string(response)}"
@@ -929,7 +992,7 @@ def unregister_model(self, model_type: str, model_name: str):
             Report failure to unregister the custom model. Provide details of failure through error message.
         """
         url = f"{self.base_url}/v1/model_registrations/{model_type}/{model_name}"
-        response = requests.delete(url)
+        response = requests.delete(url, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to register model, detail: {_get_error_string(response)}"
@@ -959,7 +1022,7 @@ def list_model_registrations(self, model_type: str) -> List[Dict[str, Any]]:
 
         """
         url = f"{self.base_url}/v1/model_registrations/{model_type}"
-        response = requests.get(url)
+        response = requests.get(url, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to list model registration, detail: {_get_error_string(response)}"
@@ -987,7 +1050,7 @@ def get_model_registration(
             The collection of registered models on the server.
         """
         url = f"{self.base_url}/v1/model_registrations/{model_type}/{model_name}"
-        response = requests.get(url)
+        response = requests.get(url, headers=self._headers)
         if response.status_code != 200:
             raise RuntimeError(
                 f"Failed to list model registration, detail: {_get_error_string(response)}"
diff --git a/xinference/client/tests/test_client_with_auth.py b/xinference/client/tests/test_client_with_auth.py
new file mode 100644
index 0000000000..5be0df8fbf
--- /dev/null
+++ b/xinference/client/tests/test_client_with_auth.py
@@ -0,0 +1,51 @@
+# Copyright 2022-2023 XProbe Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import pytest
+
+from ..restful.restful_client import Client as RESTfulClient
+from ..restful.restful_client import RESTfulEmbeddingModelHandle
+
+
+def test_client_auth(setup_with_auth):
+    endpoint, _ = setup_with_auth
+    client = RESTfulClient(endpoint)
+    with pytest.raises(RuntimeError):
+        client.list_models()
+
+    client.login("user2", "pass2")
+    assert len(client.list_models()) == 0
+
+    with pytest.raises(RuntimeError):
+        client.launch_model(
+            model_name="jina-embeddings-v2-small-en", model_type="embedding"
+        )
+
+    client.login("user3", "pass3")
+    model_uid = client.launch_model(
+        model_name="jina-embeddings-v2-small-en", model_type="embedding"
+    )
+    model = client.get_model(model_uid=model_uid)
+    assert isinstance(model, RESTfulEmbeddingModelHandle)
+
+    completion = model.create_embedding("write a poem.")
+    assert len(completion["data"][0]["embedding"]) == 512
+
+    with pytest.raises(RuntimeError):
+        client.terminate_model(model_uid=model_uid)
+
+    client.login("user1", "pass1")
+    assert len(client.list_models()) == 1
+    client.terminate_model(model_uid=model_uid)
+    assert len(client.list_models()) == 0
diff --git a/xinference/conftest.py b/xinference/conftest.py
index 4d41c326c4..5c0394d688 100644
--- a/xinference/conftest.py
+++ b/xinference/conftest.py
@@ -13,16 +13,19 @@
 # limitations under the License.
 
 import asyncio
+import json
 import logging
 import multiprocessing
 import os
 import signal
 import sys
+import tempfile
 from typing import Dict, Optional
 
 import pytest
 import xoscar as xo
 
+from .api.oauth2.types import AuthConfig, AuthStartupConfig, User
 from .constants import XINFERENCE_LOG_BACKUP_COUNT, XINFERENCE_LOG_MAX_BYTES
 from .core.supervisor import SupervisorActor
 from .deploy.utils import create_worker_actor_pool, get_log_file, get_timestamp_ms
@@ -233,3 +236,58 @@ def setup_with_file_logging():
 
     local_cluster_proc.terminate()
     restful_api_proc.terminate()
+
+
+@pytest.fixture
+def setup_with_auth():
+    from .api.restful_api import run_in_subprocess as run_restful_api
+    from .deploy.utils import health_check as cluster_health_check
+
+    logging.config.dictConfig(TEST_LOGGING_CONF)  # type: ignore
+
+    supervisor_addr = f"localhost:{xo.utils.get_next_port()}"
+    local_cluster_proc = run_test_cluster_in_subprocess(
+        supervisor_addr, TEST_LOGGING_CONF
+    )
+    if not cluster_health_check(supervisor_addr, max_attempts=10, sleep_interval=3):
+        raise RuntimeError("Cluster is not available after multiple attempts")
+
+    user1 = User(username="user1", password="pass1", permissions=["admin"])
+    user2 = User(username="user2", password="pass2", permissions=["models:list"])
+    user3 = User(
+        username="user3",
+        password="pass3",
+        permissions=["models:list", "models:read", "models:start"],
+    )
+    auth_config = AuthConfig(
+        algorithm="HS256",
+        secret_key="09d25e094faa6ca2556c818166b7a9563b93f7099f6f0f4caa6cf63b88e8d3e7",
+        token_expire_in_minutes=30,
+    )
+    startup_config = AuthStartupConfig(
+        auth_config=auth_config, user_config=[user1, user2, user3]
+    )
+    _, auth_file = tempfile.mkstemp()
+    with open(auth_file, "w") as fd:
+        fd.write(json.dumps(startup_config.dict()))
+
+    port = xo.utils.get_next_port()
+    restful_api_proc = run_restful_api(
+        supervisor_addr,
+        host="localhost",
+        port=port,
+        logging_conf=TEST_LOGGING_CONF,
+        auth_config_file=auth_file,
+    )
+    endpoint = f"http://localhost:{port}"
+    if not api_health_check(endpoint, max_attempts=10, sleep_interval=5):
+        raise RuntimeError("Endpoint is not available after multiple attempts")
+
+    yield f"http://localhost:{port}", supervisor_addr
+
+    local_cluster_proc.terminate()
+    restful_api_proc.terminate()
+    try:
+        os.remove(auth_file)
+    except:
+        pass
diff --git a/xinference/constants.py b/xinference/constants.py
index 5fee675083..294ea71e02 100644
--- a/xinference/constants.py
+++ b/xinference/constants.py
@@ -39,6 +39,7 @@ def get_xinference_home() -> str:
 XINFERENCE_MODEL_DIR = os.path.join(XINFERENCE_HOME, "model")
 XINFERENCE_LOG_DIR = os.path.join(XINFERENCE_HOME, "logs")
 XINFERENCE_IMAGE_DIR = os.path.join(XINFERENCE_HOME, "image")
+XINFERENCE_AUTH_DIR = os.path.join(XINFERENCE_HOME, "auth")
 
 XINFERENCE_DEFAULT_LOCAL_HOST = "127.0.0.1"
 XINFERENCE_DEFAULT_DISTRIBUTED_HOST = "0.0.0.0"
diff --git a/xinference/core/chat_interface.py b/xinference/core/chat_interface.py
index aa5b284a72..3adbbd36b2 100644
--- a/xinference/core/chat_interface.py
+++ b/xinference/core/chat_interface.py
@@ -14,7 +14,7 @@
 
 import logging
 import os
-from typing import Generator, List
+from typing import Generator, List, Optional
 
 import gradio as gr
 from gradio.components import Markdown, Textbox
@@ -43,6 +43,7 @@ def __init__(
         model_ability: List[str],
         model_description: str,
         model_lang: List[str],
+        access_token: Optional[str],
     ):
         self.endpoint = endpoint
         self.model_uid = model_uid
@@ -54,6 +55,9 @@ def __init__(
         self.model_ability = model_ability
         self.model_description = model_description
         self.model_lang = model_lang
+        self._access_token = (
+            access_token.replace("Bearer ", "") if access_token is not None else None
+        )
 
     def build(self) -> "gr.Blocks":
         if "chat" in self.model_ability:
@@ -102,6 +106,7 @@ def generate_wrapper(
             from ..client import RESTfulClient
 
             client = RESTfulClient(self.endpoint)
+            client._set_token(self._access_token)
             model = client.get_model(self.model_uid)
             assert isinstance(
                 model, (RESTfulChatModelHandle, RESTfulChatglmCppChatModelHandle)
@@ -198,6 +203,7 @@ def complete(text, hist, max_tokens, temperature) -> Generator:
             from ..client import RESTfulClient
 
             client = RESTfulClient(self.endpoint)
+            client._set_token(self._access_token)
             model = client.get_model(self.model_uid)
             assert isinstance(model, RESTfulGenerateModelHandle)
 
@@ -234,6 +240,7 @@ def retry(text, hist, max_tokens, temperature) -> Generator:
             from ..client import RESTfulClient
 
             client = RESTfulClient(self.endpoint)
+            client._set_token(self._access_token)
             model = client.get_model(self.model_uid)
             assert isinstance(model, RESTfulGenerateModelHandle)
 
diff --git a/xinference/deploy/cmdline.py b/xinference/deploy/cmdline.py
index 0910ae3c9b..28099bdf1a 100644
--- a/xinference/deploy/cmdline.py
+++ b/xinference/deploy/cmdline.py
@@ -24,13 +24,13 @@
 
 from .. import __version__
 from ..client import RESTfulClient
-from ..client.oscar.actor_client import ActorClient
 from ..client.restful.restful_client import (
     RESTfulChatglmCppChatModelHandle,
     RESTfulChatModelHandle,
     RESTfulGenerateModelHandle,
 )
 from ..constants import (
+    XINFERENCE_AUTH_DIR,
     XINFERENCE_DEFAULT_DISTRIBUTED_HOST,
     XINFERENCE_DEFAULT_ENDPOINT_PORT,
     XINFERENCE_DEFAULT_LOCAL_HOST,
@@ -62,10 +62,32 @@ def get_endpoint(endpoint: Optional[str]) -> str:
         return endpoint
 
 
+def get_hash_endpoint(endpoint: str) -> str:
+    import hashlib
+
+    m = hashlib.sha256()
+    m.update(bytes(endpoint, "utf-8"))
+    return m.hexdigest()
+
+
+def get_stored_token(
+    endpoint: str, client: Optional[RESTfulClient] = None
+) -> Optional[str]:
+    rest_client = RESTfulClient(endpoint) if client is None else client
+    authed = rest_client._cluster_authed
+    if not authed:
+        return None
+
+    token_path = os.path.join(XINFERENCE_AUTH_DIR, get_hash_endpoint(endpoint))
+    if not os.path.exists(token_path):
+        raise RuntimeError("Cannot find access token, please login first!")
+    with open(token_path, "r") as f:
+        access_token = str(f.read())
+    return access_token
+
+
 def start_local_cluster(
-    log_level: str,
-    host: str,
-    port: int,
+    log_level: str, host: str, port: int, auth_config_file: Optional[str] = None
 ):
     from .local import main
 
@@ -81,6 +103,7 @@ def start_local_cluster(
         host=host,
         port=port,
         logging_conf=dict_config,
+        auth_config_file=auth_config_file,
     )
 
 
@@ -159,12 +182,15 @@ def cli(
     type=int,
     help="Specify the port number for the Xinference server.",
 )
-def local(
-    log_level: str,
-    host: str,
-    port: int,
-):
-    start_local_cluster(log_level=log_level, host=host, port=port)
+@click.option(
+    "--auth-config",
+    type=str,
+    help="Specify the auth config json file.",
+)
+def local(log_level: str, host: str, port: int, auth_config: Optional[str]):
+    start_local_cluster(
+        log_level=log_level, host=host, port=port, auth_config_file=auth_config
+    )
 
 
 @click.command(
@@ -196,7 +222,18 @@ def local(
     type=int,
     help="Specify the port number for the Xinference supervisor.",
 )
-def supervisor(log_level: str, host: str, port: int, supervisor_port: Optional[int]):
+@click.option(
+    "--auth-config",
+    type=str,
+    help="Specify the auth config json file.",
+)
+def supervisor(
+    log_level: str,
+    host: str,
+    port: int,
+    supervisor_port: Optional[int],
+    auth_config: Optional[str],
+):
     from ..deploy.supervisor import main
 
     dict_config = get_config_dict(
@@ -208,7 +245,11 @@ def supervisor(log_level: str, host: str, port: int, supervisor_port: Optional[i
     logging.config.dictConfig(dict_config)  # type: ignore
 
     main(
-        host=host, port=port, supervisor_port=supervisor_port, logging_conf=dict_config
+        host=host,
+        port=port,
+        supervisor_port=supervisor_port,
+        logging_conf=dict_config,
+        auth_config_file=auth_config,
     )
 
 
@@ -288,6 +329,7 @@ def register_model(
         model = fd.read()
 
     client = RESTfulClient(base_url=endpoint)
+    client._set_token(get_stored_token(endpoint, client))
     client.register_model(
         model_type=model_type,
         model=model,
@@ -316,6 +358,7 @@ def unregister_model(
     endpoint = get_endpoint(endpoint)
 
     client = RESTfulClient(base_url=endpoint)
+    client._set_token(get_stored_token(endpoint, client))
     client.unregister_model(
         model_type=model_type,
         model_name=model_name,
@@ -343,8 +386,9 @@ def list_model_registrations(
     from tabulate import tabulate
 
     endpoint = get_endpoint(endpoint)
-
     client = RESTfulClient(base_url=endpoint)
+    client._set_token(get_stored_token(endpoint, client))
+
     registrations = client.list_model_registrations(model_type=model_type)
 
     table = []
@@ -518,8 +562,9 @@ def model_launch(
         if size_in_billions is None or "_" in size_in_billions
         else int(size_in_billions)
     )
-
     client = RESTfulClient(base_url=endpoint)
+    client._set_token(get_stored_token(endpoint, client))
+
     model_uid = client.launch_model(
         model_name=model_name,
         model_type=model_type,
@@ -550,6 +595,7 @@ def model_list(endpoint: Optional[str]):
 
     endpoint = get_endpoint(endpoint)
     client = RESTfulClient(base_url=endpoint)
+    client._set_token(get_stored_token(endpoint, client))
 
     llm_table = []
     embedding_table = []
@@ -626,8 +672,8 @@ def model_terminate(
     model_uid: str,
 ):
     endpoint = get_endpoint(endpoint)
-
     client = RESTfulClient(base_url=endpoint)
+    client._set_token(get_stored_token(endpoint, client))
     client.terminate_model(model_uid=model_uid)
 
 
@@ -657,6 +703,8 @@ def model_generate(
     stream: bool,
 ):
     endpoint = get_endpoint(endpoint)
+    client = RESTfulClient(base_url=endpoint)
+    client._set_token(get_stored_token(endpoint, client))
     if stream:
         # TODO: when stream=True, RestfulClient cannot generate words one by one.
         # So use Client in temporary. The implementation needs to be changed to
@@ -669,7 +717,7 @@ async def generate_internal():
                 if prompt == "":
                     break
                 print(f"Completion: {prompt}", end="", file=sys.stdout)
-                async for chunk in model.generate(
+                for chunk in model.generate(
                     prompt=prompt,
                     generate_config={"stream": stream, "max_tokens": max_tokens},
                 ):
@@ -680,7 +728,6 @@ async def generate_internal():
                         print(choice["text"], end="", flush=True, file=sys.stdout)
                 print("", file=sys.stdout)
 
-        client = ActorClient(endpoint=endpoint)
         model = client.get_model(model_uid=model_uid)
 
         loop = asyncio.get_event_loop()
@@ -700,8 +747,7 @@ async def generate_internal():
                 # avoid displaying exception-unhandled warnings
                 task.exception()
     else:
-        restful_client = RESTfulClient(base_url=endpoint)
-        restful_model = restful_client.get_model(model_uid=model_uid)
+        restful_model = client.get_model(model_uid=model_uid)
         if not isinstance(
             restful_model, (RESTfulChatModelHandle, RESTfulGenerateModelHandle)
         ):
@@ -744,6 +790,9 @@ def model_chat(
 ):
     # TODO: chat model roles may not be user and assistant.
     endpoint = get_endpoint(endpoint)
+    client = RESTfulClient(base_url=endpoint)
+    client._set_token(get_stored_token(endpoint, client))
+
     chat_history: "List[ChatCompletionMessage]" = []
     if stream:
         # TODO: when stream=True, RestfulClient cannot generate words one by one.
@@ -758,7 +807,7 @@ async def chat_internal():
                     break
                 print("Assistant: ", end="", file=sys.stdout)
                 response_content = ""
-                async for chunk in model.chat(
+                for chunk in model.chat(
                     prompt=prompt,
                     chat_history=chat_history,
                     generate_config={"stream": stream, "max_tokens": max_tokens},
@@ -775,7 +824,6 @@ async def chat_internal():
                     ChatCompletionMessage(role="assistant", content=response_content)
                 )
 
-        client = ActorClient(endpoint=endpoint)
         model = client.get_model(model_uid=model_uid)
 
         loop = asyncio.get_event_loop()
@@ -795,8 +843,7 @@ async def chat_internal():
                 # avoid displaying exception-unhandled warnings
                 task.exception()
     else:
-        restful_client = RESTfulClient(base_url=endpoint)
-        restful_model = restful_client.get_model(model_uid=model_uid)
+        restful_model = client.get_model(model_uid=model_uid)
         if not isinstance(
             restful_model, (RESTfulChatModelHandle, RESTfulChatglmCppChatModelHandle)
         ):
@@ -822,5 +869,31 @@ async def chat_internal():
             )
 
 
+@cli.command("login", help="Login when the cluster is authenticated.")
+@click.option("--endpoint", "-e", type=str, help="Xinference endpoint.")
+@click.option("--username", type=str, required=True, help="Username.")
+@click.option(
+    "--password",
+    type=str,
+    required=True,
+    help="Password.",
+)
+def cluster_login(
+    endpoint: Optional[str],
+    username: str,
+    password: str,
+):
+    endpoint = get_endpoint(endpoint)
+    restful_client = RESTfulClient(base_url=endpoint)
+    if restful_client._cluster_authed:
+        restful_client.login(username, password)
+        access_token = restful_client._get_token()
+        assert access_token is not None
+        os.makedirs(XINFERENCE_AUTH_DIR, exist_ok=True)
+        hashed_ep = get_hash_endpoint(endpoint)
+        with open(os.path.join(XINFERENCE_AUTH_DIR, hashed_ep), "w") as f:
+            f.write(access_token)
+
+
 if __name__ == "__main__":
     cli()
diff --git a/xinference/deploy/local.py b/xinference/deploy/local.py
index d646f80906..a152c45edc 100644
--- a/xinference/deploy/local.py
+++ b/xinference/deploy/local.py
@@ -79,7 +79,12 @@ def run_in_subprocess(
     return p
 
 
-def main(host: str, port: int, logging_conf: Optional[Dict] = None):
+def main(
+    host: str,
+    port: int,
+    logging_conf: Optional[Dict] = None,
+    auth_config_file: Optional[str] = None,
+):
     supervisor_address = f"{host}:{get_next_port()}"
     local_cluster = run_in_subprocess(supervisor_address, logging_conf)
 
@@ -98,6 +103,7 @@ def main(host: str, port: int, logging_conf: Optional[Dict] = None):
             host=host,
             port=port,
             logging_conf=logging_conf,
+            auth_config_file=auth_config_file,
         )
     finally:
         local_cluster.terminate()
diff --git a/xinference/deploy/supervisor.py b/xinference/deploy/supervisor.py
index ddc4f25224..57f03c99c6 100644
--- a/xinference/deploy/supervisor.py
+++ b/xinference/deploy/supervisor.py
@@ -75,6 +75,7 @@ def main(
     port: int,
     supervisor_port: Optional[int],
     logging_conf: Optional[Dict] = None,
+    auth_config_file: Optional[str] = None,
 ):
     supervisor_address = f"{host}:{supervisor_port or get_next_port()}"
     local_cluster = run_in_subprocess(supervisor_address, logging_conf)
@@ -94,6 +95,7 @@ def main(
             host=host,
             port=port,
             logging_conf=logging_conf,
+            auth_config_file=auth_config_file,
         )
     finally:
         local_cluster.terminate()
diff --git a/xinference/web/ui/package-lock.json b/xinference/web/ui/package-lock.json
index dfd1fc9d1b..6cbc3b3670 100644
--- a/xinference/web/ui/package-lock.json
+++ b/xinference/web/ui/package-lock.json
@@ -27,7 +27,9 @@
         "@testing-library/react": "^13.4.0",
         "@testing-library/user-event": "^13.5.0",
         "formik": "^2.4.2",
+        "jsonwebtoken": "^9.0.2",
         "react": "^18.2.0",
+        "react-cookie": "^6.1.1",
         "react-dom": "^18.2.0",
         "react-pro-sidebar": "^1.1.0-alpha.1",
         "react-router-dom": "^6.14.1",
@@ -4960,6 +4962,11 @@
         "@types/node": "*"
       }
     },
+    "node_modules/@types/cookie": {
+      "version": "0.5.4",
+      "resolved": "https://registry.npmjs.org/@types/cookie/-/cookie-0.5.4.tgz",
+      "integrity": "sha512-7z/eR6O859gyWIAjuvBWFzNURmf2oPBmJlfVWkwehU5nzIyjwBsTh7WMmEEV4JFnHuQ3ex4oyTvfKzcyJVDBNA=="
+    },
     "node_modules/@types/d3-color": {
       "version": "2.0.3",
       "resolved": "https://registry.npmjs.org/@types/d3-color/-/d3-color-2.0.3.tgz",
@@ -5069,6 +5076,15 @@
         "@types/node": "*"
       }
     },
+    "node_modules/@types/hoist-non-react-statics": {
+      "version": "3.3.5",
+      "resolved": "https://registry.npmjs.org/@types/hoist-non-react-statics/-/hoist-non-react-statics-3.3.5.tgz",
+      "integrity": "sha512-SbcrWzkKBw2cdwRTwQAswfpB9g9LJWfjtUeW/jvNwbhC8cpmmNYVePa+ncbUe0rGTQ7G3Ff6mYUN2VMfLVr+Sg==",
+      "dependencies": {
+        "@types/react": "*",
+        "hoist-non-react-statics": "^3.3.0"
+      }
+    },
     "node_modules/@types/html-minifier-terser": {
       "version": "6.1.0",
       "resolved": "https://registry.npmjs.org/@types/html-minifier-terser/-/html-minifier-terser-6.1.0.tgz",
@@ -6822,6 +6838,11 @@
         "node-int64": "^0.4.0"
       }
     },
+    "node_modules/buffer-equal-constant-time": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",
+      "integrity": "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA=="
+    },
     "node_modules/buffer-from": {
       "version": "1.1.2",
       "resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz",
@@ -8224,6 +8245,14 @@
       "resolved": "https://registry.npmjs.org/duplexer/-/duplexer-0.1.2.tgz",
       "integrity": "sha512-jtD6YG370ZCIi/9GTaJKQxWTZD045+4R4hTk/x1UyoqadyJ9x9CgSi1RlVDQF8U2sxLLSnFkCaMihqljHIWgMg=="
     },
+    "node_modules/ecdsa-sig-formatter": {
+      "version": "1.0.11",
+      "resolved": "https://registry.npmjs.org/ecdsa-sig-formatter/-/ecdsa-sig-formatter-1.0.11.tgz",
+      "integrity": "sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ==",
+      "dependencies": {
+        "safe-buffer": "^5.0.1"
+      }
+    },
     "node_modules/ee-first": {
       "version": "1.1.1",
       "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
@@ -13249,6 +13278,27 @@
         "node": ">=0.10.0"
       }
     },
+    "node_modules/jsonwebtoken": {
+      "version": "9.0.2",
+      "resolved": "https://registry.npmjs.org/jsonwebtoken/-/jsonwebtoken-9.0.2.tgz",
+      "integrity": "sha512-PRp66vJ865SSqOlgqS8hujT5U4AOgMfhrwYIuIhfKaoSCZcirrmASQr8CX7cUg+RMih+hgznrjp99o+W4pJLHQ==",
+      "dependencies": {
+        "jws": "^3.2.2",
+        "lodash.includes": "^4.3.0",
+        "lodash.isboolean": "^3.0.3",
+        "lodash.isinteger": "^4.0.4",
+        "lodash.isnumber": "^3.0.3",
+        "lodash.isplainobject": "^4.0.6",
+        "lodash.isstring": "^4.0.1",
+        "lodash.once": "^4.0.0",
+        "ms": "^2.1.1",
+        "semver": "^7.5.4"
+      },
+      "engines": {
+        "node": ">=12",
+        "npm": ">=6"
+      }
+    },
     "node_modules/jsx-ast-utils": {
       "version": "3.3.5",
       "resolved": "https://registry.npmjs.org/jsx-ast-utils/-/jsx-ast-utils-3.3.5.tgz",
@@ -13263,6 +13313,25 @@
         "node": ">=4.0"
       }
     },
+    "node_modules/jwa": {
+      "version": "1.4.1",
+      "resolved": "https://registry.npmjs.org/jwa/-/jwa-1.4.1.tgz",
+      "integrity": "sha512-qiLX/xhEEFKUAJ6FiBMbes3w9ATzyk5W7Hvzpa/SLYdxNtng+gcurvrI7TbACjIXlsJyr05/S1oUhZrc63evQA==",
+      "dependencies": {
+        "buffer-equal-constant-time": "1.0.1",
+        "ecdsa-sig-formatter": "1.0.11",
+        "safe-buffer": "^5.0.1"
+      }
+    },
+    "node_modules/jws": {
+      "version": "3.2.2",
+      "resolved": "https://registry.npmjs.org/jws/-/jws-3.2.2.tgz",
+      "integrity": "sha512-YHlZCB6lMTllWDtSPHz/ZXTsi8S00usEV6v1tjq8tOUZzw7DpSDWVXjXDre6ed1w/pd495ODpHZYSdkRTsa0HA==",
+      "dependencies": {
+        "jwa": "^1.4.1",
+        "safe-buffer": "^5.0.1"
+      }
+    },
     "node_modules/kind-of": {
       "version": "6.0.3",
       "resolved": "https://registry.npmjs.org/kind-of/-/kind-of-6.0.3.tgz",
@@ -13395,6 +13464,36 @@
       "resolved": "https://registry.npmjs.org/lodash.debounce/-/lodash.debounce-4.0.8.tgz",
       "integrity": "sha512-FT1yDzDYEoYWhnSGnpE/4Kj1fLZkDFyqRb7fNt6FdYOSxlUWAtp42Eh6Wb0rGIv/m9Bgo7x4GhQbm5Ys4SG5ow=="
     },
+    "node_modules/lodash.includes": {
+      "version": "4.3.0",
+      "resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz",
+      "integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w=="
+    },
+    "node_modules/lodash.isboolean": {
+      "version": "3.0.3",
+      "resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",
+      "integrity": "sha512-Bz5mupy2SVbPHURB98VAcw+aHh4vRV5IPNhILUCsOzRmsTmSQ17jIuqopAentWoehktxGd9e/hbIXq980/1QJg=="
+    },
+    "node_modules/lodash.isinteger": {
+      "version": "4.0.4",
+      "resolved": "https://registry.npmjs.org/lodash.isinteger/-/lodash.isinteger-4.0.4.tgz",
+      "integrity": "sha512-DBwtEWN2caHQ9/imiNeEA5ys1JoRtRfY3d7V9wkqtbycnAmTvRRmbHKDV4a0EYc678/dia0jrte4tjYwVBaZUA=="
+    },
+    "node_modules/lodash.isnumber": {
+      "version": "3.0.3",
+      "resolved": "https://registry.npmjs.org/lodash.isnumber/-/lodash.isnumber-3.0.3.tgz",
+      "integrity": "sha512-QYqzpfwO3/CWf3XP+Z+tkQsfaLL/EnUlXWVkIk5FUPc4sBdTehEqZONuyRt2P67PXAk+NXmTBcc97zw9t1FQrw=="
+    },
+    "node_modules/lodash.isplainobject": {
+      "version": "4.0.6",
+      "resolved": "https://registry.npmjs.org/lodash.isplainobject/-/lodash.isplainobject-4.0.6.tgz",
+      "integrity": "sha512-oSXzaWypCMHkPC3NvBEaPHf0KsA5mvPrOPgQWDsbg8n7orZ290M0BmC/jgRZ4vcJ6DTAhjrsSYgdsW/F+MFOBA=="
+    },
+    "node_modules/lodash.isstring": {
+      "version": "4.0.1",
+      "resolved": "https://registry.npmjs.org/lodash.isstring/-/lodash.isstring-4.0.1.tgz",
+      "integrity": "sha512-0wJxfxH1wgO3GrbuP+dTTk7op+6L41QCXbGINEmD+ny/G/eCqGzxyCsh7159S+mgDDcoarnBw6PC1PS5+wUGgw=="
+    },
     "node_modules/lodash.memoize": {
       "version": "4.1.2",
       "resolved": "https://registry.npmjs.org/lodash.memoize/-/lodash.memoize-4.1.2.tgz",
@@ -13405,6 +13504,11 @@
       "resolved": "https://registry.npmjs.org/lodash.merge/-/lodash.merge-4.6.2.tgz",
       "integrity": "sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ=="
     },
+    "node_modules/lodash.once": {
+      "version": "4.1.1",
+      "resolved": "https://registry.npmjs.org/lodash.once/-/lodash.once-4.1.1.tgz",
+      "integrity": "sha512-Sb487aTOCr9drQVL8pIxOzVhafOjZN9UU54hiN8PU3uAiSV7lx1yYNpbNmex2PK6dSJoNTSJUUswT651yww3Mg=="
+    },
     "node_modules/lodash.sortby": {
       "version": "4.7.0",
       "resolved": "https://registry.npmjs.org/lodash.sortby/-/lodash.sortby-4.7.0.tgz",
@@ -15863,6 +15967,19 @@
         "node": ">=14"
       }
     },
+    "node_modules/react-cookie": {
+      "version": "6.1.1",
+      "resolved": "https://registry.npmjs.org/react-cookie/-/react-cookie-6.1.1.tgz",
+      "integrity": "sha512-fuFRpf8LH6SfmVMowDUIRywJF5jAUDUWrm0EI5VdXfTl5bPcJ7B0zWbuYpT0Tvikx7Gs18MlvAT+P+744dUz2g==",
+      "dependencies": {
+        "@types/hoist-non-react-statics": "^3.3.1",
+        "hoist-non-react-statics": "^3.3.2",
+        "universal-cookie": "^6.0.0"
+      },
+      "peerDependencies": {
+        "react": ">= 16.3.0"
+      }
+    },
     "node_modules/react-dev-utils": {
       "version": "12.0.1",
       "resolved": "https://registry.npmjs.org/react-dev-utils/-/react-dev-utils-12.0.1.tgz",
@@ -18450,6 +18567,15 @@
         "node": ">=8"
       }
     },
+    "node_modules/universal-cookie": {
+      "version": "6.1.1",
+      "resolved": "https://registry.npmjs.org/universal-cookie/-/universal-cookie-6.1.1.tgz",
+      "integrity": "sha512-33S9x3CpdUnnjwTNs2Fgc41WGve2tdLtvaK2kPSbZRc5pGpz2vQFbRWMxlATsxNNe/Cy8SzmnmbuBM85jpZPtA==",
+      "dependencies": {
+        "@types/cookie": "^0.5.1",
+        "cookie": "^0.5.0"
+      }
+    },
     "node_modules/universalify": {
       "version": "2.0.0",
       "resolved": "https://registry.npmjs.org/universalify/-/universalify-2.0.0.tgz",
diff --git a/xinference/web/ui/package.json b/xinference/web/ui/package.json
index 86abc72cfb..2c0fe23afa 100644
--- a/xinference/web/ui/package.json
+++ b/xinference/web/ui/package.json
@@ -11,9 +11,9 @@
     "@fullcalendar/list": "^6.1.8",
     "@fullcalendar/timegrid": "^6.1.8",
     "@mui/icons-material": "^5.14.0",
+    "@mui/lab": "latest",
     "@mui/material": "^5.14.0",
     "@mui/x-data-grid": "^6.10.0",
-    "@mui/lab": "latest",
     "@nivo/bar": "^0.83.0",
     "@nivo/core": "^0.83.0",
     "@nivo/geo": "^0.83.0",
@@ -24,6 +24,7 @@
     "@testing-library/user-event": "^13.5.0",
     "formik": "^2.4.2",
     "react": "^18.2.0",
+    "react-cookie": "^6.1.1",
     "react-dom": "^18.2.0",
     "react-pro-sidebar": "^1.1.0-alpha.1",
     "react-router-dom": "^6.14.1",
@@ -58,9 +59,9 @@
     ]
   },
   "devDependencies": {
-    "@babel/plugin-proposal-private-property-in-object": "^7.21.11",
     "@babel/core": "^7.21.0",
     "@babel/eslint-parser": "^7.19.1",
+    "@babel/plugin-proposal-private-property-in-object": "^7.21.11",
     "eslint": "^7.32.0",
     "eslint-config-prettier": "^8.5.0",
     "eslint-plugin-react": "^7.24.0",
diff --git a/xinference/web/ui/src/App.js b/xinference/web/ui/src/App.js
index 693c872584..beffd7fc97 100644
--- a/xinference/web/ui/src/App.js
+++ b/xinference/web/ui/src/App.js
@@ -1,22 +1,98 @@
 import { CssBaseline, ThemeProvider } from '@mui/material'
+import Snackbar from '@mui/material/Snackbar'
+import React, { useEffect, useState } from 'react'
+import { useCookies } from 'react-cookie'
 import { HashRouter, Route, Routes } from 'react-router-dom'
 
+import { Alert } from './components/alertComponent'
 import { ApiContextProvider } from './components/apiContext'
+import AuthAlertDialog from './components/authAlertDialog'
+import { getEndpoint, isValidBearerToken } from './components/utils'
 import Layout from './scenes/_layout'
 import LaunchModel from './scenes/launch_model'
+import Login from './scenes/login/login'
 import RegisterModel from './scenes/register_model'
 import RunningModels from './scenes/running_models'
 import { useMode } from './theme'
 
 function App() {
   const [theme] = useMode()
+  const [cookie, setCookie, removeCookie] = useCookies(['token'])
+  const [msg, setMsg] = useState('')
+
+  const endPoint = getEndpoint()
+
+  const removeToken = () => {
+    removeCookie('token', { path: '/' })
+  }
+
+  useEffect(() => {
+    // token possible value: no_auth / need_auth / <real bearer token>
+    fetch(endPoint + '/v1/cluster/auth', {
+      method: 'GET',
+      headers: {
+        'Content-Type': 'application/json',
+      },
+    }).then((res) => {
+      if (!res.ok) {
+        res.json().then((errorData) => {
+          setMsg(
+            `Server error: ${res.status} - ${
+              errorData.detail || 'Unknown error'
+            }`
+          )
+        })
+      } else {
+        res.json().then((data) => {
+          if (data['auth'] === false) {
+            if (cookie.token !== 'no_auth') {
+              setCookie('token', 'no_auth', { path: '/' })
+            }
+          } else {
+            // TODO: validate bearer token
+            if (
+              cookie.token === undefined ||
+              !isValidBearerToken(cookie.token)
+            ) {
+              // not a bearer token, need a bearer token here
+              setCookie('token', 'need_auth', { path: '/' })
+            }
+          }
+        })
+      }
+    })
+    // return a function in useEffect means doing something on component unmount
+    return () => {
+      removeToken()
+    }
+  }, [])
+
+  const handleClose = (event, reason) => {
+    if (reason === 'clickaway') {
+      return
+    }
+    setMsg('')
+  }
+
   return (
     <div className="app">
+      <Snackbar
+        open={msg !== ''}
+        autoHideDuration={10000}
+        anchorOrigin={{ vertical: 'top', horizontal: 'center' }}
+        onClose={handleClose}
+      >
+        <Alert severity="error" onClose={handleClose} sx={{ width: '100%' }}>
+          {msg}
+        </Alert>
+      </Snackbar>
       <HashRouter>
         <ThemeProvider theme={theme}>
           <ApiContextProvider>
             <CssBaseline />
+            <AuthAlertDialog />
             <Routes>
+              <Route path="/login" element={<Login />} />
               <Route element={<Layout />}>
                 <Route path="/" element={<LaunchModel />} />
                 <Route path="/running_models" element={<RunningModels />} />
diff --git a/xinference/web/ui/src/components/Title.js b/xinference/web/ui/src/components/Title.js
index a28cfd5d67..05e124a281 100644
--- a/xinference/web/ui/src/components/Title.js
+++ b/xinference/web/ui/src/components/Title.js
@@ -1,19 +1,42 @@
-import { Box, Typography } from '@mui/material'
+import ExitToAppIcon from '@mui/icons-material/ExitToApp'
+import { Box, Stack, Typography } from '@mui/material'
+import Button from '@mui/material/Button'
+import { useCookies } from 'react-cookie'
+import { useNavigate } from 'react-router-dom'
+
+import { isValidBearerToken } from './utils'
+
+const Title = ({ title }) => {
+  const [cookie, , removeCookie] = useCookies(['token'])
+  const navigate = useNavigate()
+
+  const handleLogout = () => {
+    removeCookie('token', { path: '/' })
+    navigate('/login', { replace: true })
+  }
 
-const Title = ({ title, subtitle }) => {
   return (
     <Box mb="30px">
-      <Typography
-        variant="h2"
-        color="#141414"
-        fontWeight="bold"
-        sx={{ m: '0 0 5px 0' }}
-      >
-        {title}
-      </Typography>
-      <Typography variant="h5" color="#3d3d3d">
-        {subtitle}
-      </Typography>
+      <Stack direction="row" alignItems="center" justifyContent="space-between">
+        <Typography
+          variant="h2"
+          color="#141414"
+          fontWeight="bold"
+          sx={{ m: '0 0 5px 0' }}
+        >
+          {title}
+        </Typography>
+        {isValidBearerToken(cookie.token) && (
+          <Button
+            variant="outlined"
+            size="large"
+            onClick={handleLogout}
+            startIcon={<ExitToAppIcon />}
+          >
+            LOG OUT
+          </Button>
+        )}
+      </Stack>
     </Box>
   )
 }
diff --git a/xinference/web/ui/src/components/alertComponent.js b/xinference/web/ui/src/components/alertComponent.js
new file mode 100644
index 0000000000..4603b3f9c6
--- /dev/null
+++ b/xinference/web/ui/src/components/alertComponent.js
@@ -0,0 +1,8 @@
+import MuiAlert from '@mui/material/Alert'
+import React from 'react'
+
+const Alert = React.forwardRef(function Alert(props, ref) {
+  return <MuiAlert elevation={6} ref={ref} variant="filled" {...props} />
+})
+
+export { Alert }
diff --git a/xinference/web/ui/src/components/apiContext.js b/xinference/web/ui/src/components/apiContext.js
index 0b8f5aa748..f4f223e648 100644
--- a/xinference/web/ui/src/components/apiContext.js
+++ b/xinference/web/ui/src/components/apiContext.js
@@ -1,18 +1,14 @@
 import React, { createContext, useState } from 'react'
 
+import { getEndpoint } from './utils'
+
 export const ApiContext = createContext()
 
 export const ApiContextProvider = ({ children }) => {
   const [isCallingApi, setIsCallingApi] = useState(false)
   const [isUpdatingModel, setIsUpdatingModel] = useState(false)
   const [errorMsg, setErrorMsg] = useState('')
-  let endPoint = ''
-  if (!process.env.NODE_ENV || process.env.NODE_ENV === 'development') {
-    endPoint = 'http://127.0.0.1:9997'
-  } else {
-    const fullUrl = window.location.href
-    endPoint = fullUrl.split('/ui')[0]
-  }
+  const endPoint = getEndpoint()
 
   return (
     <ApiContext.Provider
diff --git a/xinference/web/ui/src/components/authAlertDialog.js b/xinference/web/ui/src/components/authAlertDialog.js
new file mode 100644
index 0000000000..4150ac54b2
--- /dev/null
+++ b/xinference/web/ui/src/components/authAlertDialog.js
@@ -0,0 +1,92 @@
+import Button from '@mui/material/Button'
+import Dialog from '@mui/material/Dialog'
+import DialogActions from '@mui/material/DialogActions'
+import DialogContent from '@mui/material/DialogContent'
+import DialogContentText from '@mui/material/DialogContentText'
+import DialogTitle from '@mui/material/DialogTitle'
+import * as React from 'react'
+import { useEffect, useState } from 'react'
+import { useCookies } from 'react-cookie'
+import { useNavigate } from 'react-router-dom'
+
+export default function AuthAlertDialog() {
+  const navigate = useNavigate()
+  const [authStatus, setAuthStatus] = useState('')
+  const [, , removeCookie] = useCookies(['token'])
+
+  const handleAuthStatus = () => {
+    const status = localStorage.getItem('authStatus')
+    if (status) {
+      setAuthStatus(status)
+    } else {
+      setAuthStatus('')
+    }
+  }
+
+  useEffect(() => {
+    localStorage.removeItem('authStatus')
+    window.addEventListener('auth-status', handleAuthStatus)
+
+    return () => {
+      window.removeEventListener('auth-status', handleAuthStatus)
+    }
+  }, [])
+
+  const handleClose = () => {
+    // trigger first
+    const code = localStorage.getItem('authStatus')
+    localStorage.removeItem('authStatus')
+    setAuthStatus('')
+    if (code === '401') {
+      removeCookie('token', { path: '/' })
+      navigate('/login', { replace: true })
+    }
+  }
+
+  const handleDialogClose = (event, reason) => {
+    if (reason && reason === 'backdropClick') {
+      return
+    }
+    localStorage.removeItem('authStatus')
+    setAuthStatus('')
+  }
+
+  return (
+    <React.Fragment>
+      <Dialog
+        fullWidth
+        maxWidth="md"
+        open={authStatus === '401' || authStatus === '403'}
+        onClose={handleDialogClose}
+        aria-labelledby="alert-dialog-title"
+        aria-describedby="alert-dialog-description"
+      >
+        {authStatus === '403' && (
+          <DialogTitle id="alert-dialog-title">
+            {'Permission Error'}
+          </DialogTitle>
+        )}
+        {authStatus === '401' && (
+          <DialogTitle id="alert-dialog-title">
+            {'Authentication Error'}
+          </DialogTitle>
+        )}
+        <DialogContent>
+          {authStatus === '403' && (
+            <DialogContentText id="alert-dialog-description">
+              {'You do not have permissions to do this!'}
+            </DialogContentText>
+          )}
+          {authStatus === '401' && (
+            <DialogContentText id="alert-dialog-description">
+              {'Invalid credentials! Please login.'}
+            </DialogContentText>
+          )}
+        </DialogContent>
+        <DialogActions>
+          <Button onClick={handleClose}>CONFIRMED</Button>
+        </DialogActions>
+      </Dialog>
+    </React.Fragment>
+  )
+}
diff --git a/xinference/web/ui/src/components/errorMessageSnackBar.js b/xinference/web/ui/src/components/errorMessageSnackBar.js
index 2c4802adea..905f84d50b 100644
--- a/xinference/web/ui/src/components/errorMessageSnackBar.js
+++ b/xinference/web/ui/src/components/errorMessageSnackBar.js
@@ -1,13 +1,9 @@
-import MuiAlert from '@mui/material/Alert'
 import Snackbar from '@mui/material/Snackbar'
 import React, { useContext } from 'react'
 
+import { Alert } from './alertComponent'
 import { ApiContext } from './apiContext'
 
-const Alert = React.forwardRef(function Alert(props, ref) {
-  return <MuiAlert elevation={6} ref={ref} variant="filled" {...props} />
-})
-
 const ErrorMessageSnackBar = () => {
   const { errorMsg, setErrorMsg } = useContext(ApiContext)
 
diff --git a/xinference/web/ui/src/components/fetcher.js b/xinference/web/ui/src/components/fetcher.js
new file mode 100644
index 0000000000..6d1544ebe0
--- /dev/null
+++ b/xinference/web/ui/src/components/fetcher.js
@@ -0,0 +1,36 @@
+import { Cookies } from 'react-cookie'
+
+import { isValidBearerToken } from './utils'
+
+const cookies = new Cookies()
+
+const updateOptions = (url, options) => {
+  const update = { ...options }
+  if (cookies.get('token') !== 'no_auth') {
+    update.headers = {
+      ...update.headers,
+      Authorization: 'Bearer ' + cookies.get('token'),
+    }
+  }
+  return update
+}
+
+export default function fetcher(url, options) {
+  return fetch(url, updateOptions(url, options)).then((res) => {
+    // For the situation that server has already been restarted, the current token may become invalid,
+    // which leads to UI hangs.
+    if (res.status === 401 && isValidBearerToken(cookies.get('token'))) {
+      if (localStorage.getItem('authStatus') !== '401') {
+        localStorage.setItem('authStatus', '401')
+        window.dispatchEvent(new Event('auth-status'))
+      }
+    } else if (res.status === 403 && isValidBearerToken(cookies.get('token'))) {
+      if (localStorage.getItem('authStatus') !== '403') {
+        localStorage.setItem('authStatus', '403')
+        window.dispatchEvent(new Event('auth-status'))
+      }
+    } else {
+      return res
+    }
+  })
+}
diff --git a/xinference/web/ui/src/components/utils.js b/xinference/web/ui/src/components/utils.js
new file mode 100644
index 0000000000..fe995efe03
--- /dev/null
+++ b/xinference/web/ui/src/components/utils.js
@@ -0,0 +1,18 @@
+const getEndpoint = () => {
+  let endPoint = ''
+  if (!process.env.NODE_ENV || process.env.NODE_ENV === 'development') {
+    endPoint = 'http://127.0.0.1:9997'
+  } else {
+    const fullUrl = window.location.href
+    endPoint = fullUrl.split('/ui')[0]
+  }
+  return endPoint
+}
+
+const isValidBearerToken = (token) => {
+  return (
+    token !== '' && token !== undefined && token !== null && token.length > 10
+  )
+}
+
+export { getEndpoint, isValidBearerToken }
diff --git a/xinference/web/ui/src/index.js b/xinference/web/ui/src/index.js
index 34d5faf9cb..eed265203d 100644
--- a/xinference/web/ui/src/index.js
+++ b/xinference/web/ui/src/index.js
@@ -1,4 +1,5 @@
 import React from 'react'
+import { CookiesProvider } from 'react-cookie'
 import ReactDOM from 'react-dom/client'
 
 import App from './App'
@@ -6,6 +7,8 @@ import App from './App'
 const root = ReactDOM.createRoot(document.getElementById('root'))
 root.render(
   <React.StrictMode>
-    <App />
+    <CookiesProvider>
+      <App />
+    </CookiesProvider>
   </React.StrictMode>
 )
diff --git a/xinference/web/ui/src/scenes/launch_model/embeddingCard.js b/xinference/web/ui/src/scenes/launch_model/embeddingCard.js
index 07e01749f9..14c37a1a7d 100644
--- a/xinference/web/ui/src/scenes/launch_model/embeddingCard.js
+++ b/xinference/web/ui/src/scenes/launch_model/embeddingCard.js
@@ -12,6 +12,7 @@ import IconButton from '@mui/material/IconButton'
 import React, { useContext, useEffect, useState } from 'react'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 
 const CARD_HEIGHT = 270
 const CARD_WIDTH = 270
@@ -46,8 +47,8 @@ const EmbeddingCard = ({
       model_type: 'embedding',
     }
 
-    // First fetch request to initiate the model
-    fetch(url + '/v1/models', {
+    // First fetcher request to initiate the model
+    fetcher(url + '/v1/models', {
       method: 'POST',
       headers: {
         'Content-Type': 'application/json',
@@ -204,7 +205,7 @@ const EmbeddingCard = ({
 
   const handeCustomDelete = (e) => {
     e.stopPropagation()
-    fetch(url + `/v1/model_registrations/embedding/${modelData.model_name}`, {
+    fetcher(url + `/v1/model_registrations/embedding/${modelData.model_name}`, {
       method: 'DELETE',
       headers: {
         'Content-Type': 'application/json',
diff --git a/xinference/web/ui/src/scenes/launch_model/index.js b/xinference/web/ui/src/scenes/launch_model/index.js
index 33aceee695..b6c89bbb94 100644
--- a/xinference/web/ui/src/scenes/launch_model/index.js
+++ b/xinference/web/ui/src/scenes/launch_model/index.js
@@ -1,6 +1,8 @@
 import { TabContext, TabList, TabPanel } from '@mui/lab'
 import { Box, Tab } from '@mui/material'
 import React, { useContext, useEffect, useState } from 'react'
+import { useCookies } from 'react-cookie'
+import { useNavigate } from 'react-router-dom'
 
 import { ApiContext } from '../../components/apiContext'
 import ErrorMessageSnackBar from '../../components/errorMessageSnackBar'
@@ -16,12 +18,22 @@ const LaunchModel = () => {
   const [gpuAvailable, setGPUAvailable] = useState(-1)
 
   const { setErrorMsg } = useContext(ApiContext)
+  const [cookie] = useCookies(['token'])
+  const navigate = useNavigate()
 
   const handleTabChange = (event, newValue) => {
     setValue(newValue)
   }
 
   useEffect(() => {
+    if (cookie.token === '' || cookie.token === undefined) {
+      return
+    }
+    if (cookie.token === 'need_auth') {
+      navigate('/login', { replace: true })
+      return
+    }
+
     if (gpuAvailable === -1) {
       fetch(endPoint + '/v1/cluster/devices', {
         method: 'GET',
@@ -45,7 +57,7 @@ const LaunchModel = () => {
         }
       })
     }
-  }, [])
+  }, [cookie.token])
 
   return (
     <Box m="20px">
diff --git a/xinference/web/ui/src/scenes/launch_model/launchCustom.js b/xinference/web/ui/src/scenes/launch_model/launchCustom.js
index eda5599680..fbbf683ab1 100644
--- a/xinference/web/ui/src/scenes/launch_model/launchCustom.js
+++ b/xinference/web/ui/src/scenes/launch_model/launchCustom.js
@@ -2,6 +2,7 @@ import { Box, FormControl, TextField } from '@mui/material'
 import React, { useContext, useEffect, useState } from 'react'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 import EmbeddingCard from './embeddingCard'
 import ModelCard from './modelCard'
 import RerankCard from './rerankCard'
@@ -33,7 +34,7 @@ const LaunchCustom = ({ gpuAvailable }) => {
     try {
       setIsCallingApi(true)
 
-      const rerankResponse = await fetch(
+      const rerankResponse = await fetcher(
         `${endPoint}/v1/model_registrations/rerank`,
         {
           method: 'GET',
@@ -44,7 +45,7 @@ const LaunchCustom = ({ gpuAvailable }) => {
         (data) => !data.is_builtin
       )
 
-      const embeddingResponse = await fetch(
+      const embeddingResponse = await fetcher(
         `${endPoint}/v1/model_registrations/embedding`,
         {
           method: 'GET',
@@ -56,7 +57,7 @@ const LaunchCustom = ({ gpuAvailable }) => {
         (data) => !data.is_builtin
       )
 
-      const llmResponse = await fetch(
+      const llmResponse = await fetcher(
         `${endPoint}/v1/model_registrations/LLM`,
         {
           method: 'GET',
@@ -69,7 +70,7 @@ const LaunchCustom = ({ gpuAvailable }) => {
 
       const newEmbeddingData = await Promise.all(
         customEmbeddingRegistrations.map(async (registration) => {
-          const desc = await fetch(
+          const desc = await fetcher(
             `${endPoint}/v1/model_registrations/embedding/${registration.model_name}`,
             {
               method: 'GET',
@@ -85,7 +86,7 @@ const LaunchCustom = ({ gpuAvailable }) => {
 
       const newLLMData = await Promise.all(
         customLLMRegistrations.map(async (registration) => {
-          const desc = await fetch(
+          const desc = await fetcher(
             `${endPoint}/v1/model_registrations/LLM/${registration.model_name}`,
             {
               method: 'GET',
@@ -101,7 +102,7 @@ const LaunchCustom = ({ gpuAvailable }) => {
 
       const newRerankData = await Promise.all(
         customRerankRegistrations.map(async (registration) => {
-          const desc = await fetch(
+          const desc = await fetcher(
             `${endPoint}/v1/model_registrations/rerank/${registration.model_name}`,
             {
               method: 'GET',
diff --git a/xinference/web/ui/src/scenes/launch_model/launchEmbedding.js b/xinference/web/ui/src/scenes/launch_model/launchEmbedding.js
index 9d54c95ebe..fecbbf9eb0 100644
--- a/xinference/web/ui/src/scenes/launch_model/launchEmbedding.js
+++ b/xinference/web/ui/src/scenes/launch_model/launchEmbedding.js
@@ -2,6 +2,7 @@ import { Box, FormControl, TextField } from '@mui/material'
 import React, { useContext, useEffect, useState } from 'react'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 import EmbeddingCard from './embeddingCard'
 
 const LaunchEmbedding = () => {
@@ -31,7 +32,7 @@ const LaunchEmbedding = () => {
     try {
       setIsCallingApi(true)
 
-      const response = await fetch(
+      const response = await fetcher(
         `${endPoint}/v1/model_registrations/embedding?detailed=true`,
         {
           method: 'GET',
@@ -41,7 +42,7 @@ const LaunchEmbedding = () => {
       const registrations = await response.json()
       const newRegistrationData = await Promise.all(
         registrations.map(async (registration) => {
-          const desc = await fetch(
+          const desc = await fetcher(
             `${endPoint}/v1/model_registrations/embedding/${registration.model_name}`,
             {
               method: 'GET',
diff --git a/xinference/web/ui/src/scenes/launch_model/launchLLM.js b/xinference/web/ui/src/scenes/launch_model/launchLLM.js
index 9755c13462..9b3eabc18d 100644
--- a/xinference/web/ui/src/scenes/launch_model/launchLLM.js
+++ b/xinference/web/ui/src/scenes/launch_model/launchLLM.js
@@ -7,19 +7,22 @@ import {
   TextField,
 } from '@mui/material'
 import React, { useContext, useEffect, useState } from 'react'
+import { useCookies } from 'react-cookie'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 import ModelCard from './modelCard'
 
 const LaunchLLM = ({ gpuAvailable }) => {
   let endPoint = useContext(ApiContext).endPoint
-  const [registrationData, setRegistrationData] = useState([])
   const { isCallingApi, setIsCallingApi } = useContext(ApiContext)
   const { isUpdatingModel } = useContext(ApiContext)
+  const { setErrorMsg } = useContext(ApiContext)
+  const [cookie] = useCookies(['token'])
 
+  const [registrationData, setRegistrationData] = useState([])
   // States used for filtering
   const [searchTerm, setSearchTerm] = useState('')
-
   const [modelAbility, setModelAbility] = useState('all')
 
   const handleChange = (event) => {
@@ -53,23 +56,39 @@ const LaunchLLM = ({ gpuAvailable }) => {
     return true
   }
 
-  const update = async () => {
-    if (isCallingApi || isUpdatingModel) return
+  const update = () => {
+    if (
+      isCallingApi ||
+      isUpdatingModel ||
+      cookie.token === '' ||
+      cookie.token === undefined ||
+      cookie.token === 'need_auth'
+    )
+      return
 
     try {
       setIsCallingApi(true)
 
-      const response = await fetch(
-        `${endPoint}/v1/model_registrations/LLM?detailed=true`,
-        {
-          method: 'GET',
+      fetcher(`${endPoint}/v1/model_registrations/LLM?detailed=true`, {
+        method: 'GET',
+      }).then((response) => {
+        if (!response.ok) {
+          response
+            .json()
+            .then((errData) =>
+              setErrorMsg(
+                `Server error: ${response.status} - ${
+                  errData.detail || 'Unknown error'
+                }`
+              )
+            )
+        } else {
+          response.json().then((data) => {
+            const builtinRegistrations = data.filter((v) => v.is_builtin)
+            setRegistrationData(builtinRegistrations)
+          })
         }
-      )
-
-      const registrations = await response.json()
-      const builtinRegistrations = registrations.filter((v) => v.is_builtin)
-
-      setRegistrationData(builtinRegistrations)
+      })
     } catch (error) {
       console.error('Error:', error)
     } finally {
@@ -78,8 +97,8 @@ const LaunchLLM = ({ gpuAvailable }) => {
   }
 
   useEffect(() => {
-    update().catch(console.error)
-  }, [])
+    update()
+  }, [cookie.token])
 
   const style = {
     display: 'grid',
diff --git a/xinference/web/ui/src/scenes/launch_model/launchRerank.js b/xinference/web/ui/src/scenes/launch_model/launchRerank.js
index a342d3c9ca..bb26b629f4 100644
--- a/xinference/web/ui/src/scenes/launch_model/launchRerank.js
+++ b/xinference/web/ui/src/scenes/launch_model/launchRerank.js
@@ -2,6 +2,7 @@ import { Box, FormControl, TextField } from '@mui/material'
 import React, { useContext, useEffect, useState } from 'react'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 import RerankCard from './rerankCard'
 
 const LaunchRerank = () => {
@@ -31,7 +32,7 @@ const LaunchRerank = () => {
     try {
       setIsCallingApi(true)
 
-      const response = await fetch(
+      const response = await fetcher(
         `${endPoint}/v1/model_registrations/rerank?detailed=true`,
         {
           method: 'GET',
@@ -41,7 +42,7 @@ const LaunchRerank = () => {
       const registrations = await response.json()
       const newRegistrationData = await Promise.all(
         registrations.map(async (registration) => {
-          const desc = await fetch(
+          const desc = await fetcher(
             `${endPoint}/v1/model_registrations/rerank/${registration.model_name}`,
             {
               method: 'GET',
diff --git a/xinference/web/ui/src/scenes/launch_model/modelCard.js b/xinference/web/ui/src/scenes/launch_model/modelCard.js
index 8f3224d28a..b4d0330b05 100644
--- a/xinference/web/ui/src/scenes/launch_model/modelCard.js
+++ b/xinference/web/ui/src/scenes/launch_model/modelCard.js
@@ -21,8 +21,10 @@ import {
 import IconButton from '@mui/material/IconButton'
 import Typography from '@mui/material/Typography'
 import React, { useContext, useEffect, useState } from 'react'
+import { useNavigate } from 'react-router-dom'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 
 const CARD_HEIGHT = 380
 const CARD_WIDTH = 300
@@ -33,6 +35,7 @@ const ModelCard = ({ url, modelData, gpuAvailable, is_custom = false }) => {
   const { isCallingApi, setIsCallingApi } = useContext(ApiContext)
   const { isUpdatingModel } = useContext(ApiContext)
   const { setErrorMsg } = useContext(ApiContext)
+  const navigate = useNavigate()
 
   // Model parameter selections
   const [modelUID, setModelUID] = useState('')
@@ -122,8 +125,8 @@ const ModelCard = ({ url, modelData, gpuAvailable, is_custom = false }) => {
         nGPU === '0' ? null : nGPU === 'auto' ? 'auto' : parseInt(nGPU, 10),
     }
 
-    // First fetch request to initiate the model
-    fetch(url + '/v1/models', {
+    // First fetcher request to initiate the model
+    fetcher(url + '/v1/models', {
       method: 'POST',
       headers: {
         'Content-Type': 'application/json',
@@ -141,7 +144,7 @@ const ModelCard = ({ url, modelData, gpuAvailable, is_custom = false }) => {
             )
           })
         } else {
-          window.open(url + '/ui/#/running_models', '_blank', 'noreferrer')
+          navigate('/running_models')
         }
         setIsCallingApi(false)
       })
@@ -281,7 +284,7 @@ const ModelCard = ({ url, modelData, gpuAvailable, is_custom = false }) => {
 
   const handeCustomDelete = (e) => {
     e.stopPropagation()
-    fetch(url + `/v1/model_registrations/LLM/${modelData.model_name}`, {
+    fetcher(url + `/v1/model_registrations/LLM/${modelData.model_name}`, {
       method: 'DELETE',
       headers: {
         'Content-Type': 'application/json',
diff --git a/xinference/web/ui/src/scenes/launch_model/rerankCard.js b/xinference/web/ui/src/scenes/launch_model/rerankCard.js
index a38ff6c39c..f5b8c1d21e 100644
--- a/xinference/web/ui/src/scenes/launch_model/rerankCard.js
+++ b/xinference/web/ui/src/scenes/launch_model/rerankCard.js
@@ -12,6 +12,7 @@ import IconButton from '@mui/material/IconButton'
 import React, { useContext, useEffect, useState } from 'react'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 
 const CARD_HEIGHT = 270
 const CARD_WIDTH = 270
@@ -45,8 +46,8 @@ const RerankCard = ({
       model_type: 'rerank',
     }
 
-    // First fetch request to initiate the model
-    fetch(url + '/v1/models', {
+    // First fetcher request to initiate the model
+    fetcher(url + '/v1/models', {
       method: 'POST',
       headers: {
         'Content-Type': 'application/json',
@@ -190,7 +191,7 @@ const RerankCard = ({
 
   const handeCustomDelete = (e) => {
     e.stopPropagation()
-    fetch(url + `/v1/model_registrations/rerank/${modelData.model_name}`, {
+    fetcher(url + `/v1/model_registrations/rerank/${modelData.model_name}`, {
       method: 'DELETE',
       headers: {
         'Content-Type': 'application/json',
diff --git a/xinference/web/ui/src/scenes/login/header.js b/xinference/web/ui/src/scenes/login/header.js
new file mode 100644
index 0000000000..247951c19a
--- /dev/null
+++ b/xinference/web/ui/src/scenes/login/header.js
@@ -0,0 +1,37 @@
+import { AppBar, Box, Toolbar } from '@mui/material'
+import Typography from '@mui/material/Typography'
+import * as React from 'react'
+
+import icon from '../../media/icon.webp'
+
+export default function Header() {
+  return (
+    <AppBar
+      elevation={0}
+      color="transparent"
+      sx={{
+        backdropFilter: 'blur(20px)',
+        borderBottom: 1,
+        borderColor: 'grey.300',
+        zIndex: (theme) => theme.zIndex.drawer + 1,
+      }}
+    >
+      <Toolbar sx={{ justifyContent: 'start' }}>
+        <Box
+          component="img"
+          alt="profile"
+          src={icon}
+          height="60px"
+          width="60px"
+          borderRadius="50%"
+          sx={{ objectFit: 'cover', mr: 1.5 }}
+        />
+        <Box textAlign="left">
+          <Typography fontWeight="bold" fontSize="1.7rem">
+            {'Xinference'}
+          </Typography>
+        </Box>
+      </Toolbar>
+    </AppBar>
+  )
+}
diff --git a/xinference/web/ui/src/scenes/login/login.js b/xinference/web/ui/src/scenes/login/login.js
new file mode 100644
index 0000000000..05a26b1b93
--- /dev/null
+++ b/xinference/web/ui/src/scenes/login/login.js
@@ -0,0 +1,112 @@
+import { Box } from '@mui/material'
+import Button from '@mui/material/Button'
+import Container from '@mui/material/Container'
+import TextField from '@mui/material/TextField'
+import Typography from '@mui/material/Typography'
+import * as React from 'react'
+import { Fragment, useContext, useState } from 'react'
+import { useCookies } from 'react-cookie'
+import { useNavigate } from 'react-router-dom'
+
+import { ApiContext } from '../../components/apiContext'
+import ErrorMessageSnackBar from '../../components/errorMessageSnackBar'
+import { getEndpoint } from '../../components/utils'
+import Header from './header'
+
+function Login() {
+  const [, setCookie] = useCookies(['token'])
+  const navigate = useNavigate()
+  const [username, setUsername] = useState('')
+  const [password, setPassword] = useState('')
+  const { setErrorMsg } = useContext(ApiContext)
+  const endpoint = getEndpoint()
+
+  const handleSubmit = () => {
+    fetch(endpoint + '/token', {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json',
+      },
+      body: JSON.stringify({
+        username: username,
+        password: password,
+      }),
+    }).then((res) => {
+      if (!res.ok) {
+        res.json().then((errorData) => {
+          setErrorMsg(
+            `Login failed: ${res.status} - ${
+              errorData.detail || 'Unknown error'
+            }`
+          )
+        })
+      } else {
+        res.json().then((data) => {
+          setCookie('token', data['access_token'], { path: '/' })
+          navigate('/')
+        })
+      }
+    })
+  }
+
+  return (
+    <Fragment>
+      <Header />
+      <Container component="main" maxWidth="xl" sx={{ marginTop: 20 }}>
+        <ErrorMessageSnackBar />
+        <Box
+          sx={{
+            marginTop: 8,
+            display: 'flex',
+            flexDirection: 'column',
+            alignItems: 'center',
+          }}
+        >
+          <Typography component="h1" variant="h5">
+            LOGIN
+          </Typography>
+          <Box component="main" noValidate sx={{ mt: 1 }}>
+            <TextField
+              margin="normal"
+              required
+              fullWidth
+              id="username"
+              label="Username"
+              name="username"
+              value={username}
+              onChange={(e) => {
+                setUsername(e.target.value)
+              }}
+              autoFocus
+            />
+            <TextField
+              margin="normal"
+              required
+              fullWidth
+              name="password"
+              label="Password"
+              type="password"
+              id="password"
+              autoComplete="current-password"
+              value={password}
+              onChange={(e) => {
+                setPassword(e.target.value)
+              }}
+            />
+            <Button
+              type="submit"
+              fullWidth
+              variant="contained"
+              sx={{ mt: 3, mb: 2 }}
+              onClick={handleSubmit}
+            >
+              Sign In
+            </Button>
+          </Box>
+        </Box>
+      </Container>
+    </Fragment>
+  )
+}
+
+export default Login
diff --git a/xinference/web/ui/src/scenes/register_model/index.js b/xinference/web/ui/src/scenes/register_model/index.js
index 85dc8c8090..5d8a760368 100644
--- a/xinference/web/ui/src/scenes/register_model/index.js
+++ b/xinference/web/ui/src/scenes/register_model/index.js
@@ -14,9 +14,12 @@ import AlertTitle from '@mui/material/AlertTitle'
 import Button from '@mui/material/Button'
 import TextField from '@mui/material/TextField'
 import React, { useContext, useEffect, useState } from 'react'
+import { useCookies } from 'react-cookie'
+import { useNavigate } from 'react-router-dom'
 
 import { ApiContext } from '../../components/apiContext'
 import ErrorMessageSnackBar from '../../components/errorMessageSnackBar'
+import fetcher from '../../components/fetcher'
 import Title from '../../components/Title'
 import { useMode } from '../../theme'
 import RegisterEmbeddingModel from './register_embedding'
@@ -54,6 +57,8 @@ const RegisterModel = () => {
   })
   const [familyLabel, setFamilyLabel] = useState('')
   const [tabValue, setTabValue] = React.useState('1')
+  const [cookie] = useCookies(['token'])
+  const navigate = useNavigate()
 
   const errorModelName = formData.model_name.trim().length <= 0
   const errorModelDescription = formData.model_description.length < 0
@@ -81,6 +86,14 @@ const RegisterModel = () => {
     errorFamily
 
   useEffect(() => {
+    if (cookie.token === '' || cookie.token === undefined) {
+      return
+    }
+    if (cookie.token === 'need_auth') {
+      navigate('/login', { replace: true })
+      return
+    }
+
     const getBuiltinFamilies = async () => {
       const response = await fetch(endPoint + '/v1/models/families', {
         method: 'GET',
@@ -147,7 +160,7 @@ const RegisterModel = () => {
         console.error('Error: ', error)
       })
     }
-  })
+  }, [cookie.token])
 
   const getFamilyByAbility = () => {
     if (formData.model_ability.includes('chat')) {
@@ -232,7 +245,7 @@ const RegisterModel = () => {
     }
 
     try {
-      const response = await fetch(endPoint + '/v1/model_registrations/LLM', {
+      const response = await fetcher(endPoint + '/v1/model_registrations/LLM', {
         method: 'POST',
         headers: {
           'Content-Type': 'application/json',
diff --git a/xinference/web/ui/src/scenes/register_model/register_embedding.js b/xinference/web/ui/src/scenes/register_model/register_embedding.js
index 29ce0335b5..ac7ab8d4ae 100644
--- a/xinference/web/ui/src/scenes/register_model/register_embedding.js
+++ b/xinference/web/ui/src/scenes/register_model/register_embedding.js
@@ -6,6 +6,7 @@ import TextField from '@mui/material/TextField'
 import React, { useContext, useState } from 'react'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 import { useMode } from '../../theme'
 
 const SUPPORTED_LANGUAGES_DICT = { en: 'English', zh: 'Chinese' }
@@ -41,7 +42,7 @@ const RegisterEmbeddingModel = () => {
     }
 
     try {
-      const response = await fetch(
+      const response = await fetcher(
         endPoint + '/v1/model_registrations/embedding',
         {
           method: 'POST',
diff --git a/xinference/web/ui/src/scenes/register_model/register_rerank.js b/xinference/web/ui/src/scenes/register_model/register_rerank.js
index ed8f7255c9..075b35ff9d 100644
--- a/xinference/web/ui/src/scenes/register_model/register_rerank.js
+++ b/xinference/web/ui/src/scenes/register_model/register_rerank.js
@@ -6,6 +6,7 @@ import TextField from '@mui/material/TextField'
 import React, { useContext, useState } from 'react'
 
 import { ApiContext } from '../../components/apiContext'
+import fetcher from '../../components/fetcher'
 import { useMode } from '../../theme'
 
 const SUPPORTED_LANGUAGES_DICT = { en: 'English', zh: 'Chinese' }
@@ -36,7 +37,7 @@ const RegisterRerankModel = () => {
     }
 
     try {
-      const response = await fetch(
+      const response = await fetcher(
         endPoint + '/v1/model_registrations/rerank',
         {
           method: 'POST',
diff --git a/xinference/web/ui/src/scenes/running_models/index.js b/xinference/web/ui/src/scenes/running_models/index.js
index e87bd57aea..7a755403b3 100644
--- a/xinference/web/ui/src/scenes/running_models/index.js
+++ b/xinference/web/ui/src/scenes/running_models/index.js
@@ -4,8 +4,12 @@ import { TabContext, TabList, TabPanel } from '@mui/lab'
 import { Box, Stack, Tab } from '@mui/material'
 import { DataGrid } from '@mui/x-data-grid'
 import React, { useContext, useEffect, useState } from 'react'
+import { useCookies } from 'react-cookie'
+import { useNavigate } from 'react-router-dom'
 
 import { ApiContext } from '../../components/apiContext'
+import ErrorMessageSnackBar from '../../components/errorMessageSnackBar'
+import fetcher from '../../components/fetcher'
 import Title from '../../components/Title'
 
 const RunningModels = () => {
@@ -16,6 +20,9 @@ const RunningModels = () => {
   const [rerankModelData, setRerankModelData] = useState([])
   const { isCallingApi, setIsCallingApi } = useContext(ApiContext)
   const { isUpdatingModel, setIsUpdatingModel } = useContext(ApiContext)
+  const { setErrorMsg } = useContext(ApiContext)
+  const [cookie] = useCookies(['token'])
+  const navigate = useNavigate()
   const endPoint = useContext(ApiContext).endPoint
 
   const handleTabChange = (event, newValue) => {
@@ -23,6 +30,13 @@ const RunningModels = () => {
   }
 
   const update = (isCallingApi) => {
+    if (cookie.token === '' || cookie.token === undefined) {
+      return
+    }
+    if (cookie.token === 'need_auth') {
+      navigate('/login', { replace: true })
+      return
+    }
     if (isCallingApi) {
       setLlmData([{ id: 'Loading, do not refresh page...', url: 'IS_LOADING' }])
       setEmbeddingModelData([
@@ -36,36 +50,47 @@ const RunningModels = () => {
       ])
     } else {
       setIsUpdatingModel(true)
-      fetch(`${endPoint}/v1/models/`, {
+      fetcher(`${endPoint}/v1/models/`, {
         method: 'GET',
       })
-        .then((response) => response.json())
-        .then((data) => {
-          const newLlmData = []
-          const newEmbeddingModelData = []
-          const newImageModelData = []
-          const newRerankModelData = []
-          Object.entries(data).forEach(([key, value]) => {
-            let newValue = {
-              ...value,
-              id: key,
-              url: key,
-            }
-            if (newValue.model_type === 'LLM') {
-              newLlmData.push(newValue)
-            } else if (newValue.model_type === 'embedding') {
-              newEmbeddingModelData.push(newValue)
-            } else if (newValue.model_type === 'image') {
-              newImageModelData.push(newValue)
-            } else if (newValue.model_type === 'rerank') {
-              newRerankModelData.push(newValue)
-            }
-          })
-          setLlmData(newLlmData)
-          setEmbeddingModelData(newEmbeddingModelData)
-          setImageModelData(newImageModelData)
-          setRerankModelData(newRerankModelData)
-          setIsUpdatingModel(false)
+        .then((response) => {
+          if (!response.ok) {
+            response.json().then((errorData) => {
+              setErrorMsg(
+                `Login failed: ${response.status} - ${
+                  errorData.detail || 'Unknown error'
+                }`
+              )
+            })
+          } else {
+            response.json().then((data) => {
+              const newLlmData = []
+              const newEmbeddingModelData = []
+              const newImageModelData = []
+              const newRerankModelData = []
+              Object.entries(data).forEach(([key, value]) => {
+                let newValue = {
+                  ...value,
+                  id: key,
+                  url: key,
+                }
+                if (newValue.model_type === 'LLM') {
+                  newLlmData.push(newValue)
+                } else if (newValue.model_type === 'embedding') {
+                  newEmbeddingModelData.push(newValue)
+                } else if (newValue.model_type === 'image') {
+                  newImageModelData.push(newValue)
+                } else if (newValue.model_type === 'rerank') {
+                  newRerankModelData.push(newValue)
+                }
+              })
+              setLlmData(newLlmData)
+              setEmbeddingModelData(newEmbeddingModelData)
+              setImageModelData(newImageModelData)
+              setRerankModelData(newRerankModelData)
+              setIsUpdatingModel(false)
+            })
+          }
         })
         .catch((error) => {
           console.error('Error:', error)
@@ -77,7 +102,7 @@ const RunningModels = () => {
   useEffect(() => {
     update(isCallingApi)
     // eslint-disable-next-line
-  }, [isCallingApi])
+  }, [isCallingApi, cookie.token])
 
   const llmColumns = [
     {
@@ -154,14 +179,14 @@ const RunningModels = () => {
 
                 setIsCallingApi(true)
 
-                fetch(openUrl, {
+                fetcher(openUrl, {
                   method: 'HEAD',
                 })
                   .then((response) => {
                     if (response.status === 404) {
                       // If web UI doesn't exist (404 Not Found)
                       console.log('UI does not exist, creating new...')
-                      return fetch(gradioUrl, {
+                      return fetcher(gradioUrl, {
                         method: 'POST',
                         headers: {
                           'Content-Type': 'application/json',
@@ -231,7 +256,7 @@ const RunningModels = () => {
                   return
                 }
                 setIsCallingApi(true)
-                fetch(closeUrl, {
+                fetcher(closeUrl, {
                   method: 'DELETE',
                 })
                   .then((response) => {
@@ -328,7 +353,7 @@ const RunningModels = () => {
                   return
                 }
                 setIsCallingApi(true)
-                fetch(closeUrl, {
+                fetcher(closeUrl, {
                   method: 'DELETE',
                 })
                   .then((response) => {
@@ -414,6 +439,7 @@ const RunningModels = () => {
       }}
     >
       <Title title="Running Models" />
+      <ErrorMessageSnackBar />
       <TabContext value={tabValue}>
         <Box sx={{ borderBottom: 1, borderColor: 'divider' }}>
           <TabList


Code review comments:
diff_hunk: @@ -0,0 +1,225 @@
+# SOME DESCRIPTIVE TITLE.
+# Copyright (C) 2023, Xorbits Inc.
+# This file is distributed under the same license as the Xinference package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, 2024.
+#
+#, fuzzy
+msgid ""
+msgstr ""
+"Project-Id-Version: Xinference \n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2024-01-10 11:33+0800\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language: zh_CN\n"
+"Language-Team: zh_CN <LL@li.org>\n"
+"Plural-Forms: nplurals=1; plural=0;\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=utf-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Generated-By: Babel 2.12.1\n"
+
+#: ../../source/user_guide/auth_system.rst:5
+msgid "Simple OAuth2 System (experimental)"
+msgstr "OAuth2 系统（实验性质）"
+
+#: ../../source/user_guide/auth_system.rst:7
+msgid ""
+"Xinference builds an In-memory OAuth2 authentication and authorization "
+"system using the account-password mode."
+msgstr ""
+"Xinference 使用了账号密码的模式构建了一个基于内存的 OAuth2 的身份验证和授权系统。"
+
+#: ../../source/user_guide/auth_system.rst:10
+msgid ""
+"If you don't have authentication and authorization requirements, you can "
+"use Xinference as before, without any changes."
+msgstr ""
+"如果没有身份验证和授权的要求，可以像之前一样使用 Xinference，无需任何改动。"
+
+#: ../../source/user_guide/auth_system.rst:14
+msgid "Permissions"
+msgstr "权限"
+
+#: ../../source/user_guide/auth_system.rst:15
+msgid ""
+"Currently, Xinference system internally defines some interface "
+"permissions:"
+msgstr ""
+"目前，Xinference 内部定义了以下几个接口权限："
+
+#: ../../source/user_guide/auth_system.rst:17
+msgid "``models:list``: Permission to list models and get models' information."
+msgstr "``models:list``: 获取模型列表和信息的权限。"
+
+#: ../../source/user_guide/auth_system.rst:18
+msgid "``models:read``: Permission to use models."
+msgstr "``models:read``: 使用模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:19
+msgid "``models:register``: Permission to register custom models."
+msgstr "``models:register``: 注册模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:20
+msgid "``models:unregister``: Permission to unregister custom models."
+msgstr "``models:unregister``: 取消注册模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:21
+msgid "``models:start``: Permission to launch models."
+msgstr "``models:start``: 启动模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:22
+msgid "``models:stop``: Permission to stop running models."
+msgstr "``models:stop``: 停止模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:23
+msgid "``admin``: Administrators have permissions for all interfaces."
+msgstr "``admin``: 管理员拥有所有接口的权限。"
+
+#: ../../source/user_guide/auth_system.rst:27
+msgid "Startup"
+msgstr "开始使用"
+
+#: ../../source/user_guide/auth_system.rst:28
+msgid ""
+"All authentication and authorization information needs to be specified "
+"and loaded into memory when Xinference is started. Xinference requires a "
+"JSON-formatted file with the following specific fields:"
+msgstr ""
+"在启动 Xinference 时，需要指定所有的验证和授权信息。当前，Xinference 需要一个"
+" JSON 文件，其中包含以下特定字段："
+
+#: ../../source/user_guide/auth_system.rst:59
+msgid ""
+"``auth_config``: This field is used to configure security-related "
+"information."
+msgstr ""
+"``auth_config``: 这个字段配置与安全相关的信息。"
+
+#: ../../source/user_guide/auth_system.rst:61
+msgid ""
+"``algorithm``: The algorithm used for token generation and parsing. "
+"``HS256`` or ``RS256`` is recommended."
+msgstr "``algorithm``: 用于令牌生成与解析的算法。推荐使用 `HS256`` 或者 ``RS256`` 。"
path: doc/source/locale/zh_CN/LC_MESSAGES/user_guide/auth_system.po
text: @author:
```suggestion
msgstr "``algorithm``: 用于令牌生成与解析的算法。推荐使用 ``HS256`` 或者 ``RS256`` 。"
```

diff_hunk: @@ -0,0 +1,225 @@
+# SOME DESCRIPTIVE TITLE.
+# Copyright (C) 2023, Xorbits Inc.
+# This file is distributed under the same license as the Xinference package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, 2024.
+#
+#, fuzzy
+msgid ""
+msgstr ""
+"Project-Id-Version: Xinference \n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2024-01-10 11:33+0800\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language: zh_CN\n"
+"Language-Team: zh_CN <LL@li.org>\n"
+"Plural-Forms: nplurals=1; plural=0;\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=utf-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Generated-By: Babel 2.12.1\n"
+
+#: ../../source/user_guide/auth_system.rst:5
+msgid "Simple OAuth2 System (experimental)"
+msgstr "OAuth2 系统（实验性质）"
+
+#: ../../source/user_guide/auth_system.rst:7
+msgid ""
+"Xinference builds an In-memory OAuth2 authentication and authorization "
+"system using the account-password mode."
+msgstr ""
+"Xinference 使用了账号密码的模式构建了一个基于内存的 OAuth2 的身份验证和授权系统。"
+
+#: ../../source/user_guide/auth_system.rst:10
+msgid ""
+"If you don't have authentication and authorization requirements, you can "
+"use Xinference as before, without any changes."
+msgstr ""
+"如果没有身份验证和授权的要求，可以像之前一样使用 Xinference，无需任何改动。"
+
+#: ../../source/user_guide/auth_system.rst:14
+msgid "Permissions"
+msgstr "权限"
+
+#: ../../source/user_guide/auth_system.rst:15
+msgid ""
+"Currently, Xinference system internally defines some interface "
+"permissions:"
+msgstr ""
+"目前，Xinference 内部定义了以下几个接口权限："
+
+#: ../../source/user_guide/auth_system.rst:17
+msgid "``models:list``: Permission to list models and get models' information."
+msgstr "``models:list``: 获取模型列表和信息的权限。"
+
+#: ../../source/user_guide/auth_system.rst:18
+msgid "``models:read``: Permission to use models."
+msgstr "``models:read``: 使用模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:19
+msgid "``models:register``: Permission to register custom models."
+msgstr "``models:register``: 注册模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:20
+msgid "``models:unregister``: Permission to unregister custom models."
+msgstr "``models:unregister``: 取消注册模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:21
+msgid "``models:start``: Permission to launch models."
+msgstr "``models:start``: 启动模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:22
+msgid "``models:stop``: Permission to stop running models."
+msgstr "``models:stop``: 停止模型的权限。"
+
+#: ../../source/user_guide/auth_system.rst:23
+msgid "``admin``: Administrators have permissions for all interfaces."
+msgstr "``admin``: 管理员拥有所有接口的权限。"
+
+#: ../../source/user_guide/auth_system.rst:27
+msgid "Startup"
+msgstr "开始使用"
+
+#: ../../source/user_guide/auth_system.rst:28
+msgid ""
+"All authentication and authorization information needs to be specified "
+"and loaded into memory when Xinference is started. Xinference requires a "
+"JSON-formatted file with the following specific fields:"
+msgstr ""
+"在启动 Xinference 时，需要指定所有的验证和授权信息。当前，Xinference 需要一个"
+" JSON 文件，其中包含以下特定字段："
+
+#: ../../source/user_guide/auth_system.rst:59
+msgid ""
+"``auth_config``: This field is used to configure security-related "
+"information."
+msgstr ""
+"``auth_config``: 这个字段配置与安全相关的信息。"
+
+#: ../../source/user_guide/auth_system.rst:61
+msgid ""
+"``algorithm``: The algorithm used for token generation and parsing. "
+"``HS256`` or ``RS256`` is recommended."
+msgstr "``algorithm``: 用于令牌生成与解析的算法。推荐使用 `HS256`` 或者 ``RS256`` 。"
+
+#: ../../source/user_guide/auth_system.rst:63
+msgid ""
+"``secret_key``: The secret_key used for token generation and parsing. Use"
+" this command to generate: ``openssl rand -hex 32``."
+msgstr "``secret_key``: 用于令牌生成和解析的密钥。可以使用该命令生成： ``openssl rand -hex 32`` 。"
+
+#: ../../source/user_guide/auth_system.rst:65
+msgid ""
+"``token_expire_in_minutes``: Reserved field indicating the expiration "
+"time of the token. The current open-source version of Xinference does not"
+" check the expiration time of tokens."
+msgstr ""
+"``token_expire_in_minutes``: 保留字段，表示令牌失效时间。目前 Xinference 开源版本"
+"不会检查令牌过期时间。"
+
+#: ../../source/user_guide/auth_system.rst:67
+msgid ""
+"``user_config``: This field is used to configure user and permission "
+"information. Each user information is composed of these fields:"
+msgstr ""
+"``user_config``: 这个字段用来配置用户和权限信息。每个用户信息由以下字段组成："
+
+#: ../../source/user_guide/auth_system.rst:69
+msgid "``username``: string field for username."
+msgstr "``username``: 字符串，表示用户名"
+
+#: ../../source/user_guide/auth_system.rst:71
+msgid "``password``: string field for password."
+msgstr "``password``: 字符串，表示密码"
+
+#: ../../source/user_guide/auth_system.rst:73
+msgid ""
+"``permissions``: A list containing strings representing the permissions "
+"that this user has. The permissions are described as above."
+msgstr ""
+"``permissions``: 字符串列表，表示该用户拥有的权限。权限描述如上权限部分文档所述。"
+
+#: ../../source/user_guide/auth_system.rst:76
+msgid ""
+"Once you have configured such a JSON file, use the ``--auth-config`` "
+"option to enable Xinference with the authentication and authorization "
+"system. For example, for local startup:"
+msgstr ""
+"配置好这样一个 JSON 文件后，可以使用 ``--auth-config`` 选项启用具有身份验证和"
+"授权系统的 Xinference。例如，本地启动的命令如下所示："
+
+#: ../../source/user_guide/auth_system.rst:83
+msgid ""
+"For distributed startup, just specify this option when starting the "
+"supervisor:"
+msgstr ""
+"在分布式环境下，只需要在启动 supervisor 的是指定这个选项："
path: doc/source/locale/zh_CN/LC_MESSAGES/user_guide/auth_system.po
text: @author:
```suggestion
"在分布式环境下，只需要在启动 supervisor 时指定这个选项："
```