Repository: mlflow/mlflow

Issue:
[BUG] `mlflow.tracking._model_registry.utils.get_default_host_creds` ignores configuration file
### Issues Policy acknowledgement

- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)

### Where did you encounter this bug?

Other

### Willingness to contribute

No. I cannot contribute a bug fix at this time.

### MLflow version

- Client: 2.10.2
- Tracking server: 2.10.2


### System information

- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: macOS
- **Python version**: 3.11.7
- **yarn version, if running the dev UI**: n/a


### Describe the problem

I've been trying out mlflow with basic auth using a credentials file at `~/.mlflow/credentials`, [per the docs](https://mlflow.org/docs/latest/auth/index.html#using-credentials-file). This works great for model tracking, however when I tried following the section of the docs to [retrieve my model artifacts](https://mlflow.org/docs/latest/auth/index.html#using-credentials-file), I got an `MlfowException` resulting from a 401 unauthorized error code.

I've tracked the problem down to the [`get_default_host_creds`](https://github.com/mlflow/mlflow/blob/master/mlflow/tracking/_model_registry/utils.py#L148) function which is partialed by `mlflow.store.model_registry.rest_store.RestStore`, at `MlflowClient._registry_client_lazy.store` after the first call to the registry.

The problem occurs because the `get_default_host_creds` function only loads the `MLFLOW_TRACKING_USERNAME` and `MLFLOW_TRACKING_PASSWORD` environment variables, and doesn't look in the config file at all.

### Tracking information

_No response_

### Code to reproduce issue

Setup as follows:

- Running mlflow with `mlflow server --app-name basic-auth`
- Add credentials to `~/.mlflow/credentials`

```
import mlflow.client

client = mlflow.client.MlflowClient(tracking_uri="[tracking URI with basic auth]")
client.search_experiments()                          # Shows that the client is correctly using the config file auth
client.get_model_version("tracking-quickstart", "1") # Raises `MlflowException`
```


### Stack trace

```
---------------------------------------------------------------------------
MlflowException                           Traceback (most recent call last)
Cell In[56], line 1
----> 1 client.get_model_version_download_uri("LightGBM Test", "2")

File ~/.pyenv/versions/3.11.7/envs/mlflow-test-3.11.7/lib/python3.11/site-packages/mlflow/tracking/client.py:3166, in MlflowClient.get_model_version_download_uri(self, name, version)
   3122 def get_model_version_download_uri(self, name: str, version: str) -> str:
   3123     """
   3124     Get the download location in Model Registry for this model version.
   3125 
   (...)
   3164         Download URI: runs:/027d7bbe81924c5a82b3e4ce979fcab7/sklearn-model
   3165     """
-> 3166     return self._get_registry_client().get_model_version_download_uri(name, version)

File ~/.pyenv/versions/3.11.7/envs/mlflow-test-3.11.7/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py:278, in ModelRegistryClient.get_model_version_download_uri(self, name, version)
    270 def get_model_version_download_uri(self, name, version):
    271     """
    272     Get the download location in Model Registry for this model version.
    273 
   (...)
    276     :return: A single URI location that allows reads for downloading.
    277     """
--> 278     return self.store.get_model_version_download_uri(name, version)

File ~/.pyenv/versions/3.11.7/envs/mlflow-test-3.11.7/lib/python3.11/site-packages/mlflow/store/model_registry/rest_store.py:309, in RestStore.get_model_version_download_uri(self, name, version)
    299 """
    300 Get the download location in Model Registry for this model version.
    301 NOTE: For first version of Model Registry, since the models are not copied over to another
   (...)
    306 :return: A single URI location that allows reads for downloading.
    307 """
    308 req_body = message_to_json(GetModelVersionDownloadUri(name=name, version=str(version)))
--> 309 response_proto = self._call_endpoint(GetModelVersionDownloadUri, req_body)
    310 return response_proto.artifact_uri

File ~/.pyenv/versions/3.11.7/envs/mlflow-test-3.11.7/lib/python3.11/site-packages/mlflow/store/model_registry/base_rest_store.py:44, in BaseRestStore._call_endpoint(self, api, json_body, call_all_endpoints, extra_headers)
     42 else:
     43     endpoint, method = self._get_endpoint_from_method(api)
---> 44     return call_endpoint(
     45         self.get_host_creds(), endpoint, method, json_body, response_proto, extra_headers
     46     )

File ~/.pyenv/versions/3.11.7/envs/mlflow-test-3.11.7/lib/python3.11/site-packages/mlflow/utils/rest_utils.py:220, in call_endpoint(host_creds, endpoint, method, json_body, response_proto, extra_headers)
    218     call_kwargs["json"] = json_body
    219     response = http_request(**call_kwargs)
--> 220 response = verify_rest_response(response, endpoint)
    221 js_dict = json.loads(response.text)
    222 parse_dict(js_dict=js_dict, message=response_proto)

File ~/.pyenv/versions/3.11.7/envs/mlflow-test-3.11.7/lib/python3.11/site-packages/mlflow/utils/rest_utils.py:158, in verify_rest_response(response, endpoint)
    153     else:
    154         base_msg = (
    155             f"API request to endpoint {endpoint} "
    156             f"failed with error code {response.status_code} != 200"
    157         )
--> 158         raise MlflowException(
    159             f"{base_msg}. Response body: '{response.text}'",
    160             error_code=get_error_code(response.status_code),
    161         )
    163 # Skip validation for endpoints (e.g. DBFS file-download API) which may return a non-JSON
    164 # response
    165 if endpoint.startswith(_REST_API_PATH_PREFIX) and not _can_parse_as_json_object(response.text):

MlflowException: API request to endpoint /api/2.0/mlflow/model-versions/get-download-uri failed with error code 401 != 200. Response body: 'You are not authenticated. Please see https://www.mlflow.org/docs/latest/auth/index.html#authenticating-to-mlflow on how to authenticate.'
```


### Other info / logs

_No response_

### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/deployments`: MLflow Deployments client APIs, server, and third-party Deployments integrations
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [X] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations

Discussion:
github-actions[bot]: <!-- documentation preview -->


Documentation preview for 1be4054a81517f3bd9c15c6a923be436f3cb5473 will be available when [this CircleCI job](https://circleci.com/gh/mlflow/mlflow/93091)
completes successfully.

- [Top page](https://output.circle-artifacts.com/output/job/8f1ab3c5-e587-4b5e-be0c-7e1efa5c922a/artifacts/0/docs/build/html/index.html)
- [Changed pages](https://output.circle-artifacts.com/output/job/8f1ab3c5-e587-4b5e-be0c-7e1efa5c922a/artifacts/0/docs/build/html/diff.html)

<details>
<summary>More info</summary>

- Ignore this comment if this PR does not change the documentation.
- It takes a few minutes for the preview to be available.
- The preview is updated when a new commit is pushed to this PR.
- This comment was created by https://github.com/mlflow/mlflow/actions/runs/8072885972.

</details>


Patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
index 164007f4f..19b342896 100644
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -360,6 +360,9 @@ nitpick_ignore = [
     ("py:class", "mlflow.types.schema.Object"),
     ("py:class", "mlflow.types.schema.ParamSchema"),
     ("py:class", "mlflow.types.schema.ParamSpec"),
+    ("py:class", "ModelSignature"),
+    ("py:class", "ModelInputExample"),
+    ("py:class", "Model"),
     ("py:class", "mlflow.models.model.Model"),
     ("py:class", "mlflow.models.signature.ModelSignature"),
     ("py:class", "mlflow.metrics.genai.base.EvaluationExample"),
diff --git a/docs/source/llms/transformers/guide/index.rst b/docs/source/llms/transformers/guide/index.rst
index 80b2b8f7b..0bf108d52 100644
--- a/docs/source/llms/transformers/guide/index.rst
+++ b/docs/source/llms/transformers/guide/index.rst
@@ -89,6 +89,61 @@ avoid failed inference requests.
 \***** If using `pyfunc` in MLflow Model Serving for realtime inference, the raw audio in bytes format must be base64 encoded prior to submitting to the endpoint. String inputs will be interpreted as uri locations.
 
 
+Saving Transformer Pipelines with an OpenAI-Compatible Inference Interface
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. note::
+
+    This feature is only available in MLflow 2.11.0 and above.
+
+
+MLflow's native transformers integration allows you to pass in the ``task`` param when saving a model 
+with :py:func:`mlflow.transformers.save_model()` and :py:func:`mlflow.transformers.log_model()`. Originally, this param 
+accepts any of the `Transformers pipeline task types <https://huggingface.co/tasks>`_, but in MLflow 2.11.0
+and above, we've added a few more MLflow-specific keys for ``text-generation`` pipelines.
+
+For ``text-generation`` pipelines, instead of specifying ``text-generation`` as the task type, you can provide 
+one of two string literals conforming to the `MLflow Deployments Server's endpoint_type specification <https://mlflow.org/docs/latest/llms/deployments/index.html#general-configuration-parameters>`_ 
+(``"llm/v1/embeddings"`` can be specified as a task on models saved with :py:func:`mlflow.sentence_transformers.save_model()`):
+
+- ``"llm/v1/chat"`` for chat-style applications
+- ``"llm/v1/completions"`` for generic completions
+
+For example:
+
+.. code-block:: python
+
+    from transformers import pipeline
+
+    import mlflow
+
+    generator = pipeline(
+        "text-generation",
+        "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
+    )
+
+    mlflow.transformers.save_model(
+        path="tinyllama-chat",
+        task="llm/v1/chat",
+        transformers_model=generator,
+    )
+
+When one of these keys is specified, MLflow will automatically handle everything required to serve a chat
+or completions model. This includes:
+
+- Setting a chat/completions compatible signature on the model
+- Performing data pre- and post-processing to ensure the inputs and outputs conform to 
+  the `Chat/Completions API spec <https://mlflow.org/docs/latest/llms/deployments/index.html#chat>`_, 
+  which is compatible with OpenAI's API spec.
+
+Note that these modifications only apply when the model is loaded with :py:func:`mlflow.pyfunc.load_model()` (e.g. when 
+serving the model with the ``mlflow models serve`` CLI tool). If you want to load just the base pipeline, you can
+always do so via :py:func:`mlflow.transformers.load_model()`.
+
+Check out the `notebook tutorial <../tutorials/conversational/pyfunc-chat-model.html>`_ to see this feature in action!
+
+
+
 Saving Prompt Templates with Transformer Pipelines
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
diff --git a/docs/source/llms/transformers/index.rst b/docs/source/llms/transformers/index.rst
index 9a8eff022..1faae1ed0 100644
--- a/docs/source/llms/transformers/index.rst
+++ b/docs/source/llms/transformers/index.rst
@@ -147,6 +147,17 @@ These more advanced tutorials are designed to showcase different applications of
                     </p>
                 </a>
             </div>
+            <div class="simple-card">
+                <a href="tutorials/conversational/pyfunc-chat-model.html">
+                    <div class="header">
+                        Building and Serving an OpenAI-Compatible Chatbot
+                    </div>
+                    <p>
+                        Learn how to build an OpenAI-compatible chatbot using a local Transformers 
+                        model and MLflow, and serve it with minimal configuration.
+                    </p>
+                </a>
+            </div>
             <div class="simple-card">
                 <a href="tutorials/fine-tuning/transformers-fine-tuning.html">
                     <div class="header">
@@ -190,6 +201,7 @@ To download the transformers tutorial notebooks to run in your environment, clic
     <a href="https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/audio-transcription/whisper.ipynb" class="notebook-download-btn">Download the Audio Transcription Notebook</a><br>
     <a href="https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/translation/component-translation.ipynb" class="notebook-download-btn">Download the Translation Notebook</a><br>        
     <a href="https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/conversational/conversational-model.ipynb" class="notebook-download-btn">Download the Chat Conversational Notebook</a><br>
+    <a href="https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/conversational/pyfunc-chat-model.ipynb" class="notebook-download-btn">Download the "Building and Serving an OpenAI-Compatible Chatbot" Notebook</a><br>
     <a href="https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/fine-tuning/transformers-fine-tuning.ipynb" class="notebook-download-btn">Download the Fine Tuning Notebook</a><br>
     <a href="https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/prompt-templating/prompt-templating.ipynb" class="notebook-download-btn">Download the Prompt Templating Notebook</a><br>
     <a href="https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/custom-pyfunc-for-llms/notebooks/custom-pyfunc-advanced-llm.ipynb" class="notebook-download-btn">Download the Custom PyFunc transformers Notebook</a><br>
@@ -202,6 +214,7 @@ To download the transformers tutorial notebooks to run in your environment, clic
     tutorials/audio-transcription/whisper.ipynb
     tutorials/translation/component-translation.ipynb
     tutorials/conversational/conversational-model.ipynb
+    tutorials/conversational/pyfunc-chat-model.ipynb
     tutorials/fine-tuning/transformers-fine-tuning.ipynb
     tutorials/prompt-templating/prompt-templating.ipynb
 
diff --git a/docs/source/llms/transformers/tutorials/conversational/pyfunc-chat-model.ipynb b/docs/source/llms/transformers/tutorials/conversational/pyfunc-chat-model.ipynb
new file mode 100644
index 000000000..1413234ea
--- /dev/null
+++ b/docs/source/llms/transformers/tutorials/conversational/pyfunc-chat-model.ipynb
@@ -0,0 +1,695 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Building and Serving an OpenAI-compatible Chatbot\n",
+    "\n",
+    "Welcome to our tutorial on using Transformers and MLflow to create an OpenAI-compatible chat model. In MLflow 2.11 and up, the [ChatModel](https://mlflow.org/docs/latest/python_api/mlflow.types.html#) class has been added, allowing for convenient creation of served models that conform to the OpenAI API spec. This enables you to seamlessly swap out your chat app's backing LLM, or to easily evaluate different models without having to edit your client-side code.\n",
+    "\n",
+    "If you haven't already seen it, you may find it helpful to go through our [introductory notebook on chat and Transformers](https://mlflow.org/docs/latest/llms/transformers/tutorials/conversational/conversational-model.html) before proceeding with this one, as this notebook is slightly higher-level and does not delve too deeply into the inner workings of Transformers or MLflow Tracking.\n",
+    "\n",
+    "### Learning objectives\n",
+    "\n",
+    "In this tutorial, you will:\n",
+    "\n",
+    "- Create an OpenAI-compatible chat model using TinyLLama-1.1B-Chat\n",
+    "- Serve the model with MLflow Model Serving\n",
+    "- Learn how to use MLflow's `pyfunc` API to add arbitrary customization to your model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "env: TOKENIZERS_PARALLELISM=false\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Disable tokenizers warnings when constructing pipelines\n",
+    "%env TOKENIZERS_PARALLELISM=false\n",
+    "\n",
+    "import warnings\n",
+    "\n",
+    "# Disable a few less-than-useful UserWarnings from setuptools and pydantic\n",
+    "warnings.filterwarnings(\"ignore\", category=UserWarning)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Building a Chat Model\n",
+    "\n",
+    "MLflow's native Transformers integration allows you to specify the `task` param when saving or logging your pipelines. Originally, this param accepts any of the [Transformers pipeline task types](https://huggingface.co/tasks), but the `mlflow.transformers` flavor adds a few more MLflow-specific keys for `text-generation` pipeline types.\n",
+    "\n",
+    "For `text-generation` pipelines, instead of specifying `text-generation` as the task type, you can provide one of two string literals conforming to the [MLflow Deployments Server's endpoint_type specification](https://mlflow.org/docs/latest/llms/deployments/index.html#general-configuration-parameters) (\"llm/v1/embeddings\" can be specified as a task on models saved with `mlflow.sentence_transformers`):\n",
+    "\n",
+    "- \"llm/v1/chat\" for chat-style applications\n",
+    "- \"llm/v1/completions\" for generic completions\n",
+    "\n",
+    "When one of these keys is specified, MLflow will automatically handle everything required to serve a chat or completions model. This includes:\n",
+    "\n",
+    "- Setting a chat/completions compatible signature on the model\n",
+    "- Performing data pre- and post-processing to ensure the inputs and outputs conform to the [Chat/Completions API spec](https://mlflow.org/docs/latest/llms/deployments/index.html#chat), which is compatible with OpenAI's API spec.\n",
+    "\n",
+    "Note that these modifications only apply when the model is loaded with `mlflow.pyfunc.load_model()` (e.g. when serving the model with the `mlflow models serve` CLI tool). If you want to load just the base pipeline, you can always do so via `mlflow.transformers.load_model()`.\n",
+    "\n",
+    "In the next few cells, we'll learn how serve a chat model with a local Transformers pipeline and MLflow, using TinyLlama-1.1B-Chat as an example. \n",
+    "\n",
+    "To begin, let's go through the original flow of saving a text generation pipeline:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 27,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/var/folders/qd/9rwd0_gd0qs65g4sdqlm51hr0000gp/T/ipykernel_55429/4268198845.py:11: FutureWarning: The 'transformers' MLflow Models integration is known to be compatible with the following package version ranges: ``4.25.1`` -  ``4.37.1``. MLflow Models integrations with transformers may not succeed when used with package versions outside of this range.\n",
+      "  mlflow.transformers.save_model(\n"
+     ]
+    }
+   ],
+   "source": [
+    "from transformers import pipeline\n",
+    "\n",
+    "import mlflow\n",
+    "\n",
+    "generator = pipeline(\n",
+    "    \"text-generation\",\n",
+    "    model=\"TinyLlama/TinyLlama-1.1B-Chat-v1.0\",\n",
+    ")\n",
+    "\n",
+    "# save the model using the vanilla `text-generation` task type\n",
+    "mlflow.transformers.save_model(\n",
+    "    path=\"tinyllama-text-generation\", transformers_model=generator, task=\"text-generation\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now, let's load the model and use it for inference. Our loaded model is a `text-generation` pipeline, and let's take a look at its signature to see its expected inputs and outputs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 28,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "5b41063f67ce4e2cac11e68b7d838f55",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024/02/26 21:06:51 WARNING mlflow.transformers: Could not specify device parameter for this pipeline type\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "d1e4fe7d982748e0b81204261de839ab",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/plain": [
+       "inputs: \n",
+       "  [string (required)]\n",
+       "outputs: \n",
+       "  [string (required)]\n",
+       "params: \n",
+       "  None"
+      ]
+     },
+     "execution_count": 28,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# load the model for inference\n",
+    "model = mlflow.pyfunc.load_model(\"tinyllama-text-generation\")\n",
+    "\n",
+    "model.metadata.signature"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Unfortunately, it only accepts `string` as input, which isn't directly compatible with a chat interface. When interacting with OpenAI's API, for example, we expect to simply be able to input a list of messages. In order to do this with our current model, we'll have to write some additional boilerplate:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['
\\nWrite me a hello world program in python
\\n
\\nHere\\'s a simple hello world program in Python:\\n\\n```python\\nprint(\"Hello, world!\")\\n```\\n\\nThis program prints the string \"Hello, world!\" to the console. You can run this program by typing it into the Python interpreter or by running the command `python hello_world.py` in your terminal.']"
+      ]
+     },
+     "execution_count": 29,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# first, apply the tokenizer's chat template, since the\n",
+    "# model is tuned to accept prompts in a chat format. this\n",
+    "# also converts the list of messages to a string.\n",
+    "messages = [{\"role\": \"user\", \"content\": \"Write me a hello world program in python\"}]\n",
+    "prompt = generator.tokenizer.apply_chat_template(\n",
+    "    messages, tokenize=False, add_generation_prompt=True\n",
+    ")\n",
+    "\n",
+    "model.predict(prompt)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we're getting somewhere, but formatting our messages prior to inference is cumbersome. \n",
+    "\n",
+    "Additionally, the output format isn't compatible with the OpenAI API spec either--it's just a list of strings. If we were looking to evaluate different model backends for our chat app, we'd have to rewrite some of our client-side code to both format the input, and to parse this new response.\n",
+    "\n",
+    "To simplify all this, let's just pass in `\"llm/v1/chat\"` as the task param when saving the model."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 30,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/var/folders/qd/9rwd0_gd0qs65g4sdqlm51hr0000gp/T/ipykernel_55429/609241782.py:3: FutureWarning: The 'transformers' MLflow Models integration is known to be compatible with the following package version ranges: ``4.25.1`` -  ``4.37.1``. MLflow Models integrations with transformers may not succeed when used with package versions outside of this range.\n",
+      "  mlflow.transformers.save_model(\n"
+     ]
+    }
+   ],
+   "source": [
+    "# save the model using the `\"llm/v1/chat\"`\n",
+    "# task type instead of `text-generation`\n",
+    "mlflow.transformers.save_model(\n",
+    "    path=\"tinyllama-chat\", transformers_model=generator, task=\"llm/v1/chat\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Once again, let's load the model and inspect the signature:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "e74df03331624bbdb7527cb813ae70bb",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024/02/26 21:10:04 WARNING mlflow.transformers: Could not specify device parameter for this pipeline type\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "d147b02249f5430dbb8780d207ba3bc1",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/plain": [
+       "inputs: \n",
+       "  ['messages': Array({content: string (required), name: string (optional), role: string (required)}) (required), 'temperature': double (optional), 'max_tokens': long (optional), 'stop': Array(string) (optional), 'n': long (optional), 'stream': boolean (optional)]\n",
+       "outputs: \n",
+       "  ['id': string (required), 'object': string (required), 'created': long (required), 'model': string (required), 'choices': Array({finish_reason: string (required), index: long (required), message: {content: string (required), name: string (optional), role: string (required)} (required)}) (required), 'usage': {completion_tokens: long (required), prompt_tokens: long (required), total_tokens: long (required)} (required)]\n",
+       "params: \n",
+       "  None"
+      ]
+     },
+     "execution_count": 31,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "model = mlflow.pyfunc.load_model(\"tinyllama-chat\")\n",
+    "\n",
+    "model.metadata.signature"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now when performing inference, we can pass our messages in a dict as we'd expect to do when interacting with the OpenAI API. Furthermore, the response we receive back from the model also conforms to the spec."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'id': '8435a57d-9895-485e-98d3-95b1cbe007c0',\n",
+       "  'object': 'chat.completion',\n",
+       "  'created': 1708949437,\n",
+       "  'model': 'TinyLlama/TinyLlama-1.1B-Chat-v1.0',\n",
+       "  'usage': {'prompt_tokens': 24, 'completion_tokens': 71, 'total_tokens': 95},\n",
+       "  'choices': [{'index': 0,\n",
+       "    'finish_reason': 'stop',\n",
+       "    'message': {'role': 'assistant',\n",
+       "     'content': 'Here\\'s a simple hello world program in Python:\\n\\n```python\\nprint(\"Hello, world!\")\\n```\\n\\nThis program prints the string \"Hello, world!\" to the console. You can run this program by typing it into the Python interpreter or by running the command `python hello_world.py` in your terminal.'}}]}]"
+      ]
+     },
+     "execution_count": 32,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "messages = [{\"role\": \"user\", \"content\": \"Write me a hello world program in python\"}]\n",
+    "\n",
+    "model.predict({\"messages\": messages})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Serving the Chat Model\n",
+    "\n",
+    "To take this example further, let's use MLflow to serve our chat model, so we can interact with it like a web API. To do this, we can use the `mlflow models serve` CLI tool. \n",
+    "\n",
+    "In a terminal shell, run:\n",
+    "```\n",
+    "$ mlflow models serve -m tinyllama-chat\n",
+    "```\n",
+    "\n",
+    "When the server has finished initializing, you should be able to interact with the model via HTTP requests. The input format is almost identical to the format described in the [MLflow Deployments Server docs](https://mlflow.org/docs/latest/llms/deployments/index.html#chat), with the exception that `temperature` defaults to `1.0` instead of `0.0`.\n",
+    "\n",
+    "Here's a quick example:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
+      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
+      "100   706  100   617  100    89     25      3  0:00:29  0:00:23  0:00:06   160\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[\n",
+      "  {\n",
+      "    \"id\": \"fc3d08c3-d37d-420d-a754-50f77eb32a92\",\n",
+      "    \"object\": \"chat.completion\",\n",
+      "    \"created\": 1708949465,\n",
+      "    \"model\": \"TinyLlama/TinyLlama-1.1B-Chat-v1.0\",\n",
+      "    \"usage\": {\n",
+      "      \"prompt_tokens\": 24,\n",
+      "      \"completion_tokens\": 71,\n",
+      "      \"total_tokens\": 95\n",
+      "    },\n",
+      "    \"choices\": [\n",
+      "      {\n",
+      "        \"index\": 0,\n",
+      "        \"finish_reason\": \"stop\",\n",
+      "        \"message\": {\n",
+      "          \"role\": \"assistant\",\n",
+      "          \"content\": \"Here's a simple hello world program in Python:\\n\\n```python\\nprint(\\\"Hello, world!\\\")\\n```\\n\\nThis program prints the string \\\"Hello, world!\\\" to the console. You can run this program by typing it into the Python interpreter or by running the command `python hello_world.py` in your terminal.\"\n",
+      "        }\n",
+      "      }\n",
+      "    ]\n",
+      "  }\n",
+      "]\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%sh\n",
+    "curl http://127.0.0.1:5000/invocations \\\n",
+    "    -H 'Content-Type: application/json' \\\n",
+    "    -d '{ \"messages\": [{\"role\": \"user\", \"content\": \"Write me a hello world program in python\"}] }' \\\n",
+    "    | jq"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It's that easy!\n",
+    "\n",
+    "You can also call the API with a few optional inference params to adjust the model's responses. These map to Transformers pipeline params, and are passed in directly at inference time.\n",
+    "\n",
+    "- `max_tokens` (maps to `max_new_tokens`): The maximum number of new tokens the model should generate.\n",
+    "- `temperature` (maps to `temperature`): Controls the creativity of the model's response. Note that this is not guaranteed to be supported by all models, and in order for this param to have an effect, the pipeline must have been created with `do_sample=True`.\n",
+    "- `stop` (maps to `stopping_criteria`): A list of tokens at which to stop generation.\n",
+    "\n",
+    "Note: `n` does not have an equivalent Transformers pipeline param, and is not supported in queries. However, you can implement a model that consumes the `n` param using Custom Pyfunc (details below)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Customizing the model\n",
+    "\n",
+    "As always, custom functionality can be achieved with MLflow's `pyfunc` API. In the cell below, we create a custom Chat-flavored `pyfunc` model by subclassing `mlflow.pyfunc.ChatModel`. \n",
+    "\n",
+    "In this example, we'll use our previously-saved TinyLlama pipeline as the backing model by loading it with `mlflow.transformers.load_model()`, and then build our own customizations on top of it. However, `ChatModel` is agnostic to how you generate the outputs. You could use another transformer, Langchain, native OpenAI integrations, or even no LLM at all! As long the result of `predict` is of type `mlflow.types.llm.ChatResponse`, you'll be able to take advantage of the automatic signature generation and input/output parsing.\n",
+    "\n",
+    "The possibilities for customization are endless here, but as a quick example, the code below simply edits the ID of the response, rather than having it be a random UUID. Of course, you could also insert any side-effects you wanted here, such as asynchronously logging some metadata for analytics."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import random\n",
+    "\n",
+    "import mlflow\n",
+    "from mlflow.types.llm import ChatResponse\n",
+    "\n",
+    "\n",
+    "class MyChatModel(mlflow.pyfunc.ChatModel):\n",
+    "    def load_context(self, context):\n",
+    "        # load our previously-saved Transformers pipeline from context.artifacts\n",
+    "        self.pipeline = mlflow.transformers.load_model(context.artifacts[\"chat_model_path\"])\n",
+    "\n",
+    "    def predict(self, context, messages, params):\n",
+    "        tokenizer = self.pipeline.tokenizer\n",
+    "        prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n",
+    "\n",
+    "        # perform inference using the loaded pipeline\n",
+    "        output = self.pipeline(prompt, return_full_text=False, generation_kwargs=params.to_dict())\n",
+    "        text = output[0][\"generated_text\"]\n",
+    "        id = f\"some_meaningful_id_{random.randint(0, 100)}\"\n",
+    "\n",
+    "        # construct token usage information\n",
+    "        prompt_tokens = len(tokenizer.encode(prompt))\n",
+    "        completion_tokens = len(tokenizer.encode(text))\n",
+    "        usage = {\n",
+    "            \"prompt_tokens\": prompt_tokens,\n",
+    "            \"completion_tokens\": completion_tokens,\n",
+    "            \"total_tokens\": prompt_tokens + completion_tokens,\n",
+    "        }\n",
+    "\n",
+    "        # here, we can do any post-processing or side-effects required.\n",
+    "        # for example, we could log the generated text to a database for\n",
+    "        # analytics, or check the output for any banned words or phrases\n",
+    "        # and return a different response if any are found.\n",
+    "\n",
+    "        # in this example, we just return the generated text as the response\n",
+    "\n",
+    "        response = {\n",
+    "            \"id\": id,\n",
+    "            \"model\": \"MyChatModel\",\n",
+    "            \"choices\": [\n",
+    "                {\n",
+    "                    \"index\": 0,\n",
+    "                    \"message\": {\"role\": \"assistant\", \"content\": text},\n",
+    "                    \"finish_reason\": \"stop\",\n",
+    "                }\n",
+    "            ],\n",
+    "            \"usage\": usage,\n",
+    "        }\n",
+    "\n",
+    "        return ChatResponse(**response)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Similar to what happens above, upon saving an instance of `MyChatModel`, MLflow will automatically recognize the `mlflow.pyfunc.ChatModel` subclass, and set chat signatures and handle input and output parsing automatically. Note that enforcement is performed on the output--MLflow will run inference on an example input, and assert that the output is of type `ChatResponse`.\n",
+    "\n",
+    "Full documentation for the `ChatResponse` type can be found in the [API reference](https://mlflow.org/docs/latest/python_api/mlflow.types.html#mlflow.types.llm.ChatResponse)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 36,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024/02/26 21:12:47 INFO mlflow.pyfunc: Predicting on input example to validate output\n",
+      "/var/folders/qd/9rwd0_gd0qs65g4sdqlm51hr0000gp/T/ipykernel_55429/2958668123.py:10: FutureWarning: The 'transformers' MLflow Models integration is known to be compatible with the following package version ranges: ``4.25.1`` -  ``4.37.1``. MLflow Models integrations with transformers may not succeed when used with package versions outside of this range.\n",
+      "  self.pipeline = mlflow.transformers.load_model(context.artifacts[\"chat_model_path\"])\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "cd12a6afe29847bcb79bdb01fc4892bd",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024/02/26 21:12:47 WARNING mlflow.transformers: Could not specify device parameter for this pipeline type\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "8986e18350c44293944a8dc63d5e8324",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "ab96f52af76949b790de16a3567619f3",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Downloading artifacts:   0%|          | 0/19 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024/02/26 21:13:19 INFO mlflow.store.artifact.artifact_repo: The progress bar can be disabled by setting the environment variable MLFLOW_ENABLE_ARTIFACTS_PROGRESS_BAR to false\n"
+     ]
+    }
+   ],
+   "source": [
+    "mlflow.pyfunc.save_model(\n",
+    "    path=\"my-model\",\n",
+    "    python_model=MyChatModel(),\n",
+    "    # provide the path to the pipeline we saved earlier\n",
+    "    artifacts={\"chat_model_path\": \"tinyllama-chat\"},\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As before, we can now serve the model by running the following in a terminal shell:\n",
+    "\n",
+    "```\n",
+    "$ mlflow models serve -m my-model\n",
+    "```\n",
+    "\n",
+    "And we should now be able to query it via HTTP request:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
+      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
+      "100   666  100   577  100    89     23      3  0:00:29  0:00:24  0:00:05   141\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\n",
+      "  \"id\": \"some_meaningful_id_33\",\n",
+      "  \"model\": \"MyChatModel\",\n",
+      "  \"choices\": [\n",
+      "    {\n",
+      "      \"index\": 0,\n",
+      "      \"message\": {\n",
+      "        \"role\": \"assistant\",\n",
+      "        \"content\": \"Here's a simple hello world program in Python:\\n\\n```python\\nprint(\\\"Hello, world!\\\")\\n```\\n\\nThis program prints the string \\\"Hello, world!\\\" to the console. You can run this program by typing it into the Python interpreter or by running the command `python hello_world.py` in your terminal.\"\n",
+      "      },\n",
+      "      \"finish_reason\": \"stop\"\n",
+      "    }\n",
+      "  ],\n",
+      "  \"usage\": {\n",
+      "    \"prompt_tokens\": 25,\n",
+      "    \"completion_tokens\": 71,\n",
+      "    \"total_tokens\": 96\n",
+      "  },\n",
+      "  \"object\": \"chat.completion\",\n",
+      "  \"created\": 1708949708\n",
+      "}\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%sh\n",
+    "curl http://127.0.0.1:5000/invocations \\\n",
+    "    -H 'Content-Type: application/json' \\\n",
+    "    -d '{ \"messages\": [{\"role\": \"user\", \"content\": \"Write me a hello world program in python\"}] }' \\\n",
+    "    | jq"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
+    "\n",
+    "In this tutorial, you learned how to create an OpenAI-compatible chat model by specifying \"llm/v1/chat\" as the task when saving Transformers pipelines. You also learned how to leverage Custom Pyfunc to add customizations that fit your specific use-case.\n",
+    "\n",
+    "### What's next?\n",
+    "\n",
+    "- [In-depth Pyfunc Walkthrough](https://mlflow.org/docs/latest/traditional-ml/creating-custom-pyfunc/index.html). We briefly touched on custom `pyfunc` in this tutorial, but if you're looking for more detail on the anatomy of a `pyfunc` model, the linked page provides an in-depth overview of all the components.\n",
+    "- [More on MLflow Deployments](https://mlflow.org/docs/latest/deployment/index.html). In this tutorial, we saw how to deploy a model using a local server, but MLflow provides many other ways to deploy your models to production. Check out this page to learn more about the different options.\n",
+    "- [More on MLflow's Transformers Integration](https://mlflow.org/docs/latest/llms/transformers/index.html). This page provides a comprehensive overview on MLflow's Transformers integrations, along with lots of hands-on guides and notebooks. Learn how to fine-tune models, use prompt templates, and more!\n",
+    "- [Other LLM Integrations](https://mlflow.org/docs/latest/llms/index.html). Aside from Transformers, MLflow has integrations with many other popular LLM libraries, such as Langchain and OpenAI."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "mlflow-dev",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/docs/source/new-features/index.rst b/docs/source/new-features/index.rst
index fb7b16703..744e37001 100644
--- a/docs/source/new-features/index.rst
+++ b/docs/source/new-features/index.rst
@@ -9,6 +9,28 @@ Find out about the details of major features, changes, and deprecations below.
 
     <section>
         <article class="new-content-grid">
+            <div class="grid-card">
+                <div class="content-container">
+                    <div class="header">
+                        ChatModel Pyfunc Subclass Added
+                    </div>
+                    <img class="card-image" src="../_static/images/logos/mlflow-logo.svg" alt="MLflow"></img>
+                    <div class="body">
+                        <p>
+                        OpenAI-compatible chat models are now easier than ever to build in MLflow! 
+                        <a href="../python_api/mlflow.pyfunc.html#mlflow.pyfunc.ChatModel">ChatModel</a> is a new
+                        Pyfunc subclass that makes it easy to deploy and serve chat models with MLflow.</p>
+                        
+                        <p>
+                        Check out the
+                        <a href="../llms/transformers/tutorials/conversational/pyfunc-chat-model.html">new tutorial</a> 
+                        on building an OpenAI-compatible chat model using TinyLlama-1.1B-Chat!</p>
+                    </div>
+                    <div class="tag">
+                        <a href="https://github.com/mlflow/mlflow/releases/tag/v2.11.0">released in 2.11.0</a>
+                    </div>
+                </div>
+            </div>
             <div class="grid-card">
                 <div class="content-container">
                     <div class="header">
diff --git a/mlflow/deployments/mlflow/__init__.py b/mlflow/deployments/mlflow/__init__.py
index 64e34a87f..42ba91f85 100644
--- a/mlflow/deployments/mlflow/__init__.py
+++ b/mlflow/deployments/mlflow/__init__.py
@@ -20,8 +20,8 @@ from mlflow.environment_variables import (
 )
 from mlflow.protos.databricks_pb2 import BAD_REQUEST
 from mlflow.store.entities.paged_list import PagedList
-from mlflow.tracking._tracking_service.utils import _get_default_host_creds
 from mlflow.utils.annotations import experimental
+from mlflow.utils.credentials import get_default_host_creds
 from mlflow.utils.rest_utils import augmented_raise_for_status, http_request
 from mlflow.utils.uri import join_paths
 
@@ -130,7 +130,7 @@ class MlflowDeploymentClient(BaseDeploymentClient):
             call_kwargs["json"] = json_body
 
         response = http_request(
-            host_creds=_get_default_host_creds(self.target_uri),
+            host_creds=get_default_host_creds(self.target_uri),
             endpoint=route,
             method=method,
             timeout=MLFLOW_HTTP_REQUEST_TIMEOUT.get() if timeout is None else timeout,
diff --git a/mlflow/gateway/client.py b/mlflow/gateway/client.py
index f92ead035..04368e229 100644
--- a/mlflow/gateway/client.py
+++ b/mlflow/gateway/client.py
@@ -22,7 +22,7 @@ from mlflow.gateway.utils import (
 )
 from mlflow.protos.databricks_pb2 import BAD_REQUEST
 from mlflow.store.entities.paged_list import PagedList
-from mlflow.tracking._tracking_service.utils import _get_default_host_creds
+from mlflow.utils.credentials import get_default_host_creds
 from mlflow.utils.databricks_utils import get_databricks_host_creds
 from mlflow.utils.rest_utils import augmented_raise_for_status, http_request
 from mlflow.utils.uri import get_uri_scheme
@@ -61,7 +61,7 @@ class MlflowGatewayClient:
         if self._is_databricks_host():
             return get_databricks_host_creds(self._gateway_uri)
         else:
-            return _get_default_host_creds(self._gateway_uri)
+            return get_default_host_creds(self._gateway_uri)
 
     @property
     def gateway_uri(self):
diff --git a/mlflow/server/auth/client.py b/mlflow/server/auth/client.py
index 86764e20a..4c1119419 100644
--- a/mlflow/server/auth/client.py
+++ b/mlflow/server/auth/client.py
@@ -14,7 +14,7 @@ from mlflow.server.auth.routes import (
     UPDATE_USER_ADMIN,
     UPDATE_USER_PASSWORD,
 )
-from mlflow.tracking._tracking_service.utils import _get_default_host_creds
+from mlflow.utils.credentials import get_default_host_creds
 from mlflow.utils.rest_utils import http_request_safe
 
 
@@ -33,7 +33,7 @@ class AuthServiceClient:
         self.tracking_uri = tracking_uri
 
     def _request(self, endpoint, method, **kwargs):
-        host_creds = _get_default_host_creds(self.tracking_uri)
+        host_creds = get_default_host_creds(self.tracking_uri)
         resp = http_request_safe(host_creds, endpoint, method, **kwargs)
         return resp.json()
 
diff --git a/mlflow/store/artifact/azure_blob_artifact_repo.py b/mlflow/store/artifact/azure_blob_artifact_repo.py
index c5e137141..2e16bdf02 100644
--- a/mlflow/store/artifact/azure_blob_artifact_repo.py
+++ b/mlflow/store/artifact/azure_blob_artifact_repo.py
@@ -14,7 +14,7 @@ from mlflow.entities.multipart_upload import (
 from mlflow.environment_variables import MLFLOW_ARTIFACT_UPLOAD_DOWNLOAD_TIMEOUT
 from mlflow.exceptions import MlflowException
 from mlflow.store.artifact.artifact_repo import ArtifactRepository, MultipartUploadMixin
-from mlflow.tracking._tracking_service.utils import _get_default_host_creds
+from mlflow.utils.credentials import get_default_host_creds
 
 
 def encode_base64(data: Union[str, bytes]) -> str:
@@ -58,14 +58,14 @@ class AzureBlobArtifactRepository(ArtifactRepository, MultipartUploadMixin):
         if "AZURE_STORAGE_CONNECTION_STRING" in os.environ:
             self.client = BlobServiceClient.from_connection_string(
                 conn_str=os.environ.get("AZURE_STORAGE_CONNECTION_STRING"),
-                connection_verify=_get_default_host_creds(artifact_uri).verify,
+                connection_verify=get_default_host_creds(artifact_uri).verify,
             )
         elif "AZURE_STORAGE_ACCESS_KEY" in os.environ:
             account_url = f"https://{account}.{api_uri_suffix}"
             self.client = BlobServiceClient(
                 account_url=account_url,
                 credential=os.environ.get("AZURE_STORAGE_ACCESS_KEY"),
-                connection_verify=_get_default_host_creds(artifact_uri).verify,
+                connection_verify=get_default_host_creds(artifact_uri).verify,
             )
         else:
             try:
@@ -80,7 +80,7 @@ class AzureBlobArtifactRepository(ArtifactRepository, MultipartUploadMixin):
             self.client = BlobServiceClient(
                 account_url=account_url,
                 credential=DefaultAzureCredential(),
-                connection_verify=_get_default_host_creds(artifact_uri).verify,
+                connection_verify=get_default_host_creds(artifact_uri).verify,
             )
 
     @staticmethod
diff --git a/mlflow/store/artifact/http_artifact_repo.py b/mlflow/store/artifact/http_artifact_repo.py
index a44b37824..0d1f21f83 100644
--- a/mlflow/store/artifact/http_artifact_repo.py
+++ b/mlflow/store/artifact/http_artifact_repo.py
@@ -23,7 +23,7 @@ from mlflow.store.artifact.artifact_repo import (
     verify_artifact_path,
 )
 from mlflow.store.artifact.cloud_artifact_repo import _complete_futures, _compute_num_chunks
-from mlflow.tracking._tracking_service.utils import _get_default_host_creds
+from mlflow.utils.credentials import get_default_host_creds
 from mlflow.utils.file_utils import read_chunk, relative_path_to_artifact_path
 from mlflow.utils.mime_type_utils import _guess_mime_type
 from mlflow.utils.rest_utils import augmented_raise_for_status, http_request
@@ -37,7 +37,7 @@ class HttpArtifactRepository(ArtifactRepository, MultipartUploadMixin):
 
     @property
     def _host_creds(self):
-        return _get_default_host_creds(self.artifact_uri)
+        return get_default_host_creds(self.artifact_uri)
 
     def log_artifact(self, local_file, artifact_path=None):
         verify_artifact_path(artifact_path)
@@ -84,7 +84,7 @@ class HttpArtifactRepository(ArtifactRepository, MultipartUploadMixin):
         url, tail = self.artifact_uri.split(endpoint, maxsplit=1)
         root = tail.lstrip("/")
         params = {"path": posixpath.join(root, path) if path else root}
-        host_creds = _get_default_host_creds(url)
+        host_creds = get_default_host_creds(url)
         resp = http_request(host_creds, endpoint, "GET", params=params)
         augmented_raise_for_status(resp)
         file_infos = []
@@ -115,7 +115,7 @@ class HttpArtifactRepository(ArtifactRepository, MultipartUploadMixin):
 
     def create_multipart_upload(self, local_file, num_parts=1, artifact_path=None):
         url, _ = self.artifact_uri.split("/mlflow-artifacts", maxsplit=1)
-        host_creds = _get_default_host_creds(url)
+        host_creds = get_default_host_creds(url)
         base_endpoint = "/mlflow-artifacts/mpu/create"
         endpoint = posixpath.join(base_endpoint, artifact_path) if artifact_path else base_endpoint
         params = {
@@ -128,7 +128,7 @@ class HttpArtifactRepository(ArtifactRepository, MultipartUploadMixin):
 
     def complete_multipart_upload(self, local_file, upload_id, parts=None, artifact_path=None):
         url, _ = self.artifact_uri.split("/mlflow-artifacts", maxsplit=1)
-        host_creds = _get_default_host_creds(url)
+        host_creds = get_default_host_creds(url)
         base_endpoint = "/mlflow-artifacts/mpu/complete"
         endpoint = posixpath.join(base_endpoint, artifact_path) if artifact_path else base_endpoint
         params = {
@@ -141,7 +141,7 @@ class HttpArtifactRepository(ArtifactRepository, MultipartUploadMixin):
 
     def abort_multipart_upload(self, local_file, upload_id, artifact_path=None):
         url, _ = self.artifact_uri.split("/mlflow-artifacts", maxsplit=1)
-        host_creds = _get_default_host_creds(url)
+        host_creds = get_default_host_creds(url)
         base_endpoint = "/mlflow-artifacts/mpu/abort"
         endpoint = posixpath.join(base_endpoint, artifact_path) if artifact_path else base_endpoint
         params = {
diff --git a/mlflow/tensorflow/__init__.py b/mlflow/tensorflow/__init__.py
index 182e1b9ab..06775851d 100644
--- a/mlflow/tensorflow/__init__.py
+++ b/mlflow/tensorflow/__init__.py
@@ -121,9 +121,9 @@ def get_global_custom_objects():
         A live reference to the global dictionary of custom objects.
     """
     try:
-        from tensorflow import keras
+        from tensorflow.keras.saving import get_custom_objects
 
-        return keras.saving.get_custom_objects()
+        return get_custom_objects()
     except Exception:
         pass
 
diff --git a/mlflow/tracking/_model_registry/utils.py b/mlflow/tracking/_model_registry/utils.py
index 077457198..127e77b20 100644
--- a/mlflow/tracking/_model_registry/utils.py
+++ b/mlflow/tracking/_model_registry/utils.py
@@ -1,17 +1,6 @@
-import logging
 from functools import partial
 
-from mlflow.environment_variables import (
-    MLFLOW_REGISTRY_URI,
-    MLFLOW_TRACKING_AUTH,
-    MLFLOW_TRACKING_AWS_SIGV4,
-    MLFLOW_TRACKING_CLIENT_CERT_PATH,
-    MLFLOW_TRACKING_INSECURE_TLS,
-    MLFLOW_TRACKING_PASSWORD,
-    MLFLOW_TRACKING_SERVER_CERT_PATH,
-    MLFLOW_TRACKING_TOKEN,
-    MLFLOW_TRACKING_USERNAME,
-)
+from mlflow.environment_variables import MLFLOW_REGISTRY_URI
 from mlflow.store.db.db_types import DATABASE_ENGINES
 from mlflow.store.model_registry.databricks_workspace_model_registry_rest_store import (
     DatabricksWorkspaceModelRegistryRestStore,
@@ -23,17 +12,14 @@ from mlflow.tracking._tracking_service.utils import (
     _resolve_tracking_uri,
     get_tracking_uri,
 )
-from mlflow.utils import rest_utils
 from mlflow.utils._spark_utils import _get_active_spark_session
+from mlflow.utils.credentials import get_default_host_creds
 from mlflow.utils.databricks_utils import (
     get_databricks_host_creds,
     warn_on_deprecated_cross_workspace_registry_uri,
 )
 from mlflow.utils.uri import _DATABRICKS_UNITY_CATALOG_SCHEME
 
-_logger = logging.getLogger(__name__)
-
-
 # NOTE: in contrast to tracking, we do not support the following ways to specify
 # the model registry URI:
 #  - via environment variables like MLFLOW_TRACKING_URI, MLFLOW_TRACKING_USERNAME, ...
@@ -145,20 +131,6 @@ def _get_sqlalchemy_store(store_uri):
     return SqlAlchemyStore(store_uri)
 
 
-def get_default_host_creds(store_uri):
-    return rest_utils.MlflowHostCreds(
-        host=store_uri,
-        username=MLFLOW_TRACKING_USERNAME.get(),
-        password=MLFLOW_TRACKING_PASSWORD.get(),
-        token=MLFLOW_TRACKING_TOKEN.get(),
-        aws_sigv4=MLFLOW_TRACKING_AWS_SIGV4.get(),
-        auth=MLFLOW_TRACKING_AUTH.get(),
-        ignore_tls_verification=MLFLOW_TRACKING_INSECURE_TLS.get(),
-        client_cert_path=MLFLOW_TRACKING_CLIENT_CERT_PATH.get(),
-        server_cert_path=MLFLOW_TRACKING_SERVER_CERT_PATH.get(),
-    )
-
-
 def _get_rest_store(store_uri, **_):
     return RestStore(partial(get_default_host_creds, store_uri))
 
diff --git a/mlflow/tracking/_tracking_service/utils.py b/mlflow/tracking/_tracking_service/utils.py
index 1b7921f5c..696ff271b 100644
--- a/mlflow/tracking/_tracking_service/utils.py
+++ b/mlflow/tracking/_tracking_service/utils.py
@@ -5,22 +5,13 @@ from functools import partial
 from pathlib import Path
 from typing import Union
 
-from mlflow.environment_variables import (
-    MLFLOW_TRACKING_AUTH,
-    MLFLOW_TRACKING_AWS_SIGV4,
-    MLFLOW_TRACKING_CLIENT_CERT_PATH,
-    MLFLOW_TRACKING_INSECURE_TLS,
-    MLFLOW_TRACKING_SERVER_CERT_PATH,
-    MLFLOW_TRACKING_TOKEN,
-    MLFLOW_TRACKING_URI,
-)
+from mlflow.environment_variables import MLFLOW_TRACKING_URI
 from mlflow.store.db.db_types import DATABASE_ENGINES
 from mlflow.store.tracking import DEFAULT_LOCAL_FILE_AND_ARTIFACT_PATH
 from mlflow.store.tracking.file_store import FileStore
 from mlflow.store.tracking.rest_store import RestStore
 from mlflow.tracking._tracking_service.registry import TrackingStoreRegistry
-from mlflow.utils import rest_utils
-from mlflow.utils.credentials import read_mlflow_creds
+from mlflow.utils.credentials import get_default_host_creds
 from mlflow.utils.databricks_utils import get_databricks_host_creds
 from mlflow.utils.file_utils import path_to_local_file_uri
 from mlflow.utils.uri import _DATABRICKS_UNITY_CATALOG_SCHEME
@@ -138,23 +129,8 @@ def _get_sqlalchemy_store(store_uri, artifact_uri):
     return SqlAlchemyStore(store_uri, artifact_uri)
 
 
-def _get_default_host_creds(store_uri):
-    creds = read_mlflow_creds()
-    return rest_utils.MlflowHostCreds(
-        host=store_uri,
-        username=creds.username,
-        password=creds.password,
-        token=MLFLOW_TRACKING_TOKEN.get(),
-        aws_sigv4=MLFLOW_TRACKING_AWS_SIGV4.get(),
-        auth=MLFLOW_TRACKING_AUTH.get(),
-        ignore_tls_verification=MLFLOW_TRACKING_INSECURE_TLS.get(),
-        client_cert_path=MLFLOW_TRACKING_CLIENT_CERT_PATH.get(),
-        server_cert_path=MLFLOW_TRACKING_SERVER_CERT_PATH.get(),
-    )
-
-
 def _get_rest_store(store_uri, **_):
-    return RestStore(partial(_get_default_host_creds, store_uri))
+    return RestStore(partial(get_default_host_creds, store_uri))
 
 
 def _get_databricks_rest_store(store_uri, **_):
diff --git a/mlflow/utils/credentials.py b/mlflow/utils/credentials.py
index a19044f6a..0a01bf065 100644
--- a/mlflow/utils/credentials.py
+++ b/mlflow/utils/credentials.py
@@ -4,8 +4,18 @@ import logging
 import os
 from typing import NamedTuple, Optional, Tuple
 
-from mlflow.environment_variables import MLFLOW_TRACKING_PASSWORD, MLFLOW_TRACKING_USERNAME
+from mlflow.environment_variables import (
+    MLFLOW_TRACKING_AUTH,
+    MLFLOW_TRACKING_AWS_SIGV4,
+    MLFLOW_TRACKING_CLIENT_CERT_PATH,
+    MLFLOW_TRACKING_INSECURE_TLS,
+    MLFLOW_TRACKING_PASSWORD,
+    MLFLOW_TRACKING_SERVER_CERT_PATH,
+    MLFLOW_TRACKING_TOKEN,
+    MLFLOW_TRACKING_USERNAME,
+)
 from mlflow.exceptions import MlflowException
+from mlflow.utils.rest_utils import MlflowHostCreds
 
 _logger = logging.getLogger(__name__)
 
@@ -48,6 +58,21 @@ def read_mlflow_creds() -> MlflowCreds:
     )
 
 
+def get_default_host_creds(store_uri):
+    creds = read_mlflow_creds()
+    return MlflowHostCreds(
+        host=store_uri,
+        username=creds.username,
+        password=creds.password,
+        token=MLFLOW_TRACKING_TOKEN.get(),
+        aws_sigv4=MLFLOW_TRACKING_AWS_SIGV4.get(),
+        auth=MLFLOW_TRACKING_AUTH.get(),
+        ignore_tls_verification=MLFLOW_TRACKING_INSECURE_TLS.get(),
+        client_cert_path=MLFLOW_TRACKING_CLIENT_CERT_PATH.get(),
+        server_cert_path=MLFLOW_TRACKING_SERVER_CERT_PATH.get(),
+    )
+
+
 def login(backend: str = "databricks", interactive: bool = True) -> None:
     """Configure MLflow server authentication and connect MLflow to tracking server.
 


Test patch:
diff --git a/tests/store/artifact/test_http_artifact_repo.py b/tests/store/artifact/test_http_artifact_repo.py
index dc23c6d1b..b1d096857 100644
--- a/tests/store/artifact/test_http_artifact_repo.py
+++ b/tests/store/artifact/test_http_artifact_repo.py
@@ -22,7 +22,7 @@ from mlflow.environment_variables import (
 from mlflow.exceptions import MlflowException
 from mlflow.store.artifact.artifact_repository_registry import get_artifact_repository
 from mlflow.store.artifact.http_artifact_repo import HttpArtifactRepository
-from mlflow.tracking._tracking_service.utils import _get_default_host_creds
+from mlflow.utils.credentials import get_default_host_creds
 from mlflow.utils.rest_utils import MlflowHostCreds
 
 
@@ -201,7 +201,7 @@ def test_list_artifacts(http_artifact_repo):
         endpoint = "/mlflow-artifacts/artifacts"
         url, _ = http_artifact_repo.artifact_uri.split(endpoint, maxsplit=1)
         mock_get.assert_called_once_with(
-            _get_default_host_creds(url),
+            get_default_host_creds(url),
             endpoint,
             "GET",
             params={"path": ""},
@@ -435,7 +435,7 @@ def test_complete_multipart_upload(http_artifact_repo, monkeypatch):
         endpoint = "/mlflow-artifacts"
         url, _ = http_artifact_repo.artifact_uri.split(endpoint, maxsplit=1)
         mock_post.assert_called_once_with(
-            _get_default_host_creds(url),
+            get_default_host_creds(url),
             "/mlflow-artifacts/mpu/complete/artifact/path",
             "POST",
             json={
@@ -463,7 +463,7 @@ def test_abort_multipart_upload(http_artifact_repo, monkeypatch):
         endpoint = "/mlflow-artifacts"
         url, _ = http_artifact_repo.artifact_uri.split(endpoint, maxsplit=1)
         mock_post.assert_called_once_with(
-            _get_default_host_creds(url),
+            get_default_host_creds(url),
             "/mlflow-artifacts/mpu/abort/artifact/path",
             "POST",
             json={
diff --git a/tests/store/artifact/test_mlflow_artifact_repo.py b/tests/store/artifact/test_mlflow_artifact_repo.py
index 1eb851b94..e081bc233 100644
--- a/tests/store/artifact/test_mlflow_artifact_repo.py
+++ b/tests/store/artifact/test_mlflow_artifact_repo.py
@@ -7,7 +7,7 @@ import pytest
 from mlflow.exceptions import MlflowException
 from mlflow.store.artifact.artifact_repository_registry import get_artifact_repository
 from mlflow.store.artifact.mlflow_artifacts_repo import MlflowArtifactsRepository
-from mlflow.tracking._tracking_service.utils import _get_default_host_creds
+from mlflow.utils.credentials import get_default_host_creds
 
 
 @pytest.fixture(scope="module", autouse=True)
@@ -200,7 +200,7 @@ def test_list_artifacts(mlflow_artifact_repo):
         endpoint = "/mlflow-artifacts/artifacts"
         url, _ = mlflow_artifact_repo.artifact_uri.split(endpoint, maxsplit=1)
         mock_get.assert_called_once_with(
-            _get_default_host_creds(url),
+            get_default_host_creds(url),
             endpoint,
             "GET",
             params={"path": ""},