Repository: SeldonIO/MLServer

Issue:
mlserver >= 1.3 doesn't work with MLflow
After the release of `mlserver==1.3.1`, one of our tests in `https://github.com/mlflow/mlflow` started failing with the following error. We're investigating the cause.

https://github.com/mlflow/mlflow/actions/runs/4716614587/jobs/8364498235?pr=8241#step:7:8138

```
2023-04-17 03:00:01,885 [mlserver.parallel] DEBUG - Starting response processing loop...
2023-04-17 03:00:01,894 [mlserver.rest] INFO - HTTP server running on http://0.0.0.0:8080/
2023-04-17 03:00:01,943 [mlserver.metrics] INFO - Metrics server running on http://0.0.0.0:8082/
2023-04-17 03:00:01,944 [mlserver.metrics] INFO - Prometheus scraping endpoint can be accessed on http://0.0.0.0:8082/metrics
2023-04-17 03:00:01,963 [mlserver.grpc] INFO - gRPC server running on http://0.0.0.0:8081/
INFO:     172.17.0.1:59636 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:59644 - "GET /ping HTTP/1.1" 404 Not Found
2023-04-17 03:00:03,801 [mlserver] INFO - Loaded model 'mlflow-model' succesfully.
2023-04-17 03:00:03,802 [mlserver] INFO - Loaded model 'mlflow-model' succesfully.
2023-04-17 03:00:03,804 [mlserver] INFO - Loaded model 'mlflow-model' succesfully.
INFO:     172.17.0.1:59646 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:59660 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:59674 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:59690 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:59702 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:59708 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:59712 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:59718 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:42498 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:42512 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:42528 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:42530 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:42536 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:42544 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:42546 - "GET /ping HTTP/1.1" 404 Not Found
INFO:     172.17.0.1:42552 - "GET /ping HTTP/1.1" 404 Not Found
...
```

The error above indicates that the `/ping` endpoint is not registered.


Discussion:
https://github.com/SeldonIO/MLServer/pull/1044 might be related
Hey @harupy ,

Massive thanks for raising this one!

We'll have a look once we get a chance. It should be possible to replicate just by running the `tests/models/test_cli.py::test_build_docker_with_env_override` test within the MLflow repo, right?
@adriangonz Thanks for the reply! Yes, `pytest 'tests/models/test_cli.py::test_build_docker[True]'` or `pytest 'tests/models/test_cli.py::test_build_docker_with_env_override[True]'` or `pytest 'tests/test_cli.py::test_mlflow_models_serve[True]' `should be able to reproduce the error.
@adriangonz 

Dockerfile that can reproduce the issue:

```dockerfile
FROM python:3.8

WORKDIR /mlflow
RUN git clone --branch master --depth 1 https://github.com/mlflow/mlflow.git /mlflow
RUN pip install -e .
RUN pip install -r requirements/test-requirements.txt
RUN pip install mlserver==1.3.1 mlserver-mlflow==1.3.1
RUN pip list | grep mlserver
RUN pytest 'tests/test_cli.py::test_mlflow_models_serve[True]' -s
```

Resolving patch:
diff --git a/mlserver/parallel/registry.py b/mlserver/parallel/registry.py
--- a/mlserver/parallel/registry.py
+++ b/mlserver/parallel/registry.py
@@ -130,6 +130,12 @@ def model_initialiser(self, model_settings: ModelSettings) -> MLModel:
             # as normal.
             return model_initialiser(model_settings)
 
+        parameters = model_settings.parameters
+        if not parameters or not parameters.environment_tarball:
+            # If model is not using a custom environment, instantiate the model
+            # as normal.
+            return model_initialiser(model_settings)
+
         # Otherwise, return a dummy model for now and wait for the load_model
         # hook to create the actual thing.
         # This avoids instantiating the model's actual class within the


Test patch:
diff --git a/runtimes/alibi-explain/tests/conftest.py b/runtimes/alibi-explain/tests/conftest.py
--- a/runtimes/alibi-explain/tests/conftest.py
+++ b/runtimes/alibi-explain/tests/conftest.py
@@ -115,6 +115,7 @@ async def model_registry(
         on_model_load=[inference_pool_registry.load_model],
         on_model_reload=[inference_pool_registry.reload_model],
         on_model_unload=[inference_pool_registry.unload_model],
+        model_initialiser=inference_pool_registry.model_initialiser,
     )
 
     await model_registry.load(custom_runtime_tf_settings)
diff --git a/runtimes/mlflow/tests/rest/conftest.py b/runtimes/mlflow/tests/rest/conftest.py
--- a/runtimes/mlflow/tests/rest/conftest.py
+++ b/runtimes/mlflow/tests/rest/conftest.py
@@ -71,6 +71,7 @@ async def model_registry(
         on_model_load=[inference_pool_registry.load_model],
         on_model_reload=[inference_pool_registry.reload_model],
         on_model_unload=[inference_pool_registry.unload_model],
+        model_initialiser=inference_pool_registry.model_initialiser,
     )
 
     await model_registry.load(model_settings)
diff --git a/tests/grpc/conftest.py b/tests/grpc/conftest.py
--- a/tests/grpc/conftest.py
+++ b/tests/grpc/conftest.py
@@ -37,6 +37,7 @@ async def model_registry(
         on_model_load=[inference_pool_registry.load_model, load_batching],
         on_model_reload=[inference_pool_registry.reload_model],
         on_model_unload=[inference_pool_registry.unload_model],
+        model_initialiser=inference_pool_registry.model_initialiser,
     )
 
     model_name = sum_model_settings.name
diff --git a/tests/rest/conftest.py b/tests/rest/conftest.py
--- a/tests/rest/conftest.py
+++ b/tests/rest/conftest.py
@@ -22,6 +22,7 @@ async def model_registry(
         on_model_load=[inference_pool_registry.load_model, load_batching],
         on_model_reload=[inference_pool_registry.reload_model],
         on_model_unload=[inference_pool_registry.unload_model],
+        model_initialiser=inference_pool_registry.model_initialiser,
     )
 
     model_name = sum_model_settings.name