Repository: kserve/kserve

Issue:
Broken v0.10.1.1 Python SDK
/kind bug

**What steps did you take and what happened:**
[A clear and concise description of what the bug is.]

1. Install `pip install kserve==0.10.1.1`
2. In the code `models.py`:

<img width="602" alt="image" src="https://github.com/kserve/kserve/assets/91221920/f8e3ad34-9b33-4036-b395-b337c6da9e6f">

<img width="1022" alt="image" src="https://github.com/kserve/kserve/assets/91221920/0dbe3137-027d-4cec-9054-f56c341c6098">

`_http_predict` call will fail because `PREDICTOR_URL_FORMAT` requires 3 positional args, but only 2 are provided.


And the custom model would return the following

```json
{
    "error": "IndexError : Replacement index 2 out of range for positional args tuple"
}
```

Something's off when cherry-pick/publish this release version:

https://github.com/kserve/kserve/blob/release-0.10/python/kserve/kserve/model.py#L37
https://github.com/kserve/kserve/blob/release-0.10/python/kserve/kserve/model.py#L256

**What did you expect to happen:**


**What's the InferenceService yaml:**
[To help us debug please run `kubectl get isvc $name -n $namespace -oyaml` and paste the output]

N/A

**Anything else you would like to add:**
[Miscellaneous information that will assist in solving the issue.]

Suggest re-release a new patch version with correct patch based on v0.10.1

**Environment:**

- Istio Version:
- Knative Version:
- KServe Version: 0.10.1.1
- Kubeflow version:
- Cloud Environment:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]
- Minikube/Kind version:
- Kubernetes version: (use `kubectl version`):
- OS (e.g. from `/etc/os-release`):



Discussion:

Resolving patch:
diff --git a/python/kserve/kserve/model.py b/python/kserve/kserve/model.py
--- a/python/kserve/kserve/model.py
+++ b/python/kserve/kserve/model.py
@@ -34,10 +34,10 @@
                                                 ModelInferResponse)
 from .protocol.infer_type import InferRequest, InferResponse
 
-PREDICTOR_URL_FORMAT = "{0}://{1}/v1/models/{2}:predict"
-EXPLAINER_URL_FORMAT = "{0}://{1}/v1/models/{2}:explain"
-PREDICTOR_V2_URL_FORMAT = "{0}://{1}/v2/models/{2}/infer"
-EXPLAINER_V2_URL_FORMAT = "{0}://{1}/v2/models/{2}/explain"
+PREDICTOR_URL_FORMAT = "http://{0}/v1/models/{1}:predict"
+EXPLAINER_URL_FORMAT = "http://{0}/v1/models/{1}:explain"
+PREDICTOR_V2_URL_FORMAT = "http://{0}/v2/models/{1}/infer"
+EXPLAINER_V2_URL_FORMAT = "http://{0}/v2/models/{1}/explain"
 
 
 class ModelType(Enum):
diff --git a/python/kserve/kserve/model_server.py b/python/kserve/kserve/model_server.py
--- a/python/kserve/kserve/model_server.py
+++ b/python/kserve/kserve/model_server.py
@@ -150,29 +150,28 @@ def start(self, models: Union[List[Model], Dict[str, Deployment]]) -> None:
             concurrent.futures.ThreadPoolExecutor(max_workers=self.max_asyncio_workers))
 
         async def serve():
-            serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-            serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-            serversocket.bind(('0.0.0.0', self.http_port))
-            serversocket.listen(5)
-
             logger.info(f"Starting uvicorn with {self.workers} workers")
             loop = asyncio.get_event_loop()
             for sig in [signal.SIGINT, signal.SIGTERM, signal.SIGQUIT]:
                 loop.add_signal_handler(
                     sig, lambda s=sig: asyncio.create_task(self.stop(sig=s))
                 )
-            self._rest_server = UvicornServer(self.http_port, [serversocket],
-                                              self.dataplane, self.model_repository_extension,
-                                              self.enable_docs_url,
-                                              log_config=self.log_config,
-                                              access_log_format=self.access_log_format)
             if self.workers == 1:
+                self._rest_server = UvicornServer(self.http_port, [],
+                                                  self.dataplane, self.model_repository_extension,
+                                                  self.enable_docs_url,
+                                                  log_config=self.log_config,
+                                                  access_log_format=self.access_log_format)
                 await self._rest_server.run()
             else:
                 # Since py38 MacOS/Windows defaults to use spawn for starting multiprocessing.
                 # https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
                 # Spawn does not work with FastAPI/uvicorn in multiprocessing mode, use fork for multiprocessing
                 # https://github.com/tiangolo/fastapi/issues/1586
+                serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+                serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+                serversocket.bind(('0.0.0.0', self.http_port))
+                serversocket.listen(5)
                 multiprocessing.set_start_method('fork')
                 server = UvicornServer(self.http_port, [serversocket],
                                        self.dataplane, self.model_repository_extension,
diff --git a/python/kserve/kserve/protocol/rest/server.py b/python/kserve/kserve/protocol/rest/server.py
--- a/python/kserve/kserve/protocol/rest/server.py
+++ b/python/kserve/kserve/protocol/rest/server.py
@@ -170,7 +170,7 @@ def run_sync(self):
         asyncio.run(server.serve(sockets=self.sockets))
 
     async def run(self):
-        await self.server.serve(sockets=self.sockets)
+        await self.server.serve()
 
     async def stop(self, sig: Optional[int] = None):
         if self.server:


Test patch:
diff --git a/test/scripts/gh-actions/build-server-runtimes.sh b/test/scripts/gh-actions/build-server-runtimes.sh
--- a/test/scripts/gh-actions/build-server-runtimes.sh
+++ b/test/scripts/gh-actions/build-server-runtimes.sh
@@ -44,33 +44,33 @@ IMAGE_TRANSFORMER_IMG=kserve/image-transformer:${GITHUB_SHA}
 pushd python >/dev/null
   if [[ " ${types[*]} " =~ "predictor" ]]; then
     echo "Building Sklearn image"
-    docker build -t ${SKLEARN_IMG} -f sklearn.Dockerfile .
+    docker buildx build -t ${SKLEARN_IMG} -f sklearn.Dockerfile .
     echo "Building XGB image"
-    docker build -t ${XGB_IMG} -f xgb.Dockerfile .
+    docker buildx build -t ${XGB_IMG} -f xgb.Dockerfile .
     echo "Building LGB image"
-    docker build -t ${LGB_IMG} -f lgb.Dockerfile .
+    docker buildx build -t ${LGB_IMG} -f lgb.Dockerfile .
     echo "Building PMML image"
-    docker build -t ${PMML_IMG} -f pmml.Dockerfile .
+    docker buildx build -t ${PMML_IMG} -f pmml.Dockerfile .
     echo "Building Paddle image"
-    docker build -t ${PADDLE_IMG} -f paddle.Dockerfile .
+    docker buildx build -t ${PADDLE_IMG} -f paddle.Dockerfile .
     echo "Building Custom model gRPC image"
-    docker build -t ${CUSTOM_MODEL_GRPC} -f custom_model_grpc.Dockerfile .
+    docker buildx build -t ${CUSTOM_MODEL_GRPC} -f custom_model_grpc.Dockerfile .
     echo "Building image transformer gRPC image"
-    docker build -t ${CUSTOM_TRANSFORMER_GRPC} -f custom_transformer_grpc.Dockerfile .
+    docker buildx build -t ${CUSTOM_TRANSFORMER_GRPC} -f custom_transformer_grpc.Dockerfile .
   fi
 
   if [[ " ${types[*]} " =~ "explainer" ]]; then
     echo "Building Alibi image"
-    docker build -t ${ALIBI_IMG} -f alibiexplainer.Dockerfile .
+    docker buildx build -t ${ALIBI_IMG} -f alibiexplainer.Dockerfile .
     echo "Building AIX image"
-    docker build -t ${AIX_IMG} -f aixexplainer.Dockerfile .
+    docker buildx build -t ${AIX_IMG} -f aixexplainer.Dockerfile .
     echo "Building ART explainer image"
-    docker build -t ${ART_IMG} -f artexplainer.Dockerfile .
+    docker buildx build -t ${ART_IMG} -f artexplainer.Dockerfile .
   fi
 
   if [[ " ${types[*]} " =~ "transformer" ]]; then
     echo "Building Image transformer image"
-    docker build -t ${IMAGE_TRANSFORMER_IMG} -f custom_transformer.Dockerfile .
+    docker buildx build -t ${IMAGE_TRANSFORMER_IMG} -f custom_transformer.Dockerfile .
   fi
 
 popd