Repository: SeldonIO/MLServer

Issue:
gRPC: Received message larger than max (11059277 vs. 4194304)
We are using mlserver (gRPC) for serving custom PyTorch ML Model and are sending images of size 640x640x3 . But because of the size we are getting the below error:

> status = StatusCode.RESOURCE_EXHAUSTED
> 	details = "Received message larger than max (11059277 vs. 4194304)"
> 	debug_error_string = "{"created":"@1630074151.799330000","description":"Error received from peer ipv4:0.0.0.0:28081","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"Received message larger than max (11059277 vs. 4194304)","grpc_status":8}"

Setting the following on client side did not work:
```
maxMsgLength = 20 * 1024 * 1024
ch_options = [('grpc.max_message_length', maxMsgLength),
              ('grpc.max_send_message_length', maxMsgLength),
              ('grpc.max_receive_message_length', maxMsgLength)]
channel = grpc.insecure_channel(f'{host}:{port}', options=ch_options)

```
However setting it on server side worked:
```
def _create_server(self):
        self._inference_servicer = InferenceServicer(self._data_plane)
        self._model_repository_servicer = ModelRepositoryServicer(
            self._model_repository_handlers
        )
        channel_opt = [('grpc.max_send_message_length', 20 * 1024 * 1024), ('grpc.max_receive_message_length', 20 * 1024 * 1024)]
        self._server = aio.server(
            ThreadPoolExecutor(max_workers=self._settings.grpc_workers), options = channel_opt
        )

        add_GRPCInferenceServiceServicer_to_server(
            self._inference_servicer, self._server
        )

```
But this is not a proper solution, since our mlserver runs in a docker and does a pip install.
This should be part of the server config in MLServer.


Discussion:

Resolving patch:
diff --git a/mlserver/grpc/server.py b/mlserver/grpc/server.py
--- a/mlserver/grpc/server.py
+++ b/mlserver/grpc/server.py
@@ -1,5 +1,6 @@
 from grpc import aio
 from concurrent.futures import ThreadPoolExecutor
+from typing import Any, List, Tuple
 
 from ..handlers import DataPlane, ModelRepositoryHandlers
 from ..settings import Settings
@@ -25,8 +26,10 @@ def _create_server(self):
         self._model_repository_servicer = ModelRepositoryServicer(
             self._model_repository_handlers
         )
+
         self._server = aio.server(
-            ThreadPoolExecutor(max_workers=self._settings.grpc_workers)
+            ThreadPoolExecutor(max_workers=self._settings.grpc_workers),
+            options=self._get_options(),
         )
 
         add_GRPCInferenceServiceServicer_to_server(
@@ -42,6 +45,19 @@ def _create_server(self):
 
         return self._server
 
+    def _get_options(self) -> List[Tuple[str, Any]]:
+        options = []
+
+        max_message_length = self._settings.grpc_max_message_length
+        if max_message_length is not None:
+            options += [
+                ("grpc.max_message_length", max_message_length),
+                ("grpc.max_send_message_length", max_message_length),
+                ("grpc.max_receive_message_length", max_message_length),
+            ]
+
+        return options
+
     async def start(self):
         self._create_server()
 
diff --git a/mlserver/settings.py b/mlserver/settings.py
--- a/mlserver/settings.py
+++ b/mlserver/settings.py
@@ -28,6 +28,7 @@ class Config:
     http_port: int = 8080
     grpc_port: int = 8081
     grpc_workers: int = 10
+    grpc_max_message_length: Optional[int] = None
 
 
 class ModelParameters(BaseSettings):


Test patch: