Repository: jina-ai/serve
Pull request: fix: multi protocol gateway supports monitoring
<!---Decomposing the complex issue into subtasks can help you build it step-by-step. Thanks for your pull request! :rocket: --->
<!---We know that dev life is hectic, but **please provide a (brief) description** of what your PR does, and how it does it. **Otherwise, your PR cannot be reviewed!** --->
<!---This policy was agreed upon in a past company retro, and makes everyone's life a little easier. Thanks for your collaboration!--->

**Goals:**
<!---https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword--->

- resolves #5569 
- [x] code
- [x] add test



Problem statement:
multi protocol gateway does not support monitoring
multi protocol gateway currently does not support monitoring:
`jina gateway --monitoring --protocol GRPC HTTP --port 54321 54322`
results in :
```
TypeError: cannot pickle '_thread.lock' object
```

Basically, the metrics_registry object cannot be copied in jina/serve/runtimes/gateway/composite/gateway.py.
Fix the issue and add tests that enable monitoring in tests/unit/orchestrate/flow/flow-construct/test_flow_multiprotocol.py

Hints from issue discussion:

Commit message: test: add test for multiprotocol with monitoring

Patch to review:
diff --git a/jina/helper.py b/jina/helper.py
index 73695148648fc..0a857a204cf2b 100644
--- a/jina/helper.py
+++ b/jina/helper.py
@@ -54,6 +54,7 @@
     'get_public_ip',
     'get_internal_ip',
     'convert_tuple_to_list',
+    'deepcopy_with_ignore_attrs',
     'run_async',
     'deprecated_alias',
     'retry',
@@ -1265,6 +1266,25 @@ def convert_tuple_to_list(d: Dict):
             convert_tuple_to_list(v)
 
 
+def deepcopy_with_ignore_attrs(
+    obj: Any, ignore_attrs: List[str]
+) -> Any:
+    """Deep copy an object and ignore some attributes
+
+    :param obj: the object to copy
+    :param ignore_attrs: the attributes to ignore
+    :return: the copied object
+    """
+    import copy
+    
+    memo = {}
+    for k in ignore_attrs:
+        if hasattr(obj, k):
+            memo[id(getattr(obj, k))] = None # getattr(obj, k)
+    
+    return copy.deepcopy(obj, memo)
+
+
 def is_jupyter() -> bool:  # pragma: no cover
     """
     Check if we're running in a Jupyter notebook, using magic command `get_ipython` that only available in Jupyter.
diff --git a/jina/serve/runtimes/gateway/composite/gateway.py b/jina/serve/runtimes/gateway/composite/gateway.py
index b93458f9477c0..478de983262f1 100644
--- a/jina/serve/runtimes/gateway/composite/gateway.py
+++ b/jina/serve/runtimes/gateway/composite/gateway.py
@@ -1,6 +1,7 @@
 import copy
 from typing import List, Optional
 
+from jina.helper import deepcopy_with_ignore_attrs
 from jina.serve.gateway import BaseGateway
 
 
@@ -21,10 +22,11 @@ def __init__(
         self.gateways: List[BaseGateway] = []
         for port, protocol in zip(self.ports, self.protocols):
             gateway_cls = _get_gateway_class(protocol)
-            runtime_args = copy.deepcopy(self.runtime_args)
+            # ignore metrics_registry since it is not copyable
+            runtime_args = deepcopy_with_ignore_attrs(self.runtime_args, ['metrics_registry'])
             runtime_args.port = [port]
             runtime_args.protocol = [protocol]
-            gateway_kwargs = copy.deepcopy(kwargs)
+            gateway_kwargs = {k: v for k, v in kwargs.items() if k != 'runtime_args'}
             gateway_kwargs['runtime_args'] = dict(vars(runtime_args))
             gateway = gateway_cls(**gateway_kwargs)
             self.gateways.append(gateway)
diff --git a/tests/unit/orchestrate/flow/flow-construct/test_flow_multiprotocol.py b/tests/unit/orchestrate/flow/flow-construct/test_flow_multiprotocol.py
index 9dbfaf4f99b64..4b7656357c3c6 100644
--- a/tests/unit/orchestrate/flow/flow-construct/test_flow_multiprotocol.py
+++ b/tests/unit/orchestrate/flow/flow-construct/test_flow_multiprotocol.py
@@ -118,3 +118,14 @@ def test_flow_multiprotocol_ports_protocols_mismatch():
         'You need to specify as much protocols as ports if you want to use a jina built-in gateway'
         in err_info.value.args[0]
     )
+
+
+def test_flow_multiprotocol_with_monitoring():
+    ports = [random_port(), random_port(), random_port()]
+    protocols = PROTOCOLS
+    flow = Flow().config_gateway(port=ports, protocol=protocols, monitoring=True)
+    
+    with flow:
+        for port, protocol in zip(ports, protocols):
+            client = Client(port=port, protocol=protocol)
+            client.post('/', inputs=[Document()])


Code review comments:
diff_hunk: @@ -1265,6 +1266,25 @@ def convert_tuple_to_list(d: Dict):
             convert_tuple_to_list(v)
 
 
+def deepcopy_with_ignore_attrs(
path: jina/helper.py
text: @user1:
we need to change it to private method

diff_hunk: @@ -21,10 +22,11 @@ def __init__(
         self.gateways: List[BaseGateway] = []
         for port, protocol in zip(self.ports, self.protocols):
             gateway_cls = _get_gateway_class(protocol)
-            runtime_args = copy.deepcopy(self.runtime_args)
+            # ignore metrics_registry since it is not copyable
+            runtime_args = deepcopy_with_ignore_attrs(self.runtime_args, ['metrics_registry'])
path: jina/serve/runtimes/gateway/composite/gateway.py
text: @user1:
are we sure `deepcopy` is needed?

@user2:
I think we need to copy the runtime_args object, otherwise, how can different gateway objects hold the same reference of `runtime_args` but have different values of port and protocol ?