Repository: kserve/kserve

Issue:
Bug with parsing optional parameters field in .proto
/kind bug

**What steps did you take and what happened:**
The bug is that the new stable version of Kserve `0.10.0` does not parse the optional parameters specifically in [InferRequest](https://github.com/kserve/kserve/blob/01853af81060cac139ea372a8786763f1c220cb1/python/kserve/kserve/protocol/infer_type.py#L237). Hence the parameters sent in the request do not exist anymore in the server side. 

Will write a dummy client and server code to further understand the problem

`client.py`
```
from __future__ import print_function

from kserve.utils.utils import generate_uuid
import json
import grpc
import kserve.protocol.grpc.grpc_predict_v2_pb2 as inference_pb2
import kserve.protocol.grpc.grpc_predict_v2_pb2_grpc as inference_pb2_grpc

def main():
    # Create a gRPC channel
    channel = grpc.insecure_channel('[::]:8081')

    # Create a stub for the gRPC service
    stub = inference_pb2_grpc.GRPCInferenceServiceStub(channel)

    request = inference_pb2.ModelInferRequest(
        model_name="custom-model",
        id=_id,
        parameters={"some_text": inference_pb2.InferParameter(string_param="Some amazing text")},
    )


    # Make the inference request
    response = stub.ModelInfer(request)

```

`server.py`
```


import kserve
from kserve.protocol.grpc.grpc_predict_v2_pb2 import ModelInferRequest, ModelInferResponse
from kserve.utils.utils import generate_uuid

import logging

logger = logging.getLogger(__name__)

class DummyModel(kserve.Model):
    def __init__(self, name: str):
        """
        Initialize the model and load the pre-trained model from the given model_id
        :param name: name of the model
        """
        super().__init__(name)
        self.name = name
        self.model = None
        self.ready = False
        self.load()

    def load(self):
        """
        Load the pre-trained model from the given model_id
        """
        ....STEPS TO LOAD MODEL. ..
        self.model = ...MODEL LOADED
        self.ready = True

    def preprocess(self, payload: ModelInferRequest, headers: Dict[str, str] = None) -> Dict:
        """
        Preprocess the payload and headers, if needed.
        :param payload: payload of the request
        :param headers: headers of the request
        :return: dictionary containing the preprocessed data
        """
        result = {"result": payload}
        return result

    def predict(self, input_dict: Dict, headers: Dict[str, str] = None) -> Dict:
        """
        Perform the prediction and return the response
        :param input_dict: input data in the form of a dictionary
        :param headers: headers of the request
        :return: response containing the generated image and other parameters
        """
        init_time = time.time()
        request_body = input_dict["result"]
        some_text = request_body.parameters["some_text"].string_param
        ...
        ...
        return response


if __name__ == "__main__":
    model = DummyModel("custom-model")
    model.load()
    kserve.ModelServer().start(models=[model])


```
And if you run the above server and send the request using the above client script it would throw an error

You get the following error

```
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNKNOWN
	details = "Unexpected <class 'AttributeError'>: 'InferRequest' object has no attribute 'parameters'"
	debug_error_string = "{"created":"@1676219343.496708127","description":"Error received from peer ipv6:[::]:8081","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Unexpected <class 'AttributeError'>: 'InferRequest' object has no attribute 'parameters'","grpc_status":2}"

```


**What did you expect to happen:**
I expected the parameters attribute to exist it was the case in the kserve version `kserve==0.10.0rc1`



**Anything else you would like to add:**
I think just modifying the [InferInput](https://github.com/kserve/kserve/blob/01853af81060cac139ea372a8786763f1c220cb1/python/kserve/kserve/protocol/infer_type.py#L27) class to parse the parameters should solve the problem

**Environment:**

- Istio Version:
- Knative Version:
- KServe Version: 0.10.0
- Kubeflow version:
- Cloud Environment:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]
- Minikube/Kind version:
- Kubernetes version: (use `kubectl version`):
- OS (e.g. from `/etc/os-release`): Ubuntu 20.04



Discussion:

Resolving patch:
diff --git a/python/kserve/kserve/protocol/infer_type.py b/python/kserve/kserve/protocol/infer_type.py
--- a/python/kserve/kserve/protocol/infer_type.py
+++ b/python/kserve/kserve/protocol/infer_type.py
@@ -30,7 +30,7 @@ class InferInput:
     _datatype: str
     _parameters: Dict
 
-    def __init__(self, name, shape, datatype, data=None):
+    def __init__(self, name, shape, datatype, data=None, parameters={}):
         """An object of InferInput class is used to describe
         input tensor for an inference request.
         Parameters
@@ -47,7 +47,7 @@ def __init__(self, name, shape, datatype, data=None):
         self._name = name
         self._shape = shape
         self._datatype = datatype
-        self._parameters = {}
+        self._parameters = parameters
         self._data = data
         self._raw_data = None
 
@@ -88,6 +88,16 @@ def shape(self):
         """
         return self._shape
 
+    @property
+    def parameters(self):
+        """Get the parameters of input associated with this object.
+        Returns
+        -------
+        dict
+            The key, value pair of string and InferParameter
+        """
+        return self._parameters
+
     def set_shape(self, shape):
         """Set the shape of input.
         Parameters
@@ -220,14 +230,16 @@ class InferRequest:
     """
     id: Optional[str]
     model_name: str
+    parameters: Optional[Dict]
     inputs: List[InferInput]
     from_grpc: bool
 
     def __init__(self, model_name: str, infer_inputs: List[InferInput],
-                 request_id=None, raw_inputs=None, from_grpc=False):
+                 request_id=None, raw_inputs=None, from_grpc=False, parameters={}):
         self.id = request_id
         self.model_name = model_name
         self.inputs = infer_inputs
+        self.parameters = parameters
         self.from_grpc = from_grpc
         if raw_inputs:
             for i, raw_input in enumerate(raw_inputs):
@@ -236,11 +248,12 @@ def __init__(self, model_name: str, infer_inputs: List[InferInput],
     @classmethod
     def from_grpc(cls, request: ModelInferRequest):
         infer_inputs = [InferInput(name=input_tensor.name, shape=list(input_tensor.shape),
-                                   datatype=input_tensor.datatype, data=get_content(input_tensor.datatype,
-                                                                                    input_tensor.contents))
+                                   datatype=input_tensor.datatype,
+                                   data=get_content(input_tensor.datatype, input_tensor.contents),
+                                   parameters=input_tensor.parameters)
                         for input_tensor in request.inputs]
         return cls(request_id=request.id, model_name=request.model_name, infer_inputs=infer_inputs,
-                   raw_inputs=request.raw_input_contents, from_grpc=True)
+                   raw_inputs=request.raw_input_contents, from_grpc=True, parameters=request.parameters)
 
     def to_rest(self) -> Dict:
         """ Converts the InferRequest object to v2 REST InferenceRequest message
@@ -296,7 +309,7 @@ def to_grpc(self) -> ModelInferRequest:
 
 
 class InferOutput:
-    def __init__(self, name, shape, datatype, data=None):
+    def __init__(self, name, shape, datatype, data=None, parameters={}):
         """An object of InferOutput class is used to describe
         input tensor for an inference request.
         Parameters
@@ -313,7 +326,7 @@ def __init__(self, name, shape, datatype, data=None):
         self._name = name
         self._shape = shape
         self._datatype = datatype
-        self._parameters = {}
+        self._parameters = parameters
         self._data = data
         self._raw_data = None
 
@@ -354,6 +367,16 @@ def shape(self):
         """
         return self._shape
 
+    @property
+    def parameters(self):
+        """Get the parameters of input associated with this object.
+        Returns
+        -------
+        dict
+            The key, value pair of string and InferParameter
+        """
+        return self._parameters
+
     def set_shape(self, shape):
         """Set the shape of input.
         Parameters
@@ -466,14 +489,16 @@ class InferResponse:
     """
     id: str
     model_name: str
+    parameters: Optional[Dict]
     outputs: List[InferOutput]
     from_grpc: bool
 
     def __init__(self, response_id: str, model_name: str, infer_outputs: List[InferOutput],
-                 raw_outputs=None, from_grpc=False):
+                 raw_outputs=None, from_grpc=False, parameters={}):
         self.id = response_id
         self.model_name = model_name
         self.outputs = infer_outputs
+        self.parameters = parameters
         self.from_grpc = from_grpc
         if raw_outputs:
             for i, raw_output in enumerate(raw_outputs):
@@ -482,9 +507,11 @@ def __init__(self, response_id: str, model_name: str, infer_outputs: List[InferO
     @classmethod
     def from_grpc(cls, response: ModelInferResponse):
         infer_outputs = [InferOutput(name=output.name, shape=list(output.shape),
-                                     datatype=output.datatype, data=get_content(output.datatype, output.contents))
+                                     datatype=output.datatype,
+                                     data=get_content(output.datatype, output.contents),
+                                     parameters=output.parameters)
                          for output in response.outputs]
-        return cls(model_name=response.model_name, response_id=response.id,
+        return cls(model_name=response.model_name, response_id=response.id, parameters=response.parameters,
                    infer_outputs=infer_outputs, raw_outputs=response.raw_output_contents, from_grpc=True)
 
     def to_rest(self) -> Dict:


Test patch: