Repository: kserve/kserve

Issue:
KServe lgbserver runtime error input for v1 endpoint
/kind bug

**What steps did you take and what happened:**
lgbserver used to support both of following inputs formats prior to 0.11 release.

```python
request = {'inputs' : [{'sepal_width_(cm)': {0: 3.5}, 
'petal_length_(cm)': {0: 1.4}, 
'petal_width_(cm)': {0: 0.2},
'sepal_length_(cm)': {0: 5.1} }]}
```

```python
request2 = {'inputs': [
[{'sepal_width_(cm)': 3.5}, 
 {'petal_length_(cm)': 1.4}, 
 {'petal_width_(cm)': 0.2}, 
 {'sepal_length_(cm)': 5.1}]
 ] }
```

KServe only documented the first input format https://kserve.github.io/website/0.11/modelserving/v1beta1/lightgbm

and in 0.11 the second input format stop working with following error

```bash
2023-11-03 09:06:02.099 32367 kserve ERROR [inference_error_handler():89] Exception:
Traceback (most recent call last):
  File "/Users/dsun20/kserve/python/lgbserver/lgbserver/model.py", line 62, in predict
    result = self._booster.predict(instances)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/lightgbm/basic.py", line 4220, in predict
    return predictor.predict(
           ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/lightgbm/basic.py", line 1047, in predict
    preds, nrow = self.__pred_for_np2d(
                  ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/lightgbm/basic.py", line 1187, in __pred_for_np2d
    return self.__inner_predict_np2d(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/lightgbm/basic.py", line 1127, in __inner_predict_np2d
    data = np.array(mat.reshape(mat.size), dtype=np.float32)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: float() argument must be a string or a real number, not 'dict'
```


**What did you expect to happen:**


**What's the InferenceService yaml:**
[To help us debug please run `kubectl get isvc $name -n $namespace -oyaml` and paste the output]

**Anything else you would like to add:**
[Miscellaneous information that will assist in solving the issue.]


**Environment:**

- Istio Version:
- Knative Version:
- KServe Version:
- Kubeflow version:
- Cloud Environment:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]
- Minikube/Kind version:
- Kubernetes version: (use `kubectl version`):
- OS (e.g. from `/etc/os-release`):



Discussion:

Resolving patch:
diff --git a/python/kserve/kserve/utils/utils.py b/python/kserve/kserve/utils/utils.py
--- a/python/kserve/kserve/utils/utils.py
+++ b/python/kserve/kserve/utils/utils.py
@@ -16,7 +16,7 @@
 import sys
 import uuid
 from kserve.protocol.grpc.grpc_predict_v2_pb2 import InferParameter
-from typing import Dict, Union
+from typing import Dict, Union, List
 
 from kserve.utils.numpy_codec import from_np_dtype
 import pandas as pd
@@ -149,6 +149,24 @@ def get_predict_input(payload: Union[Dict, InferRequest]) -> Union[np.ndarray, p
                 dfs.append(pd.DataFrame(input))
             inputs = pd.concat(dfs, axis=0)
             return inputs
+
+        # Handles the following input format
+        # {'inputs': [
+        #     [{'sepal_width_(cm)': 3.5},
+        #      {'petal_length_(cm)': 1.4},
+        #      {'petal_width_(cm)': 0.2},
+        #      {'sepal_length_(cm)': 5.1}]
+        # ]}
+        elif isinstance(instances[0], List) and len(instances[0]) != 0 and isinstance(instances[0][0], Dict):
+            data: Dict[str, List] = {}
+            for instance in instances:
+                for item in instance:
+                    for key, val in item.items():
+                        if key in data:
+                            data[key].append(val)
+                        else:
+                            data[key] = [val]
+            return pd.DataFrame(data)
         else:
             return np.array(instances)
 


Test patch:
diff --git a/python/lgbserver/lgbserver/test_model.py b/python/lgbserver/lgbserver/test_model.py
--- a/python/lgbserver/lgbserver/test_model.py
+++ b/python/lgbserver/lgbserver/test_model.py
@@ -50,6 +50,17 @@ def test_model():
 
     response = model.predict({"instances": [request, request]})
     assert numpy.argmax(response["predictions"][0]) == 2
+
+    request = [
+        {'sepal_width_(cm)': 3.5}, {'petal_length_(cm)': 1.4},
+        {'petal_width_(cm)': 0.2}, {'sepal_length_(cm)': 5.1}
+    ]
+    response = model.predict({"inputs": [request, request]})
+    assert numpy.argmax(response["predictions"][0]) == 2
+
+    response = model.predict({"instances": [request, request]})
+    assert numpy.argmax(response["predictions"][0]) == 2
+
     # test v2 handler
     infer_input = InferInput(name="input-0", shape=[2, 4], datatype="FP32",
                              data=[[6.8, 2.8, 4.8, 1.6], [6.0, 3.4, 4.5, 1.6]])
diff --git a/test/e2e/data/iris_input_v4.json b/test/e2e/data/iris_input_v4.json
new file mode 100644
--- /dev/null
+++ b/test/e2e/data/iris_input_v4.json
@@ -0,0 +1,10 @@
+{
+    "inputs": [
+        [
+            {"sepal_length_(cm)": 5.1},
+            {"sepal_width_(cm)": 3.5},
+            {"petal_length_(cm)": 1.4},
+            {"petal_width_(cm)": 0.2}
+        ]
+    ]
+}
diff --git a/test/e2e/predictor/test_lightgbm.py b/test/e2e/predictor/test_lightgbm.py
--- a/test/e2e/predictor/test_lightgbm.py
+++ b/test/e2e/predictor/test_lightgbm.py
@@ -91,6 +91,9 @@ def test_lightgbm_runtime_kserve():
 
     res = predict(service_name, "./data/iris_input_v3.json")
     assert res["predictions"][0][0] > 0.5
+
+    res = predict(service_name, "./data/iris_input_v4.json")
+    assert res["predictions"][0][0] > 0.5
     kserve_client.delete(service_name, KSERVE_TEST_NAMESPACE)