Repository: mlflow/mlflow

Issue:
[BUG] MLFlow served model fails to handle request with http url inside.
Thank you for submitting an issue. Please refer to our [issue policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md) for additional information about bug reports. For help with debugging your code, please refer to [Stack Overflow](https://stackoverflow.com/questions/tagged/mlflow).

**Please fill in this bug report template to ensure a timely and thorough response.**

### Willingness to contribute
The MLflow Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the MLflow code base?

- [ ] Yes. I can contribute a fix for this bug independently.
- [ ] Yes. I would be willing to contribute a fix for this bug with guidance from the MLflow community.
- [x] No. I cannot contribute a bug fix at this time.

### System information
pandas==1.1.4

### Describe the problem
Describe the problem clearly here. Include descriptions of the expected behavior and the actual behavior.
When a request with `http://url` in data, pandas will try to load data from the given url.

### Code to reproduce issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
```bash
curl -x POST -H 'ContentType: application/json' -d "{\"data\":[[\"http://google.com\"]],\"columns\":[\"f\",]}" $A_MLFLOW_SERVED_MODEL
```

### Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
```bash
  File "/miniconda3/envs/test/lib/python3.7/site-packages/pandas/util/_decorators.py", line 199, in wrapper
    return func(*args, **kwargs)
  File "/miniconda3/envs/test/lib/python3.7/site-packages/pandas/util/_decorators.py", line 296, in wrapper
    return func(*args, **kwargs)
  File "/miniconda3/envs/test/lib/python3.7/site-packages/pandas/io/json/_json.py", line 594, in read_json
    path_or_buf, encoding=encoding, compression=compression
  File "/miniconda3/envs/test/lib/python3.7/site-packages/pandas/io/common.py", line 203, in get_filepath_or_buffer
    fsspec = import_optional_dependency("fsspec")
  File "/miniconda3/envs/test/lib/python3.7/site-packages/pandas/compat/_optional.py", line 110, in import_optional_dependency
    raise ImportError(msg) from None
ImportError: Missing optional dependency 'fsspec'.  Use pip or conda to install fsspec.
```

### What component(s), interfaces, languages, and integrations does this bug affect?
Components 
- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/projects`: MLproject format, project running backends
- [x] `area/scoring`: Local serving, model deployment tools, spark UDFs
- [ ] `area/server-infra`: MLflow server, JavaScript dev server
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

Interface 
- [ ] `area/uiux`: Front-end, user experience, JavaScript, plotting
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

Language 
- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

Integrations
- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations



Discussion:
Seems like an issue for pandas: https://github.com/pandas-dev/pandas/issues/36271

Resolving patch:
diff --git a/mlflow/gluon.py b/mlflow/gluon.py
--- a/mlflow/gluon.py
+++ b/mlflow/gluon.py
@@ -1,6 +1,7 @@
 from distutils.version import LooseVersion
 import os
 
+import numpy as np
 import pandas as pd
 import yaml
 
@@ -76,17 +77,33 @@ class _GluonModelWrapper:
     def __init__(self, gluon_model):
         self.gluon_model = gluon_model
 
-    def predict(self, df):
+    def predict(self, data):
         """
-        :param df: A Pandas DataFrame containing input array values. A DataFrame input,
-                   `df` is converted to an MXNet ndarray via `ndarray = mx.nd.array(df.values)`.
-        :return: A Pandas DataFrame containing output array values. The underlying MXNet array
-                 can be extracted from the output DataFrame as `ndarray = mx.nd.array(df.values)`.
+        :param data: Either a pandas DataFrame or a numpy array containing input array values.
+                     If the input is a DataFrame, it will be converted to an array first by a
+                     `ndarray = df.values`.
+        :return: Model predictions. If the input is a pandas.DataFrame, the predictions are returned
+                 in a pandas.DataFrame. If the input is a numpy array, the predictions are returned
+                 as either a numpy.ndarray or a plain list for hybrid models.
+
+
         """
         import mxnet as mx
 
-        ndarray = mx.nd.array(df.values)
-        return pd.DataFrame(self.gluon_model(ndarray).asnumpy())
+        if isinstance(data, pd.DataFrame):
+            ndarray = mx.nd.array(data.values)
+            preds = self.gluon_model(ndarray)
+            if isinstance(preds, mx.ndarray.ndarray.NDArray):
+                preds = preds.asnumpy()
+            return pd.DataFrame(preds)
+        elif isinstance(data, np.ndarray):
+            ndarray = mx.nd.array(data)
+            preds = self.gluon_model(ndarray)
+            if isinstance(preds, mx.ndarray.ndarray.NDArray):
+                preds = preds.asnumpy()
+            return preds
+        else:
+            raise TypeError("Input data should be pandas.DataFrame or numpy.ndarray")
 
 
 def _load_pyfunc(path):
diff --git a/mlflow/onnx.py b/mlflow/onnx.py
--- a/mlflow/onnx.py
+++ b/mlflow/onnx.py
@@ -161,58 +161,92 @@ def __init__(self, path):
         self.inputs = [(inp.name, inp.type) for inp in self.rt.get_inputs()]
         self.output_names = [outp.name for outp in self.rt.get_outputs()]
 
-    @staticmethod
-    def _cast_float64_to_float32(dataframe, column_names):
-        for input_name in column_names:
-            if dataframe[input_name].values.dtype == np.float64:
-                dataframe[input_name] = dataframe[input_name].values.astype(np.float32)
-        return dataframe
+    def _cast_float64_to_float32(self, feeds):
+        for input_name, input_type in self.inputs:
+            if input_type == "tensor(float)":
+                feed = feeds.get(input_name)
+                if feed is not None and feed.dtype == np.float64:
+                    feeds[input_name] = feed.astype(np.float32)
+        return feeds
 
     @experimental
-    def predict(self, dataframe):
+    def predict(self, data):
         """
-        :param dataframe: A Pandas DataFrame that is converted to a collection of ONNX Runtime
-                          inputs. If the underlying ONNX model only defines a *single* input
-                          tensor, the DataFrame's values are converted to a NumPy array
-                          representation using the `DataFrame.values()
-                          <https://pandas.pydata.org/pandas-docs/stable/reference/api/
-                          pandas.DataFrame.values.html#pandas.DataFrame.values>`_ method. If the
-                          underlying ONNX model defines *multiple* input tensors, each column
+        :param data: Either a pandas DataFrame, numpy.ndarray or a dictionary.
+
+                     Dictionary input is expected to be a valid ONNX model feed dictionary.
+
+                     Numpy array input is supported iff the model has a single tensor input and is
+                     converted into an ONNX feed dictionary with the appropriate key.
+
+                     Pandas DataFrame is converted to ONNX inputs as follows:
+                        - If the underlying ONNX model only defines a *single* input tensor, the
+                          DataFrame's values are converted to a NumPy array representation using the
+                         `DataFrame.values()
+                         <https://pandas.pydata.org/pandas-docs/stable/reference/api/
+                          pandas.DataFrame.values.html#pandas.DataFrame.values>`_ method.
+                        - If the underlying ONNX model defines *multiple* input tensors, each column
                           of the DataFrame is converted to a NumPy array representation.
-                          The corresponding NumPy array representation is then passed to the
-                          ONNX Runtime. For more information about the ONNX Runtime, see
-                          `<https://github.com/microsoft/onnxruntime>`_.
-        :return: A Pandas DataFrame output. Each column of the DataFrame corresponds to an
-                 output tensor produced by the underlying ONNX model.
+
+                      For more information about the ONNX Runtime, see
+                      `<https://github.com/microsoft/onnxruntime>`_.
+        :return: Model predictions. If the input is a pandas.DataFrame, the predictions are returned
+                 in a pandas.DataFrame. If the input is a numpy array or a dictionary the
+                 predictions are returned in a dictionary.
         """
+        if isinstance(data, dict):
+            feed_dict = data
+        elif isinstance(data, np.ndarray):
+            # NB: We do allow scoring with a single tensor (ndarray) in order to be compatible with
+            # supported pyfunc inputs iff the model has a single input. The passed tensor is
+            # assumed to be the first input.
+            if len(self.inputs) != 1:
+                inputs = [x[0] for x in self.inputs]
+                raise MlflowException(
+                    "Unable to map numpy array input to the expected model "
+                    "input. "
+                    "Numpy arrays can only be used as input for MLflow ONNX "
+                    "models that have a single input. This model requires "
+                    "{0} inputs. Please pass in data as either a "
+                    "dictionary or a DataFrame with the following tensors"
+                    ": {1}.".format(len(self.inputs), inputs)
+                )
+            feed_dict = {self.inputs[0][0]: data}
+        elif isinstance(data, pd.DataFrame):
+            if len(self.inputs) > 1:
+                feed_dict = {name: data[name].values for (name, _) in self.inputs}
+            else:
+                feed_dict = {self.inputs[0][0]: data.values}
+
+        else:
+            raise TypeError(
+                "Input should be a dictionary or a numpy array or a pandas.DataFrame, "
+                "got '{}'".format(type(data))
+            )
+
         # ONNXRuntime throws the following exception for some operators when the input
-        # dataframe contains float64 values. Unfortunately, even if the original user-supplied
-        # dataframe did not contain float64 values, the serialization/deserialization between the
+        # contains float64 values. Unfortunately, even if the original user-supplied input
+        # did not contain float64 values, the serialization/deserialization between the
         # client and the scoring server can introduce 64-bit floats. This is being tracked in
         # https://github.com/mlflow/mlflow/issues/1286. Meanwhile, we explicitly cast the input to
         # 32-bit floats when needed. TODO: Remove explicit casting when issue #1286 is fixed.
-        if len(self.inputs) > 1:
-            cols = [name for (name, type) in self.inputs if type == "tensor(float)"]
-        else:
-            cols = dataframe.columns if self.inputs[0][1] == "tensor(float)" else []
-
-        dataframe = _OnnxModelWrapper._cast_float64_to_float32(dataframe, cols)
-        if len(self.inputs) > 1:
-            feed_dict = {name: dataframe[name].values for (name, _) in self.inputs}
-        else:
-            feed_dict = {self.inputs[0][0]: dataframe.values}
+        feed_dict = self._cast_float64_to_float32(feed_dict)
         predicted = self.rt.run(self.output_names, feed_dict)
 
-        def format_output(data):
-            # Output can be list and it should be converted to a numpy array
-            # https://github.com/mlflow/mlflow/issues/2499
-            data = np.asarray(data)
-            return data.reshape(-1)
+        if isinstance(data, pd.DataFrame):
 
-        response = pd.DataFrame.from_dict(
-            {c: format_output(p) for (c, p) in zip(self.output_names, predicted)}
-        )
-        return response
+            def format_output(data):
+                # Output can be list and it should be converted to a numpy array
+                # https://github.com/mlflow/mlflow/issues/2499
+                data = np.asarray(data)
+                return data.reshape(-1)
+
+            response = pd.DataFrame.from_dict(
+                {c: format_output(p) for (c, p) in zip(self.output_names, predicted)}
+            )
+            return response
+        else:
+            return dict(zip(self.output_names, predicted))
 
 
 def _load_pyfunc(path):


Test patch:
diff --git a/tests/gluon/test_gluon_model_export.py b/tests/gluon/test_gluon_model_export.py
--- a/tests/gluon/test_gluon_model_export.py
+++ b/tests/gluon/test_gluon_model_export.py
@@ -99,6 +99,9 @@ def test_model_save_load(gluon_model, model_data, model_path):
     test_pyfunc_data = pd.DataFrame(test_data.asnumpy())
     pyfunc_preds = pyfunc_loaded.predict(test_pyfunc_data)
     assert all(np.argmax(pyfunc_preds.values, axis=1) == expected.asnumpy())
+    # test with numpy array input
+    pyfunc_preds = pyfunc_loaded.predict(test_pyfunc_data.values)
+    assert all(np.argmax(pyfunc_preds, axis=1) == expected.asnumpy())
 
 
 @pytest.mark.large
diff --git a/tests/onnx/test_onnx_model_export.py b/tests/onnx/test_onnx_model_export.py
--- a/tests/onnx/test_onnx_model_export.py
+++ b/tests/onnx/test_onnx_model_export.py
@@ -16,6 +16,7 @@
 import mlflow.onnx
 import mlflow.pyfunc.scoring_server as pyfunc_scoring_server
 from mlflow import pyfunc
+from mlflow.exceptions import MlflowException
 from mlflow.models import infer_signature, Model
 from mlflow.models.utils import _read_example
 from mlflow.utils.file_utils import TempDir
@@ -25,7 +26,6 @@
 from mlflow.utils.model_utils import _get_flavor_configuration
 from mlflow.tracking._model_registry import DEFAULT_AWAIT_MAX_SLEEP_SECONDS
 
-
 TEST_DIR = "tests"
 TEST_ONNX_RESOURCES_DIR = os.path.join(TEST_DIR, "resources", "onnx")
 
@@ -94,6 +94,70 @@ def onnx_model(model, sample_input, tmpdir):
     return onnx.load(model_path)
 
 
+@pytest.fixture(scope="module")
+def multi_tensor_model(dataset):
+    class MyModel(nn.Module):
+        def __init__(self, n):
+            super(MyModel, self).__init__()
+            self.linear = torch.nn.Linear(n, 1)
+            self._train = True
+
+        def forward(self, sepal_features, petal_features):
+            if not self.training:
+                if isinstance(sepal_features, np.ndarray):
+                    sepal_features = torch.from_numpy(sepal_features)
+                if isinstance(petal_features, np.ndarray):
+                    petal_features = torch.from_numpy(petal_features)
+                with torch.no_grad():
+                    return self.linear(torch.cat((sepal_features, petal_features), dim=-1))
+            else:
+                return self.linear(torch.cat((sepal_features, petal_features), dim=-1))
+
+    model = MyModel(4)
+    model.train()
+    dataloader = DataLoader(dataset, batch_size=16, num_workers=1, shuffle=True, drop_last=False)
+    criterion = nn.MSELoss()
+    optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
+    for _ in range(5):
+        for batch in dataloader:
+            optimizer.zero_grad()
+            y_pred = model(*torch.split(batch[0], 2, 1)).squeeze(dim=1)
+            loss = criterion(y_pred, batch[1])
+            loss.backward()
+            optimizer.step()
+    model.train(False)
+    return model
+
+
+@pytest.fixture(scope="module")
+def multi_tensor_model_prediction(multi_tensor_model, data):
+    x, _ = data
+    feeds = {
+        "sepal_features": x[x.columns[:2]].values.astype(np.float32),
+        "petal_features": x[x.columns[2:4]].values.astype(np.float32),
+    }
+    return multi_tensor_model(**feeds).numpy().flatten()
+
+
+@pytest.fixture
+def multi_tensor_onnx_model(multi_tensor_model, sample_input, tmpdir):
+    model_path = os.path.join(str(tmpdir), "multi_tensor_onnx")
+    _sample_input = torch.split(sample_input, 2, 1)
+    torch.onnx.export(
+        multi_tensor_model,
+        _sample_input,
+        model_path,  # where to save the model (can be a file or file-like object)
+        dynamic_axes={"sepal_features": [0], "petal_features": [0]},
+        export_params=True,
+        # store the trained parameter weights inside the model file
+        do_constant_folding=True,
+        # whether to execute constant folding for optimization
+        input_names=["sepal_features", "petal_features"],  # the model's input names
+        output_names=["target"],  # the model's output names
+    )
+    return onnx.load(model_path)
+
+
 @pytest.fixture(scope="module")
 def onnx_sklearn_model():
     """
@@ -161,15 +225,6 @@ def onnx_custom_env(tmpdir):
     return conda_env
 
 
-@pytest.mark.large
-def test_cast_float64_to_float32():
-    df = pd.DataFrame([[1.0, 2.1], [True, False]], columns=["col1", "col2"])
-    df["col1"] = df["col1"].astype(np.float64)
-    df["col2"] = df["col2"].astype(np.bool)
-    df2 = mlflow.onnx._OnnxModelWrapper._cast_float64_to_float32(df, df.columns)
-    assert df2["col1"].dtype == np.float32 and df2["col2"].dtype == np.bool
-
-
 @pytest.mark.large
 def test_model_save_load(onnx_model, model_path):
     mlflow.onnx.save_model(onnx_model, model_path)
@@ -351,24 +406,53 @@ def test_log_model_no_registered_model_name(onnx_model, onnx_custom_env):
 def test_model_log_evaluate_pyfunc_format(onnx_model, data, predicted):
     x = data[0]
 
-    # should_start_run tests whether or not calling log_model() automatically starts a run.
-    for should_start_run in [False, True]:
-        try:
-            if should_start_run:
-                mlflow.start_run()
-            artifact_path = "onnx_model"
-            mlflow.onnx.log_model(onnx_model=onnx_model, artifact_path=artifact_path)
-            model_uri = "runs:/{run_id}/{artifact_path}".format(
-                run_id=mlflow.active_run().info.run_id, artifact_path=artifact_path
-            )
+    with mlflow.start_run() as run:
+        artifact_path = "onnx_model"
+        mlflow.onnx.log_model(onnx_model=onnx_model, artifact_path=artifact_path)
+        model_uri = "runs:/{run_id}/{artifact_path}".format(
+            run_id=run.info.run_id, artifact_path=artifact_path
+        )
 
-            # Loading pyfunc model
-            pyfunc_loaded = mlflow.pyfunc.load_pyfunc(model_uri=model_uri)
-            assert np.allclose(
-                pyfunc_loaded.predict(x).values.flatten(), predicted, rtol=1e-05, atol=1e-05
-            )
-        finally:
-            mlflow.end_run()
+        # Loading pyfunc model
+        pyfunc_loaded = mlflow.pyfunc.load_pyfunc(model_uri=model_uri)
+        assert np.allclose(
+            pyfunc_loaded.predict(x).values.flatten(), predicted, rtol=1e-05, atol=1e-05
+        )
+        # test with a single numpy array
+        np_ary = x.values
+
+        # NB: Onnx wrapper returns a dictionary for non-dataframe inputs, we want to get the
+        # numpy array belonging to the first (and only) model output.
+        def get_ary_output(args):
+            return next(iter(pyfunc_loaded.predict(args).values())).flatten()
+
+        assert np.allclose(get_ary_output(np_ary), predicted, rtol=1e-05, atol=1e-05)
+        # test with a dict with a single tensor
+        assert np.allclose(get_ary_output({"input": np_ary}), predicted, rtol=1e-05, atol=1e-05)
+
+
+@pytest.mark.large
+def test_model_save_evaluate_pyfunc_format_multi_tensor(
+    multi_tensor_onnx_model, data, multi_tensor_model_prediction
+):
+    with TempDir(chdr=True):
+        path = "onnx_model"
+        mlflow.onnx.save_model(onnx_model=multi_tensor_onnx_model, path=path)
+        # Loading pyfunc model
+        pyfunc_loaded = mlflow.pyfunc.load_pyfunc(model_uri=path)
+        data, _ = data
+        # get prediction
+        feeds = {
+            "sepal_features": data[data.columns[:2]].values,
+            "petal_features": data[data.columns[2:4]].values.astype(np.float32),
+        }
+        preds = pyfunc_loaded.predict(feeds)["target"].flatten()
+        assert np.allclose(preds, multi_tensor_model_prediction, rtol=1e-05, atol=1e-05)
+        # single numpy array input should fail with the right error message:
+        with pytest.raises(
+            MlflowException, match="Unable to map numpy array input to the expected model " "input."
+        ):
+            pyfunc_loaded.predict(data.values)
 
 
 @pytest.mark.large