Repository: mlflow/mlflow

Issue:
[BUG] Invalid Requirement error with custom wheel on dbfs path
### Issues Policy acknowledgement

- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)

### Willingness to contribute

No. I cannot contribute a bug fix at this time.

### MLflow version

- mlflow: 2.0.1


### System information

- **Databricks Runtime 12.0 for Machine Learning**



### Describe the problem

I am logging a model with a custom library via `pip_requirements ` parameter as described in the Databricks documentation in the [Use custom Python libraries with Model Serving](https://learn.microsoft.com/de-de/azure/databricks/machine-learning/model-serving/private-libraries-model-serving) section. My wheel file is stored on DBFS (for example under `/dbfs/path/to/dependency.whl`).

When loading the model the parsing of the custom library requirement fails with `InvalidRequirement('Parse error at "\'/dbfs/pa\'": Expected W:(abcd...)') `, see code example and full stack trace below.

Would appreciate any hints or advise on how I can get the wheel file recognized in my pip requirements!


### Code to reproduce issue

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```python
import numpy as np
from sklearn.linear_model import LogisticRegression

import logging
import mlflow
import mlflow.sklearn

logging.getLogger("mlflow").setLevel(logging.DEBUG)

X = np.array([-2, -1, 0, 1, 2, 1]).reshape(-1, 1)
y = np.array([0, 0, 1, 1, 1, 0])
lr = LogisticRegression()
lr.fit(X, y)

pip_requirements=["scikit-learn", "numpy", "/dbfs/path/to/dependency.whl"]
mlflow.sklearn.log_model(lr, "model", pip_requirements=pip_requirements)

logged_model = f"runs:/{mlflow.active_run().info.run_uuid}/model"

# Load model as a PyFuncModel.
loaded_model = mlflow.pyfunc.load_model(logged_model)
```


### Stack trace

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```
2023/04/14 06:45:26 WARNING mlflow.pyfunc: Encountered an unexpected error (InvalidRequirement('Parse error at "\'/dbfs/pa\'": Expected W:(abcd...)')) while detecting model dependency mismatches. Set logging level to DEBUG to see the full traceback.
2023/04/14 06:45:26 DEBUG mlflow.pyfunc: 
Traceback (most recent call last):
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 102, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 1654, in parseString
    raise exc
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 1644, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 3417, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 3739, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 3400, in parseImpl
    loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 1406, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/pyparsing.py", line 2711, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pkg_resources._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/databricks/python/lib/python3.9/site-packages/mlflow/pyfunc/__init__.py", line 458, in _warn_dependency_requirement_mismatches
    mismatch_info = _check_requirement_satisfied(req_line)
  File "/databricks/python/lib/python3.9/site-packages/mlflow/utils/requirements_utils.py", line 494, in _check_requirement_satisfied
    req = pkg_resources.Requirement.parse(requirement_str)
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3134, in parse
    req, = parse_requirements(s)
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3089, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/databricks/python/lib/python3.9/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 104, in __init__
    raise InvalidRequirement(
pkg_resources.extern.packaging.requirements.InvalidRequirement: Parse error at "'/dbfs/pa'": Expected W:(abcd...)
```



### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [X] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [X] `integrations/databricks`: Databricks integrations

Discussion:
mlflow-automation: <!-- documentation preview -->


Documentation preview for 3f795f8380ce6e7c54b6e9483a29f89bbcce3681 will be available [here](https://output.circle-artifacts.com/output/job/85c83622-04f2-4b86-8ab1-ba3f6f74b74c/artifacts/0/docs/build/html/index.html) when [this CircleCI job](https://circleci.com/gh/mlflow/mlflow/46641) completes successfully.

<details>
<summary>More info</summary>

- Ignore this comment if this PR does not change the documentation.
- It takes a few minutes for the preview to be available.
- The preview is updated when a new commit is pushed to this PR.
- This comment was created by https://github.com/mlflow/mlflow/actions/runs/4730699452.

</details>


Patch:
diff --git a/mlflow/models/utils.py b/mlflow/models/utils.py
index e83ba9881..3d9847039 100644
--- a/mlflow/models/utils.py
+++ b/mlflow/models/utils.py
@@ -466,14 +466,17 @@ def _reshape_and_cast_pandas_column_values(name, pd_series, tensor_spec):
         )
 
     if np.isscalar(pd_series[0]):
-        if tensor_spec.shape != (-1,):
-            raise MlflowException(
-                f"The input pandas dataframe column '{name}' contains scalar "
-                "values, which requires the shape to be (-1,), but got tensor spec "
-                f"shape of {tensor_spec.shape}.",
-                error_code=INVALID_PARAMETER_VALUE,
-            )
-        return _enforce_tensor_spec(np.array(pd_series, dtype=tensor_spec.type), tensor_spec)
+        for shape in [(-1,), (-1, 1)]:
+            if tensor_spec.shape == shape:
+                return _enforce_tensor_spec(
+                    np.array(pd_series, dtype=tensor_spec.type).reshape(shape), tensor_spec
+                )
+        raise MlflowException(
+            f"The input pandas dataframe column '{name}' contains scalar "
+            "values, which requires the shape to be (-1,) or (-1, 1), but got tensor spec "
+            f"shape of {tensor_spec.shape}.",
+            error_code=INVALID_PARAMETER_VALUE,
+        )
     elif isinstance(pd_series[0], list) and np.isscalar(pd_series[0][0]):
         # If the pandas column contains list type values,
         # in this case, the shape and type information is lost,
diff --git a/mlflow/utils/requirements_utils.py b/mlflow/utils/requirements_utils.py
index a2e9f898f..170207708 100644
--- a/mlflow/utils/requirements_utils.py
+++ b/mlflow/utils/requirements_utils.py
@@ -488,11 +488,19 @@ class _MismatchedPackageInfo(NamedTuple):
 
 def _check_requirement_satisfied(requirement_str):
     """
-    Returns None if the current python environment satisfies the given requirement.
-    Otherwise, returns an instance of `_MismatchedPackageInfo`.
+    Checks whether the current python environment satisfies the given requirement if it is parsable
+    as a package name and a set of version specifiers, and returns a `_MismatchedPackageInfo`
+    object containing the mismatched package name, installed version, and requirement if the
+    requirement is not satisfied. Otherwise, returns None.
     """
     _init_packages_to_modules_map()
-    req = pkg_resources.Requirement.parse(requirement_str)
+    try:
+        req = pkg_resources.Requirement.parse(requirement_str)
+    except Exception:
+        # We reach here if the requirement string is a file path or a URL.
+        # Extracting the package name from the requirement string is not trivial,
+        # so we skip the check.
+        return
     pkg_name = req.name
 
     try:


Test patch:
diff --git a/tests/pyfunc/test_dependencies_functions.py b/tests/pyfunc/test_dependencies_functions.py
index 64ca22f1b..ed2b172fa 100644
--- a/tests/pyfunc/test_dependencies_functions.py
+++ b/tests/pyfunc/test_dependencies_functions.py
@@ -243,3 +243,11 @@ def test_get_model_dependencies_with_model_version_uri():
 
     deps = get_model_dependencies("models:/linear/1", format="pip")
     assert f"scikit-learn=={sklearn.__version__}" in Path(deps).read_text()
+
+
+def test_warn_dependency_requirement_mismatches_ignores_file_path(tmp_path):
+    req_file = tmp_path / "requirements.txt"
+    req_file.write_text("/path/to/my.whl")
+    with mock.patch("mlflow.pyfunc._logger.warning") as mock_warning:
+        _warn_dependency_requirement_mismatches(model_path=tmp_path)
+        mock_warning.assert_not_called()
diff --git a/tests/pyfunc/test_model_export_with_loader_module_and_data_path.py b/tests/pyfunc/test_model_export_with_loader_module_and_data_path.py
index fcace660d..15adc1415 100644
--- a/tests/pyfunc/test_model_export_with_loader_module_and_data_path.py
+++ b/tests/pyfunc/test_model_export_with_loader_module_and_data_path.py
@@ -440,7 +440,7 @@ def test_tensor_multi_named_schema_enforcement():
         MlflowException,
         match=re.escape(
             "The input pandas dataframe column 'a' contains scalar values, which requires the "
-            "shape to be (-1,), but got tensor spec shape of (-1, 5)"
+            "shape to be (-1,) or (-1, 1), but got tensor spec shape of (-1, 5)"
         ),
     ):
         pyfunc_model.predict(pdf)
@@ -547,7 +547,7 @@ def test_schema_enforcement_named_tensor_schema_1d():
         expected_exception=MlflowException,
         match=re.escape(
             "The input pandas dataframe column 'a' contains scalar "
-            "values, which requires the shape to be (-1,), but got tensor spec "
+            "values, which requires the shape to be (-1,) or (-1, 1), but got tensor spec "
             "shape of (-1, 2)."
         ),
     ):
@@ -928,3 +928,20 @@ def test_log_model_without_specified_conda_env_uses_default_env_with_expected_de
         )
         model_uri = mlflow.get_artifact_uri(pyfunc_artifact_path)
     _assert_pip_requirements(model_uri, mlflow.pyfunc.get_default_pip_requirements())
+
+
+def test_schema_enforcement_single_column_2d_array():
+    X = np.array([[1], [2], [3]])
+    y = np.array([1, 2, 3])
+    model = sklearn.linear_model.LinearRegression()
+    model.fit(X, y)
+    signature = infer_signature(X, y)
+    assert signature.inputs.inputs[0].shape == (-1, 1)
+    assert signature.outputs.inputs[0].shape == (-1,)
+
+    with mlflow.start_run():
+        model_info = mlflow.sklearn.log_model(model, "model", signature=signature)
+
+    loaded_model = mlflow.pyfunc.load_model(model_info.model_uri)
+    pdf = pd.DataFrame(X)
+    np.testing.assert_almost_equal(loaded_model.predict(pdf), model.predict(pdf))