Repository: mlflow/mlflow

Issue:
[BUG] 2.4.0 regression: `import mlflow` fails if a `datasets` directory exists
### Issues Policy acknowledgement

- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)

### Willingness to contribute

No. I cannot contribute a bug fix at this time.

### MLflow version

- Client: 2.4.0
- Tracking server: N/A


### System information

- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Ubuntu 22.04 LTS
- **Python version**: 3.10.6
- **yarn version, if running the dev UI**: N/A


### Describe the problem

* Actual behavior: `import mlflow` fails for the 2.4.0 `mlflow` Python package if a directory named `datasets` exists in the current directory. The traceback shows an `AttributeError` on line 23 of `mlflow/data/huggingface_dataset_source.py`.
* Expected behavior: `import mlflow` does not fail. It's not far-fetched that a user working with MLflow has a directory with the name `datasets`, so this should be supported by MLflow.
* The error does not appear with version 2.3.2, so this is a regression.
* This is probably related to #8186.

### Tracking information

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```shell
```


### Code to reproduce issue

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```console
$ mkdir mlflow-bug
$ cd mlflow-bug
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install mlflow==2.4.0
$ mkdir datasets
$ python -c "import mlflow"
```


### Stack trace

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/__init__.py", line 41, in <module>
    from mlflow import projects  # pylint: disable=unused-import
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/projects/__init__.py", line 9, in <module>
    import mlflow.projects.databricks
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/projects/databricks.py", line 11, in <module>
    from mlflow import tracking
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/tracking/__init__.py", line 20, in <module>
    from mlflow.tracking.fluent import _EXPERIMENT_ID_ENV_VAR, _EXPERIMENT_NAME_ENV_VAR, _RUN_ID_ENV_VAR
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py", line 13, in <module>
    from mlflow.data.dataset import Dataset
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/data/__init__.py", line 5, in <module>
    from mlflow.data import dataset_registry
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/data/dataset_registry.py", line 137, in <module>
    from mlflow.data.huggingface_dataset import from_huggingface
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/data/huggingface_dataset.py", line 11, in <module>
    from mlflow.data.huggingface_dataset_source import HuggingFaceDatasetSource
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/data/huggingface_dataset_source.py", line 10, in <module>
    class HuggingFaceDatasetSource(DatasetSource):
  File "/home/JulianJvn/mlflow-bug/.venv/lib/python3.10/site-packages/mlflow/data/huggingface_dataset_source.py", line 23, in HuggingFaceDatasetSource
    split: Optional[Union[str, datasets.Split]] = None,
AttributeError: module 'datasets' has no attribute 'Split'
```


### Other info / logs

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```
```


### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [X] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations


Discussion:
@JulianJvn Thanks for reporting the issue. I was able to make a repro.
To repro, `datasets` must be uninstalled.

Resolving patch:
diff --git a/mlflow/data/huggingface_dataset.py b/mlflow/data/huggingface_dataset.py
--- a/mlflow/data/huggingface_dataset.py
+++ b/mlflow/data/huggingface_dataset.py
@@ -1,9 +1,8 @@
 import json
 import logging
 from functools import cached_property
-from typing import Any, Union, Optional, Mapping, Sequence, Dict
+from typing import Any, Union, Optional, Mapping, Sequence, Dict, TYPE_CHECKING
 
-import datasets
 
 from mlflow.data.dataset import Dataset
 from mlflow.data.digest_utils import compute_pandas_digest
@@ -20,6 +19,9 @@
 
 _MAX_ROWS_FOR_DIGEST_COMPUTATION_AND_SCHEMA_INFERENCE = 10000
 
+if TYPE_CHECKING:
+    import datasets
+
 
 @experimental
 class HuggingFaceDataset(Dataset, PyFuncConvertibleDatasetMixin):
@@ -29,7 +31,7 @@ class HuggingFaceDataset(Dataset, PyFuncConvertibleDatasetMixin):
 
     def __init__(
         self,
-        ds: datasets.Dataset,
+        ds: "datasets.Dataset",
         source: HuggingFaceDatasetSource,
         targets: Optional[str] = None,
         name: Optional[str] = None,
@@ -85,7 +87,7 @@ def _to_dict(self, base_dict: Dict[str, str]) -> Dict[str, str]:
         }
 
     @property
-    def ds(self) -> datasets.Dataset:
+    def ds(self) -> "datasets.Dataset":
         """
         The Hugging Face ``datasets.Dataset`` instance.
 
@@ -219,6 +221,7 @@ def from_huggingface(
     :param digest: The digest (hash, fingerprint) of the dataset. If unspecified, a digest
                    is automatically computed.
     """
+    import datasets
     from mlflow.data.code_dataset_source import CodeDatasetSource
     from mlflow.tracking.context import registry
 
diff --git a/mlflow/data/huggingface_dataset_source.py b/mlflow/data/huggingface_dataset_source.py
--- a/mlflow/data/huggingface_dataset_source.py
+++ b/mlflow/data/huggingface_dataset_source.py
@@ -1,11 +1,13 @@
-import datasets
-
-from typing import Any, Union, Optional, Mapping, Sequence, Dict
+from typing import Any, Union, Optional, Mapping, Sequence, Dict, TYPE_CHECKING
 
 from mlflow.data.dataset_source import DatasetSource
 from mlflow.utils.annotations import experimental
 
 
+if TYPE_CHECKING:
+    import datasets
+
+
 @experimental
 class HuggingFaceDatasetSource(DatasetSource):
     """
@@ -20,9 +22,9 @@ def __init__(
         data_files: Optional[
             Union[str, Sequence[str], Mapping[str, Union[str, Sequence[str]]]]
         ] = None,
-        split: Optional[Union[str, datasets.Split]] = None,
-        revision: Optional[Union[str, datasets.Version]] = None,
-        task: Optional[Union[str, datasets.TaskTemplate]] = None,
+        split: Optional[Union[str, "datasets.Split"]] = None,
+        revision: Optional[Union[str, "datasets.Version"]] = None,
+        task: Optional[Union[str, "datasets.TaskTemplate"]] = None,
     ):
         """
         :param path: The path of the Hugging Face dataset.
@@ -56,6 +58,8 @@ def load(self, **kwargs):
                        ``data_dir``, ``data_files``, ``split``, ``revision``, ``task``.
         :return: An instance of ``datasets.Dataset``.
         """
+        import datasets
+
         load_kwargs = {
             "path": self._path,
             "name": self._config_name,


Test patch: