Repository: zenml-io/zenml

Issue:
Update `sklearn` Integration to Support Versions >1.3.0 and Resolve MLflow Autologging Issues
## Open Source Contributors Welcomed!
Please comment below if you would like to work on this issue!

### Contact Details [Optional]

support@zenml.io

### What happened?
The current ZenML Sklearn integration is restricted to versions of Sklearn <1.3.0, as defined in `src/zenml/integrations/sklearn/__init__.py`. However, the release of Sklearn 1.3.0 necessitates an update to this constraint. Additionally, this Sklearn version upgrade appears to cause issues with MLflow autologging, likely due to compatibility conflicts.

### Task Description
Update the Sklearn integration in ZenML to support Sklearn versions >1.3.0. Additionally, identify and resolve any issues arising in MLflow autologging due to this version update.

### Expected Outcome
- The Sklearn integration in ZenML should allow for the use of Sklearn versions >1.3.0.
- Any compatibility issues, especially with MLflow autologging, should be identified and resolved.
- Ensure that all tests, including CI pipelines, pass with the updated Sklearn version.

### Steps to Implement

- Modify the Sklearn version constraint in src/zenml/integrations/sklearn/__init__.py to allow for versions >1.3.0.
- Investigate and identify the root cause of the issues with MLflow autologging when using Sklearn 1.3.0.
- Implement necessary fixes or updates to ensure compatibility with the new Sklearn version.
- Thoroughly test the changes, especially focusing on MLflow autologging functionality.
- Update documentation and examples as necessary to reflect the support for the new Sklearn version.

### Additional Context
This update is crucial for keeping ZenML compatible with the latest machine learning tools and libraries, ensuring that users can leverage the newest features and improvements in Sklearn.

### Code of Conduct
- [ ] I agree to follow this project's Code of Conduct


Discussion:
Hello, I am trying to work on this but I getting an error while testing after upgrading the sklearn version.
The error:
```
ImportError while loading conftest 'C:\Users\<MyUser>\Github\zenml\tests\conftest.py'.
tests\conftest.py:23: in <module>
    from py._builtin import execfile
E   ModuleNotFoundError: No module named 'py._builtin'; 'py' is not a package
```
Okay, I found out why it was giving me an issue.
The  `from py._builtin import execfile` is deprecated from version 3.0.0 and it was replaced by `from pytest import File`

Replacing the import solves the error!
@strickvl I ran the MLFlow integration tests after updating the sklearn version > 1.3.0 and I don't see any issues.
All the test cases passed without any errors.
> @strickvl I ran the MLFlow integration tests after updating the sklearn version > 1.3.0 and I don't see any issues. All the test cases passed without any errors.

Thanks for taking a look at this. The key thing is to check that autologging is happening as expected. Just checking the tests pass probably isn't enough.
Okay, I'll check whether Autologging is correctly working or not.
@strickvl I ran a test file that autologs the score and everything ran fine.
``` 
import numpy as np
from sklearn.linear_model import LogisticRegression

import mlflow
import mlflow.sklearn
from mlflow.models import infer_signature

if __name__ == "__main__":
    X = np.array([-2, -1, 0, 1, 2, 1]).reshape(-1, 1)
    y = np.array([0, 0, 1, 1, 1, 0])
    lr = LogisticRegression()
    lr.fit(X, y)
    score = lr.score(X, y)
    print(f"Score: {score}")
    mlflow.log_metric("score", score)
    predictions = lr.predict(X)
    signature = infer_signature(X, predictions)
    mlflow.sklearn.log_model(lr, "model", signature=signature)
    print(f"Model saved in run {mlflow.active_run().info.run_uuid}")
```
Above is the code I used for testing the auto-logging function of MLFlow using the latest sklearn version.
My Experiment got logged properly and I was able to everything on the MLFlow dashboard.
![image](https://github.com/zenml-io/zenml/assets/66507933/4d03f276-ebe0-4758-b3f3-876a6b5bddf1)
 
 
@strickvl should I make a PR for this change?
@Vishal-Padia sorry I think maybe I was unclear. The autologging to be tested is the zenml autologging functionality. i.e. what is [described here](https://docs.zenml.io/stacks-and-components/component-guide/experiment-trackers/mlflow#how-do-you-use-it). So the real check to be done would be to use a simple step doing something with the new sklearn and then use [the `autolog()` method](https://mlflow.org/docs/latest/python_api/mlflow.sklearn.html) within a ZenML step and then see if the relevant logging/tracking happens both inside ZenML as well as inside the MLflow interface.
Ohh okay understood, let me see what I can do.
@strickvl I ran the below:
```
import pandas as pd
import logging
import mlflow
from typing import Annotated
from typing import Tuple
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.base import ClassifierMixin
from sklearn.svm import SVC
from mlflow.tracking import MlflowClient

from zenml import pipeline, step


@step
def training_data_loader() -> (
    Tuple[
        # Notice we use a Tuple and Annotated to return
        # multiple named outputs
        Annotated[pd.DataFrame, "X_train"],
        Annotated[pd.DataFrame, "X_test"],
        Annotated[pd.Series, "y_train"],
        Annotated[pd.Series, "y_test"],
    ]
):
    """Load the iris dataset as a tuple of Pandas DataFrame / Series."""
    mlflow.sklearn.autolog(
        log_datasets=True,
        log_models=True,
    )
    logging.info("Loading iris...")
    iris = load_iris(as_frame=True)
    logging.info("Splitting train and test...")
    X_train, X_test, y_train, y_test = train_test_split(
        iris.data, iris.target, test_size=0.2, shuffle=True, random_state=42
    )
    return X_train, X_test, y_train, y_test


@step
def svc_trainer(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    gamma: float = 0.001,
) -> Tuple[
    Annotated[ClassifierMixin, "trained_model"],
    Annotated[float, "training_acc"],
]:
    """Train a sklearn SVC classifier."""

    model = SVC(gamma=gamma)
    model.fit(X_train.to_numpy(), y_train.to_numpy())

    train_acc = model.score(X_train.to_numpy(), y_train.to_numpy())
    print(f"Train accuracy: {train_acc}")

    return model, train_acc





@pipeline
def training_pipeline(gamma: float = 0.002):
    X_train, X_test, y_train, y_test = training_data_loader()
    with mlflow.start_run() as run:
        svc_trainer(gamma=gamma, X_train=X_train, y_train=y_train)


if __name__ == "__main__":
    training_pipeline(gamma=0.0015)

```

I can see the MLFlow dashboard as well as the same thing on ZenML Dashboard
ZenML DashBoard:
![image](https://github.com/zenml-io/zenml/assets/66507933/beeff849-b063-4453-92eb-fe4e23e40a17)

MLFlow DashBoard:
![image](https://github.com/zenml-io/zenml/assets/66507933/f39edc8e-6e7c-49d6-9210-a677fd2da600)



Ok looks like everything is working as expected. Then I'd say you're probably good to make a PR to the core repo off the `develop` branch which allows mlflow >1.3.0 and we can run it through our CI system / tests.
okay sure, I'll make a PR for this now!

Resolving patch:
diff --git a/src/zenml/integrations/sklearn/__init__.py b/src/zenml/integrations/sklearn/__init__.py
--- a/src/zenml/integrations/sklearn/__init__.py
+++ b/src/zenml/integrations/sklearn/__init__.py
@@ -21,7 +21,7 @@ class SklearnIntegration(Integration):
     """Definition of sklearn integration for ZenML."""
 
     NAME = SKLEARN
-    REQUIREMENTS = ["scikit-learn<1.3"]
+    REQUIREMENTS = ["scikit-learn>1.3"]
 
     @classmethod
     def activate(cls) -> None:


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -20,7 +20,7 @@
 from uuid import uuid4
 
 import pytest
-from py._builtin import execfile
+from pytest import File
 from pytest_mock import MockerFixture
 
 from tests.harness.environment import TestEnvironment
@@ -325,9 +325,7 @@ def virtualenv(
                 "tests"
             )
 
-        execfile(
-            str(activate_this_file), dict(__file__=str(activate_this_file))
-        )
+        File(str(activate_this_file), dict(__file__=str(activate_this_file)))
 
         # Set new system executable
         sys.executable = tmp_path / env_bin_dir / "python"
@@ -346,7 +344,7 @@ def virtualenv(
                 "your virtual environment to run integration "
                 "tests"
             )
-        execfile(str(activate_this_f), dict(__file__=str(activate_this_f)))
+        File(str(activate_this_f), dict(__file__=str(activate_this_f)))
 
     else:
         yield ""