Repository: optuna/optuna

Issue:
Review constrain scikit-learn< 0.23
scikit-optimize 0.8.1 has been released which works together with scikit-learn >= 0.23

https://github.com/scikit-optimize/scikit-optimize/releases/tag/v0.8.1


Discussion:
This issue has not seen any recent activity.
@holgern I'm sorry for the delayed response.

> works together with scikit-learn >= 0.23

Thank you for letting us know. The version restriction was introduced by #1236 to resolve #1235, but it is no longer required as you mentioned.

The change seems to be obvious, and let add `contribution-welcome` label to this issue.

## Task

Please remove the version constraint of `scikit-learn<0.23.0` in [`setup.py`](https://github.com/optuna/optuna/blob/master/setup.py). Please note that the constraint `scikit-learn>=0.19.0` is still required by the other part of the code base.
We might need to update codes for `scikit-learn>=0.23.0`. I removed the constraint of `scikit-learn<0.23.0` in `setup.py`, then ran tests:

```bash
$ circleci build --job tests-python37

=========================== short test summary info ============================
FAILED tests/integration_tests/test_sklearn.py::test_objective_error_score_nan
FAILED tests/samplers_tests/test_samplers.py::test_nan_objective_value[<lambda>2]
===== 2 failed, 1189 passed, 3 skipped, 935 warnings in 405.94s (0:06:45) ======
Error:
Exited with code exit status 1

Step failed
Error: runner failed (exited with 101)
Task failed
Step canceled
Error: task failed
```
@nzw0301 Thank you for your investigation. I removed the `contribution-welcome` label since you are working on this issue. As you mentioned in #1890, `SkoptSampler` with `scikit-learn>=0.23.0` did not work as expected and this isn't a good-first issue in fact.
This issue has not seen any recent activity.
It works now...?

```bash
 pytest tests/integration_tests/test_skopt.py            [18:52:05]
====================================== test session starts ======================================
platform darwin -- Python 3.8.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/nzw/PycharmProjects/optuna
plugins: anyio-2.1.0
collected 8 items

tests/integration_tests/test_skopt.py ........                                            [100%]

======================================= warnings summary ========================================
tests/integration_tests/test_skopt.py::test_get_trials
  /Users/nzw/PycharmProjects/optuna/optuna/integration/skopt.py:125: ExperimentalWarning: `consider_pruned_trials` option is an experimental feature. The interface can change in the future.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================================= 8 passed, 1 warning in 9.62s ==================================
```

```bash
$ pip list | grep -i scikit
scikit-image                  0.18.1
scikit-learn                  0.24.1
scikit-optimize               0.8.1
```


Ok, not yet :(
Several PRs of scikit-learn are trying to fix the source of this issue (`0` division of `std`). We will come back to this issue if the PRs have been merged into a future stable version.
This issue has not seen any recent activity.
Scikit-learn 0.24.2 is out. It seems to contain the bug fixes related to Gaussian process according to [the release note](https://scikit-learn.org/stable/whats_new/v0.24.html#sklearn-gaussian-process).

I locally confirmed that the pytest's errors disappear by using this following libraries:

```zsh
scikit-learn                  0.24.2
scikit-optimize               0.8.1
```


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -58,7 +58,7 @@ def get_extras_require() -> Dict[str, List[str]]:
             "matplotlib>=3.0.0",
             "pandas",
             "plotly>=4.0.0",
-            "scikit-learn>=0.19.0,<0.23.0",
+            "scikit-learn>=0.24.2",
             "scikit-optimize",
             "mlflow",
         ],
@@ -83,7 +83,7 @@ def get_extras_require() -> Dict[str, List[str]]:
         ],
         "example": [
             "nbval",
-            "scikit-learn>=0.19.0,<0.23.0 ; python_version<'3.9'",
+            "scikit-learn>=0.24.2",
             # optuna/visualization/param_importances.py.
             "thop",
             "torch==1.8.0 ; sys_platform=='darwin'",
@@ -112,7 +112,7 @@ def get_extras_require() -> Dict[str, List[str]]:
             "pandas",
             "plotly>=4.0.0",
             "pytest",
-            "scikit-learn>=0.19.0,<0.23.0",
+            "scikit-learn>=0.24.2",
             "scikit-optimize",
             "xgboost",
             "keras",
@@ -143,7 +143,7 @@ def get_extras_require() -> Dict[str, List[str]]:
             "pandas",  # optuna/study.py
             "plotly>=4.0.0",  # optuna/visualization.
             "redis",  # optuna/storages/redis.py.
-            "scikit-learn>=0.19.0,<0.23.0 ; python_version<'3.9'",
+            "scikit-learn>=0.24.2",
             # optuna/visualization/param_importances.py.
         ],
         "integration": [
@@ -156,7 +156,7 @@ def get_extras_require() -> Dict[str, List[str]]:
             "mpi4py",
             "mxnet",
             "pandas",
-            "scikit-learn>=0.19.0,<0.23.0 ; python_version<'3.9'",
+            "scikit-learn>=0.24.2",
             "scikit-optimize",
             "xgboost",
             "keras ; python_version<'3.9'",


Test patch:
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -70,13 +70,6 @@ jobs:
         pip install --use-deprecated=legacy-resolver scikit-optimize
         pip install --use-deprecated=legacy-resolver cma
 
-    # TODO: Add the deselected tests back after fixing version constraint of sklearn.
     - name: Tests
       run: |
-        if [ ${{ matrix.python-version }} = 3.9 ]; then
-          pytest tests --ignore tests/integration_tests \
-            --deselect tests/samplers_tests/test_samplers.py::test_nan_objective_value \
-            --deselect tests/samplers_tests/test_samplers.py::test_partial_fixed_sampling
-        else
-          pytest tests --ignore tests/integration_tests
-        fi
+        pytest tests --ignore tests/integration_tests
diff --git a/tests/integration_tests/test_sklearn.py b/tests/integration_tests/test_sklearn.py
--- a/tests/integration_tests/test_sklearn.py
+++ b/tests/integration_tests/test_sklearn.py
@@ -294,7 +294,10 @@ def test_objective_error_score_nan() -> None:
         return_train_score=True,
     )
 
-    with pytest.raises(ValueError, match="y cannot be None"):
+    with pytest.raises(
+        ValueError,
+        match="This SGDClassifier estimator requires y to be passed, but the target y is None.",
+    ):
         optuna_search.fit(X)