Issue 1:
Describe the issue linked to the documentation

The homepage has a link name "Maling list: scikit-learn@python.org" which points to https://mail.python.org/mailman/listinfo/scikit-learn

(Would prefer it indicate the sign-up page for mailing list instead of an email address for posting to mailing list.)

Suggest a potential alternative/fix

Update this text "Mailing list: scikit-learn@python.org" to something like: "Subscribe to mailing list"

and link would go to: https://mail.python.org/mailman/listinfo/scikit-learn

Note for Beginners

Here are notes on how to get started:

Read the documentation: https://scikit-learn.org/stable/developers/contributing.html

Read notes on Git: https://www.dataschool.io/how-to-contribute-on-github/

Watch videos: https://www.youtube.com/playlist?list=PLBKcU7Ik-ir-b1fwjNabO3b8ebs9ez5ga

Go through the code base, explore it and find the file(s) where the edits need to be made.

Read Stack Overflow's "How do I ask a good question?"

Be inclusive in your communications: inclusive language

Do not use salutations such as "Hello Sir", or "Hey guys" (Why: these terms exclude women and non-binary people.)

Do not use salutations such as "Hello Sir/Madam" (Why: these terms assume gender is binary, and they exclude non-binary people.)

In general, no need to address people here as "Sir" or "Maam" or "Madam".

If you begin work on an issue, indicate so on the issue, so others do not duplicate the work. Comment on the issue, say "I'm working on this" or there's also a command now where you can just say "take" and the bot will assign the issue to you.

Note that due to limited resources, it may take many days before we can review your pull request. Please be patient, and if you don't receive any feedback within two weeks, you can give us a reminder on gitter.

Issue 2:
Describe the bug

I will double-cross-validation with GroupKFold, LeaveOneGroupOut.

What Is Nested Cross-Validation

In the example of KFold, Double-CV can be executed by the following simple code.

X, y, groups = something defined

estimator = RandomForestRegressor()
param_grid = {'n_estimators':[5,10], 'max_depth':[3,5]}
opt_cv = KFold(n_splits=2)
eval_cv = KFold(n_splits=2)
gsv_cv = GridSearchCV(estimator , param_grid, cv=opt_cv)
double_cv_score = cross_val_score(gsv_cv, X,y ,groups=groups, cv = eval_cv )

By the way, attempting to evaluate the same with GroupKFold does not work because of the specification that requires groups in cv.split.

X, y, groups = something defined

estimator = RandomForestRegressor()
param_grid = {'n_estimators':[5,10], 'max_depth':[3,5]}
opt_cv = LeaveOneGroupOut()
eval_cv = LeaveOneGroupOut()
gsv_cv = GridSearchCV(estimator , param_grid, cv=opt_cv)
double_cv_score = cross_val_score(gsv_cv, X,y ,groups=groups, cv = eval_cv )

raise ValueError: The 'groups' parameter should not be None.

This is because the argument of the function "_fit_and_predict" of cross_val_predict is limited to "estimator, X, y, train, test, verbose, fit_params, method" and [groups] cannot be taken as an argument.

def _fit_and_predict(estimator, X, y, train, test, verbose, fit_params, method):

It should be

def _fit_and_predict(estimator, X, y, train, test, verbose, fit_params, method, groups=None):

Also, GridSearchCV does not give groups to the argument of cross_val_score, which also needs to be modified.

scores = np.ndarray.flatten (
cross_val_score (estimator, X, y, cv = self.cv, scoring = scorers))

It should be

scores = np.ndarray.flatten (
cross_val_score (estimator, X, y, groups=groups, cv = self.cv, scoring = scorers))

Alternatively, you can use the notation that gives cv a CV generator.

An error occurs because the generator cannot be cloned.

**opt_splitter = opt_cv.split(X, y, groups)**

**gsv_cv = GridSearchCV(estimator , param_grid, cv=opt_splitter )**
gsv_cv.fit(X,y, groups=group)
gsv_cv.score(X,y)

splitter_ = eval_cv.split(X, y, groups = group_s)
print(type(splitter_))
cr = cross_val_predict(gsv_cv, X,y ,groups=group, cv = splitter_)

\lib\copy.py in deepcopy(x, memo, _nil)
167 reductor = getattr(x, "__reduce_ex__", None)
168 if reductor:
--> 169 rv = reductor(4)
170 else:
171 reductor = getattr(x, "__reduce__", None)

TypeError: can't pickle generator objects

Steps/Code to Reproduce

param_grid = {'n_estimators':[10], 'max_depth':[3]}
estimator = RandomForestRegressor()
opt_cv= LeaveOneGroupOut()
eval_cv = LeaveOneGroupOut()
gsv_cv = GridSearchCV(estimator , param_grid, cv=opt_cv)
cr = cross_val_predict(gsv_cv, X,y ,groups=group, cv = eval_cv )

or

param_grid = {'n_estimators':[10], 'max_depth':[3]}
estimator = RandomForestRegressor()
opt_cv= LeaveOneGroupOut()
eval_cv = LeaveOneGroupOut()
opt_splitter = inner_cv.split(X, y, group)
gsv_cv = GridSearchCV(estimator , param_grid, cv=opt_splitter )
cr = cross_val_predict(gsv_cv, X,y ,groups=group, cv = eval_cv )

Expected Results

No Error

Actual Results

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_11376/3990998576.py in <module>
20 splitter_ = eval_cv.split(X, y, groups = group_s)
21 print(type(splitter_))
---> 22 cr = cross_val_predict(gsv_cv, X,y ,groups=group, cv = splitter_)

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs)
61 extra_args = len(args) - len(all_args)
62 if extra_args <= 0:
---> 63 return f(*args, **kwargs)
64
65 # extra_args > 0

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\model_selection\_validation.py in cross_val_predict(estimator, X, y, groups, cv, n_jobs, verbose, fit_params, pre_dispatch, method)
880 predictions = parallel(delayed(_fit_and_predict)(
881 clone(estimator), X, y, train, test, verbose, fit_params, method)
--> 882 for train, test in splits)
883
884 inv_test_indices = np.empty(len(test_indices), dtype=int)

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\parallel.py in __call__(self, iterable)
1041 # remaining jobs.
1042 self._iterating = False
-> 1043 if self.dispatch_one_batch(iterator):
1044 self._iterating = self._original_iterator is not None
1045

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\parallel.py in dispatch_one_batch(self, iterator)
859 return False
860 else:
--> 861 self._dispatch(tasks)
862 return True
863

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\parallel.py in _dispatch(self, batch)
777 with self._lock:
778 job_idx = len(self._jobs)
--> 779 job = self._backend.apply_async(batch, callback=cb)
780 # A job can complete so quickly than its callback is
781 # called before we get here, causing self._jobs to

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\_parallel_backends.py in apply_async(self, func, callback)
206 def apply_async(self, func, callback=None):
207 """Schedule a func to be run"""
--> 208 result = ImmediateResult(func)
209 if callback:
210 callback(result)

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\_parallel_backends.py in __init__(self, batch)
570 # Don't delay the application, to avoid keeping the input
571 # arguments in memory
--> 572 self.results = batch()
573
574 def get(self):

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\parallel.py in __call__(self)
261 with parallel_backend(self._backend, n_jobs=self._n_jobs):
262 return [func(*args, **kwargs)
--> 263 for func, args, kwargs in self.items]
264
265 def __reduce__(self):

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\parallel.py in <listcomp>(.0)
261 with parallel_backend(self._backend, n_jobs=self._n_jobs):
262 return [func(*args, **kwargs)
--> 263 for func, args, kwargs in self.items]
264
265 def __reduce__(self):

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\utils\fixes.py in __call__(self, *args, **kwargs)
220 def __call__(self, *args, **kwargs):
221 with config_context(**self.config):
--> 222 return self.function(*args, **kwargs)

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\model_selection\_validation.py in _fit_and_predict(estimator, X, y, train, test, verbose, fit_params, method)
958 estimator.fit(X_train, **fit_params)
959 else:
--> 960 estimator.fit(X_train, y_train, **fit_params)
961 func = getattr(estimator, method)
962 predictions = func(X_test)

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs)
61 extra_args = len(args) - len(all_args)
62 if extra_args <= 0:
---> 63 return f(*args, **kwargs)
64
65 # extra_args > 0

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\model_selection\_search.py in fit(self, X, y, groups, **fit_params)
761
762 cv_orig = check_cv(self.cv, y, classifier=is_classifier(estimator))
--> 763 n_splits = cv_orig.get_n_splits(X, y, groups)
764
765 base_estimator = clone(self.estimator)

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\model_selection\_split.py in get_n_splits(self, X, y, groups)
967 """
968 if groups is None:
--> 969 raise ValueError("The 'groups' parameter should not be None.")
970 groups = check_array(groups, ensure_2d=False, dtype=None)
971 return len(np.unique(groups))

ValueError: The 'groups' parameter should not be None.

or

---------------------------------------------------------------------------
Empty Traceback (most recent call last)
~\Anaconda3\envs\py37_test\lib\site-packages\joblib\parallel.py in dispatch_one_batch(self, iterator)
821 try:
--> 822 tasks = self._ready_batches.get(block=False)
823 except queue.Empty:

~\Anaconda3\envs\py37_test\lib\queue.py in get(self, block, timeout)
166 if not self._qsize():
--> 167 raise Empty
168 elif timeout is None:

Empty:

During handling of the above exception, another exception occurred:

TypeError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_11376/2630706207.py in <module>
16 splitter_ = eval_cv.split(X, y, groups = group_s)
17 print(type(splitter_))
---> 18 cr = cross_val_predict(gsv_cv, X,y ,groups=group, cv = eval_cv)

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs)
61 extra_args = len(args) - len(all_args)
62 if extra_args <= 0:
---> 63 return f(*args, **kwargs)
64
65 # extra_args > 0

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\model_selection\_validation.py in cross_val_predict(estimator, X, y, groups, cv, n_jobs, verbose, fit_params, pre_dispatch, method)
880 predictions = parallel(delayed(_fit_and_predict)(
881 clone(estimator), X, y, train, test, verbose, fit_params, method)
--> 882 for train, test in splits)
883
884 inv_test_indices = np.empty(len(test_indices), dtype=int)

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\parallel.py in __call__(self, iterable)
1041 # remaining jobs.
1042 self._iterating = False
-> 1043 if self.dispatch_one_batch(iterator):
1044 self._iterating = self._original_iterator is not None
1045

~\Anaconda3\envs\py37_test\lib\site-packages\joblib\parallel.py in dispatch_one_batch(self, iterator)
831 big_batch_size = batch_size * n_jobs
832
--> 833 islice = list(itertools.islice(iterator, big_batch_size))
834 if len(islice) == 0:
835 return False

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\model_selection\_validation.py in <genexpr>(.0)
880 predictions = parallel(delayed(_fit_and_predict)(
881 clone(estimator), X, y, train, test, verbose, fit_params, method)
--> 882 for train, test in splits)
883
884 inv_test_indices = np.empty(len(test_indices), dtype=int)

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs)
61 extra_args = len(args) - len(all_args)
62 if extra_args <= 0:
---> 63 return f(*args, **kwargs)
64
65 # extra_args > 0

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\base.py in clone(estimator, safe)
74 new_object_params = estimator.get_params(deep=False)
75 for name, param in new_object_params.items():
---> 76 new_object_params[name] = clone(param, safe=False)
77 new_object = klass(**new_object_params)
78 params_set = new_object.get_params(deep=False)

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs)
61 extra_args = len(args) - len(all_args)
62 if extra_args <= 0:
---> 63 return f(*args, **kwargs)
64
65 # extra_args > 0

~\Anaconda3\envs\py37_test\lib\site-packages\sklearn\base.py in clone(estimator, safe)
58 elif not hasattr(estimator, 'get_params') or isinstance(estimator, type):
59 if not safe:
---> 60 return copy.deepcopy(estimator)
61 else:
62 if isinstance(estimator, type):

~\Anaconda3\envs\py37_test\lib\copy.py in deepcopy(x, memo, _nil)
167 reductor = getattr(x, "__reduce_ex__", None)
168 if reductor:
--> 169 rv = reductor(4)
170 else:
171 reductor = getattr(x, "__reduce__", None)

TypeError: can't pickle generator objects

Versions

System:

python: 3.7.11 (default, Jul 27 2021, 09:42:29) [MSC v.1916 64 bit (AMD64)]

executable: C:\Users\11665307\Anaconda3\envs\py37_test\python.exe

machine: Windows-10-10.0.19041-SP0

Python dependencies:

pip: 21.2.4

setuptools: 58.0.4

sklearn: 0.24.2

numpy: 1.20.3

scipy: 1.6.3

Cython: 0.29.26

pandas: 1.3.4

matplotlib: 3.5.0

joblib: 1.1.0

threadpoolctl: 3.0.0

Built with OpenMP: True