Repository: feature-engine/feature_engine
Pull request #804 (2024-08-27 13:38:47)

Groups support in feature selection for cross validate
Currently feature selection methods: `BaseRecursiveSelector`, `ProbeFeatureSelection`, `RecursiveFeatureAddition`, `RecursiveFeatureElimination`, `SelectByShuffling`, `SelectBySingleFeaturePerformance`, `SelectByTargetMeanPerformance` use `cross_validate` from `sklearn`, which support `groups` argument for group based splits, but feature_engine doesn't support it. Purpose of this PR is to enable feature_engine to have feature selection with cross_validation based on groups too. Example usage: While working with multiple time series data, for example, store item sales, same dates are repeated for multiple items, and during cross-validation it is good practice to use time series splits considering groups of items for each date. 

**Update:**

- Imports were put into original format, but that format doesn't align with isort, and running tox doesn't use isort
- Groups are given default None value for every class and function it is used, so current tests do not have to pass groups
- If everything is ok with current changes, I will write unit tests for groups usage

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/feature-engine/feature_engine/pull/804?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 97.98%. Comparing base [(`e3f99db`)](https://app.codecov.io/gh/feature-engine/feature_engine/commit/e3f99dbe61d133aa8e557236ac2844cd72eaa43d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine) to head [(`000c8d1`)](https://app.codecov.io/gh/feature-engine/feature_engine/commit/000c8d18ca5555b764219134aef6cf149a4f65f7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine).

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##             main     #804   +/-   ##
=======================================
  Coverage   97.98%   97.98%           
=======================================
  Files         107      107           
  Lines        4312     4319    +7     
  Branches      857      857           
=======================================
+ Hits         4225     4232    +7     
  Misses         54       54           
  Partials       33       33           
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/feature-engine/feature_engine/pull/804?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=feature-engine).

KananMahammadli: - Tests have been added to check if groups perform exactly same as passing indices
- groups option has been removed from ShuffeFeatureSelection, as it randomly changes values of features, which changes group order too, causing abnormal behaviour

@solegalli can you please check if everything is ok?

Patch:
diff --git a/feature_engine/_docstrings/selection/_docstring.py b/feature_engine/_docstrings/selection/_docstring.py
index 4344d6a..21dc4cf 100644
--- a/feature_engine/_docstrings/selection/_docstring.py
+++ b/feature_engine/_docstrings/selection/_docstring.py
@@ -53,6 +53,11 @@ _cv_docstring = """cv: int, cross-validation generator or an iterable, default=3
         documentation.
         """.rstrip()
 
+_groups_docstring = """groups: array-like of shape (n_samples,), default=None
+        Group labels for the samples used while splitting the dataset into train/test set.
+        Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
+        """.rstrip()
+
 _initial_model_performance_docstring = """initial_model_performance_:
         The model's performance when trained with the original dataset.
         """.rstrip()
diff --git a/feature_engine/selection/base_recursive_selector.py b/feature_engine/selection/base_recursive_selector.py
index f84c5d9..79bb628 100644
--- a/feature_engine/selection/base_recursive_selector.py
+++ b/feature_engine/selection/base_recursive_selector.py
@@ -66,6 +66,11 @@ class BaseRecursiveSelector(BaseSelector):
         across calls. For more details check Scikit-learn's `cross_validate`'s
         documentation.
 
+    groups: Array-like of shape (n_samples,), default=None
+        Group labels for the samples used while splitting
+        the dataset into train/test set. Only used in conjunction with a
+        “Group” cv instance (e.g., GroupKFold).
+
     confirm_variables: bool, default=False
         If set to True, variables that are not present in the input dataframe will be
         removed from the list of variables. Only used when passing a variable list to
@@ -105,6 +110,7 @@ class BaseRecursiveSelector(BaseSelector):
         estimator,
         scoring: str = "roc_auc",
         cv=3,
+        groups=None,
         threshold: Union[int, float] = 0.01,
         variables: Variables = None,
         confirm_variables: bool = False,
@@ -119,6 +125,7 @@ class BaseRecursiveSelector(BaseSelector):
         self.scoring = scoring
         self.threshold = threshold
         self.cv = cv
+        self.groups = groups
 
     def fit(self, X: pd.DataFrame, y: pd.Series):
         """
@@ -155,10 +162,11 @@ class BaseRecursiveSelector(BaseSelector):
 
         # train model with all features and cross-validation
         model = cross_validate(
-            self.estimator,
-            X[self.variables_],
-            y,
+            estimator=self.estimator,
+            X=X[self.variables_],
+            y=y,
             cv=self._cv,
+            groups=self.groups,
             scoring=self.scoring,
             return_estimator=True,
         )
diff --git a/feature_engine/selection/base_selection_functions.py b/feature_engine/selection/base_selection_functions.py
index 9bde522..f44f7d4 100644
--- a/feature_engine/selection/base_selection_functions.py
+++ b/feature_engine/selection/base_selection_functions.py
@@ -167,6 +167,7 @@ def single_feature_performance(
     estimator,
     cv,
     scoring,
+    groups=None,
 ):
     """
     Trains one estimator per feature and determines the performance of that estimator.
@@ -191,6 +192,11 @@ def single_feature_performance(
     scoring:
         The performance metric. Any supported by the Scikit-learn estimator.
 
+    groups: Array-like of shape (n_samples,), default=None
+        Group labels for the samples used while splitting
+        the dataset into train/test set. Only used in conjunction with a
+        “Group” cv instance (e.g., GroupKFold).
+
     Returns
     -------
     feature_performance: dict
@@ -213,6 +219,7 @@ def single_feature_performance(
             X[feature].to_frame(),
             y,
             cv=cv,
+            groups=groups,
             return_estimator=False,
             scoring=scoring,
         )
@@ -228,6 +235,7 @@ def find_feature_importance(
     estimator,
     cv,
     scoring,
+    groups=None,
 ):
     """
     Trains an estimator using cross-validation and derives feature importance from it.
@@ -253,6 +261,11 @@ def find_feature_importance(
     scoring:
         The performance metric. Any supported by the Scikit-learn estimator.
 
+    groups: Array-like of shape (n_samples,), default=None
+        Group labels for the samples used while splitting
+        the dataset into train/test set. Only used in conjunction with a
+        “Group” cv instance (e.g., GroupKFold).
+
     Returns
     -------
     feature_importance: pd.Series
@@ -271,6 +284,7 @@ def find_feature_importance(
         X,
         y,
         cv=cv,
+        groups=groups,
         scoring=scoring,
         return_estimator=True,
     )
diff --git a/feature_engine/selection/probe_feature_selection.py b/feature_engine/selection/probe_feature_selection.py
index d1fa613..985f4ac 100644
--- a/feature_engine/selection/probe_feature_selection.py
+++ b/feature_engine/selection/probe_feature_selection.py
@@ -14,6 +14,7 @@ from feature_engine._docstrings.init_parameters.selection import (
 from feature_engine._docstrings.methods import _fit_transform_docstring
 from feature_engine._docstrings.selection._docstring import (
     _cv_docstring,
+    _groups_docstring,
     _features_to_drop_docstring,
     _fit_docstring,
     _get_support_docstring,
@@ -40,6 +41,7 @@ Variables = Union[None, int, str, List[Union[str, int]]]
     estimator=_estimator_docstring,
     scoring=_scoring_docstring,
     cv=_cv_docstring,
+    groups=_groups_docstring,
     confirm_variables=_confirm_variables_docstring,
     variables=_variables_numerical_docstring,
     feature_names_in_=_feature_names_in_docstring,
@@ -104,6 +106,8 @@ class ProbeFeatureSelection(BaseSelector):
 
     {cv}
 
+    {groups}
+
     Attributes
     ----------
     probe_features_:
@@ -173,6 +177,7 @@ class ProbeFeatureSelection(BaseSelector):
         n_probes: int = 1,
         distribution: str = "normal",
         cv=5,
+        groups=None,
         random_state: int = 0,
         confirm_variables: bool = False,
     ):
@@ -203,6 +208,7 @@ class ProbeFeatureSelection(BaseSelector):
         self.scoring = scoring
         self.distribution = distribution
         self.cv = cv
+        self.groups = groups
         self.n_probes = n_probes
         self.random_state = random_state
 
@@ -238,7 +244,12 @@ class ProbeFeatureSelection(BaseSelector):
         if self.collective is True:
             # train model using entire dataset and derive feature importance
             f_importance_mean, f_importance_std = find_feature_importance(
-                X_new, y, self.estimator, self.cv, self.scoring,
+                X=X_new,
+                y=y,
+                estimator=self.estimator,
+                cv=self.cv,
+                groups=self.groups,
+                scoring=self.scoring,
             )
             self.feature_importances_ = f_importance_mean
             self.feature_importances_std_ = f_importance_std
@@ -246,12 +257,13 @@ class ProbeFeatureSelection(BaseSelector):
         else:
             # trains a model per feature (single feature models)
             f_importance_mean, f_importance_std = single_feature_performance(
-                X_new,
-                y,
-                X_new.columns,
-                self.estimator,
-                self.cv,
-                self.scoring,
+                X=X_new,
+                y=y,
+                variables=X_new.columns,
+                estimator=self.estimator,
+                cv=self.cv,
+                groups=self.groups,
+                scoring=self.scoring,
             )
             self.feature_importances_ = pd.Series(f_importance_mean)
             self.feature_importances_std_ = pd.Series(f_importance_std)
diff --git a/feature_engine/selection/recursive_feature_addition.py b/feature_engine/selection/recursive_feature_addition.py
index 499f2d1..155d71b 100644
--- a/feature_engine/selection/recursive_feature_addition.py
+++ b/feature_engine/selection/recursive_feature_addition.py
@@ -19,6 +19,7 @@ from feature_engine._docstrings.selection._docstring import (
     _features_to_drop_docstring,
     _fit_docstring,
     _get_support_docstring,
+    _groups_docstring,
     _initial_model_performance_docstring,
     _scoring_docstring,
     _threshold_docstring,
@@ -35,6 +36,7 @@ from feature_engine.selection.base_recursive_selector import BaseRecursiveSelect
     scoring=_scoring_docstring,
     threshold=_threshold_docstring,
     cv=_cv_docstring,
+    groups=_groups_docstring,
     variables=_variables_numerical_docstring,
     confirm_variables=_confirm_variables_docstring,
     initial_model_performance_=_initial_model_performance_docstring,
@@ -87,6 +89,8 @@ class RecursiveFeatureAddition(BaseRecursiveSelector):
 
     {cv}
 
+    {groups}
+
     {confirm_variables}
 
     Attributes
@@ -167,10 +171,11 @@ class RecursiveFeatureAddition(BaseRecursiveSelector):
 
         # Run baseline model using only the most important feature
         baseline_model = cross_validate(
-            self.estimator,
-            X[first_most_important_feature].to_frame(),
-            y,
+            estimator=self.estimator,
+            X=X[first_most_important_feature].to_frame(),
+            y=y,
             cv=self._cv,
+            groups=self.groups,
             scoring=self.scoring,
             return_estimator=True,
         )
@@ -194,10 +199,11 @@ class RecursiveFeatureAddition(BaseRecursiveSelector):
 
             # Add feature and train new model
             model_tmp = cross_validate(
-                self.estimator,
-                X[_selected_features + [feature]],
-                y,
+                estimator=self.estimator,
+                X=X[_selected_features + [feature]],
+                y=y,
                 cv=self._cv,
+                groups=self.groups,
                 scoring=self.scoring,
                 return_estimator=True,
             )
diff --git a/feature_engine/selection/recursive_feature_elimination.py b/feature_engine/selection/recursive_feature_elimination.py
index 1320a8c..034a9e0 100644
--- a/feature_engine/selection/recursive_feature_elimination.py
+++ b/feature_engine/selection/recursive_feature_elimination.py
@@ -19,6 +19,7 @@ from feature_engine._docstrings.selection._docstring import (
     _features_to_drop_docstring,
     _fit_docstring,
     _get_support_docstring,
+    _groups_docstring,
     _initial_model_performance_docstring,
     _scoring_docstring,
     _threshold_docstring,
@@ -35,6 +36,7 @@ from feature_engine.selection.base_recursive_selector import BaseRecursiveSelect
     scoring=_scoring_docstring,
     threshold=_threshold_docstring,
     cv=_cv_docstring,
+    groups=_groups_docstring,
     variables=_variables_numerical_docstring,
     confirm_variables=_confirm_variables_docstring,
     initial_model_performance_=_initial_model_performance_docstring,
@@ -88,6 +90,8 @@ class RecursiveFeatureElimination(BaseRecursiveSelector):
 
     {cv}
 
+    {groups}
+
     {confirm_variables}
 
     Attributes
@@ -186,10 +190,11 @@ class RecursiveFeatureElimination(BaseRecursiveSelector):
 
             # remove feature and train new model
             model_tmp = cross_validate(
-                self.estimator,
-                X_tmp.drop(columns=feature),
-                y,
+                estimator=self.estimator,
+                X=X_tmp.drop(columns=feature),
+                y=y,
                 cv=self._cv,
+                groups=self.groups,
                 scoring=self.scoring,
                 return_estimator=False,
             )
@@ -213,10 +218,11 @@ class RecursiveFeatureElimination(BaseRecursiveSelector):
                 X_tmp = X_tmp.drop(columns=feature)
 
                 baseline_model = cross_validate(
-                    self.estimator,
-                    X_tmp,
-                    y,
+                    estimator=self.estimator,
+                    X=X_tmp,
+                    y=y,
                     cv=self._cv,
+                    groups=self.groups,
                     return_estimator=False,
                     scoring=self.scoring,
                 )
diff --git a/feature_engine/selection/shuffle_features.py b/feature_engine/selection/shuffle_features.py
index 6dd9e0c..62c61b3 100644
--- a/feature_engine/selection/shuffle_features.py
+++ b/feature_engine/selection/shuffle_features.py
@@ -226,9 +226,9 @@ class SelectByShuffling(BaseSelector):
 
         # train model with all features and cross-validation
         model = cross_validate(
-            self.estimator,
-            X[self.variables_],
-            y,
+            estimator=self.estimator,
+            X=X[self.variables_],
+            y=y,
             cv=cv,
             return_estimator=True,
             scoring=self.scoring,
diff --git a/feature_engine/selection/single_feature_performance.py b/feature_engine/selection/single_feature_performance.py
index 65a4fac..4b0fd9a 100644
--- a/feature_engine/selection/single_feature_performance.py
+++ b/feature_engine/selection/single_feature_performance.py
@@ -16,6 +16,7 @@ from feature_engine._docstrings.init_parameters.selection import (
 from feature_engine._docstrings.methods import _fit_transform_docstring
 from feature_engine._docstrings.selection._docstring import (
     _cv_docstring,
+    _groups_docstring,
     _estimator_docstring,
     _features_to_drop_docstring,
     _fit_docstring,
@@ -45,6 +46,7 @@ Variables = Union[None, int, str, List[Union[str, int]]]
     scoring=_scoring_docstring,
     threshold=_threshold_docstring,
     cv=_cv_docstring,
+    groups=_groups_docstring,
     variables=_variables_numerical_docstring,
     confirm_variables=_confirm_variables_docstring,
     initial_model_performance_=_initial_model_performance_docstring,
@@ -83,6 +85,8 @@ class SelectBySingleFeaturePerformance(BaseSelector):
 
     {cv}
 
+    {groups}
+
     {confirm_variables}
 
     Attributes
@@ -147,6 +151,7 @@ class SelectBySingleFeaturePerformance(BaseSelector):
         estimator,
         scoring: str = "roc_auc",
         cv=3,
+        groups=None,
         threshold: Union[int, float, None] = None,
         variables: Variables = None,
         confirm_variables: bool = False,
@@ -177,6 +182,7 @@ class SelectBySingleFeaturePerformance(BaseSelector):
         self.scoring = scoring
         self.threshold = threshold
         self.cv = cv
+        self.groups = groups
 
     def fit(self, X: pd.DataFrame, y: pd.Series):
         """
@@ -209,7 +215,13 @@ class SelectBySingleFeaturePerformance(BaseSelector):
             )
 
         self.feature_performance_, _ = single_feature_performance(
-            X, y, self.variables_, self.estimator, self.cv, self.scoring
+            X=X,
+            y=y,
+            variables=self.variables_,
+            estimator=self.estimator,
+            cv=self.cv,
+            groups=self.groups,
+            scoring=self.scoring,
         )
 
         # select features
diff --git a/feature_engine/selection/smart_correlation_selection.py b/feature_engine/selection/smart_correlation_selection.py
index 1396e57..8fd9afa 100644
--- a/feature_engine/selection/smart_correlation_selection.py
+++ b/feature_engine/selection/smart_correlation_selection.py
@@ -16,6 +16,7 @@ from feature_engine._docstrings.init_parameters.selection import (
 from feature_engine._docstrings.methods import _fit_transform_docstring
 from feature_engine._docstrings.selection._docstring import (
     _cv_docstring,
+    _groups_docstring,
     _estimator_docstring,
     _get_support_docstring,
     _missing_values_docstring,
@@ -44,6 +45,7 @@ Variables = Union[None, int, str, List[Union[str, int]]]
     estimator=_estimator_docstring,
     scoring=_scoring_docstring,
     cv=_cv_docstring,
+    groups=_groups_docstring,
     confirm_variables=_confirm_variables_docstring,
     variables=_variables_numerical_docstring,
     missing_values=_missing_values_docstring,
@@ -119,6 +121,8 @@ class SmartCorrelatedSelection(BaseSelector):
 
     {cv}
 
+    {groups}
+
     {confirm_variables}
 
     Attributes
@@ -206,6 +210,7 @@ class SmartCorrelatedSelection(BaseSelector):
         estimator=None,
         scoring: str = "roc_auc",
         cv=3,
+        groups=None,
         confirm_variables: bool = False,
     ):
         if not isinstance(threshold, float) or threshold < 0 or threshold > 1:
@@ -253,6 +258,7 @@ class SmartCorrelatedSelection(BaseSelector):
         self.estimator = estimator
         self.scoring = scoring
         self.cv = cv
+        self.groups = groups
 
     def fit(self, X: pd.DataFrame, y: pd.Series = None):
         """
@@ -321,12 +327,13 @@ class SmartCorrelatedSelection(BaseSelector):
             cv = list(self.cv) if isinstance(self.cv, GeneratorType) else self.cv
             for feature_group in correlated_groups:
                 feature_performance, _ = single_feature_performance(
-                    X,
-                    y,
-                    feature_group,
-                    self.estimator,
-                    cv,
-                    self.scoring,
+                    X=X,
+                    y=y,
+                    variables=feature_group,
+                    estimator=self.estimator,
+                    cv=cv,
+                    groups=self.groups,
+                    scoring=self.scoring,
                 )
                 # get most important feature
                 f_i = (
diff --git a/feature_engine/selection/target_mean_selection.py b/feature_engine/selection/target_mean_selection.py
index 625263f..c6f5d20 100644
--- a/feature_engine/selection/target_mean_selection.py
+++ b/feature_engine/selection/target_mean_selection.py
@@ -21,6 +21,7 @@ from feature_engine._docstrings.selection._docstring import (
     _features_to_drop_docstring,
     _fit_docstring,
     _get_support_docstring,
+    _groups_docstring,
     _scoring_docstring,
     _threshold_docstring,
     _transform_docstring,
@@ -46,6 +47,7 @@ Variables = Union[None, int, str, List[Union[str, int]]]
     scoring=_scoring_docstring,
     threshold=_threshold_docstring,
     cv=_cv_docstring,
+    groups=_groups_docstring,
     confirm_variables=_confirm_variables_docstring,
     features_to_drop_=_features_to_drop_docstring,
     variables_=_variables_attribute_docstring,
@@ -125,6 +127,8 @@ class SelectByTargetMeanPerformance(BaseSelector):
 
     {cv}
 
+    {groups}
+
     regression: boolean, default=True
         Indicates whether the target is one for regression or a classification.
 
@@ -213,6 +217,7 @@ class SelectByTargetMeanPerformance(BaseSelector):
         strategy: str = "equal_width",
         scoring: str = "roc_auc",
         cv=3,
+        groups=None,
         threshold: Union[int, float, None] = None,
         regression: bool = False,
         confirm_variables: bool = False,
@@ -252,6 +257,7 @@ class SelectByTargetMeanPerformance(BaseSelector):
         self.strategy = strategy
         self.scoring = scoring
         self.cv = cv
+        self.groups = groups
         self.threshold = threshold
         self.regression = regression
 
@@ -310,10 +316,11 @@ class SelectByTargetMeanPerformance(BaseSelector):
             estimator.set_params(variables=variable)
 
             model = cross_validate(
-                estimator,
-                X,
-                y,
+                estimator=estimator,
+                X=X,
+                y=y,
                 cv=cv,
+                groups=self.groups,
                 scoring=self.scoring,
             )
 


Test patch:
diff --git a/tests/test_selection/conftest.py b/tests/test_selection/conftest.py
index c3ac27c..e41d7ce 100644
--- a/tests/test_selection/conftest.py
+++ b/tests/test_selection/conftest.py
@@ -25,6 +25,37 @@ def df_test():
     return X, y
 
 
+@pytest.fixture(scope="module")
+def df_test_with_groups():
+    # Parameters
+    n_samples = 100  # Total number of samples
+    n_groups = 10    # Total number of groups
+    n_features = 5   # Number of features
+
+    # Generate random features
+    np.random.seed(1)
+    features = np.random.randn(n_samples, n_features)
+
+    # Generate random target variable
+    target = np.random.randint(0, 100, size=n_samples)
+
+    # Generate groups
+    groups = np.repeat(np.arange(1, n_groups + 1), n_samples // n_groups)
+    np.random.shuffle(groups)
+
+    # Create DataFrame
+    df = pd.DataFrame(features, columns=[f'var_{i+1}' for i in range(n_features)])
+    df['target'] = target
+    df['group'] = groups
+
+    features = [col for col in df.columns if col.startswith('var')]
+    X = df[features]
+    y = df['target']
+    groups = df['group']
+
+    return X, y, groups
+
+
 @pytest.fixture(scope="module")
 def load_diabetes_dataset():
     # Load the diabetes dataset from sklearn
diff --git a/tests/test_selection/test_base_selection_functions.py b/tests/test_selection/test_base_selection_functions.py
index 317a69c..b2345a5 100644
--- a/tests/test_selection/test_base_selection_functions.py
+++ b/tests/test_selection/test_base_selection_functions.py
@@ -1,7 +1,8 @@
 import pandas as pd
 import pytest
 from sklearn.ensemble import RandomForestClassifier
-from sklearn.model_selection import StratifiedKFold
+from sklearn.model_selection import StratifiedKFold, GroupKFold
+
 
 from feature_engine.selection.base_selection_functions import (
     _select_all_variables,
@@ -125,7 +126,14 @@ def test_single_feature_performance(df_test):
     rf = RandomForestClassifier(n_estimators=5, random_state=1)
     variables = X.columns.to_list()
 
-    mean_, std_ = single_feature_performance(X, y, variables, rf, 3, "roc_auc")
+    mean_, std_ = single_feature_performance(
+        X=X,
+        y=y,
+        variables=variables,
+        estimator=rf,
+        cv=3,
+        scoring="roc_auc",
+    )
 
     expected_mean = {
         "var_0": 0.5813469607144305,
@@ -165,7 +173,14 @@ def test_single_feature_performance_cv_generator(df_test):
     variables = X.columns.to_list()
     cv = StratifiedKFold(n_splits=3)
     for cv_ in [cv, cv.split(X, y)]:
-        mean_, _ = single_feature_performance(X, y, variables, rf, cv_, "roc_auc")
+        mean_, _ = single_feature_performance(
+            X=X,
+            y=y,
+            variables=variables,
+            estimator=rf,
+            cv=cv_,
+            scoring="roc_auc",
+        )
 
         expected_mean = {
             "var_0": 0.5813469607144305,
@@ -184,6 +199,37 @@ def test_single_feature_performance_cv_generator(df_test):
         assert mean_ == expected_mean
 
 
+def test_single_feature_performance_with_groups(df_test_with_groups):
+    X, y, groups = df_test_with_groups
+    rf = RandomForestClassifier(n_estimators=5, random_state=1)
+    variables = X.columns.to_list()
+    scoring = "neg_mean_absolute_error"
+    cv = GroupKFold(n_splits=3)
+    cv_indices = cv.split(X=X, y=y, groups=groups)
+
+    expected_mean_, expected_std_ = single_feature_performance(
+        X=X,
+        y=y,
+        variables=variables,
+        estimator=rf,
+        cv=cv_indices,
+        scoring=scoring,
+    )
+
+    mean_, std_ = single_feature_performance(
+        X=X,
+        y=y,
+        variables=variables,
+        estimator=rf,
+        cv=cv,
+        scoring=scoring,
+        groups=groups,
+    )
+
+    assert mean_ == expected_mean_
+    assert std_ == expected_std_
+
+
 def test_find_feature_importance(df_test):
     X, y = df_test
     rf = RandomForestClassifier(n_estimators=3, random_state=3)
@@ -238,10 +284,50 @@ def test_find_feature_importance(df_test):
         ],
     )
 
-    mean_, std_ = find_feature_importance(X, y, rf, cv, scoring)
+    mean_, std_ = find_feature_importance(
+        X=X,
+        y=y,
+        estimator=rf,
+        cv=cv,
+        scoring=scoring,
+    )
     pd.testing.assert_series_equal(mean_.round(2), expected_mean)
     pd.testing.assert_series_equal(std_.round(4), expected_std)
 
-    mean_, std_ = find_feature_importance(X, y, rf, cv.split(X, y), scoring)
+    mean_, std_ = find_feature_importance(
+        X=X,
+        y=y,
+        estimator=rf,
+        cv=cv.split(X, y),
+        scoring=scoring,
+    )
     pd.testing.assert_series_equal(mean_.round(2), expected_mean)
     pd.testing.assert_series_equal(std_.round(4), expected_std)
+
+
+def test_find_feature_importancewith_groups(df_test_with_groups):
+    X, y, groups = df_test_with_groups
+    rf = RandomForestClassifier(n_estimators=3, random_state=1)
+    cv = GroupKFold(n_splits=3)
+    scoring = "neg_mean_absolute_error"
+    cv_indices = cv.split(X=X, y=y, groups=groups)
+
+    expected_mean_, expected_std_ = find_feature_importance(
+        X=X,
+        y=y,
+        estimator=rf,
+        cv=cv_indices,
+        scoring=scoring,
+    )
+
+    mean_, std_ = find_feature_importance(
+        X=X,
+        y=y,
+        estimator=rf,
+        cv=cv,
+        scoring=scoring,
+        groups=groups
+    )
+
+    pd.testing.assert_series_equal(mean_, expected_mean_)
+    pd.testing.assert_series_equal(std_, expected_std_)
diff --git a/tests/test_selection/test_probe_feature_selection.py b/tests/test_selection/test_probe_feature_selection.py
index 1d55434..a21a012 100644
--- a/tests/test_selection/test_probe_feature_selection.py
+++ b/tests/test_selection/test_probe_feature_selection.py
@@ -1,8 +1,8 @@
 import pandas as pd
 import pytest
-from sklearn.ensemble import RandomForestClassifier
+from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
 from sklearn.linear_model import Lasso, LogisticRegression
-from sklearn.model_selection import StratifiedKFold
+from sklearn.model_selection import StratifiedKFold, GroupKFold
 from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor
 
 from feature_engine.selection import ProbeFeatureSelection
@@ -432,3 +432,41 @@ def test_single_feature_importance_generation(df_test):
     )
 
     assert sel.feature_importances_.round(4).equals(expected_)
+
+
+def test_probe_feature_selector_with_groups(df_test_with_groups):
+    X, y, groups = df_test_with_groups
+    cv = GroupKFold(n_splits=3)
+    cv_indices = cv.split(X=X, y=y, groups=groups)
+
+    estimator = RandomForestRegressor(n_estimators=3, random_state=3)
+    distribution = "normal"
+    n_probes = 2
+    scoring = "neg_mean_absolute_error"
+    random_state = 3
+    confirm_variables = True
+
+    sel_expected = ProbeFeatureSelection(
+        estimator=estimator,
+        distribution=distribution,
+        n_probes=n_probes,
+        scoring=scoring,
+        cv=cv_indices,
+        random_state=random_state,
+        confirm_variables=confirm_variables,
+    )
+    X_tr_expected = sel_expected.fit_transform(X, y)
+
+    sel = ProbeFeatureSelection(
+        estimator=estimator,
+        distribution=distribution,
+        n_probes=n_probes,
+        scoring=scoring,
+        cv=cv,
+        groups=groups,
+        random_state=random_state,
+        confirm_variables=confirm_variables,
+    )
+    X_tr = sel.fit_transform(X, y)
+
+    pd.testing.assert_frame_equal(X_tr_expected, X_tr)
diff --git a/tests/test_selection/test_recursive_feature_addition.py b/tests/test_selection/test_recursive_feature_addition.py
index 5c768c9..0acd558 100644
--- a/tests/test_selection/test_recursive_feature_addition.py
+++ b/tests/test_selection/test_recursive_feature_addition.py
@@ -2,7 +2,7 @@ import pandas as pd
 import pytest
 from sklearn.ensemble import RandomForestClassifier
 from sklearn.linear_model import Lasso, LogisticRegression, LinearRegression
-from sklearn.model_selection import KFold
+from sklearn.model_selection import KFold, GroupKFold
 from sklearn.tree import DecisionTreeRegressor
 
 from feature_engine.selection import RecursiveFeatureAddition
@@ -275,3 +275,36 @@ def test_cv_generator(load_diabetes_dataset):
     ).fit(X, y)
     test2 = sel.transform(X)
     pd.testing.assert_frame_equal(expected, test2)
+
+
+def test_recursive_feature_addition_with_groups(df_test_with_groups):
+    X, y, groups = df_test_with_groups
+    cv = GroupKFold(n_splits=3)
+    cv_indices = cv.split(X=X, y=y, groups=groups)
+    threshold = 0.5
+
+    estimator = LinearRegression()
+    scoring = "neg_mean_absolute_error"
+
+    sel_expected = RecursiveFeatureAddition(
+        estimator=estimator,
+        scoring=scoring,
+        cv=cv_indices,
+        threshold=threshold,
+    )
+
+    X_tr_expected = sel_expected.fit_transform(X, y)
+
+    sel = RecursiveFeatureAddition(
+        estimator=estimator,
+        scoring=scoring,
+        cv=cv,
+        groups=groups,
+        threshold=threshold,
+    )
+    X_tr = sel.fit_transform(X, y)
+
+    pd.testing.assert_frame_equal(
+        X_tr_expected,
+        X_tr,
+    )
diff --git a/tests/test_selection/test_recursive_feature_elimination.py b/tests/test_selection/test_recursive_feature_elimination.py
index 3170984..598efba 100644
--- a/tests/test_selection/test_recursive_feature_elimination.py
+++ b/tests/test_selection/test_recursive_feature_elimination.py
@@ -2,7 +2,7 @@ import pandas as pd
 import pytest
 from sklearn.ensemble import RandomForestClassifier
 from sklearn.linear_model import Lasso, LinearRegression, LogisticRegression
-from sklearn.model_selection import KFold
+from sklearn.model_selection import KFold, GroupKFold
 from sklearn.tree import DecisionTreeRegressor
 
 from feature_engine.selection import RecursiveFeatureElimination
@@ -299,3 +299,34 @@ def test_cv_generator(load_diabetes_dataset):
     ).fit(X, y)
     test2 = sel.transform(X)
     pd.testing.assert_frame_equal(expected, test2)
+
+
+def test_recursive_feature_elimination_with_groups(df_test_with_groups):
+    X, y, groups = df_test_with_groups
+    cv = GroupKFold(n_splits=3)
+    cv_indices = cv.split(X=X, y=y, groups=groups)
+
+    estimator = LinearRegression()
+    scoring = "neg_mean_absolute_error"
+
+    sel_expected = RecursiveFeatureElimination(
+        estimator=estimator,
+        scoring=scoring,
+        cv=cv_indices,
+    )
+
+    X_tr_expected = sel_expected.fit_transform(X, y)
+
+    sel = RecursiveFeatureElimination(
+        estimator=estimator,
+        scoring=scoring,
+        cv=cv,
+        groups=groups,
+    )
+
+    X_tr = sel.fit_transform(X, y)
+
+    pd.testing.assert_frame_equal(
+        X_tr_expected,
+        X_tr,
+    )
diff --git a/tests/test_selection/test_single_feature_performance.py b/tests/test_selection/test_single_feature_performance.py
index b4369ca..f81f6f4 100644
--- a/tests/test_selection/test_single_feature_performance.py
+++ b/tests/test_selection/test_single_feature_performance.py
@@ -1,9 +1,10 @@
 import numpy as np
 import pandas as pd
 import pytest
-from sklearn.ensemble import RandomForestClassifier
+from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
 from sklearn.linear_model import LinearRegression
 from sklearn.tree import DecisionTreeRegressor
+from sklearn.model_selection import GroupKFold
 
 from feature_engine.selection import SelectBySingleFeaturePerformance
 
@@ -247,3 +248,31 @@ def test_test_selector_with_one_variable(df_test):
     }
     # test transform output
     pd.testing.assert_frame_equal(sel.transform(X), Xtransformed)
+
+
+def test_single_feature_importance_with_groups(df_test_with_groups):
+    X, y, groups = df_test_with_groups
+    cv = GroupKFold(n_splits=3)
+    cv_indices = cv.split(X=X, y=y, groups=groups)
+
+    estimator = RandomForestRegressor(n_estimators=3, random_state=3)
+    scoring = "neg_mean_absolute_error"
+
+    sel_expected = SelectBySingleFeaturePerformance(
+        estimator=estimator,
+        scoring=scoring,
+        cv=cv_indices,
+    )
+
+    X_tr_expected = sel_expected.fit_transform(X, y)
+
+    sel = SelectBySingleFeaturePerformance(
+        estimator=estimator,
+        scoring=scoring,
+        cv=cv,
+        groups=groups,
+    )
+
+    X_tr = sel.fit_transform(X, y)
+
+    pd.testing.assert_frame_equal(X_tr_expected, X_tr)
diff --git a/tests/test_selection/test_smart_correlation_selection.py b/tests/test_selection/test_smart_correlation_selection.py
index af60b16..c7a3254 100644
--- a/tests/test_selection/test_smart_correlation_selection.py
+++ b/tests/test_selection/test_smart_correlation_selection.py
@@ -2,8 +2,8 @@ import numpy as np
 import pandas as pd
 import pytest
 from sklearn.datasets import make_classification
-from sklearn.ensemble import RandomForestClassifier
-from sklearn.model_selection import KFold
+from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
+from sklearn.model_selection import KFold, GroupKFold
 
 from feature_engine.selection import SmartCorrelatedSelection
 from tests.estimator_checks.init_params_allowed_values_checks import (
@@ -403,3 +403,34 @@ def test_callable_method(df_test, random_uniform_method):
     assert len(transformer.features_to_drop_) > 0
     assert len(transformer.variables_) > 0
     assert transformer.n_features_in_ == len(X.columns)
+
+
+def test_smart_correlation_selection_with_groups(df_test_with_groups):
+    X, y, groups = df_test_with_groups
+    cv = GroupKFold(n_splits=3)
+    cv_indices = cv.split(X=X, y=y, groups=groups)
+
+    estimator = RandomForestRegressor(n_estimators=3, random_state=1)
+    scoring = "neg_mean_absolute_error"
+    selection_method = "variance"
+
+    transformer_expected = SmartCorrelatedSelection(
+        estimator=estimator,
+        scoring=scoring,
+        selection_method=selection_method,
+        cv=cv_indices,
+    )
+
+    X_tr_expected = transformer_expected.fit_transform(X, y)
+
+    transformer = SmartCorrelatedSelection(
+        estimator=estimator,
+        scoring=scoring,
+        selection_method=selection_method,
+        cv=cv,
+        groups=groups,
+    )
+
+    X_tr = transformer.fit_transform(X, y)
+
+    pd.testing.assert_frame_equal(X_tr_expected, X_tr)
diff --git a/tests/test_selection/test_target_mean_selection.py b/tests/test_selection/test_target_mean_selection.py
index 11de202..f686cbf 100644
--- a/tests/test_selection/test_target_mean_selection.py
+++ b/tests/test_selection/test_target_mean_selection.py
@@ -1,7 +1,7 @@
 import pandas as pd
 import pytest
 
-from sklearn.model_selection import StratifiedKFold
+from sklearn.model_selection import StratifiedKFold, GroupKFold
 
 from feature_engine.selection import SelectByTargetMeanPerformance
 
@@ -246,3 +246,31 @@ def test_test_selector_with_one_variable():
     assert sel.features_to_drop_ == ["cat_var_B"]
     assert sel.feature_performance_ == performance_dict
     pd.testing.assert_frame_equal(sel.transform(X), Xtransformed)
+
+
+def test_target_mean_selection_with_groups(df_test_with_groups):
+    X, y, groups = df_test_with_groups
+    cv = GroupKFold(n_splits=3)
+    cv_indices = cv.split(X=X, y=y, groups=groups)
+
+    scoring = "neg_mean_absolute_error"
+    regression = True
+
+    sel_expected = SelectByTargetMeanPerformance(
+        scoring=scoring,
+        cv=cv_indices,
+        regression=regression,
+    )
+
+    X_tr_expected = sel_expected.fit_transform(X, y)
+
+    sel = SelectByTargetMeanPerformance(
+        scoring=scoring,
+        cv=cv,
+        groups=groups,
+        regression=regression,
+    )
+
+    X_tr = sel.fit_transform(X, y)
+
+    pd.testing.assert_frame_equal(X_tr, X_tr_expected)