Repository: feature-engine/feature_engine

Issue:
Feature Request: Add Correlation with target as selection method for `SmartCorrelatedSelection`
**Is your feature request related to a problem? Please describe.**
No.

**Describe the solution you'd like**
I would like to add as a selection method of `SmartCorrelatedSelection` a new selection method based on the correlation of the features with the target variable.

**Describe alternatives you've considered**
I have not considered other alternatives.

**Additional context**
For linear models it can make a lot of sense to select features using correlation with target and it can be faster than computing single model performances.

I would like to contribute to this project by implementing this feature by myself.


Discussion:

Patch:
diff --git a/docs/user_guide/outliers/Winsorizer.rst b/docs/user_guide/outliers/Winsorizer.rst
index 5fb07c9..7416902 100644
--- a/docs/user_guide/outliers/Winsorizer.rst
+++ b/docs/user_guide/outliers/Winsorizer.rst
@@ -127,7 +127,7 @@ The default values for fold are as follows:
 - 'gaussian': `fold` is set to 3.0;
 - 'iqr': `fold` is set to 1.5;
 - 'mad': `fold` is set to 3.29;
-- 'percentiles': `fold` is set to 0.05.
+- 'quantiles': `fold` is set to 0.05.
 
 You can manually adjust the `fold` value to make the outlier detection process more or less
 conservative, thus customizing the extent of outlier capping.
@@ -187,4 +187,4 @@ Or read our book:
 |
 
 Both our book and course are suitable for beginners and more advanced data scientists
-alike. By purchasing them you are supporting Sole, the main developer of Feature-engine.
\ No newline at end of file
+alike. By purchasing them you are supporting Sole, the main developer of Feature-engine.
diff --git a/docs/user_guide/selection/SmartCorrelatedSelection.rst b/docs/user_guide/selection/SmartCorrelatedSelection.rst
index e29c308..682d199 100644
--- a/docs/user_guide/selection/SmartCorrelatedSelection.rst
+++ b/docs/user_guide/selection/SmartCorrelatedSelection.rst
@@ -19,6 +19,8 @@ such us:
 
 3. **Missing Data**: Features with less missing data are generally more reliable and informative.
 
+4. **Correlation with Target**: Features that show a stronger correlation with the target variable are often more predictive and should be prioritized.
+
 We can apply this selection strategies out of the box with the :class:`SmartCorrelatedSelection`.
 
 From a group of correlated variables, the :class:`SmartCorrelatedSelection` will retain
@@ -28,6 +30,7 @@ the variable with:
 - the highest cardinality
 - the least missing data
 - the best performing model (based on a single feature)
+- the strongest correlation with the target variable
 
 The remaining features within each correlated group will be dropped.
 
@@ -62,6 +65,10 @@ larger scales will dominate the selection procedure**, unless you have a scaled
 If the criteria is based on missing data, :class:`SmartCorrelatedSelection` will determine the
 number of NA in each feature from the correlated group and keep the one with less NA.
 
+If the criteria is based on correlation with the target, :class:`SmartCorrelatedSelection`
+will calculate the correlation coefficient between each feature and the target, and retain
+the one with the highest absolute correlation.
+
 Variance
 ~~~~~~~~
 
@@ -344,6 +351,181 @@ the dataframe:
 
     [True, True, True, True, False, True, False, True, False, False, True, True]
 
+
+Correlation with Target
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Finally, let's select the feature that shows the strongest correlation with the target
+variable from each group.
+
+Let's create another toy dataframe with 4 features and a target variable:
+
+.. code:: python
+
+    import pandas as pd
+    from sklearn.datasets import make_classification
+    from feature_engine.selection import SmartCorrelatedSelection
+
+    # make dataframe with some correlated variables
+    def make_data():
+        X, y = make_classification(n_samples=1000,
+                                   n_features=12,
+                                   n_redundant=4,
+                                   n_clusters_per_class=1,
+                                   weights=[0.50],
+                                   class_sep=2,
+                                   random_state=1)
+
+        # transform arrays into pandas df and series
+        colnames = ['var_'+str(i) for i in range(12)]
+        X = pd.DataFrame(X, columns=colnames)
+        y = pd.Series(y)
+        return X, y
+
+    X, y = make_data()
+
+Now, we set up :class:`SmartCorrelatedSelection` to find feature groups with absolute
+correlation coefficient >0.8 and retain the feature with the strongest correlation with
+the target variable:
+
+.. code:: python
+
+    # set up the selector
+    tr = SmartCorrelatedSelection(
+        variables=None,
+        method="pearson",
+        threshold=0.8,
+        missing_values="raise",
+        selection_method="corr_with_target",
+        estimator=None,
+    )
+
+With `fit()`, the transformer finds the correlated variables and selects the ones to
+keep. With `transform()`, it drops the remaining features in the correlated group from
+the dataset:
+
+.. code:: python
+
+    Xt = tr.fit_transform(X, y)
+
+Let's explore the correlated feature groups:
+
+.. code:: python
+
+    tr.correlated_feature_sets_
+
+Similarly to the previous examples, we see that the groups of correlated features are
+slightly different. Here, what happened is that the features were ordered first based 
+on their absolute correlation with the target, before carrying on the search of correlation
+with the other features in the dataset. Like this, the first feature of the group is retained, 
+which is the one with highest correlation with the target.
+
+.. code:: python
+
+    [{'var_4', 'var_6', 'var_7', 'var_9'}, {'var_0', 'var_8'}]
+
+We can find the features that will be retained as keys in the following dictionary:
+
+.. code:: python
+
+    tr.correlated_feature_dict_
+
+The variables `var_7` and `var_0` will be retained, and the remaining ones will be dropped.
+
+.. code:: python
+
+    {'var_7': {'var_4', 'var_6', 'var_9'}, 'var_0': {'var_8'}}
+
+We can check the correlation of the features with the target variable as follows:
+
+.. code:: python
+
+    print(X.corrwith(y).abs())
+
+.. code:: python
+
+    var_0     0.270913
+    var_1     0.088358
+    var_2     0.038257
+    var_3     0.027320
+    var_4     0.838361
+    var_5     0.028020
+    var_6     0.834925
+    var_7     0.916045
+    var_8     0.007724
+    var_9     0.797149
+    var_10    0.006742
+    var_11    0.023710
+    dtype: float64
+
+
+We notice that indeed the the `var_0` has a much higher correlation with the target than
+the `var_8`, which is why `var_0` was retained:
+
+.. code:: python
+
+    print(X.corrwith(y).abs().loc[['var_0', 'var_8']])
+
+.. code:: python
+
+    var_0    0.270913
+    var_8    0.007724
+    dtype: float64
+
+The variables that will be dropped are available in the following attribute:
+
+.. code:: python
+
+    tr.features_to_drop_
+
+.. code:: python
+
+    ['var_4', 'var_6', 'var_9', 'var_8']
+
+And now we can print the resulting dataframe after the transformation:
+
+.. code:: python
+
+    print(Xt.head())
+
+.. code:: python
+
+        var_0     var_1     var_2     var_3     var_5     var_7    var_10  \
+    0  1.471061 -2.376400 -0.247208  1.210290  0.091527 -2.230170  2.070526
+    1  1.819196  1.969326 -0.126894  0.034598 -0.186802 -1.447490  1.184820
+    2  1.625024  1.499174  0.334123 -2.233844 -0.313881 -2.240741 -0.066448
+    3  1.939212  0.075341  1.627132  0.943132 -0.468041 -3.534861  0.713558
+    4  1.579307  0.372213  0.338141  0.951526  0.729005 -2.053965  0.398790
+
+        var_11
+    0 -1.989335
+    1 -1.309524
+    2 -0.852703
+    3  0.484649
+    4 -0.186530
+
+We can also obtain the names of the features in the resulting dataframe as follows:
+
+.. code:: python
+
+    tr.get_feature_names_out()
+
+.. code:: python
+
+    ['var_0', 'var_1', 'var_2', 'var_3', 'var_5', 'var_7', 'var_10', 'var_11']
+
+We also find the `get_support` method that flags the features that will be retained from
+the dataframe:
+
+.. code:: python
+
+    tr.get_support()
+
+.. code:: python
+
+    [True, True, True, True, False, True, False, True, False, False, True, True]
+
+
 And that's it!
 
 Additional resources
diff --git a/feature_engine/selection/smart_correlation_selection.py b/feature_engine/selection/smart_correlation_selection.py
index 8fd9afa..bf86293 100644
--- a/feature_engine/selection/smart_correlation_selection.py
+++ b/feature_engine/selection/smart_correlation_selection.py
@@ -29,6 +29,7 @@ from feature_engine.dataframe_checks import (
     _check_contains_inf,
     _check_contains_na,
     check_X,
+    check_y,
 )
 from feature_engine.selection.base_selector import BaseSelector
 
@@ -64,6 +65,7 @@ class SmartCorrelatedSelection(BaseSelector):
     - Feature with the highest cardinality (greatest number of unique values).
     - Feature with the highest variance.
     - Feature with the highest importance according to an estimator.
+    - Feature with the highest correlation with the target variable.
 
     SmartCorrelatedSelection() returns a dataframe containing from each group of
     correlated features, the selected variable, plus all the features that were
@@ -99,8 +101,8 @@ class SmartCorrelatedSelection(BaseSelector):
     {missing_values}
 
     selection_method: str, default= "missing_values"
-        Takes the values "missing_values", "cardinality", "variance" and
-        "model_performance".
+        Takes the values "missing_values", "cardinality", "variance",
+        "model_performance", and "corr_with_target".
 
         **"missing_values"**: keeps the feature from the correlated group with the least
         missing observations.
@@ -115,6 +117,11 @@ class SmartCorrelatedSelection(BaseSelector):
         features in a correlated group and retains the feature with the highest
         importance.
 
+        **"corr_with_target"**: keeps the feature from the correlated group that has the
+        highest (absolute) correlation with the target variable. The same correlation
+        method defined in the `method` parameter is used to calculate the correlation
+        between the features and the target.
+
     {estimator}
 
     {scoring}
@@ -229,10 +236,12 @@ class SmartCorrelatedSelection(BaseSelector):
             "cardinality",
             "variance",
             "model_performance",
+            "corr_with_target",
         ]:
             raise ValueError(
                 "selection_method takes only values 'missing_values', 'cardinality', "
-                f"'variance' or 'model_performance'. Got {selection_method} instead."
+                "'variance', 'model_performance' or 'corr_with_target'. "
+                f"Got {selection_method} instead."
             )
 
         if selection_method == "model_performance" and estimator is None:
@@ -271,7 +280,8 @@ class SmartCorrelatedSelection(BaseSelector):
             The training dataset.
 
         y: pandas series. Default = None
-            y is needed if selection_method == 'model_performance'.
+            y is needed if selection_method == 'model_performance' or
+                'corr_with_target'.
         """
 
         # check input dataframe
@@ -289,9 +299,11 @@ class SmartCorrelatedSelection(BaseSelector):
             _check_contains_na(X, self.variables_)
             _check_contains_inf(X, self.variables_)
 
-        if self.selection_method == "model_performance" and y is None:
+        if (
+            self.selection_method in ["model_performance", "corr_with_target"]
+        ) and y is None:
             raise ValueError(
-                "When `selection_method = 'model_performance'` y is needed to "
+                f"When `selection_method = '{self.selection_method}'` y is needed to "
                 "fit the transformer."
             )
 
@@ -314,6 +326,15 @@ class SmartCorrelatedSelection(BaseSelector):
                 .sort_values(ascending=False)
                 .index.to_list()
             )
+        elif self.selection_method == "corr_with_target":
+            y = check_y(y)
+            features = (
+                X[self.variables_]
+                .corrwith(y, method=self.method)
+                .abs()
+                .sort_values(ascending=False)
+                .index.to_list()
+            )
         else:
             features = sorted(self.variables_)
 


Test patch:
diff --git a/tests/test_selection/test_smart_correlation_selection.py b/tests/test_selection/test_smart_correlation_selection.py
index c7a3254..d40f797 100644
--- a/tests/test_selection/test_smart_correlation_selection.py
+++ b/tests/test_selection/test_smart_correlation_selection.py
@@ -117,7 +117,7 @@ def test_raises_error_when_threshold_not_permitted(_threshold):
 def test_raises_error_when_selection_method_not_permitted(_method):
     msg = (
         "selection_method takes only values 'missing_values', 'cardinality', "
-        f"'variance' or 'model_performance'. Got {_method} instead."
+        f"'variance', 'model_performance' or 'corr_with_target'. Got {_method} instead."
     )
     with pytest.raises(ValueError) as record:
         SmartCorrelatedSelection(selection_method=_method)
@@ -252,6 +252,21 @@ def test_error_if_select_model_performance_and_y_is_none(df_single):
     assert record.value.args[0] == msg
 
 
+def test_error_if_select_corr_with_target_and_y_is_none(df_single):
+    X, _ = df_single
+
+    transformer = SmartCorrelatedSelection(
+        selection_method="corr_with_target",
+    )
+    msg = (
+        "When `selection_method = 'corr_with_target'` y is needed to fit "
+        "the transformer."
+    )
+    with pytest.raises(ValueError) as record:
+        transformer.fit(X)
+    assert record.value.args[0] == msg
+
+
 def test_selection_method_variance(df_var_car):
     X = df_var_car
 
@@ -434,3 +449,27 @@ def test_smart_correlation_selection_with_groups(df_test_with_groups):
     X_tr = transformer.fit_transform(X, y)
 
     pd.testing.assert_frame_equal(X_tr_expected, X_tr)
+
+
+def test_corr_with_target_single_corr_group(df_single):
+    X, y = df_single
+
+    transformer = SmartCorrelatedSelection(
+        variables=None,
+        method="pearson",
+        threshold=0.8,
+        missing_values="raise",
+        selection_method="corr_with_target",
+    )
+
+    Xt = transformer.fit_transform(X, y)
+
+    # expected result
+    df = X[["var_0", "var_2", "var_3", "var_5"]].copy()
+
+    # test fit attrs
+    assert transformer.correlated_feature_sets_ == [{"var_1", "var_2", "var_4"}]
+    assert transformer.features_to_drop_ == ['var_4', 'var_1']
+    assert transformer.correlated_feature_dict_ == {"var_2": {"var_1", "var_4"}}
+    # test transform output
+    pd.testing.assert_frame_equal(Xt, df)