Repository: scikit-learn/scikit-learn
Pull request: ENH add support for `'passthrough'` in FeatureUnion
<!--
Thanks for contributing a pull request! Please ensure you have taken a look at
the contribution guidelines: https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md
-->

#### Reference Issues/PRs
Fixes #20699 
<!--
Example: Fixes #1234. See also #3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests
-->


#### What does this implement/fix? Explain your changes.
* Adds a feature to 'passthrough' to pass the features without a transformation and added test and documentation for it.
* Changes made in `FeatureUnion._validate_transformers()` to allow 'passthrough'
* Changes made in `FeatureUnion._iter()` to use `FunctionTransformer(none)` in place of passthrough to represent the identity transformer.
* Added `FeatureUnion._passthrough_function()` which returns a `FunctionTransformer(none)` with `get_feature_names` set. This separate function is needed as the `FunctionTransformer` class does not have a `get_feature_names` function which is needed by `FeatureUnion`.
* Wrote a test function `test_set_feature_union_passthrough()` to test the functionality.

#### Any other comments?
It may be better to check if more than one transformers are not set to passthrough as having a repetition of  the same features is redundant.

<!--
Please be aware that we are a loose team of volunteers so patience is
necessary; assistance handling other issues is very welcome. We value
all user contributions, no matter how minor they are. If we are slow to
review, either the pull request needs some benchmarking, tinkering,
convincing, etc. or more likely the reviewers are simply busy. In either
case, we ask for your understanding during the review process.
For more information, see our FAQ on this topic:
http://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.

Thanks for contributing!
-->


Problem statement:
Support "passthrough" in FeatureUnion
Right now setting an estimator to ``"passthrough"`` in ``FeatureUnion`` is not supported (the validation errors).
I think that is a bit inconsistent as we have it in `ColumnTransformer` and `Pipeline`. It's actually quite natural in ``FeatureUnion`` to keep the original features, so I think it's more a historical oversight that we don't allow this right now.

Hints from issue discussion:
I read about FeatureUnion and find it it's a real problem. I think the passthrough feature can be done if we do little modification of FeatureSelection constructor. Here we can introduce a new parameter "passthrough" and by default  value  can be set to None  and whenever any value is passes as a parameter of "passthrough" we can slice the array and then we can apply the operations of  FeatureUnion and after that we can concatenate it. Beside  that, we can inherit the class FeatureUnion and there  we can implement the function.  I am not very much experienced in machine learning. If I tell something wrong, please correct me. Thank you.  
If you want me to work on it, I think I can do it.
>  I think the passthrough feature can be done if we do little modification of FeatureSelection constructor

@das-ankur you will need to modify `_validate_transformers` to let `"passthrough"` to be accepted and then during the union, we need to concatenate the feature as-is.

> Here we can introduce a new parameter "passthrough" and by default value can be set to None and whenever any value is passes as a parameter of "passthrough" we can slice the array and then we can apply the operations of FeatureUnion and after that we can concatenate it. Beside that, we can inherit the class FeatureUnion and there we can implement the function.

It is difficult to follow :) A PR with the proposal will be easier to give a feedback. The only thing to keep in mind is that by default, `FeatureUnion` should behave as it does now.

> If you want me to work on it, I think I can do it.

Please go ahead. I would be happy to review your PR.
@glemaitre I am new to open source contribution. I would like to contribute to this. I want to clarify if I understood the issue properly as I have not used `pipeline` much. As far as I have understood, `"passthrough"` should let the feature be concatenated as it is. If that is the case, the passthrough should be set to at most one feature, right? Or it would concatenate the same features twice as the documentation says "A FeatureUnion has no way of checking whether two transformers might produce identical features. It only produces a union when the feature sets are disjoint, and making sure they are is the caller’s responsibility." But since it is trivial to know here that passthrough if used twice will give the same features, I think it is better to include a check for that it only takes one "passthrough".

Commit message: Add documentation for passthrough

Patch to review:
diff --git a/sklearn/pipeline.py b/sklearn/pipeline.py
index 34788d64de9e9..d73529d90a42b 100644
--- a/sklearn/pipeline.py
+++ b/sklearn/pipeline.py
@@ -17,6 +17,7 @@
 from joblib import Parallel
 
 from .base import clone, TransformerMixin
+from .preprocessing import FunctionTransformer
 from .utils._estimator_html_repr import _VisualBlock
 from .utils.metaestimators import available_if
 from .utils import (
@@ -828,8 +829,9 @@ class FeatureUnion(TransformerMixin, _BaseComposition):
 
     Parameters of the transformers may be set using its name and the parameter
     name separated by a '__'. A transformer may be replaced entirely by
-    setting the parameter with its name to another transformer,
-    or removed by setting to 'drop'.
+    setting the parameter with its name to another transformer, removed by
+    setting to 'drop' or disabled by setting to 'passthrough' (features are passed
+    without transformation).
 
     Read more in the :ref:`User Guide <feature_union>`.
 
@@ -839,8 +841,9 @@ class FeatureUnion(TransformerMixin, _BaseComposition):
     ----------
     transformer_list : list of (string, transformer) tuples
         List of transformer objects to be applied to the data. The first
-        half of each tuple is the name of the transformer. The tranformer can
-        be 'drop' for it to be ignored.
+        half of each tuple is the name of the transformer. The transformer can
+        be 'drop' for it to be ignored or can be 'passthrough' for features to
+        be passed as it is.
 
         .. versionchanged:: 0.22
            Deprecated `None` as a transformer in favor of 'drop'.
@@ -942,7 +945,7 @@ def _validate_transformers(self):
 
         # validate estimators
         for t in transformers:
-            if t == "drop":
+            if t == "drop" or t == "passthrough":
                 continue
             if not (hasattr(t, "fit") or hasattr(t, "fit_transform")) or not hasattr(
                 t, "transform"
@@ -964,14 +967,26 @@ def _validate_transformer_weights(self):
                     "but it is not present in transformer_list."
                 )
 
+    def _passthrough_function(self):
+        ft = FunctionTransformer(accept_sparse=True, check_inverse=False)
+        ft.get_feature_names = lambda: []
+        return ft
+
     def _iter(self):
         """
         Generate (name, trans, weight) tuples excluding None and
         'drop' transformers.
         """
         get_weight = (self.transformer_weights or {}).get
+
         return (
-            (name, trans, get_weight(name))
+            (
+                name,
+                self._passthrough_function(),
+                get_weight(name),
+            )
+            if trans == "passthrough"
+            else (name, trans, get_weight(name))
             for name, trans in self.transformer_list
             if trans != "drop"
         )
diff --git a/sklearn/tests/test_pipeline.py b/sklearn/tests/test_pipeline.py
index 4ec5c7f081a15..49abe9e4ed92f 100644
--- a/sklearn/tests/test_pipeline.py
+++ b/sklearn/tests/test_pipeline.py
@@ -983,6 +983,50 @@ def test_set_feature_union_step_drop():
     assert not record
 
 
+def test_set_feature_union_passthrough():
+    mult2 = Mult(2)
+    mult2.get_feature_names = lambda: ["x2"]
+    mult3 = Mult(3)
+    mult3.get_feature_names = lambda: ["x3"]
+    X = np.asarray([[1]])
+
+    ft = FeatureUnion([("m2", mult2), ("m3", mult3)])
+    assert_array_equal([[2, 3]], ft.fit(X).transform(X))
+    assert_array_equal([[2, 3]], ft.fit_transform(X))
+    assert ["m2__x2", "m3__x3"] == ft.get_feature_names()
+
+    with pytest.warns(None) as record:
+        ft.set_params(m2="passthrough")
+        assert_array_equal([[1, 3]], ft.fit(X).transform(X))
+        assert_array_equal([[1, 3]], ft.fit_transform(X))
+    assert ["m3__x3"] == ft.get_feature_names()
+    assert not record
+
+    with pytest.warns(None) as record:
+        ft.set_params(m3="passthrough")
+        assert_array_equal([[1, 1]], ft.fit(X).transform(X))
+        assert_array_equal([[1, 1]], ft.fit_transform(X))
+    assert [] == ft.get_feature_names()
+    assert not record
+
+    with pytest.warns(None) as record:
+        # check we can change back
+        ft.set_params(m3=mult3)
+        assert_array_equal([[1, 3]], ft.fit(X).transform(X))
+        assert_array_equal([[1, 3]], ft.fit_transform(X))
+    assert not record
+
+    with pytest.warns(None) as record:
+        # Check 'drop' step at construction time
+        ft = FeatureUnion([("m2", "passthrough"), ("m3", mult3)])
+        assert_array_equal([[1, 3]], ft.fit(X).transform(X))
+        assert_array_equal([[1, 3]], ft.fit_transform(X))
+    assert ["m3__x3"] == ft.get_feature_names()
+    assert not record
+
+    pass
+
+
 def test_step_name_validation():
     error_message_1 = r"Estimator names must not contain __: got \['a__q'\]"
     error_message_2 = r"Names provided are not unique: \['a', 'a'\]"


Code review comments:
diff_hunk: @@ -942,7 +945,7 @@ def _validate_transformers(self):
 
         # validate estimators
         for t in transformers:
-            if t == "drop":
+            if t == "drop" or t == "passthrough":
path: sklearn/pipeline.py
text: @user1:
nit: ``if t in ["drop", "passthrough"]``?

@author:
Changed it now

diff_hunk: @@ -964,14 +967,26 @@ def _validate_transformer_weights(self):
                     "but it is not present in transformer_list."
                 )
 
+    def _passthrough_function(self):
+        ft = FunctionTransformer(accept_sparse=True, check_inverse=False)
+        ft.get_feature_names = lambda: []
path: sklearn/pipeline.py
text: @user1:
Shouldn't it be ``lambda x: x`` as we keep the feature names? This is a bit of a non-standard way of doing this but it seems helpful here.

diff_hunk: @@ -964,14 +967,26 @@ def _validate_transformer_weights(self):
                     "but it is not present in transformer_list."
                 )
 
+    def _passthrough_function(self):
+        ft = FunctionTransformer(accept_sparse=True, check_inverse=False)
+        ft.get_feature_names = lambda: []
+        return ft
+
     def _iter(self):
         """
         Generate (name, trans, weight) tuples excluding None and
         'drop' transformers.
         """
         get_weight = (self.transformer_weights or {}).get
+
         return (
-            (name, trans, get_weight(name))
+            (
+                name,
+                self._passthrough_function(),
path: sklearn/pipeline.py
text: @user1:
I would have just put the FunctionTransformer here directly. Ideally we'd share code between this and ColumnTransformer but it looks like it might not be worth it.

@author:
Yeah, I added it directly initially. I misinterpreted the function of `get_feature_names` and assumed that it should return a value for all transforms. But since it is expected to return error if the `get_feature_names` is not implemented by the transformer, I understand that it is not necessary to implement the `get_feature_names` explicitly.

diff_hunk: @@ -983,6 +983,50 @@ def test_set_feature_union_step_drop():
     assert not record
 
 
+def test_set_feature_union_passthrough():
+    mult2 = Mult(2)
+    mult2.get_feature_names = lambda: ["x2"]
+    mult3 = Mult(3)
+    mult3.get_feature_names = lambda: ["x3"]
+    X = np.asarray([[1]])
+
+    ft = FeatureUnion([("m2", mult2), ("m3", mult3)])
+    assert_array_equal([[2, 3]], ft.fit(X).transform(X))
+    assert_array_equal([[2, 3]], ft.fit_transform(X))
+    assert ["m2__x2", "m3__x3"] == ft.get_feature_names()
+
+    with pytest.warns(None) as record:
+        ft.set_params(m2="passthrough")
+        assert_array_equal([[1, 3]], ft.fit(X).transform(X))
+        assert_array_equal([[1, 3]], ft.fit_transform(X))
+    assert ["m3__x3"] == ft.get_feature_names()
path: sklearn/tests/test_pipeline.py
text: @user1:
the shape of the result and the feature names doesn't match here, see above.

@author:
Since, passthrough does not have `get_feature_names`, I have removed the feature names tests from this. I initially did those because they were done fot `drop` but there it makes sense. I think checking the feature names is not needed here.