Repository: pyjanitor-devs/pyjanitor

Issue:
[ENH] Warn the user of non-existent column names passed to .select_columns()
It would be nice if the user was given a warning when non-existent column names are passed to `.select_columns()`. For example, it would be helpful if the following chunk

```python
>>> test_df = pd.DataFrame(data=np.random.randn(20, 5), columns=["A", "B", "C", "col_D", "col_E"])
>>> test_df.select_columns(["A", "B", "G", "col_*"]).rename_column("A", "a")

          a         B     col_D     col_E
0 -0.307637 -0.057561 -0.355314  0.050262
1  1.263703  0.050407 -0.300649  1.057278
2  1.133523 -0.742104 -0.866652 -1.503412
3  1.776966 -0.106066  1.221996 -0.901457
4  0.841195  0.503199  0.064804  0.181194
```

also produced a warning (without raising an exception), à la

```bash
/Users/smu095/Documents/github/pyjanitor/janitor/functions.py:2780: UserWarning: Column 'G' not found.
```

Occasionally I find myself selecting columns, only to realise later on that I have misspelled the column name and it hasn't been selected at all. I would like to be warned ASAP if this is the case.

My back of the envelope suggestion is something like this:

```python
@pf.register_dataframe_method
@deprecated_alias(search_cols="search_column_names")
def select_columns(
    df: pd.DataFrame, search_column_names: Iterable, invert: bool = False
) -> pd.DataFrame:
    """
    Method-chainable selection of columns.

    This method does not mutate the original DataFrame.

    Optional ability to invert selection of columns available as well.

    Method-chaining example:

    .. code-block:: python

        df = pd.DataFrame(...).select_columns(['a', 'b', 'col_*'], invert=True)

    :param df: A pandas DataFrame.
    :param search_column_names: A list of column names or search strings to be
        used to select. Valid inputs include:
        1) an exact column name to look for
        2) a shell-style glob string (e.g., `*_thing_*`)
    :param invert: Whether or not to invert the selection.
        This will result in selection of the complement of the columns
        provided.
    :returns: A pandas DataFrame with the specified columns selected.
    """
    wildcards = [col for col in search_column_names if "*" in col]
    non_wildcards = set(search_column_names) - set(wildcards)

    for col in non_wildcards:
        if col not in df:
            warnings.warn(f"Column '{col}' not found.")

    full_column_list = []

    for col in search_column_names:
        search_string = translate(col)
        columns = [col for col in df if re.match(search_string, col)]
        full_column_list.extend(columns)

    return (
        df.drop(columns=full_column_list) if invert else df[full_column_list]
    )
```

This is my first foray into contributing to open-source projects, any feedback is welcome. Thanks!


Discussion:
Hello @smu095! Thanks for pinging in here :smile:.

Turns out I've also been hit by the same problem before, but hadn't had the time to put in a PR. I like what you're suggesting here - wondering what you think about loudly raising an error instead? The relevant error raising error might look something like this:


```python
non_wildcards = set(search_column_names) - set(wildcards)

if not non_wildcards.issubset(df.columns):
    # Get the nonexistent column names
    nonexistent_column_names = non_wildcards.difference(df.columns)
    raise NameError(f"The following column names are not present in the dataframe: {nonexistent_column_names}")
```

Please let me know what you think! I'd also love to let you have the credit for this contribution, so if you'd like to make a pull request in, please let me know!

Resolving patch:
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -2747,7 +2747,7 @@ def currency_column_to_numeric(
 @pf.register_dataframe_method
 @deprecated_alias(search_cols="search_column_names")
 def select_columns(
-    df: pd.DataFrame, search_column_names: Iterable, invert: bool = False
+    df: pd.DataFrame, search_column_names: List[str], invert: bool = False
 ) -> pd.DataFrame:
     """
     Method-chainable selection of columns.
@@ -2771,13 +2771,41 @@ def select_columns(
         This will result in selection of the complement of the columns
         provided.
     :returns: A pandas DataFrame with the specified columns selected.
-    """
+    :raises:
+        TypeError: if input is not passed as a list.
+    :raises:
+        NameError: if one or more of the specified column names or
+        search strings are not found in DataFrame columns.
+    """
+    if not isinstance(search_column_names, list):
+        raise TypeError(
+            "Column name(s) or search string(s) must be passed as list"
+        )
+
+    wildcards = {col for col in search_column_names if "*" in col}
+    non_wildcards = set(search_column_names) - wildcards
+
+    if not non_wildcards.issubset(df.columns):
+        nonexistent_column_names = non_wildcards.difference(df.columns)
+        raise NameError(
+            f"{list(nonexistent_column_names)} missing from DataFrame"
+        )
+
+    missing_wildcards = []
     full_column_list = []
 
     for col in search_column_names:
         search_string = translate(col)
         columns = [col for col in df if re.match(search_string, col)]
-        full_column_list.extend(columns)
+        if len(columns) == 0:
+            missing_wildcards.append(col)
+        else:
+            full_column_list.extend(columns)
+
+    if len(missing_wildcards) > 0:
+        raise NameError(
+            f"Search string(s) {missing_wildcards} not found in DataFrame"
+        )
 
     return (
         df.drop(columns=full_column_list) if invert else df[full_column_list]


Test patch:
diff --git a/tests/functions/test_select_columns.py b/tests/functions/test_select_columns.py
--- a/tests/functions/test_select_columns.py
+++ b/tests/functions/test_select_columns.py
@@ -30,3 +30,29 @@ def test_select_columns_glob_inputs(dataframe, invert, expected):
     df = dataframe.select_columns(search_column_names=columns, invert=invert)
 
     pd.testing.assert_frame_equal(df, dataframe[expected])
+
+
+@pytest.mark.functions
+@pytest.mark.parametrize(
+    "columns",
+    [
+        ["a", "Bell__Chart", "foo"],
+        ["a", "Bell__Chart", "foo", "bar"],
+        ["a*", "Bell__Chart", "foo"],
+        ["a*", "Bell__Chart", "foo", "bar"],
+    ],
+)
+def test_select_columns_missing_columns(dataframe, columns):
+    """Check that passing non-existent column names or search strings raises NameError"""  # noqa: E501
+    with pytest.raises(NameError):
+        dataframe.select_columns(search_column_names=columns)
+
+
+@pytest.mark.functions
+@pytest.mark.parametrize(
+    "columns", ["a", ("a", "Bell__Chart"), {"a", "Bell__Chart"}]
+)
+def test_select_columns_input(dataframe, columns):
+    """Check that passing an iterable that is not a list raises TypeError."""
+    with pytest.raises(TypeError):
+        dataframe.select_columns(search_column_names=columns)