Repository: pyjanitor-devs/pyjanitor

Issue:
[ENH] Allow substring and regex matching in find_replace
# Brief Description

<!-- Please provide a brief description of what you'd like to propose. -->

Currently `find_replace` only allows exact matching. I have run into use cases for fuzzy/substring match and replace both in my work and in the process of making a pyjanitor example. Was able to implement it by combining pyjanitor function `update_where` and pandas-flavor, but we potentially could make this as a pyjanitor feature--either as part of `find_replace` (for example, by adding a kwarg flag `exact=True/False`), or as a separate function `fuzzy_find_replace`.

# Example API


<!-- One of the selling points of pyjanitor is the API. Hence, we guard the API very carefully, and want to
make sure that it is accessible and understandable to many people. Please provide a few examples of what the API
of the new function you're proposing will look like. We have provided an example that you should modify. -->

For example, in the example [Tidy Up Web-Scraped Media Franchise Data](https://pyjanitor.readthedocs.io/notebooks/medium_franchise.html), when trying to implement a python equivalence of the following R snippet:
```R
clean_df <- clean_category %>%
  mutate(revenue_category = case_when(
    str_detect(str_to_lower(revenue_category), "box office") ~ "Box Office",
    str_detect(str_to_lower(revenue_category), "dvd|blu|vhs|home video|video rentals|video sales|streaming|home entertainment") ~ "Home Video/Entertainment",
    str_detect(str_to_lower(revenue_category), "video game|computer game|mobile game|console|game|pachinko|pet|card") ~ "Video Games/Games",
    str_detect(str_to_lower(revenue_category), "comic|manga") ~ "Comic or Manga",
    str_detect(str_to_lower(revenue_category), "music|soundtrack") ~ "Music",
    str_detect(str_to_lower(revenue_category), "tv") ~ "TV",
    str_detect(str_to_lower(revenue_category), "merchandise|licens|mall|stage|retail") ~ "Merchandise, Licensing & Retail",
    TRUE ~ revenue_category))
```
A mapper dictionary could be set up as:
```python
mapper = {
    'box office': 'Box Office',
    'dvd|blu|vhs|home video|video rentals|video sales|streaming|home entertainment': 'Home Video/Entertainment',
    'video game|computer game|mobile game|console|game|pachinko|pet|card': 'Video Games/Games',
    'comic|manga': 'Comic or Manga',
    'music|soundtrac': 'Music',
    'tv': 'TV',
    'merchandise|licens|mall|stage|retail': 'Merchandise, Licensing & Retail',
}
```

If we enable substring matching and regex, the find_replace call could look like this:
```python
df.find_replace(column_name, mapper, exact=False, regex=True)
```

If this feature extension could clutter `find_replace`, a separate function could be set up.


Discussion:
This is a great idea, @shandou! I think the API could be simplified by using strings. An example might be:

```python
# One of the following
df.find_replace(df, column_name, mapper, match=‘exact’)
df.find_replace(df, column_name, mapper, match=‘regex’)
```

If we default to ‘exact’, this preserves the current behaviour, and we can add in a ‘regex’ string as an option as well.
Don't want to overpromise, but I can take this issue as well and get onto it once I am done with the `contribution.rst` improvements--only if no one else wants to claim it of course 😄
Alright! I will keep it open and available for others until you officially claim it for yourself. :smile:
@ericmjl Okay I officially claim this issue 😃 

Resolving patch:
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -2908,42 +2908,85 @@ def dropnotnull(df: pd.DataFrame, column_name) -> pd.DataFrame:
 
 @pf.register_dataframe_method
 @deprecated_alias(column="column_name")
-def find_replace(df: pd.DataFrame, column_name, mapper: Dict) -> pd.DataFrame:
+def find_replace(
+    df: pd.DataFrame, column_name, mapper: Dict, match: str = "exact"
+) -> pd.DataFrame:
     """
     Perform a find-and-replace action on a column of data.
 
     This method mutates the original DataFrame.
 
-    For example, let's say we have a column for which we want to replace all
-    of the values 'a' with 1, 'b' with 2, 'c' with 3. We would use the
-    following function call:
+    Depending on use cases, users can choose either exact, full value matching
+    or regular-expression-based fuzzy matching (substring matching is allowed
+    in the latter case). For strings, the matching is always case sensitive.
+
+    Note that default value for keyword argument `match` is "exact".
+
+    For instance, given a dataframe containing orders at a coffee shop:
 
     .. code-block:: python
 
-        df = (
-            pd.DataFrame(...)
-            .find_replace('column_name', {'a': 1, 'b': 2, 'c': 3})
+        df = pd.DataFrame({
+            'customer': ['Mary', 'Tom', 'Lila'],
+            'order': ['ice coffee', 'lemonade', 'regular coffee']
+        })
+
+    Our task is to replace values `'ice coffee'` and `'regular coffee'`
+    of the `'order'` column into `'latte'`.
+
+    Example 1: Exact matching
+
+    .. code-block:: python
+
+        # Functional usage
+        df = find_replace(
+            df, 'order', {'ice coffee': 'latte', 'regular coffee': 'latte},
+            match='exact'
         )
 
-    This find-and-replace functionality does an exact match only. Hence,
-    substring matches do not work. The value of a cell in the dataframe
-    must be exactly 'a', 'b', or 'c', otherwise the replacement will not
-    happen and the original data will be left in-place.
+        # Method chaining usage
+        df = df.find_replace(
+            'order', {'ice coffee': 'latte', 'regular coffee': 'latte},
+            match='exact'
+        )
+
+
+    Example 2: Regular-expression-based matching
+
+    .. code-block:: python
+
+        # Functional usage
+        df = find_replace(df, 'order', {'coffee$': 'latte'}, match='regex')
+
+        # Method chaining usage
+        df = df.find_replace('order', {'coffee$': 'latte'}, match='regex')
+
 
     :param df: A pandas DataFrame.
     :param column_name: The column on which the find/replace action is to be
         made.
     :param mapper: A dictionary that maps "thing to find" -> "thing to
         replace".  Note: Does not support null-value replacement.
+    :param match: A string that dictates whether exact match or
+        regular-expression-based fuzzy match will be used for finding patterns.
+        Default to "exact". Can only be "exact" or "regex".
     :returns: A pandas DataFrame.
+    :raises: ValueError
     """
     if any(map(pd.isna, mapper.keys())):
         raise ValueError(
             "find_replace() does not support null replacement. "
             "Use DataFrame.fillna() instead."
         )
-
-    df[column_name] = df[column_name].apply(lambda x: mapper.get(x, x))
+    if match.lower() not in ("exact", "regex"):
+        raise ValueError("`match` can only be 'exact' or 'regex'.")
+
+    if match.lower() == "exact":
+        df[column_name] = df[column_name].apply(lambda x: mapper.get(x, x))
+    if match.lower() == "regex":
+        for k, v in mapper.items():
+            condition = df[column_name].str.contains(k, regex=True)
+            df = df.update_where(condition, column_name, v)
     return df
 
 


Test patch:
diff --git a/tests/functions/test_find_replace.py b/tests/functions/test_find_replace.py
--- a/tests/functions/test_find_replace.py
+++ b/tests/functions/test_find_replace.py
@@ -3,7 +3,7 @@
 import pytest
 
 
-@pytest.fixture()
+@pytest.fixture
 def df():
     return pd.DataFrame(
         {"a": [1, np.nan, 3], "b": [2, 3, 1], "c": [2, np.nan, 2]}
@@ -27,3 +27,29 @@ def test_find_replace_single(df):
 def test_find_replace_null_raises_error(df):
     with pytest.raises(ValueError):
         df.find_replace("a", {np.nan: 5})
+
+
+@pytest.fixture
+def df_orders():
+    return pd.DataFrame(
+        {
+            "customer": ["Mary", "Tom", "Lila"],
+            "order": ["ice coffee", "lemonade", "regular coffee"],
+        }
+    )
+
+
+@pytest.mark.functions
+def test_find_replace_regex(df_orders):
+    df_orders.find_replace("order", {"coffee$": "latte"}, match="regex")
+    assert df_orders["order"].iloc[0] == "latte"
+    assert df_orders["order"].iloc[1] == "lemonade"
+    assert df_orders["order"].iloc[-1] == "latte"
+
+
+@pytest.mark.functions
+def test_find_replace_regex_match_rases_error(df_orders):
+    with pytest.raises(ValueError):
+        df_orders.find_replace(
+            "order", {"lemonade": "orange juice"}, match="bla"
+        )