Repository: pyjanitor-devs/pyjanitor

Issue:
[ENH] Decorator utility for handling NAs and errors (for transform_column)
# Brief Description

If say `df["x"]` is a string column with NAs, then the routine `df["x"].apply(lambda x : x+"1")` doesn't work due to the NAs. It would be useful to have a decorator such that 
```python
df.transform_column("x", skipna(lambda x : x + "1"))
``` 
would work

# Example API

```python
from janitor.func_utils import skipna, skiperror

df.transform_column("x", skipna(lambda x : x + "1"))
df.transform_column("y", skiperror(func))

@skiperror
def foo(x):
    ...
    pass
```

# Example implementation

```python

def skipna(f):
    def _wrapped(x):
        if type(x) is float and np.isnan(x) or x is None:
            return np.nan
        else:
            return f(x)

    return _wrapped


def skiperror(f, return_val=None, return_x=False):
    def _wrapped(x):
        try:
            return f(x)
        except:
            if return_x:
                return x
            return return_val

    return _wrapped
```



Discussion:
@jiafengkevinchen thanks for pinging in! I think the example implementation looks great. Are you up for a PR?
Sure. I'm not sure what namespace to put these in though
Is `from janitor.func_utils import skipna, skiperror` reasonable?

@jiafengkevinchen we have a `janitor.utils`, and I think that is a reasonable first place to put it. 

I will assign it to you; looking forward to reviewing the PR! (Don’t forget tests, and to also add your contribution to the CHANGELOG :smile:)

Resolving patch:
diff --git a/janitor/utils.py b/janitor/utils.py
--- a/janitor/utils.py
+++ b/janitor/utils.py
@@ -4,6 +4,7 @@
 import warnings
 from typing import Callable, Dict, List, Union
 
+import numpy as np
 import pandas as pd
 
 from .errors import JanitorError
@@ -289,3 +290,66 @@ def check_column(
                 raise ValueError(
                     f"{column_name} already present in dataframe columns!"
                 )
+
+
+def skipna(f: Callable) -> Callable:
+    """
+    Decorator for escaping np.nan and None in a function
+
+    Should be used like this::
+
+        df[column].apply(skipna(transform))
+
+    or::
+
+        @skipna
+        def transform(x):
+            pass
+
+    :param f: the function to be wrapped
+    :returns: _wrapped, the wrapped function
+    """
+
+    def _wrapped(x, *args, **kwargs):
+        if (type(x) is float and np.isnan(x)) or x is None:
+            return np.nan
+        else:
+            return f(x, *args, **kwargs)
+
+    return _wrapped
+
+
+def skiperror(
+    f: Callable, return_x: bool = False, return_val=np.nan
+) -> Callable:
+    """
+    Decorator for escaping errors in a function
+
+    Should be used like this::
+
+        df[column].apply(
+            skiperror(transform, return_val=3, return_x=False))
+
+    or::
+
+        @skiperror(return_val=3, return_x=False)
+        def transform(x):
+            pass
+
+    :param f: the function to be wrapped
+    :param return_x: whether or not the original value that caused error
+        should be returned
+    :param return_val: the value to be returned when an error hits.
+        Ignored if return_x is True
+    :returns: _wrapped, the wrapped function
+    """
+
+    def _wrapped(x, *args, **kwargs):
+        try:
+            return f(x, *args, **kwargs)
+        except Exception:
+            if return_x:
+                return x
+            return return_val
+
+    return _wrapped


Test patch:
diff --git a/tests/utils/test_skiperror.py b/tests/utils/test_skiperror.py
new file mode 100644
--- /dev/null
+++ b/tests/utils/test_skiperror.py
@@ -0,0 +1,28 @@
+import pandas as pd
+from janitor.utils import skiperror
+import numpy as np
+
+import pytest
+
+
+@pytest.mark.functions
+def test_skiperror():
+    df = pd.DataFrame({"x": [1, 2, 3, "a"], "y": [1, 2, 3, "b"]})
+
+    def func(s):
+        return s + 1
+
+    # Verify that applying function causes error
+    with pytest.raises(Exception):
+        df["x"].apply(func)
+
+    result = df["x"].apply(skiperror(func))
+    assert (result.values[:-1] == np.array([2, 3, 4])).all() and np.isnan(
+        result.values[-1]
+    )
+
+    result = df["x"].apply(skiperror(func, return_x=True))
+    assert (result.values == np.array([2, 3, 4, "a"], dtype=object)).all()
+
+    result = df["x"].apply(skiperror(func, return_x=False, return_val=5))
+    assert (result.values == np.array([2, 3, 4, 5])).all()
diff --git a/tests/utils/test_skipna.py b/tests/utils/test_skipna.py
new file mode 100644
--- /dev/null
+++ b/tests/utils/test_skipna.py
@@ -0,0 +1,22 @@
+import pandas as pd
+from janitor.utils import skipna
+import numpy as np
+
+import pytest
+
+
+@pytest.mark.functions
+def test_skipna():
+    df = pd.DataFrame({"x": ["a", "b", "c", np.nan], "y": [1, 2, 3, np.nan]})
+
+    def func(s):
+        return s + "1"
+
+    # Verify that applying function causes error
+    with pytest.raises(Exception):
+        df["x"].apply(func)
+
+    result = df["x"].apply(skipna(func))
+    assert (
+        result.values[:-1] == np.array(["a1", "b1", "c1"])
+    ).all() and np.isnan(result.values[-1])