internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pyjanitor-devs/pyjanitor

Issue:
[INF] Allow `import_message()` to be Python distribution flexible
# Brief Description

<!-- Please provide a brief description of what you'd like to propose. -->

Currently, if a user attempts to user a feature of an optional external package (`rdkit`, `biopython`, `unyt`, `pyspark`) which is not installed, the user receives an error that directs them on how to install it. The error message is displayed by `import_message()` which passes instructions on how to install it. Ex:
```
To use the janitor submodule spark, you need to install pyspark.
To do so, use the following command:
    conda install -c conda-forge pyspark
```
With the exception of `rdkit`, I think all of these packages are `pip` installable. It would be nice if this message could decide whether to provide `conda` vs `pip` instructions to the user. Or tell them that the package can only be installed with `conda`. 

This is how the function is currently called:
```python
import_message(submodule="spark", package="pyspark", 
    installation="conda install -c conda-forge pyspark")
```

Not all `conda` installs will use the same channel. One option is to provide both `conda` and `pip` instructions as arguments in the call, and it figures out which to send to the user. If either are `None`, then it is understood to be `pip` or `conda` only.

# Example API

One verbose option would be to extend what currently exists:
```python
import_message(submodule="spark", package="pyspark", 
    conda_installation="conda install -c conda-forge pyspark", 
    pip_installation="pip install pyspark")
```

A more succinct version could be:
```python
import_message(submodule="spark", package="pyspark", 
    conda_channel="conda-forge", pip_install=True)
```
which would use the provided `package` argument, and `conda_channel` could be `None` if it doesn't exist on `conda`.


Discussion:
This answer on [StackOverflow](https://stackoverflow.com/questions/21282363/any-way-to-tell-if-users-python-environment-is-anaconda/21282816#21282816) should work to determine if the user has an Anaconda distribution.
This is great, @hectormz! Are you interested in taking this on? If not, please go ahead and tag it as available for hacking on!
If anyone else is interested, they can claim it. Otherwise I can check back in claim it!
I have this 😃, PR coming soon

Resolving patch:
diff --git a/janitor/spark/functions.py b/janitor/spark/functions.py
--- a/janitor/spark/functions.py
+++ b/janitor/spark/functions.py
@@ -11,7 +11,7 @@
 try:
     from pyspark.sql import DataFrame
 except ImportError:
-    import_message(
+    janitor_utils.import_message(
         submodule="spark",
         package="pyspark",
         conda_channel="conda-forge",


Test patch:
diff --git a/tests/spark/functions/test_clean_names_spark.py b/tests/spark/functions/test_clean_names_spark.py
--- a/tests/spark/functions/test_clean_names_spark.py
+++ b/tests/spark/functions/test_clean_names_spark.py
@@ -4,7 +4,6 @@
 
 try:
     import pyspark
-    from pyspark.sql import SparkSession
 
     import janitor.spark
 except ImportError:
Repository: pyjanitor-devs/pyjanitor

Issue:
[BUG] Error message on "new column names" is incorrect in deconcatenate_columns
# Brief Description

If I pass in the wrong number of column names in deconcatenate_columns, the error message simply reflects back the number of columns that I provided, rather than the number of columns I need to provide.

Specifically, these two lines need to be changed:

```python
-> 1033             f"you need to provide {len(new_column_names)} names"
   1034             "to new_column_names"
```

to:

```python
-> 1033             f"you need to provide {len(deconcat.shape[1])} names "
   1034             "to new_column_names"
```

(Extra space needed at the end of the first line as well.)


Discussion:
I would like to work on this issue.

Resolving patch:
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -997,9 +997,8 @@ def deconcatenate_column(
 
     if not len(new_column_names) == df_deconcat.shape[1]:
         raise JanitorError(
-            f"Number of supplied column names was not equal to the number of "
-            f"deconcatenated columns. You need to provide "
-            f"{len(new_column_names)} names to new_column_names."
+            f"you need to provide {len(deconcat.shape[1])} names "
+            "to new_column_names"
         )
 
     df_deconcat.columns = new_column_names


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[DOC] transform_column dest_column_name kwarg description needs to be clearer on defaults
# Brief Description of Fix

The current wording on `dest_column_name` is:


`dest_column_name – The column name to store the transformation result in. By default, replaces contents of original column.`

I would like to propose to change it to:

`dest_column_name – The column name to store the transformation result in. Defaults to None, which will result in the original column name being overwritten. If a name is provided here, then a new column with the transformed values will be created.`


# Relevant Context

- [Link to documentation page](https://pyjanitor.readthedocs.io/reference/janitor.functions/janitor.transform_column.html)
- [Link to exact file to be edited](https://github.com/ericmjl/pyjanitor/blob/dev/janitor/functions.py#L2034)



Discussion:
It looks like this issue is still open - I'd be happy to take care of it! 
@evan-anderson I'm happy to assign it to you! :smile: Hope you've been having fun!

Resolving patch:
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -2248,7 +2248,9 @@ def transform_column(
     :param column_name: The column to transform.
     :param function: A function to apply on the column.
     :param dest_column_name: The column name to store the transformation result
-        in. By default, replaces contents of original column.
+        in. Defaults to None, which will result in the original column
+        name being overwritten. If a name is provided here, then a new column
+        with the transformed values will be created.
     :returns: A pandas DataFrame with a transformed column.
     """
     if dest_column_name is None:


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
Version on GitHub different from version on PyPI
# Brief Description of Fix

<!-- Please describe the fix in terms of a "before" and "after". In other words, what's not so good about the current docs
page, and what you would like to see it become.

Example starter wording is provided. -->

Currently, the version in the repo is "0.19.0", whereas it's "0.20.0" on PyPI.

I would like to propose a change, such that the version is updated here.

# Relevant Context

<!-- Please put here, in bullet points, links to the relevant docs page. A few starting template points are available
to get you started. -->

- [Link to PyPI](https://pypi.org/project/pyjanitor/)



Discussion:

Resolving patch:
diff --git a/janitor/__init__.py b/janitor/__init__.py
--- a/janitor/__init__.py
+++ b/janitor/__init__.py
@@ -19,4 +19,4 @@ def get_features_targets(*args, **kwargs):
     return _get_features_targets(*args, **kwargs)
 
 
-__version__ = "0.19.0"
+__version__ = "0.20.0"
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -46,7 +46,7 @@ def generate_long_description() -> str:
 
 setup(
     name="pyjanitor",
-    version="0.19.0",
+    version="0.20.0",
     description="Tools for cleaning pandas DataFrames",
     author="Eric J. Ma",
     author_email="ericmajinglong@gmail.com",


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
Natsort import error
# Brief Description

The module `natsort` isn't found because it was added (in #627) to dev requirements but needs to be in the main requirements file. It is imported with all functions from the init script. 

Rather than requiring it, perhaps it could also be brought in with a `try`, `except` per #97 

# Error Messages

```
 /usr/local/lib/python3.7/site-packages/janitor/functions.py:25: in <module>
     from natsort import index_natsorted, natsorted
 E   ModuleNotFoundError: No module named 'natsort'
```


Discussion:
Wonderful, thanks for the catch, @adamzev! I am going to cut a patch release to get this out.

Resolving patch:
diff --git a/janitor/__init__.py b/janitor/__init__.py
--- a/janitor/__init__.py
+++ b/janitor/__init__.py
@@ -19,4 +19,4 @@ def get_features_targets(*args, **kwargs):
     return _get_features_targets(*args, **kwargs)
 
 
-__version__ = "0.20.1"
+__version__ = "0.20.2"
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -46,7 +46,7 @@ def generate_long_description() -> str:
 
 setup(
     name="pyjanitor",
-    version="0.20.1",
+    version="0.20.2",
     description="Tools for cleaning pandas DataFrames",
     author="Eric J. Ma",
     author_email="ericmajinglong@gmail.com",


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
`df` annotation lacking in `find_replace` function of `functions.py` submodule
# Brief Description

`df` annotation lacking in `find_replace` function of `functions.py` submodule.
It seems to be the only function for which there is no type annotation for the dataframe. Clearly not critical.


Discussion:

Resolving patch:
diff --git a/janitor/finance.py b/janitor/finance.py
--- a/janitor/finance.py
+++ b/janitor/finance.py
@@ -347,7 +347,7 @@ def _convert_currency(
     from_currency: str = None,
     to_currency: str = None,
     historical_date: date = None,
-):
+) -> float:
     """
     Currency conversion for Pandas DataFrame column.
 
@@ -492,7 +492,7 @@ def convert_currency(
 @lru_cache(maxsize=32)
 def _inflate_currency(
     country: str = None, currency_year: int = None, to_year: int = None
-):
+) -> float:
     """
     Currency inflation for Pandas DataFrame column.
     Helper function for `inflate_currency` method.
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -2978,7 +2978,9 @@ def dropnotnull(df: pd.DataFrame, column_name: Hashable) -> pd.DataFrame:
 
 
 @pf.register_dataframe_method
-def find_replace(df, match: str = "exact", **mappings):
+def find_replace(
+    df: pd.DataFrame, match: str = "exact", **mappings
+) -> pd.DataFrame:
     """Perform a find-and-replace action on provided columns.
 
     Depending on use case, users can choose either exact, full-value matching,
@@ -3471,7 +3473,9 @@ def shuffle(
 
 
 @pf.register_dataframe_method
-def join_apply(df: pd.DataFrame, func: Callable, new_column_name: str):
+def join_apply(
+    df: pd.DataFrame, func: Callable, new_column_name: str
+) -> pd.DataFrame:
     """Join the result of applying a function across dataframe rows.
 
     This method does not mutate the original DataFrame.
diff --git a/janitor/ml.py b/janitor/ml.py
--- a/janitor/ml.py
+++ b/janitor/ml.py
@@ -17,7 +17,7 @@ def get_features_targets(
     df: pd.DataFrame,
     target_column_names: Union[str, Union[List, Tuple], Hashable],
     feature_column_names: Union[str, Iterable[str], Hashable] = None,
-):
+) -> Tuple[pd.DataFrame, pd.DataFrame]:
     """
     Get the features and targets as separate DataFrames/Series.
 


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
groupby_agg mutates dataframe
# Brief Description

When using groupby_agg the source dataframe is changed
Issue may be related to https://github.com/ericmjl/pyjanitor/issues/645

# System Information

- Operating system: Windows 
- OS details (optional):  Windows Pro 19041.450
- Python version (required): 3.7
- Using pyjanitor 0.20.9 on a clean environment

# Minimally Reproducible Code

```python
import pandas as pd
import janitor

data = {
    "item": ["shoe", "shoe", "bag", "shoe", "bag"],
    "MRP": [220, 450, 320, 200, 305],
    "number_sold": [100, 40, 56, 38, 25],
}

df = pd.DataFrame(data)

print(df.head())

df.groupby_agg(by="item", agg="mean", agg_column_name="MRP", new_column_name="Avg_MRP")

print(df.head())
```

# Error Messages

The second print statement displays the new column Avg_MRP

```

```



Discussion:

Resolving patch:
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -3246,7 +3246,7 @@ def groupby_agg(
     :param axis: Split along rows (0) or columns (1).
     :returns: A pandas DataFrame.
     """
-
+    df = df.copy()
     # convert to list
     # needed when creating a mapping through the iteration
     if isinstance(by, str):
@@ -4015,6 +4015,11 @@ def expand_grid(
     # if there is a dataframe, for the method chaining,
     # it must have a key, to create a name value pair
     if df is not None:
+        df = df.copy()
+        if isinstance(df.index, pd.MultiIndex) or isinstance(
+            df.columns, pd.MultiIndex
+        ):
+            raise TypeError("`expand_grid` does not work with pd.MultiIndex")
         if not df_key:
             raise KeyError(
                 """
@@ -4113,6 +4118,7 @@ def process_text(
     :raises: KeyError if ``string_function`` is not a Pandas string method.
     :raises: TypeError if wrong ``arg`` or ``kwarg`` is supplied.
     """
+    df = df.copy()
 
     pandas_string_methods = [
         func.__name__
@@ -4123,7 +4129,9 @@ def process_text(
     if string_function not in pandas_string_methods:
         raise KeyError(f"{string_function} is not a Pandas string method.")
 
-    df[column] = getattr(df[column].str, string_function)(*args, **kwargs)
+    df.loc[:, column] = getattr(df.loc[:, column].str, string_function)(
+        *args, **kwargs
+    )
 
     return df
 
@@ -4220,7 +4228,7 @@ def fill_direction(
     :raises: ValueError if direction supplied is not one of `down`,`up`,
         `updown`, or `downup`.
     """
-
+    df = df.copy()
     # check that dictionary is not empty
     if not directions:
         raise ValueError("A mapping of columns with directions is required.")


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[DOC] Update docs to have timeseries section
# Brief Description of Fix

<!-- Please describe the fix in terms of a "before" and "after". In other words, what's not so good about the current docs
page, and what you would like to see it become.

Example starter wording is provided. -->

Currently, the docs do not have a section for the timeseries module.

I would like to propose a change, such that now the docs will have details on the timerseries functions.

# Relevant Context

<!-- Please put here, in bullet points, links to the relevant docs page. A few starting template points are available
to get you started. -->

- [Link to documentation page](https://ericmjl.github.io/pyjanitor/reference/index.html)
- [Link to exact file to be edited](https://github.com/ericmjl/pyjanitor/blob/dev/docs/reference/index.rst)



Discussion:
I can take this if you'd like 😄 
Thanks @loganthomas! Assigning!

Resolving patch:
diff --git a/janitor/timeseries.py b/janitor/timeseries.py
--- a/janitor/timeseries.py
+++ b/janitor/timeseries.py
@@ -1,6 +1,4 @@
-"""
-Time series-specific data testing and cleaning functions.
-"""
+""" Time series-specific data cleaning functions. """
 
 import pandas as pd
 import pandas_flavor as pf
@@ -22,9 +20,28 @@ def fill_missing_timestamps(
     this function will reindex the dataframe.
     If timestamps are not missing,
     then the function will return the dataframe unmodified.
-    Example usage:
+
+    Functional usage example:
+
+    .. code-block:: python
+
+        import pandas as pd
+        import janitor.timeseries
+
+        df = pd.DataFrame(...)
+
+        df = janitor.timeseries.fill_missing_timestamps(
+            df=df,
+            frequency="1H",
+        )
+
+    Method chaining example:
+
     .. code-block:: python
 
+        import pandas as pd
+        import janitor.timeseries
+
         df = (
             pd.DataFrame(...)
             .fill_missing_timestamps(frequency="1H")
@@ -102,13 +119,29 @@ def sort_timestamps_monotonically(
     If timestamps are not monotonic,
     then the function will sort the dataframe.
 
-    Example usage:
+    Functional usage example:
 
     .. code-block:: python
 
+        import pandas as pd
+        import janitor.timeseries
+
+        df = pd.DataFrame(...)
+
+        df = janitor.timeseries.sort_timestamps_monotonically(
+            direction="increasing"
+        )
+
+    Method chaining example:
+
+    .. code-block:: python
+
+        import pandas as pd
+        import janitor.timeseries
+
         df = (
             pd.DataFrame(...)
-            .sort_timestamps_monotonically(direction='increasing')
+            .sort_timestamps_monotonically(direction="increasing")
         )
 
     :param df: Dataframe which needs to be tested for monotonicity


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
janitor.utils.check_column issue when passed single column
If `check_column` is passed a string instead of an iterable list/tuple/etc of column names, as can happen with `label_encode`, it will iterate through the string's characters, checking whether each is present as a column:

https://github.com/ericmjl/pyjanitor/blob/87ee618d3764a8c0bf9cf5c277b926cbc0f3e14e/janitor/utils.py#L340-L371

There should be some form of type checking here to prevent this from happening:

```python
if isinstance(column_names, str) or not isinstance(column_names, Iterable):
    column_names = [column_names]
```

Will submit PR for this later


Discussion:

Resolving patch:
diff --git a/janitor/utils.py b/janitor/utils.py
--- a/janitor/utils.py
+++ b/janitor/utils.py
@@ -42,7 +42,7 @@ def check(varname: str, value, expected_types: list):
 
         check('x', x, [int, float])
 
-    :param varname: The name of the variable.
+    :param varname: The name of the variable (for diagnostic error message).
     :param value: The value of the varname.
     :param expected_types: The types we expect the item to be.
     :raises TypeError: if data is not the expected type.
@@ -360,6 +360,9 @@ def check_column(
         in df.columns.
     :raises ValueError: if data is not the expected type.
     """
+    if isinstance(column_names, str) or not isinstance(column_names, Iterable):
+        column_names = [column_names]
+
     for column_name in column_names:
         if present and column_name not in df.columns:  # skipcq: PYL-R1720
             raise ValueError(


Test patch:
diff --git a/tests/utils/test_check_column.py b/tests/utils/test_check_column.py
--- a/tests/utils/test_check_column.py
+++ b/tests/utils/test_check_column.py
@@ -11,6 +11,27 @@ def test_check_column(dataframe):
     assert check_column(dataframe, ["a"]) is None
 
 
+@pytest.mark.utils
+def test_check_column_single(dataframe):
+    """
+    Check works with a single input
+    """
+
+    assert check_column(dataframe, "a") is None
+
+    with pytest.raises(ValueError):
+        check_column(dataframe, "b")
+
+    # should also work with non-string inputs
+
+    with pytest.raises(ValueError):
+        check_column(dataframe, 2)
+
+    dataframe[2] = "asdf"
+
+    assert check_column(dataframe, 2) is None
+
+
 @pytest.mark.utils
 def test_check_column_absent_column(dataframe):
     """
Repository: pyjanitor-devs/pyjanitor

Issue:
[ENH] Add `softmax` to math submodule
In reviewing the `math` module, I'd like to add `softmax` to the functions. This will complete the "activation" functions :) as we have sigmoids but no softmax.
[ENH] Add `softmax` to math submodule
In reviewing the `math` module, I'd like to add `softmax` to the functions. This will complete the "activation" functions :) as we have sigmoids but no softmax.


Discussion:

Resolving patch:
diff --git a/janitor/math.py b/janitor/math.py
--- a/janitor/math.py
+++ b/janitor/math.py
@@ -8,6 +8,7 @@
 import pandas_flavor as pf
 from pandas.api.types import is_numeric_dtype
 from scipy.special import expit
+from scipy.special import softmax as scipy_softmax
 from scipy.stats import norm
 
 
@@ -65,6 +66,27 @@ def sigmoid(s: pd.Series) -> pd.Series:
     return expit(s)
 
 
+@pf.register_series_method
+def softmax(s: pd.Series) -> pd.Series:
+    """
+    Take the softmax transform of the series.
+
+    The softmax function transforms each element of a collection by
+    computing the exponential of each element divided by the sum of the
+    exponentials of all the elements.
+
+    That is, if x is a one-dimensional numpy array or pandas Series:
+
+    ```python
+    softmax(x) = exp(x)/sum(exp(x))
+    ```
+
+    :param s: Input Series.
+    :return: Transformed Series.
+    """
+    return scipy_softmax(s)
+
+
 @pf.register_series_method
 def logit(s: pd.Series, error: str = "warn") -> pd.Series:
     """


Test patch:
diff --git a/tests/math/test_softmax.py b/tests/math/test_softmax.py
new file mode 100644
--- /dev/null
+++ b/tests/math/test_softmax.py
@@ -0,0 +1,12 @@
+import pandas as pd
+import pytest
+from scipy.special import softmax as scipy_softmax
+
+
+@pytest.mark.functions
+def test_softmax():
+    s = pd.Series([0, 1, 2, 3, -1])
+    out = s.softmax()
+    assert (out == scipy_softmax(s)).all()
+    assert (s.index == out.index).all()
+    assert out.sum() == 1.0
Repository: pyjanitor-devs/pyjanitor

Issue:
Some modules are missing from the top-level import
Issue referenced during PR here: https://github.com/pyjanitor-devs/pyjanitor/pull/977#discussion_r781732964

For example, now running:

```python
import janitor as jn
jn.io.read_csvs("")  # throws AttributeError: module 'janitor' has no attribute 'io'
```

Similarly for other modules like biology or timeseries.


Discussion:
Apart from IO, I think the others were intentionally shielded from the top-level namespace to prevent their imports from running on first import of janitor. Sort of like a namespace shielding. 

That said, I think it's a good idea for the IO module to be top-level, as it'll allow for `jn.read_csvs()` just like `pd.read_csv()` and other IO functions are top-level.
@ericmjl I see! yes, namespace shielding is a good point. Then, I'll just add `io` to top-level imports.

Resolving patch:
diff --git a/janitor/__init__.py b/janitor/__init__.py
--- a/janitor/__init__.py
+++ b/janitor/__init__.py
@@ -5,6 +5,7 @@
     pass
 
 from .functions import *  # noqa: F403, F401
+from .io import *  # noqa: F403, F401
 from .math import *  # noqa: F403, F401
 from .ml import get_features_targets as _get_features_targets
 from .utils import refactored_function
diff --git a/janitor/io.py b/janitor/io.py
--- a/janitor/io.py
+++ b/janitor/io.py
@@ -88,7 +88,7 @@ def read_commandline(cmd: str, **kwargs) -> pd.DataFrame:
 
     ```python
     import janitor as jn
-    df = jn.io.read_commandline("cat data.csv | grep .SEA1AA")
+    df = jn.read_commandline("cat data.csv | grep .SEA1AA")
 
     This function assumes that your command line command will return
     an output that is parsable using pandas.read_csv and StringIO.


Test patch: