internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pyjanitor-devs/pyjanitor

Issue:
[ENH] Shuffle
Dataframe shuffling of rows is done by an unintuitive syntax:

```python
df.sample(frac=1)
```

I propose adding a `shuffle` method:

```python
df.shuffle()
```

This should just wrap `df.sample(frac=1)`, and would be considered super-sugary syntax for the dataframe operation. I also think `shuffle` is a more intuitive name than `sample(frac=1)`. Thoughts here?


Discussion:
I think that's a good idea, but I think that we should be including all the arguments from the aliased function if people want to override defaults.

For example, I can imagine people would want to be able to set `random_state` when running `df.shuffle()`.

Resolving patch:
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -2574,7 +2574,7 @@ def impute(
 @pf.register_dataframe_method
 def then(df: pd.DataFrame, func: Callable) -> pd.DataFrame:
     """
-    Add an arbitrary function to run in the pyJanitor method chain.
+    Add an arbitrary function to run in the ``pyjanitor`` method chain.
 
     :param df: A pandas dataframe.
     :param func: A function you would like to run in the method chain.
@@ -2960,3 +2960,23 @@ def take_first(
     )
 
     return result
+
+
+@pf.register_dataframe_method
+def shuffle(df: pd.DataFrame, random_state=None) -> pd.DataFrame:
+    """
+    Shuffle the rows of the DataFrame.
+
+    Super-sugary syntax! Underneath the hood, we use ``df.sample(frac=1)``,
+    with the option to set the random state.
+
+    Example usage:
+
+    .. code-block:: python
+
+        df = pd.DataFrame(...).shuffle()
+
+    :param df: A pandas DataFrame
+    :param random_state: (optional) A seed for the random number generator.
+    """
+    return df.sample(frac=1, random_state=random_state)


Test patch:
diff --git a/tests/functions/test_shuffle.py b/tests/functions/test_shuffle.py
new file mode 100644
--- /dev/null
+++ b/tests/functions/test_shuffle.py
@@ -0,0 +1,16 @@
+import pandas as pd
+import pytest
+
+
+@pytest.mark.functions
+@pytest.mark.test
+def test_shuffle(dataframe):
+    """
+    Test the shuffle function.
+
+    This test checks that the set of indices in the shuffled dataframe are
+    identical to the set of indices in the original.
+    """
+    df = dataframe.shuffle()
+
+    assert set(df.index) == set(dataframe.index)
Repository: pyjanitor-devs/pyjanitor

Issue:
Copyright notice is hardcoded to 2018
# Brief Description of Fix

<!-- Please describe the fix in terms of a "before" and "after". In other words, what's not so good about the current docs
page, and what you would like to see it become. 

copyright notice is hard-coded to 2018. eg: 
 **©2018, Eric J. Ma. | Powered by Sphinx 2.1.2 & Alabaster 0.7.12 | Page source**
I propose modifying /pyjanitor/docs/conf.py to calculate and display current year.

Example starter wording is provided. -->

Currently, the docs...

I would like to propose a change, such that now the docs...

# Relevant Context

<!-- Please put here, in bullet points, links to the relevant docs page. A few starting template points are available
to get you started. -->

- [Link to documentation page](http://pyjanitor.readthedocs.io)
- [Link to exact file to be edited](https://github.com/ericmjl/pyjanitor/AUTHORS.rst)



Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -16,6 +16,7 @@
 #
 import os
 import sys
+import datetime
 from pathlib import Path
 
 sys.path.insert(0, os.path.abspath("."))
@@ -32,7 +33,11 @@
 # -- Project information -----------------------------------------------------
 
 project = "pyjanitor"
-copyright = "2018, Eric J. Ma"
+
+
+now = datetime.datetime.now()
+CurrentYear = str(now.year)
+copyright = CurrentYear+", PyJanitor devs"
 author = "Eric J. Ma"
 
 # The short X.Y version


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[DOC] Adding context about method chaining  
# Brief Description of Fix

<!-- Please describe the fix in terms of a "before" and "after". In other words, what's not so good about the current docs
page, and what you would like to see it become. 

Example starter wording is provided. -->

Currently, the docs don't explain what method chaining is. A beginner would benefit from reading about it.

I would like to propose a change, such that now the docs include a link to a medium post explaining the topic.

# Relevant Context

<!-- Please put here, in bullet points, links to the relevant docs page. A few starting template points are available
to get you started. -->

- [Link to documentation page](http://pyjanitor.readthedocs.io)
- [Link to exact file to be edited](https://github.com/ericmjl/pyjanitor/README.rst)



Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -10,13 +10,14 @@
 
 # -- Path setup --------------------------------------------------------------
 
+import datetime
+
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
 import os
 import sys
-import datetime
 from pathlib import Path
 
 sys.path.insert(0, os.path.abspath("."))
@@ -37,7 +38,7 @@
 
 now = datetime.datetime.now()
 CurrentYear = str(now.year)
-copyright = CurrentYear+", PyJanitor devs"
+copyright = CurrentYear + ", PyJanitor devs"
 author = "Eric J. Ma"
 
 # The short X.Y version


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[DOC] Clarify original-dataframe mutation behavior in pyjanitor function docstrings
# Brief Description of Fix
Currently, some pyjanitor functions mutate the original dataframe and others return a copy. Solutions are heavily discussed in #79 but no conclusion has been reached. At the moment, it is unclear, without experimentation from the user, which behavior applies in each function. 

In the interim, I propose to explicitly clarify this behavior in each function's docstring so the user has a clear idea regarding the function's mutating behavior. Below is a sample of what this could look like for `.clean_names()`:

"""
Clean column names.
    Takes all column names, converts them to lowercase, then replaces all
    spaces with underscores. <b>Does not mutate original dataframe.</b>
"""

Happy to add this line somewhere else in the docstring if inappropriate here. 

- [Link to documentation page](https://pyjanitor.readthedocs.io/reference/index.html)
- [Link to exact file to be edited](https://github.com/ericmjl/pyjanitor/blob/dev/janitor/functions.py)



Discussion:

Resolving patch:
diff --git a/janitor/finance.py b/janitor/finance.py
--- a/janitor/finance.py
+++ b/janitor/finance.py
@@ -126,6 +126,8 @@ def convert_currency(
     Converts a column from one currency to another, with an option to
     convert based on historical exchange values.
 
+    This method mutates the original DataFrame.
+
     :param df: A pandas dataframe.
     :param column_name: Name of the new column. Should be a string, in order
         for the column name to be compatible with the Feather binary


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[DOC] Clarify original-dataframe mutation behavior in pyjanitor function docstrings
# Brief Description of Fix
Currently, some pyjanitor functions mutate the original dataframe and others return a copy. Solutions are heavily discussed in #79 but no conclusion has been reached. At the moment, it is unclear, without experimentation from the user, which behavior applies in each function. 

In the interim, I propose to explicitly clarify this behavior in each function's docstring so the user has a clear idea regarding the function's mutating behavior. Below is a sample of what this could look like for `.clean_names()`:

"""
Clean column names.
    Takes all column names, converts them to lowercase, then replaces all
    spaces with underscores. <b>Does not mutate original dataframe.</b>
"""

Happy to add this line somewhere else in the docstring if inappropriate here. 

- [Link to documentation page](https://pyjanitor.readthedocs.io/reference/index.html)
- [Link to exact file to be edited](https://github.com/ericmjl/pyjanitor/blob/dev/janitor/functions.py)



Discussion:

Resolving patch:
diff --git a/janitor/chemistry.py b/janitor/chemistry.py
--- a/janitor/chemistry.py
+++ b/janitor/chemistry.py
@@ -80,6 +80,8 @@ def smiles2mol(
 
     Automatically drops invalid SMILES, as determined by RDKIT.
 
+    This method mutates the original DataFrame.
+
     Method chaining usage:
 
     .. code-block:: python
@@ -143,6 +145,8 @@ def morgan_fingerprint(
     intentional, as Morgan fingerprints are usually high-dimensional
     features.
 
+    This method does not mutate the original DataFrame.
+
     Method chaining usage:
 
     .. code-block:: python
@@ -203,6 +207,8 @@ def molecular_descriptors(df: pd.DataFrame, mols_column_name) -> pd.DataFrame:
     Returns a new dataframe without any of the original data. This is
     intentional to leave the user only with the data requested.
 
+    This method does not mutate the original DataFrame.
+
     The molecular descriptors are from the rdkit.Chem.rdMolDescriptors:
 
         Chi0n, Chi0v, Chi1n, Chi1v, Chi2n, Chi2v, Chi3n, Chi3v,
@@ -294,6 +300,8 @@ def maccs_keys_fingerprint(df: pd.DataFrame, mols_column_name) -> pd.DataFrame:
     Returns a new dataframe without any of the original data.
     This is intentional to leave the user with the data requested.
 
+    This method does not mutate the original DataFrame.
+
     Method chaining usage:
 
     .. code-block:: python


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[DOC] Clarify original-dataframe mutation behavior in pyjanitor function docstrings
# Brief Description of Fix
Currently, some pyjanitor functions mutate the original dataframe and others return a copy. Solutions are heavily discussed in #79 but no conclusion has been reached. At the moment, it is unclear, without experimentation from the user, which behavior applies in each function. 

In the interim, I propose to explicitly clarify this behavior in each function's docstring so the user has a clear idea regarding the function's mutating behavior. Below is a sample of what this could look like for `.clean_names()`:

"""
Clean column names.
    Takes all column names, converts them to lowercase, then replaces all
    spaces with underscores. <b>Does not mutate original dataframe.</b>
"""

Happy to add this line somewhere else in the docstring if inappropriate here. 

- [Link to documentation page](https://pyjanitor.readthedocs.io/reference/index.html)
- [Link to exact file to be edited](https://github.com/ericmjl/pyjanitor/blob/dev/janitor/functions.py)



Discussion:

Resolving patch:
diff --git a/janitor/biology.py b/janitor/biology.py
--- a/janitor/biology.py
+++ b/janitor/biology.py
@@ -26,12 +26,14 @@ def join_fasta(
     This allows us to add the string sequence of a FASTA file as a new column
     of data in the dataframe.
 
-    This function only attaches the string representation of the SeqRecord.Seq
+    This method only attaches the string representation of the SeqRecord.Seq
     object from Biopython. Does not attach the full SeqRecord. Alphabet is
     also not stored, under the assumption that the data scientist has domain
     knowledge of what kind of sequence is being read in (nucleotide vs. amino
     acid.)
 
+    This method mutates the original DataFrame.
+
     For more advanced functions, please use phylopandas.
 
     :param df: A pandas DataFrame.


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[DOC] Broken link on encode_categorical function
# Brief Description of Fix

Before -  the link below is broken
<http://pandas.pydata.org/pandas-docs/stable/user_guide/categorical.html> # noqa: E501 dtype.

After - correct working link to the pandas documentation

[Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/user_guide/categorical.html)

Link to documentation page"
- [link to page to be edited](https://pyjanitor.readthedocs.io/reference/janitor.functions/janitor.encode_categorical.html#janitor.encode_categorical)



Discussion:
Great catch! 

Resolving patch:
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -202,8 +202,7 @@ def encode_categorical(
 ) -> pd.DataFrame:
     """
     Encode the specified columns with Pandas'
-    `category <http://pandas.pydata.org/pandas-docs/stable/user_guide/categorical.html>`  # noqa: E501
-    dtype.
+    `category dtype <http://pandas.pydata.org/pandas-docs/stable/user_guide/categorical.html>`_. # noqa:E501
 
     This method mutates the original DataFrame.
 


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[BUG] find_replace() doesn't replace null values
# Brief Description

When replacing null values in a dataframe, `find_replace()` doesn't perform the replacement.

# System Information

- Operating system: Linux
- OS details (optional):  Ubuntu 
- Python version (required): 3.7

# Minimally Reproducible Code

```python
import pandas as pd
import numpy as np
import janitor
df = pd.DataFrame({'a': [1, 2, np.nan]})
df.find_replace('a', {np.nan: 5})  # Result is an unchanged dataframe
```

# Error Messages

None



Discussion:
Thanks for raising this, @nickdelgrosso! 

Yes, in fact, null values should be out of scope of find_replace, because `fillna()` can handle this. (That said, pandas-native `fillna()` doesn't provide a mechanism to fill NA values on selected columns - we can talk about this at another time.)

Resolving patch:
diff --git a/janitor/functions.py b/janitor/functions.py
--- a/janitor/functions.py
+++ b/janitor/functions.py
@@ -2715,9 +2715,15 @@ def find_replace(df: pd.DataFrame, column_name, mapper: Dict) -> pd.DataFrame:
     :param column_name: The column on which the find/replace action is to be
         made.
     :param mapper: A dictionary that maps "thing to find" -> "thing to
-        replace".
+        replace".  Note: Does not support null-value replacement.
     :returns: A pandas DataFrame.
     """
+    if any(map(pd.isna, mapper.keys())):
+        raise ValueError(
+            "find_replace() does not support null replacement. "
+            "Use DataFrame.fillna() instead."
+        )
+
     df[column_name] = df[column_name].apply(lambda x: mapper.get(x, x))
     return df
 


Test patch:
diff --git a/tests/functions/test_find_replace.py b/tests/functions/test_find_replace.py
--- a/tests/functions/test_find_replace.py
+++ b/tests/functions/test_find_replace.py
@@ -3,11 +3,29 @@
 import pytest
 
 
+@pytest.fixture()
+def df():
+    return pd.DataFrame(
+        {"a": [1, np.nan, 3], "b": [2, 3, 1], "c": [2, np.nan, 2]}
+    )
+
+
 @pytest.mark.test
 @pytest.mark.functions
-def test_find_replace():
-    df = pd.DataFrame(
-        {"a": [1, np.nan, 3], "b": [2, 3, 1], "c": [2, np.nan, 9]}
-    ).find_replace("a", {1: 2, 3: 4, np.nan: 5})
+def test_find_replace_single(df):
+    assert df["a"].iloc[2] == 3
+    df.find_replace("a", {3: 5})
+    assert df["a"].iloc[2] == 5
+
+    assert sum(df["c"] == 2) == 2
+    assert sum(df["c"] == 5) == 0
+    df.find_replace("c", {2: 5})
+    assert sum(df["c"] == 2) == 0
+    assert sum(df["c"] == 5) == 2
 
-    assert not all(pd.isnull(df["a"]))
+
+@pytest.mark.test
+@pytest.mark.functions
+def test_find_replace_null_raises_error(df):
+    with pytest.raises(ValueError):
+        df.find_replace("a", {np.nan: 5})
Repository: pyjanitor-devs/pyjanitor

Issue:
[DOC] Clarify Python version requirements
# Brief Description of Fix

I was looking through documentation (for users and contributors), and it was unclear to me which python versions we actually support. It seems that we support python 3.6 + 3.7. This arose as I was updating the `pyproject.toml` file to avoid the warning:
```
--py36 is deprecated and will be removed in a future version. Use --target-version py36 instead.
```

Our current locations of explicit python versions are in:
- `pyproject.toml`
    - `py36 = true`
- `environment-dev.yml`
    - `- python >= 3.6`
- `.azure-pipelines/pipeline-master.yml`
    - `python.version: "3.7"`

# Proposed Fix

If `pyjanitor` is in fact meant to function on 3.6+, we should
- Explicitly inform contributors that their code should be 3.6+ compatible
- Inform users which python versions the package requires, on the documentation site, PyPI etc
- Add `python_requires=">=3.6"` to `setup.py`



Discussion:
Thanks for the catch, @hectormz! 

I think we only test on the latest version of Python, which means that is the only “officially” supported version. (I disabled 3.6, to allow more CI pipelines to run simultaneously.)

That said, we do use language features that only show up in Python 3.6 onwards, I think.

Let’s definitely move forward with what you suggested.
Great, I'll make some changes to establish in documentation and code that this supports Python 3.6+ and submit a PR

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -15,4 +15,5 @@ def requirements():
     url="https://github.com/ericmjl/pyjanitor",
     packages=["janitor"],
     install_requires=requirements(),
+    python_requires=">=3.6",
 )


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[ENH] Enable chiral morgans by default
# Brief Description

I would like to propose changing `janitor.chemistry.morgan_fingerprints` to pass `useChirality=True` to both `GetMorganFingerprintAsBitVect` and `GetHashedMorganFingerprint `. This should be exposed directly as default to `True` in the Python API.


Discussion:

Resolving patch:
diff --git a/janitor/chemistry.py b/janitor/chemistry.py
--- a/janitor/chemistry.py
+++ b/janitor/chemistry.py
@@ -244,12 +244,12 @@ def morgan_fingerprint(
 
     if kind == "bits":
         fps = [
-            GetMorganFingerprintAsBitVect(m, radius, nbits)
+            GetMorganFingerprintAsBitVect(m, radius, nbits, useChirality=True)
             for m in df[mols_column_name]
         ]
     elif kind == "counts":
         fps = [
-            GetHashedMorganFingerprint(m, radius, nbits)
+            GetHashedMorganFingerprint(m, radius, nbits, useChirality=True)
             for m in df[mols_column_name]
         ]
 


Test patch:
Repository: pyjanitor-devs/pyjanitor

Issue:
[INF] Make requirements.txt smaller
Follow-up from #257 

The idea is to have feature-specific requirements.txt. Such as for biology, specifically requires biopython.

so we can install the package per feature as needed, such as with extra biology. It goes `pip install "pyjanitor[biology]"`

The example of such implementations in `setup.py` is available at this link: https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
[INF] Make requirements.txt smaller
Follow-up from #257 

The idea is to have feature-specific requirements.txt. Such as for biology, specifically requires biopython.

so we can install the package per feature as needed, such as with extra biology. It goes `pip install "pyjanitor[biology]"`

The example of such implementations in `setup.py` is available at this link: https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies


Discussion:
Makes a lot of sense to me. We also have that chemistry module in there, and I'm guessing both of those might be somewhat niche for what the average user of pyjanitor's use case is.

Also, the current usage for `import_message` in chemistry and biology only references conda packages to be installed (rather than both pip & conda depending on the user's env). Might be something else to include in this general goal of making pyjanitor more modularizable.
I would love to work on this :)
Assigning this to you, @eyaltrabelsi!
Makes a lot of sense to me. We also have that chemistry module in there, and I'm guessing both of those might be somewhat niche for what the average user of pyjanitor's use case is.

Also, the current usage for `import_message` in chemistry and biology only references conda packages to be installed (rather than both pip & conda depending on the user's env). Might be something else to include in this general goal of making pyjanitor more modularizable.
I would love to work on this :)
Assigning this to you, @eyaltrabelsi!

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -38,6 +38,12 @@ def generate_long_description() -> str:
     return long_description
 
 
+extra_spark = ["pyspark"]
+extra_biology = ["biopython"]
+extra_chemistry = ["rdkit"]
+extra_engineering = ["unyt"]
+extra_all = extra_biology + extra_engineering + extra_spark
+
 setup(
     name="pyjanitor",
     version="0.18.2",
@@ -47,6 +53,14 @@ def generate_long_description() -> str:
     url="https://github.com/ericmjl/pyjanitor",
     packages=["janitor"],
     install_requires=requirements(),
+    extras_require={
+        "all": extra_all,
+        "biology": extra_biology,
+        # "chemistry": extra_chemistry, should be inserted once rdkit
+        # fixes https://github.com/rdkit/rdkit/issues/1812
+        "engineering": extra_engineering,
+        "spark": extra_spark,
+    },
     python_requires=">=3.6",
     long_description=generate_long_description(),
     long_description_content_type="text/x-rst",


Test patch: