Repository: pyjanitor-devs/pyjanitor (python)

Issue:
Bug with complement in `filter_on` with multiple criteria
# Brief Description

`filter_on` function does not return the complement properly. (see MWE below) Came across this bug when I was writing the MWE docstring for it.

# Minimally Reproducible Code

```python
import janitor
import pandas as pd
df = pd.DataFrame({
    "student_id": ["S1", "S2", "S3"],
    "score": [40, 60, 85],
})

df.filter_on("(score < 50) & (score > 70)", complement=True)
# the above should return students whose scores are between 50 and 70 included.
# but the result is

  student_id  score
2         S3     85
```

Reason is simple, see the source code implementation:

```python
if complement:
    return df.query("not " + criteria)
```

criteria should be enclosed with parenthesis to ensure complement of the whole criteria is taken instead.


Perhaps a test should also be added to cover this case as well (as in, multiple criteria).

Pull request:
[ENH] Fix filter on bug with complement option
# PR Description

Please describe the changes proposed in the pull request:

- As per title.
- Added an additional test with filtering by multiple criteria to prevent this bug from happening in the future.
- Also sneaked in a small formatting fix for `read_commandline` documentation.

<!-- Doing so provides maintainers with context on what the PR is, and can help us more effectively review your PR. -->

<!-- Please also identify below which issue that has been raised that you are going to close. -->

**This PR resolves #988.**

<!-- As you go down the PR template, please feel free to delete sections that are irrelevant. -->

# PR Checklist

<!-- This checklist exists for newcomers who are not yet familiar with our requirements. If you are experienced with
the project, please feel free to delete this section. -->

Please ensure that you have done the following:

1. [ ] PR in from a fork off your branch. Do not PR from `<your_username>`:`dev`, but rather from `<your_username>`:`<feature-branch_name>`.
<!-- Doing this helps us keep the commit history much cleaner than it would otherwise be. -->
2. [x] If you're not on the contributors list, add yourself to `AUTHORS.rst`.
<!-- We'd like to acknowledge your contributions! -->
3. [x] Add a line to `CHANGELOG.md` under the latest version header (i.e. the one that is "on deck") describing the contribution.
    - Do use some discretion here; if there are multiple PRs that are related, keep them in a single line.

# Automatic checks

There will be automatic checks run on the PR. These include:

- Building a preview of the docs on Netlify
- Automatically linting the code
- Making sure the code is documented
- Making sure that all tests are passed
- Making sure that code coverage doesn't go down.

# Relevant Reviewers

<!-- Finally, please tag relevant maintainers to review. -->

Please tag maintainers to review.

- @ericmjl


Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 31e93be..0b670ff 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,7 @@
 -   [ENH] Fix `concatenate_columns` not working on category inputs @zbarry
 -   [INF] Simplify CI system @ericmjl
 -   [ENH] Added "read_commandline" function to janitor.io @BaritoneBeard
+-   [BUG] Fix bug with the complement parameter of `filter_on`. Issue #988. @thatlittleboy
 
 ## [v0.22.0] - 2021-11-21
 
diff --git a/janitor/functions/filter.py b/janitor/functions/filter.py
index 75e5310..3669004 100644
--- a/janitor/functions/filter.py
+++ b/janitor/functions/filter.py
@@ -107,7 +107,7 @@ def filter_on(
     :returns: A filtered pandas DataFrame.
     """
     if complement:
-        return df.query("not " + criteria)
+        return df.query(f"not ({criteria})")
     return df.query(criteria)
 
 
diff --git a/janitor/io.py b/janitor/io.py
index 7a7f400..d8383f5 100644
--- a/janitor/io.py
+++ b/janitor/io.py
@@ -89,15 +89,16 @@ def read_commandline(cmd: str, **kwargs) -> pd.DataFrame:
     ```python
     import janitor as jn
     df = jn.read_commandline("cat data.csv | grep .SEA1AA")
+    ```
 
     This function assumes that your command line command will return
-    an output that is parsable using pandas.read_csv and StringIO.
-    We default to using pd.read_csv underneath the hood.
+    an output that is parsable using `pandas.read_csv` and StringIO.
+    We default to using `pd.read_csv` underneath the hood.
     Keyword arguments are passed through to read_csv.
-    ```
 
     :param cmd: Shell command to preprocess a file on disk.
-    :param kwargs: Keyword arguments that are passed through to pd.read_csv().
+    :param kwargs: Keyword arguments that are passed through to
+        `pd.read_csv()`.
     :raises JanitorError: If commandline command is malformed or invalid.
     :returns: A pandas DataFrame parsed from the stdout of the underlying
         shell.


Test patch:
diff --git a/tests/functions/test_filter_on.py b/tests/functions/test_filter_on.py
index 2bfda02..db1c696 100644
--- a/tests/functions/test_filter_on.py
+++ b/tests/functions/test_filter_on.py
@@ -6,3 +6,10 @@ import pytest
 def test_filter_on(dataframe, complement, expected):
     df = dataframe.filter_on("a == 3", complement=complement)
     assert len(df) == expected
+
+
+@pytest.mark.functions
+@pytest.mark.parametrize("complement,expected", [(True, 3), (False, 6)])
+def test_filter_on_with_multiple_criteria(dataframe, complement, expected):
+    df = dataframe.filter_on("(a == 3) | (a == 1)", complement=complement)
+    assert len(df) == expected