Repository: glotzerlab/signac
Pull request #974 (2024-02-01 09:06:48)

Bump pytest from 7.4.4 to 8.0.0
Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.4 to 8.0.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/pytest-dev/pytest/releases">pytest's releases</a>.</em></p>
<blockquote>
<h2>pytest 8.0.0 (2024-01-27)</h2>
<p>See <a href="https://github.com/pytest-dev/pytest/releases/tag/8.0.0rc1">8.0.0rc1</a> and <a href="https://github.com/pytest-dev/pytest/releases/tag/8.0.0rc2">8.0.0rc2</a> for the full changes since pytest 7.4!</p>
<h2>Bug Fixes</h2>
<ul>
<li><a href="https://redirect.github.com/pytest-dev/pytest/issues/11842">#11842</a>: Properly escape the <code>reason</code> of a <code>skip &lt;pytest.mark.skip ref&gt;</code>{.interpreted-text role=&quot;ref&quot;} mark when writing JUnit XML files.</li>
<li><a href="https://redirect.github.com/pytest-dev/pytest/issues/11861">#11861</a>: Avoid microsecond exceeds <code>1_000_000</code> when using <code>log-date-format</code> with <code>%f</code> specifier, which might cause the test suite to crash.</li>
</ul>
<h2>8.0.0rc2</h2>
<h1>pytest 8.0.0rc2 (2024-01-17)</h1>
<h2>Improvements</h2>
<ul>
<li><a href="https://redirect.github.com/pytest-dev/pytest/issues/11233">#11233</a>: Improvements to <code>-r</code> for xfailures and xpasses:
<ul>
<li>Report tracebacks for xfailures when <code>-rx</code> is set.</li>
<li>Report captured output for xpasses when <code>-rX</code> is set.</li>
<li>For xpasses, add <code>-</code> in summary between test name and reason, to match how xfail is displayed.</li>
</ul>
</li>
<li><a href="https://redirect.github.com/pytest-dev/pytest/issues/11825">#11825</a>: The <code>pytest_plugin_registered</code>{.interpreted-text role=&quot;hook&quot;} hook has a new <code>plugin_name</code> parameter containing the name by which <code>plugin</code> is registered.</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>
<p><a href="https://redirect.github.com/pytest-dev/pytest/issues/11706">#11706</a>: Fix reporting of teardown errors in higher-scoped fixtures when using [--maxfail]{.title-ref} or [--stepwise]{.title-ref}.</p>
</li>
<li>
<p><a href="https://redirect.github.com/pytest-dev/pytest/issues/11758">#11758</a>: Fixed <code>IndexError: string index out of range</code> crash in <code>if highlighted[-1] == &quot;\n&quot; and source[-1] != &quot;\n&quot;</code>.
This bug was introduced in pytest 8.0.0rc1.</p>
</li>
<li>
<p><a href="https://redirect.github.com/pytest-dev/pytest/issues/9765">#9765</a>, <a href="https://redirect.github.com/pytest-dev/pytest/issues/11816">#11816</a>: Fixed a frustrating bug that afflicted some users with the only error being <code>assert mod not in mods</code>. The issue was caused by the fact that <code>str(Path(mod))</code> and <code>mod.__file__</code> don't necessarily produce the same string, and was being erroneously used interchangably in some places in the code.</p>
<p>This fix also broke the internal API of <code>PytestPluginManager.consider_conftest</code> by introducing a new parameter -- we mention this in case it is being used by external code, even if marked as <em>private</em>.</p>
</li>
</ul>
<h2>pytest 8.0.0rc1 (2023-12-30)</h2>
<p>See <a href="https://docs.pytest.org/en/latest/changelog.html#pytest-8-0-0rc1-2023-12-30">https://docs.pytest.org/en/latest/changelog.html#pytest-8-0-0rc1-2023-12-30</a> for the rendered changelog.</p>
<h2>Breaking Changes</h2>
<h3>Old Deprecations Are Now Errors</h3>
<ul>
<li>
<p><a href="https://redirect.github.com/pytest-dev/pytest/issues/7363">#7363</a>: <strong>PytestRemovedIn8Warning deprecation warnings are now errors by default.</strong></p>
<p>Following our plan to remove deprecated features with as little disruption as possible, all warnings of type <code>PytestRemovedIn8Warning</code> now generate errors instead of warning messages by default.</p>
<p><strong>The affected features will be effectively removed in pytest 8.1</strong>, so please consult the <code>deprecations</code>{.interpreted-text role=&quot;ref&quot;} section in the docs for directions on how to update existing code.</p>
<p>In the pytest <code>8.0.X</code> series, it is possible to change the errors back into warnings as a stopgap measure by adding this to your <code>pytest.ini</code> file:</p>
<pre lang="ini"><code>[pytest]
</code></pre>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pytest-dev/pytest/commit/478f8233bca8147445f0c5129f04ada892cc6c91"><code>478f823</code></a> Prepare release version 8.0.0</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/608590097a6542768099dd371b84d8b37a1990da"><code>6085900</code></a> [8.0.x] fix: avoid rounding microsecond to <code>1_000_000</code> (<a href="https://redirect.github.com/pytest-dev/pytest/issues/11863">#11863</a>)</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/3b41c65c81d649d962be5ec469f44104b8d09748"><code>3b41c65</code></a> [8.0.x] Escape skip reason in junitxml (<a href="https://redirect.github.com/pytest-dev/pytest/issues/11845">#11845</a>)</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/747072ad26f2443dc8a62eb88db8cbf56fa95470"><code>747072a</code></a> [8.0.x] Update docstring of scripts/generate-gh-release-notes.py (<a href="https://redirect.github.com/pytest-dev/pytest/issues/11768">#11768</a>)</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/011a475baf6e1d0e9ec30c5996d9cbcbe7c95475"><code>011a475</code></a> Properly attach packages to the GH release notes (<a href="https://redirect.github.com/pytest-dev/pytest/issues/11839">#11839</a>) (<a href="https://redirect.github.com/pytest-dev/pytest/issues/11840">#11840</a>)</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/97960bdd148972b2f26bd9b336163e590bbc4c6b"><code>97960bd</code></a> Merge pull request <a href="https://redirect.github.com/pytest-dev/pytest/issues/11835">#11835</a> from pytest-dev/release-8.0.0rc2</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/6be0a3cbf7e014834610139421a0d9804d4a3eae"><code>6be0a3c</code></a> Prepare release version 8.0.0rc2</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/44ffe071658f5ac609fe8d3b967e8dba93abc819"><code>44ffe07</code></a> Merge pull request <a href="https://redirect.github.com/pytest-dev/pytest/issues/11837">#11837</a> from pytest-dev/backport-11836-to-8.0.x</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/14ecb049732bed4dc913e2da55c616882432c978"><code>14ecb04</code></a> [8.0.x] testing: temporarily disable test due to hypothesis issue</li>
<li><a href="https://github.com/pytest-dev/pytest/commit/41c8dabee3c40a5d363bf03a3ca2370ee27cbcd0"><code>41c8dab</code></a> Merge pull request <a href="https://redirect.github.com/pytest-dev/pytest/issues/11831">#11831</a> from bluetech/backport-11825-to-8.0.x</li>
<li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest/compare/7.4.4...8.0.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest&package-manager=pip&previous-version=7.4.4&new-version=8.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/glotzerlab/signac/pull/974?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=glotzerlab) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base [(`667a47d`)](https://app.codecov.io/gh/glotzerlab/signac/commit/667a47dd0855fed142b965b33ffffadd8f050b56?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=glotzerlab) 85.65% compared to head [(`51ea6a3`)](https://app.codecov.io/gh/glotzerlab/signac/pull/974?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=glotzerlab) 76.70%.


<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main     #974      +/-   ##
==========================================
- Coverage   85.65%   76.70%   -8.95%     
==========================================
  Files          20       20              
  Lines        3464     3464              
  Branches      759      731      -28     
==========================================
- Hits         2967     2657     -310     
- Misses        338      655     +317     
+ Partials      159      152       -7     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/glotzerlab/signac/pull/974?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=glotzerlab).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=glotzerlab).


Patch:
diff --git a/changelog.txt b/changelog.txt
index 801b6a3b..671ccdd9 100644
--- a/changelog.txt
+++ b/changelog.txt
@@ -22,6 +22,11 @@ Changed
  - linked views now can contain spaces and other characters except directory separators (#926).
  - linked views now can be created on Windows, if 'Developer mode' is enabled (#430).
 
+Fixed
++++++
+
+ - Fixed parsing of ``$not`` query expressions on command line (#970).
+
 [2.1.0] -- 2023-07-12
 ---------------------
 
diff --git a/signac/filterparse.py b/signac/filterparse.py
index 86540a40..c495ced6 100644
--- a/signac/filterparse.py
+++ b/signac/filterparse.py
@@ -195,6 +195,7 @@ def parse_filter_arg(args):
 
 def _add_prefix(filter):
     """Add prefix "sp." to a (possibly nested) filter."""
+    # Logical operators ($and, $or, $not) should not be prefixed, but their values should.
     for key, value in filter.items():
         if key in ("$and", "$or"):
             if isinstance(value, list) or isinstance(value, tuple):
@@ -203,6 +204,8 @@ def _add_prefix(filter):
                 raise ValueError(
                     "The argument to a logical operator must be a list or a tuple!"
                 )
+        elif key == "$not":
+            yield key, dict(_add_prefix(value))
         elif "." in key and key.split(".", 1)[0] in ("sp", "doc"):
             yield key, value
         elif key in ("sp", "doc"):


Test patch:
diff --git a/requirements/requirements-test.txt b/requirements/requirements-test.txt
index 26dec239..6476582e 100644
--- a/requirements/requirements-test.txt
+++ b/requirements/requirements-test.txt
@@ -1,4 +1,4 @@
 coverage==7.4.1
-pytest==7.4.4
+pytest==8.0.0
 pytest-cov==4.1.0
 pytest-xdist==3.5.0
diff --git a/tests/conftest.py b/tests/conftest.py
index fac004d0..bda36c69 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -6,3 +6,34 @@ import pytest
 @pytest.fixture
 def testdata():
     return str(uuid.uuid4())
+
+
+@pytest.fixture
+def find_filter():
+    return [
+        {"a": 0},
+        {"a.b": 0},
+        {"a.b": {"$lt": 42}},
+        {"a.b.$lt": 42},
+        {"$or": [{"a.b": 41}, {"a.b.$lt": 42}]},
+        {"$or": [{"a.b": 42}, {"a.b.$lt": 42}]},
+        {"$and": [{"a.b": 42}, {"a.b.$lt": 42}]},
+        {"$and": [{"a.b": 0}, {"a.b.$lt": 42}]},
+        {"$and": [{"a.b.$gte": 0}, {"a.b.$lt": 42}]},
+        {"$not": {"a.b": 0}},
+        {"$and": [{"a.b.$gte": 0}, {"$not": {"a.b.$lt": 42}}]},
+        {"$not": {"$not": {"a.b": 0}}},
+        {"a.b": {"$in": [0, 1]}},
+        {"a.b": {"$nin": [0, 1]}},
+        {"$not": {"a.b": {"$in": [0, 1]}}},
+        {"a.b": {"$exists": True}},
+        {"a.b": {"$exists": False}},
+        {"a": {"$exists": True}},
+        {"a": {"$exists": False}},
+        {"c": {"$regex": r"^\d$"}},
+        {"c": {"$type": "str"}},
+        {"d": {"$type": "list"}},
+        {"a.b": {"$where": "lambda x: x < 10"}},
+        {"a.b": {"$where": "lambda x: isinstance(x, int)"}},
+        {"a": {"$regex": "[a][b][c]"}},
+    ]
diff --git a/tests/test_find_command_line_interface.py b/tests/test_find_command_line_interface.py
index e28fd884..5513d377 100644
--- a/tests/test_find_command_line_interface.py
+++ b/tests/test_find_command_line_interface.py
@@ -11,35 +11,6 @@ import pytest
 
 from signac.filterparse import parse_filter_arg, parse_simple
 
-FILTERS = [
-    {"a": 0},
-    {"a.b": 0},
-    {"a.b": {"$lt": 42}},
-    {"a.b.$lt": 42},
-    {"$or": [{"a.b": 41}, {"a.b.$lt": 42}]},
-    {"$or": [{"a.b": 42}, {"a.b.$lt": 42}]},
-    {"$and": [{"a.b": 42}, {"a.b.$lt": 42}]},
-    {"$and": [{"a.b": 0}, {"a.b.$lt": 42}]},
-    {"$and": [{"a.b.$gte": 0}, {"a.b.$lt": 42}]},
-    {"$not": {"a.b": 0}},
-    {"$and": [{"a.b.$gte": 0}, {"$not": {"a.b.$lt": 42}}]},
-    {"$not": {"$not": {"a.b": 0}}},
-    {"a.b": {"$in": [0, 1]}},
-    {"a.b": {"$nin": [0, 1]}},
-    {"$not": {"a.b": {"$in": [0, 1]}}},
-    {"a.b": {"$exists": True}},
-    {"a.b": {"$exists": False}},
-    {"a": {"$exists": True}},
-    {"a": {"$exists": False}},
-    {"c": {"$regex": r"^\d$"}},
-    {"c": {"$type": "str"}},
-    {"d": {"$type": "list"}},
-    {"a.b": {"$where": "lambda x: x < 10"}},
-    {"a.b": {"$where": "lambda x: isinstance(x, int)"}},
-    {"a": {"$regex": "[a][b][c]"}},
-]
-
-
 VALUES = {"1": 1, "1.0": 1.0, "abc": "abc", "true": True, "false": False, "null": None}
 
 ARITHMETIC_EXPRESSIONS = [
@@ -68,20 +39,24 @@ class TestFindCommandLineInterface:
         with redirect_stderr(StringIO()):
             return parse_filter_arg(args)
 
-    def test_interpret_json(self):
+    @pytest.mark.usefixtures("find_filter")
+    def test_interpret_json(self, find_filter):
         def _assert_equal(q):
+            # TODO: full code path not tested with this test.
+            # _assert_equal and _find_expression, are not tested
             assert q == self._parse([json.dumps(q)])
 
-        for f in FILTERS:
+        for f in find_filter:
             _assert_equal(f)
 
-    def test_interpret_simple(self):
+    @pytest.mark.usefixtures("find_filter")
+    def test_interpret_simple(self, find_filter):
         assert self._parse(["a"]) == {"a": {"$exists": True}}
         assert next(parse_simple(["a"])) == ("a", {"$exists": True})
 
         for s, v in VALUES.items():
             assert self._parse(["a", s]) == {"a": v}
-        for f in FILTERS:
+        for f in find_filter:
             f_ = f.copy()
             key, value = f.popitem()
             if key.startswith("$"):
diff --git a/tests/test_shell.py b/tests/test_shell.py
index 695fdada..172fb1e4 100644
--- a/tests/test_shell.py
+++ b/tests/test_shell.py
@@ -217,7 +217,8 @@ class TestBasicShell:
         )
         assert "duplicate paths" in err
 
-    def test_find(self):
+    @pytest.mark.usefixtures("find_filter")
+    def test_find(self, find_filter):
         self.call("python -m signac init".split())
         project = signac.Project()
         sps = [{"a": i} for i in range(3)]
@@ -280,6 +281,12 @@ class TestBasicShell:
                 == [job.id for job in project.find_jobs({"doc.b": i})][0]
             )
 
+        # ensure that there are no errors due to adding sp and doc prefixes
+        # by testing on all the example complex expressions
+        for f in find_filter:
+            command = "python -m signac find ".split() + [json.dumps(f)]
+            self.call(command).strip()
+
     def test_diff(self):
         self.call("python -m signac init".split())
         project = signac.Project()