Repository: pytest-dev/pytest
Pull request: Add more set assertion rewrites e.g. ``>=``
<!--
Thanks for submitting a PR, your contribution is really appreciated!

Here is a quick checklist that should be present in PRs.

- [ ] Include documentation when adding new features.
- [ ] Include new tests or update existing tests when applicable.
- [X] Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself.

If this change fixes an issue, please:

- [ ] Add text like ``closes #XYZW`` to the PR description and/or commits (where ``XYZW`` is the issue number). See the [github docs](https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) for more information.

Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:

- [ ] Create a new changelog file in the `changelog` folder, with a name like `<ISSUE NUMBER>.<TYPE>.rst`. See [changelog/README.rst](https://github.com/pytest-dev/pytest/blob/main/changelog/README.rst) for details.

  Write sentences in the **past or present tense**, examples:

  * *Improved verbose diff output with sequences.*
  * *Terminal summary statistics now use multiple colors.*

  Also make sure to end the sentence with a `.`.

- [ ] Add yourself to `AUTHORS` in alphabetical order.
-->
This resolves #10617 and adds more powerful set assertion rewrites for comparisons other than equality ``==``. Previously, only ``==`` was supported for sets. Now, the additional following operations are supported: ``!=``, ``<=``, ``>=``, ``<``, and ``>``.


Problem statement:
Add more powerful set assertion rewrites for comparisons other than equality `==` (eg: `<=`, etc.)
<!--
Thanks for suggesting a feature!

Quick check-list while suggesting features:
-->

#### What's the problem this feature will solve?
<!-- What are you trying to do, that you are unable to achieve with pytest as it currently stands? -->
Set comparisons that do not check for _pure equality_ do not have nice assertion rewriting.

Examples:

```
# Set equality assertion message is nice! 👍
>       assert {1, 2, 3} == {1, 2, 4}
E       assert {1, 2, 3} == {1, 2, 4}
E         Extra items in the left set:
E         3
E         Extra items in the right set:
E         4
E         Use -v to get more diff

# But others are not nice... 👎
# inequality
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}

# proper subset
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}

# proper superset
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}

# superset
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}

# subset
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}
```

#### Describe the solution you'd like
<!-- A clear and concise description of what you want to happen. -->
When asserting sets are unequal, such as `set_a <= set_b`, the assertion text should be more intelligent only displaying the values that fail the check.

<!-- Provide examples of real-world use cases that this would enable and how it solves the problem described above. -->
This might look like:

```
# inequality
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}
E         Sets are equal

# proper subset
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}
E         Sets are equal

# proper superset
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}
E         Sets are equal

# superset
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}
E         Extra items in the right set:
E         4
E         Use -v to get more diff

# subset
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}
E         Extra items in the left set:
E         3
E         Use -v to get more diff
```


#### Alternative Solutions
<!-- Have you tried to workaround the problem using a pytest plugin or other tools? Or a different approach to solving this issue? Please elaborate here. -->

I'm not aware of any plugins or other tools that implement this, but I have creating my own using `pytest_assertrepr_compare`:

```python
def pytest_assertrepr_compare(op, left, right):
    if isinstance(left, set) and isinstance(right, set) and op == "!=":
        return [
            f"{left} {op} {right}",  # N.B.: I know this isn't the _actual_ way to do this, but it's good enough for the example!
            "Sets are equal"
        ]
    if isinstance(left, set) and isinstance(right, set) and op == "<=":
        extra = map(str, left - right)
        return [
            f"{left} {op} {right}",
            "Extra items in left set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == "<":
        if left == right:
            return [
                f"{left} {op} {right}",
                "Sets are equal"
            ]
        extra = map(str, left - right)
        return [
            f"{left} {op} {right}",
            "Extra items in left set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == ">=":
        extra = map(str, right - left)
        return [
            f"{left} {op} {right}",
            "Extra items in right set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == ">":
        if left == right:
            return [
                f"{left} {op} {right}",
                " Sets are equal",
            ]
        extra = map(str, right - left)
        return [
            f"{left} {op} {right}",
            "Extra items in right set:",
            *extra,
        ]
```

We then get the following:

```pytest
$ pytest pytest_set_assertion.py 
======================================== test session starts =========================================
platform linux -- Python 3.9.13, pytest-7.2.0, pluggy-1.0.0
rootdir: /usr/local/google/home/dthor/dev/pytest_set_assertrepr
collected 8 items                                                                                    

pytest_set_assertion.py FFFFFFFF                                                               [100%]

============================================== FAILURES ==============================================
___________________________________________ test_equality ____________________________________________

    def test_equality():
>       assert {1, 2, 3} == {1, 2, 4, 5}
E       assert {1, 2, 3} == {1, 2, 4, 5}
E         Extra items in the left set:
E         3
E         Extra items in the right set:
E         4
E         5
E         Use -v to get more diff

pytest_set_assertion.py:2: AssertionError
__________________________________________ test_inequality ___________________________________________

    def test_inequality():
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}
E         Sets are equal

pytest_set_assertion.py:6: AssertionError
____________________________________________ test_subset _____________________________________________

    def test_subset():
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}
E         Extra items in left set:
E         3

pytest_set_assertion.py:10: AssertionError
_________________________________________ test_proper_subset _________________________________________

    def test_proper_subset():
>       assert {1, 2, 3} < {1, 2, 4}
E       assert {1, 2, 3} < {1, 2, 4}
E         Extra items in left set:
E         3

pytest_set_assertion.py:14: AssertionError
___________________________________ test_proper_subset_equal_sets ____________________________________

    def test_proper_subset_equal_sets():
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}
E         Sets are equal

pytest_set_assertion.py:18: AssertionError
___________________________________________ test_superset ____________________________________________

    def test_superset():
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}
E         Extra items in right set:
E         4

pytest_set_assertion.py:22: AssertionError
________________________________________ test_proper_superset ________________________________________

    def test_proper_superset():
>       assert {1, 2, 3} > {1, 2, 4}
E       assert {1, 2, 3} > {1, 2, 4}
E         Extra items in right set:
E         4

pytest_set_assertion.py:26: AssertionError
__________________________________ test_proper_superset_equal_sets ___________________________________

    def test_proper_superset_equal_sets():
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}
E          Sets are equal

pytest_set_assertion.py:30: AssertionError
====================================== short test summary info =======================================
FAILED pytest_set_assertion.py::test_equality - assert {1, 2, 3} == {1, 2, 4, 5}
FAILED pytest_set_assertion.py::test_inequality - assert {1, 2, 3} != {1, 2, 3}
FAILED pytest_set_assertion.py::test_subset - assert {1, 2, 3} <= {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_subset - assert {1, 2, 3} < {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_subset_equal_sets - assert {1, 2, 3} < {1, 2, 3}
FAILED pytest_set_assertion.py::test_superset - assert {1, 2, 3} >= {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_superset - assert {1, 2, 3} > {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_superset_equal_sets - assert {1, 2, 3} > {1, 2, 3}
========================================= 8 failed in 0.06s ==========================================
```

#### Additional context
<!-- Add any other context, links, etc. about the feature here. -->

+ Python 3.9.13 on Debian
+ pytest 7.2.0

Hints from issue discussion:

Commit message: cleanup

Patch to review:
diff --git a/changelog/10617.feature.rst b/changelog/10617.feature.rst
new file mode 100644
index 00000000000..9eb2cce4041
--- /dev/null
+++ b/changelog/10617.feature.rst
@@ -0,0 +1,3 @@
+This change resolves :issue:`10617`` and adds more powerful set assertion rewrites for comparisons
+other than equality ``==``. Previously, only ``==`` was supported for sets. Now, the additional
+following operations are supported: ``!=``, ``<=``, ``>=``, ``<``, and ``>``.
diff --git a/src/_pytest/assertion/util.py b/src/_pytest/assertion/util.py
index 01534797d73..057352cd387 100644
--- a/src/_pytest/assertion/util.py
+++ b/src/_pytest/assertion/util.py
@@ -193,6 +193,22 @@ def assertrepr_compare(
         elif op == "not in":
             if istext(left) and istext(right):
                 explanation = _notin_text(left, right, verbose)
+        elif op == "!=":
+            if isset(left) and isset(right):
+                explanation = ["Both sets are equal"]
+        elif op == ">=":
+            if isset(left) and isset(right):
+                explanation = _compare_gte_set(left, right, verbose)
+        elif op == "<=":
+            if isset(left) and isset(right):
+                explanation = _compare_lte_set(left, right, verbose)
+        elif op == ">":
+            if isset(left) and isset(right):
+                explanation = _compare_gt_set(left, right, verbose)
+        elif op == "<":
+            if isset(left) and isset(right):
+                explanation = _compare_lt_set(left, right, verbose)
+
     except outcomes.Exit:
         raise
     except Exception:
@@ -392,15 +408,49 @@ def _compare_eq_set(
     left: AbstractSet[Any], right: AbstractSet[Any], verbose: int = 0
 ) -> List[str]:
     explanation = []
-    diff_left = left - right
-    diff_right = right - left
-    if diff_left:
-        explanation.append("Extra items in the left set:")
-        for item in diff_left:
-            explanation.append(saferepr(item))
-    if diff_right:
-        explanation.append("Extra items in the right set:")
-        for item in diff_right:
+    explanation.extend(_set_one_sided_diff("left", left, right))
+    explanation.extend(_set_one_sided_diff("right", right, left))
+    return explanation
+
+
+def _compare_gt_set(
+    left: AbstractSet[Any], right: AbstractSet[Any], verbose: int = 0
+) -> List[str]:
+    explanation = _compare_gte_set(left, right, verbose)
+    if not explanation:
+        return ["Both sets are equal"]
+    return explanation
+
+
+def _compare_lt_set(
+    left: AbstractSet[Any], right: AbstractSet[Any], verbose: int = 0
+) -> List[str]:
+    explanation = _compare_lte_set(left, right, verbose)
+    if not explanation:
+        return ["Both sets are equal"]
+    return explanation
+
+
+def _compare_gte_set(
+    left: AbstractSet[Any], right: AbstractSet[Any], verbose: int = 0
+) -> List[str]:
+    return _set_one_sided_diff("right", right, left)
+
+
+def _compare_lte_set(
+    left: AbstractSet[Any], right: AbstractSet[Any], verbose: int = 0
+) -> List[str]:
+    return _set_one_sided_diff("left", left, right)
+
+
+def _set_one_sided_diff(
+    posn: str, set1: AbstractSet[Any], set2: AbstractSet[Any]
+) -> List[str]:
+    explanation = []
+    diff = set1 - set2
+    if diff:
+        explanation.append(f"Extra items in the {posn} set:")
+        for item in diff:
             explanation.append(saferepr(item))
     return explanation
 
diff --git a/testing/test_assertion.py b/testing/test_assertion.py
index c04c31f31b6..4bcfbe6b771 100644
--- a/testing/test_assertion.py
+++ b/testing/test_assertion.py
@@ -1345,48 +1345,98 @@ def test_reprcompare_whitespaces() -> None:
     ]
 
 
-def test_pytest_assertrepr_compare_integration(pytester: Pytester) -> None:
+@pytest.mark.parametrize("op", [">=", "<="])
+def test_operators_not_set_for_full_coverage(op, pytester: Pytester) -> None:
     pytester.makepyfile(
-        """
+        f"""
         def test_hello():
-            x = set(range(100))
-            y = x.copy()
-            y.remove(50)
-            assert x == y
+            x = ["hello x"]
+            y = ["hello y"]
+            assert x {op} y
     """
     )
+
     result = pytester.runpytest()
-    result.stdout.fnmatch_lines(
-        [
-            "*def test_hello():*",
-            "*assert x == y*",
-            "*E*Extra items*left*",
-            "*E*50*",
-            "*= 1 failed in*",
-        ]
-    )
+    result.stdout.no_fnmatch_line(f"assert x {op} y")
 
 
-def test_sequence_comparison_uses_repr(pytester: Pytester) -> None:
-    pytester.makepyfile(
+class TestSetAssertions:
+    @pytest.mark.parametrize("op", [">=", ">", "<=", "<", "=="])
+    def test_set_extra_item(self, op, pytester: Pytester) -> None:
+        pytester.makepyfile(
+            f"""
+            def test_hello():
+                x = set("hello x")
+                y = set("hello y")
+                assert x {op} y
         """
-        def test_hello():
-            x = set("hello x")
-            y = set("hello y")
-            assert x == y
-    """
-    )
-    result = pytester.runpytest()
-    result.stdout.fnmatch_lines(
-        [
-            "*def test_hello():*",
-            "*assert x == y*",
-            "*E*Extra items*left*",
-            "*E*'x'*",
-            "*E*Extra items*right*",
-            "*E*'y'*",
-        ]
-    )
+        )
+
+        result = pytester.runpytest()
+        result.stdout.fnmatch_lines(
+            [
+                "*def test_hello():*",
+                f"*assert x {op} y*",
+            ]
+        )
+        if op in [">=", ">", "=="]:
+            result.stdout.fnmatch_lines(
+                [
+                    "*E*Extra items in the right set:*",
+                    "*E*'y'",
+                ]
+            )
+        if op in ["<=", "<", "=="]:
+            result.stdout.fnmatch_lines(
+                [
+                    "*E*Extra items in the left set:*",
+                    "*E*'x'",
+                    # "*E*50*",
+                ]
+            )
+        assert True  # only reached if no error above
+
+    @pytest.mark.parametrize("op", [">", "<", "!="])
+    def test_set_proper_superset_equal(self, pytester: Pytester, op) -> None:
+        pytester.makepyfile(
+            f"""
+            def test_hello():
+                x = set([1, 2, 3])
+                y = x.copy()
+                assert x {op} y
+        """
+        )
+
+        result = pytester.runpytest()
+        result.stdout.fnmatch_lines(
+            [
+                "*def test_hello():*",
+                f"*assert x {op} y*",
+                "*E*Both sets are equal*",
+            ]
+        )
+        assert True  # only reached if no error above
+
+    def test_pytest_assertrepr_compare_integration(self, pytester: Pytester) -> None:
+        pytester.makepyfile(
+            """
+            def test_hello():
+                x = set(range(100))
+                y = x.copy()
+                y.remove(50)
+                assert x == y
+        """
+        )
+        result = pytester.runpytest()
+        result.stdout.fnmatch_lines(
+            [
+                "*def test_hello():*",
+                "*assert x == y*",
+                "*E*Extra items*left*",
+                "*E*50*",
+                "*= 1 failed in*",
+            ]
+        )
 
 
 def test_assertrepr_loaded_per_dir(pytester: Pytester) -> None:


Code review comments:
diff_hunk: @@ -1345,48 +1345,98 @@ def test_reprcompare_whitespaces() -> None:
     ]
 
 
-def test_pytest_assertrepr_compare_integration(pytester: Pytester) -> None:
+@pytest.mark.parametrize("op", [">=", "<="])
+def test_operators_not_set_for_full_coverage(op, pytester: Pytester) -> None:
     pytester.makepyfile(
-        """
+        f"""
         def test_hello():
-            x = set(range(100))
-            y = x.copy()
-            y.remove(50)
-            assert x == y
+            x = ["hello x"]
+            y = ["hello y"]
+            assert x {op} y
     """
     )
+
     result = pytester.runpytest()
-    result.stdout.fnmatch_lines(
-        [
-            "*def test_hello():*",
-            "*assert x == y*",
-            "*E*Extra items*left*",
-            "*E*50*",
-            "*= 1 failed in*",
-        ]
-    )
+    result.stdout.no_fnmatch_line(f"assert x {op} y")
 
 
-def test_sequence_comparison_uses_repr(pytester: Pytester) -> None:
-    pytester.makepyfile(
+class TestSetAssertions:
+    @pytest.mark.parametrize("op", [">=", ">", "<=", "<", "=="])
+    def test_set_extra_item(self, op, pytester: Pytester) -> None:
+        pytester.makepyfile(
+            f"""
+            def test_hello():
+                x = set("hello x")
+                y = set("hello y")
+                assert x {op} y
         """
-        def test_hello():
-            x = set("hello x")
-            y = set("hello y")
-            assert x == y
-    """
-    )
-    result = pytester.runpytest()
-    result.stdout.fnmatch_lines(
-        [
-            "*def test_hello():*",
-            "*assert x == y*",
-            "*E*Extra items*left*",
-            "*E*'x'*",
-            "*E*Extra items*right*",
-            "*E*'y'*",
-        ]
-    )
+        )
+
+        result = pytester.runpytest()
+        result.stdout.fnmatch_lines(
+            [
+                "*def test_hello():*",
+                f"*assert x {op} y*",
+            ]
+        )
+        if op in [">=", ">", "=="]:
+            result.stdout.fnmatch_lines(
+                [
+                    "*E*Extra items in the right set:*",
+                    "*E*'y'",
+                ]
+            )
+        if op in ["<=", "<", "=="]:
+            result.stdout.fnmatch_lines(
+                [
+                    "*E*Extra items in the left set:*",
+                    "*E*'x'",
+                    # "*E*50*",
path: testing/test_assertion.py
text: @user1:
```suggestion
```

diff_hunk: @@ -1345,48 +1345,98 @@ def test_reprcompare_whitespaces() -> None:
     ]
 
 
-def test_pytest_assertrepr_compare_integration(pytester: Pytester) -> None:
+@pytest.mark.parametrize("op", [">=", "<="])
+def test_operators_not_set_for_full_coverage(op, pytester: Pytester) -> None:
     pytester.makepyfile(
-        """
+        f"""
         def test_hello():
-            x = set(range(100))
-            y = x.copy()
-            y.remove(50)
-            assert x == y
+            x = ["hello x"]
+            y = ["hello y"]
+            assert x {op} y
     """
     )
+
     result = pytester.runpytest()
-    result.stdout.fnmatch_lines(
-        [
-            "*def test_hello():*",
-            "*assert x == y*",
-            "*E*Extra items*left*",
-            "*E*50*",
-            "*= 1 failed in*",
-        ]
-    )
+    result.stdout.no_fnmatch_line(f"assert x {op} y")
 
 
-def test_sequence_comparison_uses_repr(pytester: Pytester) -> None:
-    pytester.makepyfile(
+class TestSetAssertions:
+    @pytest.mark.parametrize("op", [">=", ">", "<=", "<", "=="])
+    def test_set_extra_item(self, op, pytester: Pytester) -> None:
+        pytester.makepyfile(
+            f"""
+            def test_hello():
+                x = set("hello x")
+                y = set("hello y")
+                assert x {op} y
         """
-        def test_hello():
-            x = set("hello x")
-            y = set("hello y")
-            assert x == y
-    """
-    )
-    result = pytester.runpytest()
-    result.stdout.fnmatch_lines(
-        [
-            "*def test_hello():*",
-            "*assert x == y*",
-            "*E*Extra items*left*",
-            "*E*'x'*",
-            "*E*Extra items*right*",
-            "*E*'y'*",
-        ]
-    )
+        )
+
+        result = pytester.runpytest()
+        result.stdout.fnmatch_lines(
+            [
+                "*def test_hello():*",
+                f"*assert x {op} y*",
+            ]
+        )
+        if op in [">=", ">", "=="]:
+            result.stdout.fnmatch_lines(
+                [
+                    "*E*Extra items in the right set:*",
+                    "*E*'y'",
+                ]
+            )
+        if op in ["<=", "<", "=="]:
+            result.stdout.fnmatch_lines(
+                [
+                    "*E*Extra items in the left set:*",
+                    "*E*'x'",
+                    # "*E*50*",
+                ]
+            )
+        assert True  # only reached if no error above
+
+    @pytest.mark.parametrize("op", [">", "<", "!="])
+    def test_set_proper_superset_equal(self, pytester: Pytester, op) -> None:
+        pytester.makepyfile(
+            f"""
+            def test_hello():
+                x = set([1, 2, 3])
+                y = x.copy()
+                assert x {op} y
+        """
+        )
+
+        result = pytester.runpytest()
+        result.stdout.fnmatch_lines(
+            [
+                "*def test_hello():*",
+                f"*assert x {op} y*",
+                "*E*Both sets are equal*",
+            ]
+        )
+        assert True  # only reached if no error above
path: testing/test_assertion.py
text: @user1:
This is not really needed.

```suggestion
```