Repository: microsoft/pybryt

Issue:
Invariants page blank
https://microsoft.github.io/pybryt/html/annotations/invariants.html

Assuming "TODO", but it is blank :-)
Additional invariants
Currently, the invariant structure is set up but there is only one proof-of-concept invariant, `string_capitalization`. More invariants need to be added.

An (incomplete) list:

- [x] numeric type
- [x] matrix transpose
- [x] list permutation

Discussion:

Patch:
diff --git a/docs/annotations/invariants.rst b/docs/annotations/invariants.rst
index 65fde8f..6a69ed6 100644
--- a/docs/annotations/invariants.rst
+++ b/docs/annotations/invariants.rst
@@ -3,4 +3,34 @@
 Invariants
 ==========
 
-This page is currently under construction.
+Invariants provide logic for determining whether a value satisfies a value annotation. They can be
+used to ensure that annotations can still be satisfied independent of the different ways students
+format their answers.
+
+
+Invariant Structure
+-------------------
+
+Invariants are subclasses of the abstract base class 
+:py:class:`invariants.invariant<pybryt.annotations.invariants.invariant>`. All subclasses implement
+the static :py:meth:`run<pybryt.annotations.invariants.invariant.run>` method, which takes in a list of
+objects and returns a transformed list of objects that contains all versions of every object in the
+input list which are considered equal.
+
+Consider an invariant for string capitalization. The ``run`` method of this invariant takes in a list 
+of objects and returns the same list but with every string lowercased. For a matrix transposition
+invariant, the ``run`` method might return a list with every 2D array's transpose included, as well
+as the original array.
+
+Invariants have a custom ``__new__`` method that calls the ``run`` method, so that they function as
+callables rather than classes.
+
+PyBryt supports custom invariants. To create your own invariant, subclass 
+:py:class:`invariants.invariant<pybryt.annotations.invariants.invariant>` and implement the ``run``
+method.
+
+
+Built-In Invariants
+-------------------
+
+A list of built-in invariants can be found :ref:`here<invariants_ref>`.
diff --git a/docs/annotations/value_annotations.rst b/docs/annotations/value_annotations.rst
index 0c98d96..3c418f4 100644
--- a/docs/annotations/value_annotations.rst
+++ b/docs/annotations/value_annotations.rst
@@ -87,7 +87,7 @@ types should be considered equal. PyBryt supports defining these conditions
 using :ref:`invariants<invariants>`. To use invariants on a value, we need to
 pass the invariant objects as a list to the ``invariants`` argument of the
 :py:class:`Value<pybryt.Value>` constructor. For instance, let's say we want to
-allow the student's solution (string) to be case-insensitive.
+allow the student's solution (a string) to be case-insensitive.
 
 .. code-block:: python
 
diff --git a/docs/api_reference.rst b/docs/api_reference.rst
index e8aedec..ee58aa6 100644
--- a/docs/api_reference.rst
+++ b/docs/api_reference.rst
@@ -20,18 +20,20 @@ Value Annotations
    :undoc-members:
 
 
-Relational Annotations
-++++++++++++++++++++++
+.. _invariants_ref:
 
-.. automodule:: pybryt.annotations.relation
+Invariants
+??????????
+
+.. automodule:: pybryt.annotations.invariants
    :members:
    :undoc-members:
 
 
-Invariants
-??????????
+Relational Annotations
+++++++++++++++++++++++
 
-.. automodule:: pybryt.annotations.invariants
+.. automodule:: pybryt.annotations.relation
    :members:
    :undoc-members:
 
diff --git a/pybryt/annotations/invariants.py b/pybryt/annotations/invariants.py
index 1503215..f5f3a01 100644
--- a/pybryt/annotations/invariants.py
+++ b/pybryt/annotations/invariants.py
@@ -62,7 +62,7 @@ class string_capitalization(invariant):
                 of the annotation or from the results of other invariants
         
         Returns:
-            ``list[object]``: the elements of ``values`` with all strings lowercased
+            ``list[object]``: the transformed values
         """
         ret = []
         for v in values:
@@ -71,3 +71,66 @@ class string_capitalization(invariant):
             else:
                 ret.append(v.lower())
         return ret
+
+
+class matrix_transpose(invariant):
+    """
+    An invariant that compares 2-dimensional arrays ignoring transposition.
+    """
+
+    @staticmethod
+    def run(values: List[Any]) -> List[Any]:
+        """
+        Returns a list of values in which all 2D iterables have been converted to NumPy arrays and
+        have had their transpose added
+
+        Args:
+            values (``list[object]``): acceptable values, either from the initial constructor call
+                of the annotation or from the results of other invariants
+        
+        Returns:
+            ``list[object]``: the transformed values
+        """
+        ret = []
+        for v in values:
+            if isinstance(v, np.ndarray):
+                ret.append(v)
+                ret.append(v.T)
+            elif isinstance(v, Iterable) and not isinstance(v, str):
+                try:
+                    arr = np.array(v)
+                    ret.append(arr)
+                    ret.append(arr.T)
+                except:
+                    ret.append(v)
+            else:
+                ret.append(v)
+        return ret
+
+
+class list_permutation(invariant):
+    """
+    An invariant that compares iterables (except strings) ignoring ordering, using ``sorted``.
+    """
+
+    @staticmethod
+    def run(values: List[Any]) -> List[Any]:
+        """
+        Returns a list of values in which all iterables have been sorted.
+
+        Args:
+            values (``list[object]``): acceptable values, either from the initial constructor call
+                of the annotation or from the results of other invariants
+        
+        Returns:
+            ``list[object]``: the transformed values
+        """
+        ret = []
+        for v in values:
+            if isinstance(v, np.ndarray):
+                ret.append(np.sort(v))
+            elif isinstance(v, Iterable) and not isinstance(v, str):
+                ret.append(sorted(v))
+            else:
+                ret.append(v)
+        return ret
diff --git a/pybryt/execution/tracing.py b/pybryt/execution/tracing.py
index c64b4dd..7aab9f4 100644
--- a/pybryt/execution/tracing.py
+++ b/pybryt/execution/tracing.py
@@ -216,6 +216,7 @@ def tracing_on(frame=None, tracing_func=None):
     vn2 = f"sys_{make_secret()}"
     frame.f_globals[vn] = tracing_func
     exec(f"import sys as {vn2}\n{vn2}.settrace({vn})", frame.f_globals, frame.f_locals)
+    frame.f_trace = tracing_func
 
 
 class no_tracing:


Test patch:
diff --git a/tests/annotations/test_invariants.py b/tests/annotations/test_invariants.py
index 34752a6..cc7d344 100644
--- a/tests/annotations/test_invariants.py
+++ b/tests/annotations/test_invariants.py
@@ -22,6 +22,25 @@ def test_string_capitalization():
     np.testing.assert_equal(string_capitalization(values), expected_values)
 
 
+def test_matrix_transpose():
+    np.random.seed(42)
+    arr = np.random.uniform(size=(10,10))
+    values = ["someString", "some_otherO!I(I*$NDdnnfkf", 120484, ["hi", "there"], arr, [[1, 2, 3], [4, 5, 6]]]
+    print(matrix_transpose(values))
+    expected_values = ["someString", "some_otherO!I(I*$NDdnnfkf", 120484, ["hi", "there"], ["hi", "there"], \
+        arr, arr.T, [[1, 2, 3], [4, 5, 6]], [[1, 4], [2, 5], [3, 6]]]
+    np.testing.assert_equal(matrix_transpose(values), expected_values)
+
+
+def test_list_permutation():
+    np.random.seed(42)
+    arr = np.random.uniform(size=(10,10))
+    values = ["someString", "some_otherO!I(I*$NDdnnfkf", 120484, ["there", "hi", "z"], arr, [[1, 2, 3], [4, 5, 6]]]
+    expected_values = ["someString", "some_otherO!I(I*$NDdnnfkf", 120484, ["hi", "there", "z"], np.sort(arr),  \
+        [[1, 2, 3], [4, 5, 6]]]
+    np.testing.assert_equal(list_permutation(values), expected_values)
+
+
 def test_alias():
     import pybryt.invariants as inv2
     assert inv2.string_capitalization is string_capitalization