Repository: nunit/nunit

Issue #3841: Breaking change: Is.SupersetOf with ValueTuple requires IComparable in NUnit 3.13.2
Created: 2021-04-28T08:46:10Z
The following test passed in 3.13.1 and earlier, but fails in 3.13.2.
This is a simplified repro, of course. ValueTuple equality is useful for simplifying more complex comparisons.
[Test]
public void Test()
{
    var a = new S { C = 'a' };
    var b = new S { C = 'b' };
    var c = new S { C = 'c' };
    var d = new S { C = 'd' };

    var actual = new[] { (a, 1), (b, 2), (c, 3), (d, 4) };
    var expected = new[] { (b, 2), (c, 3) };
    Assert.That(actual, Is.SupersetOf(expected));
}

class S
{
    public char C;
}

Here is the stack trace:
System.InvalidOperationException: Failed to compare two elements in the array.
 ---> System.ArgumentException: At least one object must implement IComparable.
   at System.Collections.Comparer.Compare(Object a, Object b)
   at System.Collections.Generic.ObjectComparer`1.Compare(T x, T y)
   at System.ValueTuple`2.CompareTo(ValueTuple`2 other)
   at System.ValueTuple`2.System.IComparable.CompareTo(Object other)
   at System.Collections.Comparer.Compare(Object a, Object b)
   at System.Array.SorterObjectArray.InsertionSort(Int32 lo, Int32 hi)
   at System.Array.SorterObjectArray.IntroSort(Int32 lo, Int32 hi, Int32 depthLimit)
   at System.Array.SorterObjectArray.IntrospectiveSort(Int32 left, Int32 length)
   --- End of inner exception stack trace ---
   at System.Array.SorterObjectArray.IntrospectiveSort(Int32 left, Int32 length)
   at System.Array.Sort(Array keys, Array items, Int32 index, Int32 length, IComparer comparer)
   at System.Collections.ArrayList.Sort(Int32 index, Int32 count, IComparer comparer)
   at System.Collections.ArrayList.Sort()
   at NUnit.Framework.Constraints.CollectionTally..ctor(NUnitEqualityComparer comparer, IEnumerable c)
   at NUnit.Framework.Constraints.CollectionItemsEqualConstraint.Tally(IEnumerable c)
   at NUnit.Framework.Constraints.CollectionSupersetConstraint.Matches(IEnumerable actual)
   at NUnit.Framework.Constraints.CollectionSupersetConstraint.ApplyTo[TActual](TActual actual)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression)

Note that ValueTuple's CompareTo works as expected:
(a, 1).CompareTo((a, 1)) // 0

but trying to use this on the assertion does not help. This still fails, presumably because it's not using this delegate as the comparer for the sorting.
Assert.That(actual, Is.SupersetOf(expected).Using<(S, int)>((t1, t2) => t1.CompareTo(t2)));

Implementing IComparable on the inner type works, but having to implement this all over the place just to make an assertion in the unit tests work, when that assertion used to work fine until a patch change in NUnit is not very nice. The objects are already reference equal.
class S : IComparable
{
    public char C;
    public int CompareTo(object s) => C.CompareTo(((S)s).C);
}

Also note that HashSet<T>.IsSupersetOf still works as expected, so this works:
Assert.That(new HashSet<(S, int)>(actual).IsSupersetOf(new HashSet<(S, int)>(expected)));

but one of the reasons NUnit is so good is that Assert.That(actual, Is.SupersetOf(expected)) generates more useful messages when the assertion fails than "expected true but was false".
Using anonymous types rather than ValueTuples also still works as expected.

Duplicate issue #3877: Is.EquivalentTo no longer works for tuples that contain non-comparable elements
Created: 2021-06-16T11:00:50Z
The performance improvements concerning equivalence of large collections done in issue #2799 breaks checking equivalence for arrays with tuples that contain non-comparable elements.
The following test succeeds in 3.13.1 but fails in 3.13.2:
[Test]
public void Test1()
{
    var tuple1 = new Tuple<StringBuilder>(new StringBuilder());
    var tuple2 = new Tuple<StringBuilder>(new StringBuilder());
      
    var tuples = new[] { tuple1, tuple2 };
    var tuples2 = new[] { tuple1, tuple2 };
      
    Assert.That(tuples, Is.EquivalentTo(tuples2));
}
Exception:
System.InvalidOperationException : Failed to compare two elements in the array.
  ----> System.ArgumentException : At least one object must implement IComparable.
   at System.Array.SorterObjectArray.IntrospectiveSort(Int32 left, Int32 length)
   at System.Array.Sort(Array keys, Array items, Int32 index, Int32 length, IComparer comparer)
   at System.Collections.ArrayList.Sort(Int32 index, Int32 count, IComparer comparer)
   at System.Collections.ArrayList.Sort()
   at NUnit.Framework.Constraints.CollectionTally..ctor(NUnitEqualityComparer comparer, IEnumerable c)
   at NUnit.Framework.Constraints.CollectionItemsEqualConstraint.Tally(IEnumerable c)
   at NUnit.Framework.Constraints.CollectionEquivalentConstraint.Matches(IEnumerable actual)
   at NUnit.Framework.Constraints.CollectionEquivalentConstraint.ApplyTo[TActual](TActual actual)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression)
   at TestProject1.Tests.Test1() in C:\Users\patrick.widauer\RiderProjects\Testit\TestProject1\UnitTest1.cs:line 21
--ArgumentException
   at System.Collections.Comparer.Compare(Object a, Object b)
   at System.Collections.Generic.ObjectComparer`1.Compare(T x, T y)
   at System.Collections.Generic.Comparer`1.System.Collections.IComparer.Compare(Object x, Object y)
   at System.Tuple`1.System.Collections.IStructuralComparable.CompareTo(Object other, IComparer comparer)
   at System.Tuple`1.System.IComparable.CompareTo(Object obj)
   at System.Collections.Comparer.Compare(Object a, Object b)
   at System.Array.SorterObjectArray.SwapIfGreater(Int32 a, Int32 b)
   at System.Array.SorterObjectArray.IntroSort(Int32 lo, Int32 hi, Int32 depthLimit)
   at System.Array.SorterObjectArray.IntrospectiveSort(Int32 left, Int32 length)

This is a problem for both Tuple<> and ValueTuple<>. They implement IComparable but this comparison might not work at runtime. If none of the tuple's elements implements IComparable it will fail with an exception as seen above.
In #2799 arrays are sorted if the array type implements IComparable, which in cases of tuple types will fail when trying to sort the internal array.

Closing comment (2021-06-16T12:21:14Z): This have also been reported in #3841, so closing this as a duplicate.