scala/scala — pull request #3367 by retronym (2014-01-15 08:14:33+00:00)
Title: [nomaster] SI-8146 Fix non-deterministic <:< for deeply nested types

Backported from master. This is a squashed commmit comprising:

SI-8146 Pending test, diagnosis for bug in decidability of <:<

(cherry picked from commit 8beeef339ad65f3308ece6fb0440cdb31b1ad404)

SI-8146 Test cases for typechecking decidability

Taken from "On Decidability of Nominal Subtyping with Variance"
(Pierce, Kennedy), which was implemented in 152563b.

Part of the implementation (SubTypePair) will be changed in the
following commit to fix the non-deterministic errors typechecking
heavily nested types involving aliases or annotations.

(cherry picked from commit 2e28cf7f76c3d5fd0c2df4274f1af9acb42de699)

SI-8146 Fix non-deterministic <:< for deeply nested types

In the interests of keeping subtyping decidable [1], 152563b
added some bookkeeping to `isSubType` to detect cycles.

However, this was based on a hash set containing instances of
`SubTypePair`, and that class had inconsistencies between its
`hashCode` (in terms of `Type#hashCode`) and `equals`
(in terms of `=:=`).

This inconsistency can be seen in:

scala> trait C { def apply: (Int @unchecked) }
defined trait C

scala> val intUnchecked = typeOf[C].decls.head.info.finalResultType
intUnchecked: $r.intp.global.Type = Int @unchecked

scala> val p1 = new SubTypePair(intUnchecked, intUnchecked)
p1: $r.intp.global.SubTypePair = Int @unchecked <: val p2 = new SubTypePair(intUnchecked.withoutAnnotations, intUnchecked.withoutAnnotations)
p2: $r.intp.global.SubTypePair = Int <: p1 == p2
res0: Boolean = true

scala> p1.hashCode == p2.hashCode
res1: Boolean = false

This commit switches to using `Type#==`, by way of the standard
case class equality.

The risk here is that you could find a subtyping computation that
progresses in such a manner that we don't detect the cycle. It would
need to produce an infinite stream of representations for types that
were `=:=` but not `==`. If that happened, we'd fail to terminate,
rather than judging the relationship as `false`.

[1] http://research.microsoft.com/pubs/64041/fool2007.pdf

(cherry picked from commit a09e143b7fd1c6b433386d45e9c5ae3548819b59)

Conflicts:
src/reflect/scala/reflect/internal/tpe/TypeComparers.scala
src/reflect/scala/reflect/runtime/JavaUniverseForce.scala

Review by @adriaanm