[nomaster] SI-8152 Backport variance validator performance fix

    % time qbin/scalac test/files/pos/t8146-performance.scala

	real	0m2.015s
	user	0m2.892s
	sys	0m0.215s

	% time scalac-hash v2.10.3 test/files/pos/t8146-performance.scala

	real	1m13.652s
	user	1m14.245s
	sys	0m0.508s

Cherry-picks one hunk from 882f8e64.

Affects Slick, so I'm considering for 2.10.4. Review by @adriaanm