I think I found and fixed the bug responsible for the terrible references() performance - the hashCode() method was calling hashCode() on the map