Do not remove player in world change. Fixes BUKKIT-4561 and BUKKIT-4082