Do not remove player in world change
Fixes BUKKIT-4561 and BUKKIT-4082 and BUKKIT-2094