[Client] Properly clean up world map