Remove the spawnPlayer packet.

I asked on freenode's minecraft development channel and Thinkofdeath has
informed me to send that packet to other users, not the player it
belongs to. It seems to work now.