server: fix revoking of players in 2 player 4 color mode