Do not validate the color of players which have the LockColor flag set

Fixes #6572
I expect that this will crash if the client joins a full game (spectators have `Slot == null`).