Copy the color pallet before modifying it

The prevention code (that would choose a random color if all colors were
already used) was falling into the same problem because we were modifying the
set instead of creating a copy of it before removing the ignored ones.

Change-Id: Id3e6c9664a1b87310ebaf7bff9ca293b281e9d00
Merge "Copy the color pallet before modifying it"