Luminary now keeps a hashmap of all colors/pips and deletes according to
color rather than by fetching actual pip