Actually color the scoreboxes