Only unpack the color that will be used when the pass is either pick or color.