Remove color swatch for last-removed tag color