Only replace if the color can be converted to HSBA