This check is probably a little more reasonable for category encoders.