Fox: Comedian, father of three girls, says men in women's sports was final straw for many voters