If your ex- was allotted visitation at a certain time on a certain day and he failed to excercise that visitation because of his own issues, then he forfeits that time with the child.  If he continously fails to appear for periods of visitation, then this can be used as an argument against him for 50% parent time-- because repeatedly abandoning the child on his visitation days can be emotionally damaging-- and giving more time only increases the potential for more emotional damage.  He will still have a chance to ask for a fifty-fifty plan, but his lack of diligence will certainly negate his chances.