Return early if we can't evalf the summand.