Remove .is_finite assumption

fixes #5393
fixes #5385
fixes #7007