Fixed rounding error angle calculation of parallel and anti-parallel vec...

This fixes angle convexity calculation for parallel and anti-parallel normals, where a rounding error occasionally caused NaN angles.
This solves issue  #1033.