Add the failing test case and a fix

cc @leeping. This is the same test as #474, but just put in a different file and without a needing an extra file in the reference data. I tracked down the bug, and the problem was just from floating point roundoff, the angle is computed from the law of cosines, and the argument was like 1.000004, which is just outside the range. clipping here seems to make the most sense to me.