g-truc/glm — pull request #145 by jsb (2013-12-19 15:34:36+00:00)
Title: angle and oriendtedAngle sometimes return NaN values

Apparently, it is possible that the dot product of two normalized vectors is not within [-1, 1], due to numerical inaccuracies. Thus, the result can lie outside of the domain of `acos`, resulting in NaN return values.