[math] cleanup

headers, extern C, etc...