Enabled -ffast-math
Only use fast-math in exes
Fixed clang compile
Merge pull request #765 from worktycho/fastmath

Enabled -ffast-math