Revert "Use CMAKE_BUILD_TYPE"

This reverts commit e7dc495157ec87afacd309ca369d8f734ea1fb3d.

It would ignore -flto cflags.