Remove compiler flags

It causes errors especially in clang.