Move -m32 flags to top level CMakeLists.txt