Only provide 32-bit compile flags on OS X