[Build][CMAKE] Fix SIMD has no effect (or effect to all).