Revert "build: Cleanup CPU-specific optimizations"

This reverts commit 14c9937013f35cfc50ebd043c6199cdef39282c4.