Revert "build: Relax Cortex CFLAGS"

This reverts commit 58dc899c94a4d501a992ad432975285b7c154464

we already have what this does. it also reverts a commit from linaro.