Revert "Add these compilter flags back for Linaro builds"

This reverts commit 9ed6ffe946fb9c0937c6e9d35abbb2da57d871ff.