Revert "build: Don't work around cortex-a8 issues when targeting a9 only"

This reverts commit cab2ca159e9db59fa24a49433de1e41183b7ec0f.

Let's see if this fixes something...