Revert "ARM: move VFP init to an earlier boot stage"

This reverts commit 1ee36cbe190e4c894cea18eee330e515fa7224f7.
Fix my derp