Revert "build: Use hard-float abi"

* Will optimize flags usage later.

This reverts commit 9bf68f8d67a5d80006d0cc8300d818d6683e6090.

Dorimanx:
Possible performance drop and stability problems.