u-boot-linaro: partially revert soft-float patch

The patch broke hardfloat, remove the hardcodes and have OE take care of
the flags.

Signed-off-by: Koen Kooi <koen@dominion.thruhere.net>