[rotorcraft] horizontal guidance fixedpoint fixes

Significantly reduce the quantization error of the horizontal guidance cmds.
This should result in nicer position hold and better trajectory tracking.
Verified in NPS sim.