use int to avoid overflows, new direction algorithm

uint16 was overflowing and causing all kinds of weird values, so
now everything uses int and it works fine and removes some type
casting

Rewrote the direction algorithm to be more dynamic. See comments
in the code.

Added a bounds check on calibration data so it doesn't cause
negative values.