fix a rounding problem in SHP(TS) offset floating point math

closes #5765