fix precision handling