one more fix to the precision calculating code