Patent ID: 7752250

Claim:
A computer program product stored in a non-transitory computer-readable medium to program a floating point processor to round floating point numbers, comprising: (a) instructions to calculate an intermediate quotient of two floating point numbers, A and B; (b) instructions to determine a unit of least precision (ULP) of the intermediate quotient; (c) instructions to calculate a first remainder using the two floating point numbers and the unit of least precision; (d) instructions to calculate a second remainder using the two floating point numbers and the unit of least precision plus one; (e) instructions to compare the signs and magnitudes of the first and second remainders; (f) instructions using the comparison of the signs and magnitudes of the first and second remainders to: when rounding mode is round-to-nearest: when a sign of the first remainder is “0” and a sign of the second remainder is “0”, selecting the intermediate quotient plus a ULP as the final intermediate quotient, setting a guard bit to “0”, and setting a sticky bit to “1”; when the sign of the first remainder is “0” and the sign of the second remainder is “1” and the first remainder is larger than the second remainder, selecting the intermediate quotient as the final intermediate quotient, setting the guard bit to “1” and setting the sticky bit to “1”; when the sign of the first remainder is “0” and the sign of the second remainder is “1” and the second remainder is larger than the first remainder, selecting the intermediate quotient as the final intermediate quotient, setting the guard bit to “0” and setting the sticky bit to “1”; and when the sign of the first remainder is “1” and the sign of the second remainder is “1” selecting the intermediate quotient minus a ULP as the final intermediate quotient, setting the guard bit to “1” and setting the sticky bit to “1”; when rounding mode is round-to-zero and when rounding mode is round to +/− infinity: when the sign of the first remainder is “0” and the sign of the second remainder is “0”, selecting the intermediate quotient plus a ULP as the final intermediate result, setting the guard bit to “0” and setting the sticky bit to “1”; when the sign of the first remainder is “0” and the sign of the second remainder is “1” selecting the intermediate quotient as the final intermediate quotient, setting the guard bit to “0”, and setting the sticky bit to “1”; when the sign of the first remainder is “1” and the sign of the second remainder is “1” selecting the intermediate quotient minus a ULP as the final intermediate quotient, setting the guard bit to “0” and setting the sticky bit to “1”; and (g) instructions to apply the final intermediate quotient and the selected correct guard and sticky bits for correct rounding of the floating point number.