fix floating point approximation error

This fixes issue #2867.