> xwax just isn't precise enough for that many significant figures. If I play a serato CD at 1.00 rate, xwax sees changes in rate below > .0001 -- including in xwax itself. So rounding out the noise improves the pitch stability, which has been an ongoing concern.

Can you hear these pitch changes, or is it only a display issue? 

In general rounding will put additional noise to a measured value. (http://en.wikipedia.org/wiki/Quantization_%28signal_processing%29) You will get a staircase like behavior.
Is it really an issue of xwax or an issue of the turntable/vinyl hardware? 
From signal theories a IIR filter will suit better. 
If you have good results, with it it is fine for me, but it should be well documented inside the code.