reduce precision in rmsd test