TST: fix checking for less_precise in floats

really close #6982