Remove some un needed code, go down all the way when testing for floating point types too