tiny_psnr: switch f32 handling to floating point

Also add support for f64.
FATE: use proper comparison mode in the lavr tests