topfunky/gruff — pull request #19 by vicvega (2011-12-23 15:23:55+00:00)
Title: Fixed float maths

I fixed another case of imprecise maths with float numbers (remember https://github.com/topfunky/gruff/pull/15)
This time is in the method ```significant``` (in ```lib/gruff/base.rb```)
The original code

def significant(inc) # :nodoc:
return 1.0 if inc == 0 # Keep from going into infinite loop
factor = 1.0
while (inc < 10)
inc *= 10
factor /= 10
end
while (inc > 100)
inc /= 10
factor *= 10
end
res = inc.floor * factor
if (res.to_i.to_f == res)
res.to_i
else
res
end
end

doesn't work fine. Try it in console

ruby-1.9.2-p290 :150 > significant 1.2
=> 1.2000000000000002

If you use BigDecimal instead of Float

def significant(i) # :nodoc:
return 1.0 if i == 0 # Keep from going into infinite loop
inc = BigDecimal(i.to_s)
factor = BigDecimal('1.0')
while (inc < 10)
inc *= 10
factor /= 10
end

while (inc > 100)
inc /= 10
factor *= 10
end

res = inc.floor * factor
if (res.to_i.to_f == res)
res.to_i
else
res
end
end

Everything goes right

ruby-1.9.2-p290 :174 > r = significant 1.2
=> #
ruby-1.9.2-p290 :175 > r.to_f
=> 1.2