Casio FX-5000F incorrect roots for quadratic? - Page 1
Forum section: electronics / general

author: corrado33
author_rank: Frequent Contributor
timestamp: March 23, 2015, 09:39:22 pm
content: Hi all,I figured with Dave's fondness of casios here, I'd be able to find an answer.I have a bunch of old casio calculators (thanks Dave...) My most recent addition is a Fx-5000F. I absolutely love it, so much more useful than newer calculators, and in a much smaller package. I especially love using the scientific formulae. That's what this question is about. I use the quadratic formula often, so I very often use the first formula on the calculator. Sure, I could solve it by hand, but using the formula on the calculator is much quicker. That's the point.Anyway, I'm trying to solve a quadratic with A=1, B=0.15, C=-7.7E-13. Now, I realize C is quite small, but I've done many other calculations on the calculator which turned out accurate, even at those orders of magnitude.When put into the first formula (Press 1, then FMLA), it gives roots of -0.15 (correct) and 4.5E-13(wrong). The actual other root is 5.1E-13. Any ideas?Please don't berate me for using a calculator to do this. This is more of a question of WHY it's wrong, not that it IS wrong. It's easy to calculate this other ways obviously.EDIT: I realize that it probably has something to do with the rounding in the calculator, but I'm not about to dig through the entire 200 page manual to find the specs on it.In fact, if I change C to = -7.7E-12, the formula actually gives the correct answer. Gahh, maybe I will dig through the manual. I've read it once, I don't remember anything about accuracy in formulae though.

author: IanB
author_rank: Super Contributor
timestamp: March 23, 2015, 09:49:16 pm
content: This is where you need to write the formula down on paper.If you do that, you will see this calculation:-0.15 + sqrt(0.15^2 - 4*1*-7.7E-13)You will notice a difference between two terms almost equal to each other, which will lead to round-off error. Work it out step by step and see what happens.This is a classic kind of problem in numerical computation.There is a way of re-ordering the calculations to avoid this problem with the quadratic formula. Google will probably find it for you.

author: corrado33
author_rank: Frequent Contributor
timestamp: March 23, 2015, 10:06:48 pm
content: Ian,I did work it out by hand and went "What the heck?" Most calculators will give you 0.15^2 when you do (0.15^2 - 4*-7.7E-13). Therefore, without another technique, using the quadratic equation with normal calculators won't give you the correct answer. I actually had to go to wolfram alpha to give me the correct answer. I believe it's supposed to be solved using completing the square.I'm just flabbergasted. I've never had a calculator give me a blatantly wrong answer. (At least that I know of.) (Yes, the calculator doesn't "give" me an answer, but in this case it's not user error. I'm using it as intended.)I THINK the reason for the rounding is because internally, the FX-5000F uses a 12 digit mantissa, and when you subtract the two numbers, the last digit (or two) is the 13th digit, and gets cut off/rounded. The calculator I was using to compare it to, a TI-83 Plus, uses a 14 digit mantissa, and therefore computes the correct answer. (However, if you use -7.7E-15, it will NOT give the right answer, because it too, rounds the last digit of the mantissa.)I hope I haven't marked any student's answers wrong because of this.Note to self: Be careful when subtracting very small numbers from relatively large numbers (especially when the difference between the two is ~12-14 orders of magnitude.)

author: corrado33
author_rank: Frequent Contributor
timestamp: March 23, 2015, 10:51:17 pm
content: I may as well spell it out now that I've (we've) figured it out.Inside of the square root for the quadratic equation is b^2-4ac.B^2 = 0.02254ac = -3.08E-12 or .00000000000308If you add (subtracting a negative = addition) B^2 and 4ac you get .02250000000308 or 2.250000000308E-2.Now if you count, that's a 13 digit mantissa (the part before the "E", not including the decimal point.... actually, I'm not sure if the decimal point is included in the "counting" for the mantissa. If it is, then it's a 14 point mantissa.)Therefore, the calculator internally either rounds or truncates it to 2.25000000031E-2 or 2.25000000030E-2 (assuming the decimal point isn't included in the mantissa counting.)/or/If the decimal point IS included in the mantissa counting it'd be 2.2500000003E-2That rounding/truncation is the reason for the error. (And the reason that the TI-83 plus gets it right, with a 14 digit mantissa. Admittedly, the TI-83 plus is a MUCH newer calculator, so I'll let this one pass TI... Apparently the TI-89 uses a 16 digit mantissa!)

author: IanB
author_rank: Super Contributor
timestamp: March 23, 2015, 11:03:44 pm
content: My venerable TI-58 (1977) had1210 display digits and 3 "guard" digits, making1513 in total. So even old calculators were capable of good accuracy (the TI-58/59 were aimed at engineers though, rather than students).As to the round-off problem, this can be solved by observing that x1*x2 = c/a. Hence we can compute:x1 = [- 0.15^2 - sqrt(0.15^2 - 4 * -7.7e-13) ] / 2 = -0.15x2 = -7.7e-13 / -0.15 = 5.13E-12This will work with even 8 digits of precision.

author: corrado33
author_rank: Frequent Contributor
timestamp: March 23, 2015, 11:16:51 pm
content: Quote from: IanB on March 23, 2015, 11:03:44 pmAh, that IS an easy way to do it. Even when trying to complete the square you still run into the same problem. Hmmm... I like these math puzzles where you have to think of different ways to do things. (Solve this problem using only X digits of precision.) Wonder if I can find a book that talks about this kind of stuff.By the way, is the decimal point included in the counting for the mantissa digits? I mean, this number has to be stored in a register somewhere on the chip. So either the decimal point position has it's own register, or they represent it some other way all in one register. (Or a few registers, since it's so long, but you catch my drift.) Geeze, you'd need 10 bits of space to store a 12 digit number. I guess that's not too many now-a-day (even tiny micros have 12 bit ADCs sometimes), but I don't know how spectacular that was back in the 80s. (Or even if it was spectacular at all!)

author: Len
author_rank: Frequent Contributor
timestamp: March 24, 2015, 12:35:36 am
content: Quote from: corrado33 on March 23, 2015, 11:16:51 pmYou could get a textbook on numerical analysis if you want tokill your enthusiasmlearn all the techniques of numeric computation.QuoteThe exponent specifies the position of the decimal point. For example, with -7.7e-13, -7.7 is the mantissa (the significant digits) and -13 is the exponent that scales the mantissa by powers of 10 (which is the same as moving the decimal point).QuoteIn the 80's microprocessors only did integer arithmetic. Floating point was done either in software or by a separate co-processor chip. Floating point numbers are usually 32, 64 or 80 bits.

author: IanB
author_rank: Super Contributor
timestamp: March 24, 2015, 01:23:38 am
content: Quote from: Len on March 24, 2015, 12:35:36 amHowever, hand-held calculators did not necessarily do binary arithmetic. It was more common to use some form of BCD for storage and computation.

author: corrado33
author_rank: Frequent Contributor
timestamp: March 24, 2015, 03:41:07 pm
content: Quote from: mojo-chan on March 24, 2015, 10:48:33 amYep, I've run across that site before. Very interesting stuff. I bet the majority of people don't realize that different calculators can give you different values for the same calculation. I mean, honestly you could use most calculations involving irrational numbers, as that would very easily display where the irrational number was rounded/truncated based on the number of digits the calculator uses. For example, two of my calculators (that I tested) give different answers for sqrt(2) - e + e - sqrt(2) (Done as three calculations. Root2-e then + e then -root2) (I get 0 for my Fx-991W and -3.7E-10 for my Fx-5000F Both give 0 when done in a single calculation) Admittedly, the algorithm used there is more "accurate" or rather shows differences to a higher degree of precision. (And theirs doesn't depend on the stored values for e.)

author: corrado33
author_rank: Frequent Contributor
timestamp: March 24, 2015, 08:19:20 pm
content: Here's the reply from casio if you're interested.Quote

author: TimFox
author_rank: Super Contributor
timestamp: March 25, 2015, 03:26:00 pm
content: When dealing with the quadratic equation where you get the small difference between two larger numbers, if is often useful to make an approximation and get most of the large values to cancel directly, leaving you with what you want.I wrote out these equations in Visio, making the usual approximation for the square root of unity plus a small number.