Talk:Wilkinson's polynomial

message: I made some additional changes to the article to try to match up the effect to what is being said. I don't have access to texts, so I am at best guessing what it is that Wilkinson tried to say, and I probably guessed wrong. The logic behind the reasoning still sounds wooly and off-kilter. (differences of large numbers are inherent sources of round-off errors; this is not news, not even in 1963. The article on ill-conditioned talks about matrices, but what the matrix is in this problem remains unclear.) linas 05:12, 26 July 2006 (UTC)
replies: text: Hi Linas. The infinite set of real numbers, providing an unlimited number of significant digits and unlimited range, were on the computer implemented by a finite set of numbers, providing unavoidable roundoff errors and an occasional arithmetic overflow. Still the approach was so amazingly successfull that people got surprised by its limitations. Wilkinson got frustrated when his program failed. He should rather be amazed that it sometimes works. Bo Jacoby 07:22, 26 July 2006 (UTC)
replies: text: I'm not sure whether he got frustrated or interested, but Wilkinson is one of the founding fathers of the field, and I doubt he would need any lessons from us on the dangers of floating-point arithmetic. -- Jitse Niesen (talk) 11:10, 26 July 2006 (UTC)

text: Hi Jitse. The words: "the most traumatic experience", indicate that mr Wilkinson got frustrated, if not traumatized, by discovering the dangers of floating-point arithmetic. It was not clear at his time that the precision needed for computation far exceeds the precision of measurement. The butterfly effect had not yet been discovered. Of course I don't express disrespect towards mr Wilkinson. Bo Jacoby 14:04, 26 July 2006 (UTC)
thread_title:  please review my changes 

message: I wanted to make some small changes, which turned out to be rather big. When saving, I got an edit conflict. Having no time at the moment, I just overwrote KSmrq edits and reverted myself immediately. Anyway, my text needs at least some links and a good look over. I'll have a look when I've time, but feel free to use anything that I wrote (that's why I saved it). -- Jitse Niesen (talk) 14:09, 26 July 2006 (UTC)
replies: text: The  tag is your freind. linas 00:38, 27 July 2006 (UTC)
thread_title:  My rewrite 

message: I'd like to stress a point from the numerical analysis perspective that may not be obvious from the pure mathematics perspective. In a practical setting, data is presented in some form which may not be under our control. In particular, we are often asked to find the roots of a polynomial presented in power-basis form. Much as we might prefer or recommend a different basis if possible, such as a Bernstein-Bézier basis, we must write software that works with what we are given. A glib response might be to first convert from an inconvenient basis to a preferred one, but that conversion itself is subject to instability!
So in this article it matters a great deal that we always respect the presentation of the data. To speak of "the polynomial" as if it were an abstract mathematical object we can freely represent or perturb as we wish, is to completely miss the point of the numerical analyst's dilemma!
I highly recommend that pure mathematicians who are tempted to edit articles concerning numerical analysis topics read George Forsythe's  classical caution, Pitfalls in computation, or why a math book isn't enough (PDF), which appeared in AMM 77, 931–956. As it happens, the Wilkinson polynomial example is mentioned.
One basic example that may come as a shock is that floating-point arithmetic is not associative. Thus simply evaluating a polynomial, never mind finding its roots, can be treacherous. --KSmrqT 14:20, 26 July 2006 (UTC)
thread_title:  Cleanup comments 

message: Note that the Wilkinson problem is (x−1 ! 20) = 0, where (n ! k) is the binomial coefficient. Your coefficients above can all be divided by 1017. They do not have to be big.  
The problem of ill condition of Wilkinson's polynomial is solved by using sufficient accuracy, so that is not a problem any more. Are you saying that it can be solved without using sufficient accuracy? What is the point of that attempt? Why not just use sufficient accuracy and stop talking nonsense?
I'm not surprised that computing the intersection of nearly parallel lines needs high accuracy. Some people may be surprised, and it is nice to explain the fact to these people so that they can update their intuition, rather than the fact. You seem to hope that the sensitivity can be got rid of by choosing a suitable basis. Well, it can, but not without cheating. Choose the system of coordinates with origo at the point of intersection and with basis vectors along the lines, then the ill conditioned problem is transformed into a well conditioned problem which is readily solved: the point of intersection between the two coordinate axes is (0,0). But one cannot do it without knowing the solution to the problem, so it is cheating. Using knowledge of the roots (1,2,3,..,20) of the Wilkinson polynomial to find a better basis, is also cheating. 
You write that the sensitivity is extraordinary. Well, it is not. It is perfectly normal for polynomials of high degree to assume huge values which require high accuracy in order to compute the roots. 
You and I agree that the Horner scheme should be used, but you reverted my edit showing the Horner scheme back into a misleading expression that is not the Horner scheme. 
Three conjectures are by Wilkinson's example shown to be false:
#The roots rk depend approximately linearly on the coefficients cj. (This is false)
#The condition numbers drk/dcj are small. (This is false)
#32 significant bits of accuracy is sufficient for practical purposes. (This is false)
You adress the second point. Some people get surprised and enlightened. That is great. You do not seem to understand the first point, however, as you still use the concept of a condition number, which is useful only for very small dcj. In Wilkinsons's example, −210>c19>−210−2−23, the roots are not at all approximately linear functions of the coefficient. They are not even differentiable, as pairs of roots fusion into double roots which immediately disintegrate into single roots moving apart with infinite speed in directions perpendicular to the original motion. This behavior also occurs with low degree equations such as x 2+t = 0 for −1<t<+1. The whole point of your writing seems to be that you still trust the third conjecture, despite the facts. Bo Jacoby 09:55, 7 August 2006 (UTC)
thread_title:  Answer to KSmrq 

message: :I think that Wilkinson's polynomial by itself does not show that the statement "32 significant bits of accuracy is sufficient for practical purposes" is false, because solving Wilkinson's polynomial is not a practical but a theoretical problem. It does show that if you have a practical problem, it is probably not a good idea to reduce the problem to solving a high-degree polynomial in the monomial basis, even though this might theoretically be possible. -- Jitse Niesen (talk) 01:13, 8 August 2006 (UTC)
thread_title: can you solve an equation without solving the equation?

message: Wow! What a difference attention has made. It reminds me of the old movie cliché where the homely librarian removes her glasses and lets down her hair to reveal a hidden beauty. Though I suppose for most of the world a mathematics article will always be homely. :-) --KSmrqT 17:59, 30 July 2006 (UTC)
replies: text: I'd second that! :)  Featured article, anyone?  Lunch 15:27, 2 August 2006 (UTC)
thread_title:  Thanks 

message: For anyone who's interested:
:...
:The history of mathematics has, in a sense, contrived to make mathematicians feel 'at home' with polynomials.  This is well illustrated by Weierstrass' theorem.  Why should one be interested in the fact that continuous functions can be satisfactorily approximated by polynomials?  The assumption is implicit in the theorem that this is a desirable thing to do since polynomials are such agreeable functions.
:2. NUMERICAL EVALUATION OF POLYNOMIALS
:The cosy relationship that mathematicians enjoyed with polynomials suffered a severe setback in the early fifties when electronic computers came into general use.  Speaking for myself I regard it as the most traumatic experience in my career as a numerical analyst.
:The early electronic computers had no hardware facilities for computation in floating-point arithmetic and operation in this mode was effected by means of subroutines.  Having constructed such a set of subroutines for our first electronic computer, PILOT ACE, I looked for a simple problem on which to give them a field trial.  Computing the largest real root of a real polynomial equation seemed tailor-made for the exercise.  Moreover since polynomials of even quite a modest degree can assume values covering a very wide range over the interval including their zeros, the use of floating-point arithmetic is particularly appropriate.My (Lunch's) note: floating point is appropriate as compared to fixed point.  I was well aware that there were practical difficulties associated with the problem but imagined that they were mainly confined to equations with multiple roots or pathologically close roots.  I felt that it was not the time to concern myself with these difficulties and I decided, therefore, to make two programs with quite limited objectives.
:The first took n prescribed real numbers z_1 > z_2 > \cdots > z_n and constructed the explicit polynomial \prod (x-z_i) by multiplying the factors together.  This in itself provided a simple test of the basic subroutines.
:The second took the resulting polynomial f(x) and, starting with the initial approximation z_1+1, used the Newton Raphson iterative method
::x_{r+1}=x_r-f(x_r)/f'(x_r)
:to locate the largest root.
:It was my intention to avoid difficulties by choosing well separated zi.  An attractive feature of this problem is that the background theory is trivial.  With exact computation the approximations tend monotonically to the root z1 and convergence is ultimately quadratic.
:After using these programs with complete success on examples of order three, I decided to give them their first genuine test.  I took n=20 and the polynomial with roots 20, 19, ..., 2, 1 and submitted it to the Newton Raphson program expecting immediate success.  To my disappointment the iteration continued indefinitely and no root emerged.  Since the polynomial was clearly devoid of all guile, I assumed that there was an error either in the floating-point subroutines or the two polynomial programs.
:Failing to find an error in the subroutines and verifying that the computed polynomial was correct to working accuracy, I returned to the zero finder and examined the successive iterates xr and the computed values f(xr).  The xr appeared to be virtually random numbers in the neighbourhood of 20 while the computed values were of arbitrary sign and were in any case much larger than the true values.  (Notice that the true values of f(xr) could be computed directly via \prod (x_r-z_i) since we were in the privileged position of knowing the roots a priori.Me again: it seems that Wilkinson is implying that he expanded the polynomial to compute (its value and) its derivative in his coding of the Newton-Raphson method, and that therein lay the bugaboo.)
:Now the interesting thing about this behaviour is that it would not have been difficult to predict, but it was so unexpected that it was some time before I could bring myself to attempt the analyis; I was convinced that either one of the programs or the computer was at fault.
:3. ELEMENTARY ERROR ANALYSIS
:We wish to avoid any detailed rounding error analysis in this article.Me again: read that sentence twice.  Fortunately a very simplified analysis is adequate to demonstrate that the observed behaviour is to be expected.  Let us concentrate on the evaluation of the polynomial
::f(z)=z^n+a_{n-1}z^{n-1}+\cdots +a_1 z+a_0
:at z=x.  This was acheived via the algorithm
::sn=1
::sp=xsp+1+ap  (p=n-1, ..., 1,0),
::f(x)=s0,
:usually known as nested multiplication. ...
Lunch 23:18, 30 July 2006 (UTC)
thread_title:  Context of "traumatic" quote 

message: I have removed two major alterations by Bo Jacoby, for two different reasons.
# The carefully formatted, easy-to-read display of the power-series form of the polynomial was changed to a wiki format. The only dubious benefit visible in the change was an explicit application of Horner's rule for evaluation. Not only is that a distraction here, the reformatting made the polynomial and its coefficients a huge run-on mess. Sorry, but this was a really bad idea.
# A new section was inserted before the stability analysis discussing round-off error. This is unsatisfactory for several reasons. First, the heart of the example — as stated by Wilkinson himself — is not roundoff, and to interpose this in the logical flow before a discussion of the essential "stability analysis" is not appropriate. Discussions on the talk page include comments from three different editors contesting your desire to treat this as about round-off. Since Jitse Niesen happens to be trained in numerical analysis (as he states on his user page), perhaps you should respect his dissent.
There is no question that floating point issues are important in polynomial root-finding, as I think we all agree. Where we disagree is that Wilkinson's example is about round-off. We have his own words saying it is not. If you don't respect us, at least respect the published source. --KSmrqT 16:04, 1 August 2006 (UTC)
replies: text: I don't know about the PILOT ACE, but Wilkinson's 1963 book says that in the floating-point subroutines on the ACE, the mantissa is stored in one 48-bit word, and that another word is used for the exponent. This makes the computations faster. -- Jitse Niesen (talk) 02:59, 2 August 2006 (UTC)
replies: text: Huh.  OK.  Forget what I said about the mantissa.  :)  Lunch 06:52, 2 August 2006 (UTC)
replies: text: Actually, I think I still have a couple of questions.  Was the sign bit stored among the 48?  And was the leading "1" assumed (or stored)?  I suppose if it's "yes" to both, then there's still 48 bits of precision.  (And a PS: I noticed on rereading what I first wrote, I used "you" to mean collective "you" and not anyone in particular so I changed it to "someone".  Sorry if I caused any offense.)  Lunch 15:23, 2 August 2006 (UTC)
replies: text: The answer is "yes" to both: The sign is stored with the mantissa, and the leading "1" is hidden. (I'm doing a bit of reading between the lines here as Wilkinson is not very explicit on the floating-point format being used). -- Jitse Niesen (talk) 03:02, 3 August 2006 (UTC)

text: Are you reading the interview published in Byte magazine (Feb. '85)?  Also, I noticed that R.e.b. added a comment to the article explaining why Wilkinson chose 2^-23 to perturb the coefficient of x^19: this was one ULP for 210.0 with a 30-bit binary mantissa. -- Lunch 05:16, 3 August 2006 (UTC)
replies: text: My source is the 1963 book mentioned in the references. I don't know anything about the interview. I also don't know where the 30-bit mantissa is coming from. I read in Pilot ACE that that machine has 32-bit words, while the ACE has 48-bit words. -- Jitse Niesen (talk) 05:52, 3 August 2006 (UTC)

text: If the coefficient of x19 is decreased from −210 by 2−23 to −210.0000001192, then the polynomial value w(20) is decreased from 0 to −2−232019 = −6.25×1017. 
Bo Jacoby 09:59, 2 August 2006 (UTC)
thread_title:  reversion 

message: An important application is that of finding eigenvalues, rj , of an n×n matrix A. If each matrix element is represented by k bits, then A is represented by kn2 bits. The eigenvalues are the roots of the characteristic polynomial : f(x) = determinant(Ex−A) = Σ cjxj , where E is the unit matrix. 
(Note that if A is a diagonal matrix, then f(x) = Πj=1..n (x − rj ) )
.  So the polynomial is represented by the n coefficients (cj)j=0..n−1 and cn = 1. If each coefficient is represented by k bits, then only kn bits represents the essentials of the matrix, which used to be represented by kn2 bits. Information is thrown away. In order to preserve the information, each coefficient must be represented by kn bits. As n-double precision is not generally available, inferior methods of finding eigenvalues are used in practice, see Eigenvalue#Numerical_computations. This point is very important. 
Bo Jacoby 09:59, 2 August 2006 (UTC)
replies: text: In practice, methods like the QR algorithm are used to calculate the eigenvalues of a matrix. Are you saying that calculating the eigenvalues by finding the zeros of its characteristic polynomial is a better method? That would be an interesting opinion for which I'd like to see some evidence. -- Jitse Niesen (talk) 03:02, 3 August 2006 (UTC)
replies: text:  Coincidentally, and perhaps ironically, the very same Wilkinson is the author of one of the most important texts in numerical analysis, The Algebraic Eigenvalue Problem (), which taught us all the benefit of the QR approach, that we use with preliminary reduction to upper Hessenberg (or tridiagonal) form, implicit shifts, and the Gershgorin circle theorem convergence test. It reliably finds eigenvalues and eigenvectors with remarkably few iterations per value. Except for tiny matrices or perhaps special cases, forming the characteristic polynomial and computing its roots is unappealing, for a variety of reasons. --KSmrqT 04:58, 3 August 2006 (UTC)
replies: text:  Of course, how could I have forgotten that. I think the "which taught us all" betrays your age wealth of experience ;) -- Jitse Niesen (talk) 05:52, 3 August 2006 (UTC)
thread_title:  Eigenvalues 

message: I have a small nit to pick, and I was wondering if I could get the opinion of others whether it's OK to make a few small changes in wording.  My problem is with the way the word "ill-conditioned" is used.
To be super pedantic (bear with me here), a problem can be ill-conditioned but data for the problem is not.  Some data will bear the brunt of the ill-conditioning, and approximate solutions to the problem for the data might not be near the true solution.  (However, some other data might produce approximate solutions that are just fine.)
For example, in the problem of solving the linear system Au=f for u given A and f, we say that the problem is ill-conditioned if the matrix A has a large condition number (say the ratio of largest to smallest singular values of A is much bigger than one).  For a given matrix A with big condition number, applying an approximate solution procedure (such as Gaussian elimination with floating-point arithmetic) to most right-hand-side data f will give a poor approximate u.  However, not all data f will give a poor u.  For instance, with f=0, Gaussian elimination will give the exact correct answer u=0 (no matter the condition number of A).
Of course, most people get sick and tired of saying over and over again that "the problem of solving Au=f with Gaussian elimination in floating-point arithmetic is ill-conditioned (when the ratio of largest to smallest singular values of A is large)".  They'll just say "A is ill-conditioned".  This is fine when it's understood what's implied, but I think the whole shebang needs to be said at least once.
I'll make some parallels with the article.  The problem of finding the roots of a polynomial given its coefficients is an ill-conditioned problem.  Some polynomials vividly demonstrate this ill-conditioning (such as Wilkinson's polynomial).  Other polynomials (like x+5 or Wilkinson's second example with the geometrically spaced roots) do not (like f=0 for the linear system above).  Of course, people get lazy and say that Wilkinson's polynomial is ill-conditioned.  But I think this sort of phrasing was at least partly responsible for some of the misunderstanding in the discussion above.
In the article, the first mention of ill-conditioning is mentioned in the "Background" section.  I might suggest rewording the paragraph:
:Like all problems in numerical analysis, it is natural to ask whether it's well-conditioned. In this context, well-conditioned means that a small change in the coefficients ci leads to a correspondingly small change in the roots of the polynomial.
to something like:
:It is a natural question in numerical analysis to ask whether the problem of finding the roots of p from the coefficients ci is well-conditioned.  That is, we hope that a small change in the coefficients will lead to a small change in the roots.  Unfortunately, that is not the case here.
I'd also change the first sentence in the next paragraph, "Polynomials with multiple roots are ill-conditioned.  For instance, ..." to something like, "Polynomials with multiple roots show the effect of this ill-conditioning.  For instance, ...".
I understand that it's much more succinct to say "Wilkinson's polynomial is ill-conditioned" so I don't intend on beating the article to death with my nitpicking.  Just a few changes here and there.  (And I'm gonna get that comma splice; it's just been razzing me. ;)
Comments or suggestions?  Thanks for listening to my rant.  :)  Lunch 01:54, 3 August 2006 (UTC)
replies: text: I agree. -- Jitse Niesen (talk) 03:02, 3 August 2006 (UTC)
thread_title:  A change in wording? 

message: The article
*J. H. Wilkinson (1984). The perfidious polynomial. Studies in Numerical Analysis, ed. by G. H. Golub, pp. 1–28. (Studies in Mathematics, vol. 24). Washington, D.C.: Mathematical Association of America,
states on page 6 :
*"In floating-point arithmetic on the PILOT ACE the mantissa had 30 binary digits, ...."
and on page 12 states:
*"The true a_19 is 210, which requires 8 binary digits for its representation; on a 30-binary digit computer the perturbation 2^-23 therefore represents a unit in the first position beyond the end of the computer representation of a_19."
R.e.b. 16:21, 3 August 2006 (UTC)
replies: text: The formulation, "2−23 is an error of 1 in the first digit that the computer cannot represent", stinks.
No, actually, it is correct.  Lunch 21:09, 4 August 2006 (UTC)

text: The formulation, "2−23 is an error of 1 in the first binary digit position that the computer does not represent", is better. The computer can represent any binary digit, both 0 and 1.  Bo Jacoby 06:59, 7 August 2006 (UTC)
thread_title:  Precision of Pilot ACE 

message: Wilkinson is a better author than any of us, and so I included his own explanation in the background subsection of the article. However, it was immediately removed. What is left is not clear and not true. For example: "It is a natural question in numerical analysis to ask whether the problem of finding the roots of p from the coefficients ci is well-conditioned". No sir, that is not natural question. Wilkinson explains that he expected the software to work, and it did not, and he did not begin to perform the analysis until he had convinced himself that there was no error in the software. He did not ask such a "natural question". The statement is simply not true, nor is it clear to the reader why it should be the case. Bo Jacoby 11:39, 10 August 2006 (UTC)
thread_title:  Background 

message: Hi KSmrq. I inserted, and you deleted, this sentence: "However, approximate knowledge of the roots is required in order to construct a well-conditioned basis, so this is not a path for solving algebraic equations without using high precision arithmetic". As you think it is wrong, you must argue for a low precision path to solving equations which otherwise can only be solved using high precision arithmetic. Bo Jacoby 12:30, 14 August 2006 (UTC)
replies: text: I think you're turning it around. Statements should not be included unless supported by a reference (see WP:V for the relevant policy). So, first you should explain why the statement should be inserted. -- Jitse Niesen (talk) 14:57, 14 August 2006 (UTC)

text: No, transformation has never been the point; it won't work, as has been stated here explicitly. --KSmrqT 12:53, 15 August 2006 (UTC)

text: The question has been answered over and over and over again. We're tired of repeating ourselves. The section is important and stays. --KSmrqT 13:18, 18 August 2006 (UTC)
thread_title:  well-conditioned basis 

message: Sometimes the Wilkinson polynomial is called Wilkinson monster. Actually I know the Wilkinson polynomial as "monster" as not as just a mere polynomial. --84.58.232.187 20:31, 16 July 2007 (UTC)
thread_title:  Monster? 

message: This section created to put the phrase "arithmetic progression" on the talk page.  Note that zn moves around the unit circle with constant velocity as z moves up the imaginary axis.  Granted we only care about z moving along the real axis when searching for roots of the polynomial f(z), but one very easy way of proving the fundamental theorem of arithmetic is to consider a growing circle around which z runs.  The fact that the root-producing radii form an arithmetic progression should tell us something about how f(z) is behaving in the complex plane, yes?  If you have a degree n polynomial whose roots don't form an arithmetic progression is that any less of a monster?  Maybe or maybe not, but why is the fact that the roots form an arithmetic progression being so uniformly ignored?  --76.102.51.128 (talk) 14:00, 19 September 2009 (UTC)
thread_title: Arithmetic progression

message: Wilkinson's polynomial is about the ill conditioning of finding the roots given the coefficients.  I fail to see the relevance of the Lagrange section to this question, since the Lagrange basis employed here presupposes that you know where the roots are.  Trivially, the roots are a well-conditioned (identity) function of the roots!
More generally, this article analyzes the problem in more detail and shows that the Lagrange basis leads to ill-conditioned root computations if the Lagrange interpolation nodes are far from the roots:
* Robert Corless, On a Generalized Companion Matrix Pencil for Matrix Polynomials Expressed in the Lagrange Basis, Symbolic-Numeric Computation (Springer, 2007). free at citeseer.  "We have experimentally observed that roots far away from the interpolation nodes are poorly conditioned. This can be explained analytically, as follows..."
My inclination is that the Lagrange section as it stands now, which is both unsourced and confusing, should be scrapped.
— Steven G. Johnson (talk) 16:39, 10 June 2011 (UTC)
thread_title:  What is the point of the Lagrange section?