Polynomial interpolation

In numerical analysis, polynomial interpolation is the interpolation of a given bivariate data set by the polynomial of lowest possible degree that passes through the points of the dataset. 

Given a set of  data points , with no two  the same, a polynomial function  is said to interpolate the data if  for each .

There is always a unique such polynomial, commonly given by two explicit formulas, the Lagrange polynomials and Newton polynomials.

Applications 
The original use of interpolation polynomials was to approximate values of important transcendental functions such as natural logarithm and trigonometric functions. Starting with a few accurately computed data points, the corresponding interpolation polynomial will approximate the function at an arbitrary nearby point. Polynomial interpolation also forms the basis for algorithms in numerical quadrature (Simpson's rule) and numerical ordinary differential equations (multigrid methods). 

In computer graphics, polynomials can be used to approximate complicated plane curves given a few specified points, for example the shapes of letters in typography. This is usually done with Bézier curves, which are a simple generalization of interpolation polynomials (having specified tangents as well as specified points).

In numerical analysis, polynomial interpolation is essential to perform sub-quadratic multiplication and squaring, such as Karatsuba multiplication and Toom–Cook multiplication, where interpolation through points on a product polynomial yields the specific product required. For example, given a = f(x) = a0x0 + a1x1 + ··· and b = g(x) = b0x0 + b1x1 + ···, the product ab is a specific value of W(x) = f(x)g(x). One may easily find points along W(x) at small values of x, and interpolation based on those points will yield the terms of W(x) and the specific product ab. As fomulated in Karatsuba multiplication, this technique is substantially faster than quadratic multiplication, even for modest-sized inputs, especially on parallel hardware.

In computer science, polynomial interpolation also leads to algorithms for secure multi party computation and secret sharing.

Interpolation theorem
For any  bivariate data points , where no two  are the same, there exists a unique polynomial  of degree at most  that interpolates these points, i.e. . 

Equivalently, for a fixed choice of interpolation nodes , polynomial interpolation defines a linear bijection  between the (n+1)-tuples of real-number values  and the vector space  of real polynomials of degree at most n:

This is a type of unisolvence theorem. The theorem is also valid over any infinite field in place of the real numbers , for example the rational or complex numbers.

First proof
Consider the Lagrange basis functions  given by:

Notice that  is a polynomial of degree , and we have  for each , while . It follows that the linear combination:

has , so  is an interpolating polynomial of degree .

To prove uniqueness, assume that there exists another interpolating polynomial  of degree at most , so that   for all . Then  is a polynomial of degree at most  which has  distinct zeros (the ). But a non-zero polynomial of degree at most  can have at most  zeros, so  must be the zero polynomial, i.e. .

Second proof
Write out the interpolation polynomial in the form

Substituting this into the interpolation equations , we get a system of linear equations in the coefficients , which reads in matrix-vector form as the following multiplication:

An interpolant  corresponds to a solution  of the above matrix equation .  The matrix X on the left is a Vandermonde matrix, whose determinant is known to be  which is non-zero since the nodes  are all distinct. This ensures that the matrix is invertible and the equation has the unique solution ; that is,  exists and is unique.

Corollary
If  is a polynomial of degree at most , then the interpolating polynomial of  at  distinct points is  itself.

Constructing the interpolation polynomial

The Vandermonde matrix in the second proof above may have large condition number, causing large errors when computing the coefficients  if the system of equations is solved using Gaussian elimination.

Several authors have therefore proposed algorithms which exploit the structure of the Vandermonde matrix to compute numerically stable solutions in O(n2) operations instead of the O(n3) required by Gaussian elimination. These methods rely on constructing first a Newton interpolation of the polynomial and then converting it to the monomial form above.

Alternatively, we may write down the polynomial immediately in terms of Lagrange polynomials:

For matrix arguments, this formula is called Sylvester's formula and the matrix-valued Lagrange polynomials are the Frobenius covariants.

Non-Vandermonde algorithms
To find the interpolation polynomial p(x) in the vector space P(n) of polynomials of degree , we may use the usual monomial basis for P(n) and invert the Vandermonde matrix by Gaussian elimination, giving a computational cost of O(n3) operations. To improve this algorithm, a more convenient basis for P(n) can simplify the calculation of the coefficients, which must then be translated back in terms of the monomial basis.

One method is to write the interpolation polynomial in the Newton form and use the method of divided differences to construct the coefficients, e.g. Neville's algorithm. The cost is O(n2) operations. Furthermore, you only need to do O(n) extra work if an extra point is added to the data set, while for the other methods, you have to redo the whole computation.

Another method is preferred when the aim is not to compute the coefficients of p(x), but only a single value p(a) at a point x = a not in the original data set. The Lagrange form computes the value p(a) with complexity O(n2).

The Bernstein form was used in a constructive proof of the Weierstrass approximation theorem by Bernstein and has gained great importance in computer graphics in the form of Bézier curves.

Interpolations as linear combinations of values 
Given a set of (position, value) data points  where no two positions  are the same, the interpolating polynomial  may be considered as a linear combination of the values , using coefficients which are polynomials in  depending on the . For example the interpolation polynomial in the Lagrange form is the linear combination

with each coefficient  given by the corresponding Lagrange basis polynomial on the given positions :

Since the coefficients depend only on the positions , not the values , we can use the same coefficients to find the interpolating polynomial for a second set of data points  at the same positions:

Furthermore, the coefficients  only depend on the relative spaces  between the positions. Thus, given a third set of data whose points are given by the new variable   (an affine transformation of , inverted by ):

we can use a transformed version of the previous coefficient polynomials: and write the interpolation polynomial as:Data points  often have equally spaced positions, which may be normalized by an affine transformation to . For example, consider the data points.The interpolation polynomial in the Lagrange form is the linear combination

For example,  and  .

The case of equally spaced points can also be treated by the method of finite differences. The first difference of a sequence of values  is the sequence  defined by . Iterating this operation gives the nth difference operation , defined explicitly by:where the coefficients form a signed version of Pascal's triangle, the triangle of binomial transform coefficients:

A polynoimial  of degree d defines a sequence of values at positive integer points, , and the  difference of this sequence is identically zero: . Thus, given values  at equally spaced points, where , we have: For example, 4 equally spaced data points  of a quadratic  obey , and solving for  gives the same interpolation equation obtained above using the Lagrange method.

Interpolation error: Lagrange remainder formula

When interpolating a given function f by a polynomial  of degree  at the nodes x0,…, xn we get the error

where  is the (n+1)st divided difference of the data points .Furthermore, there is a Lagrange remainder form of the error, for a function f which is  times continuously differentiable on a closed interval , and a polynomial  of degree at most  that interpolates f at  distinct points . For each  there exists  such that

This error bound suggests choosing the interpolation points  to minimize the product , which is achieved by the Chebyshev nodes.

Proof of Lagrange remainder 
Set the error term as , and define an auxiliary function:Thus:

But since  is a polynomial of degree at most , we have , and: 

Now, since  are roots of  and , we have , which means  has at least  roots. From Rolle's theorem,  has at least  roots, and iteratively  has at least one root  in the interval . Thus:

and:

This parallels the resoning behind the Lagrange remainder term in the Taylor theorem; in fact, the Taylor remainder is a special case of interpolation error when all interpolation nodes  are identical. Note that the error will be zero when  for any i. Thus, the maximum error will occur at some point in the interval between two successive nodes.

Equally spaced intervals
In the case of equally spaced interpolation nodes where , for  and where  the product term in the interpolation error formula can be bound as

Thus the error bound can be given as

However, this assumes that  is dominated by , i.e. . In several cases, this is not true and the error actually increases as  (see Runge's phenomenon). That question is treated in the section Convergence properties.

Lebesgue constants

We fix the interpolation nodes x0, ..., xn and an interval [a, b] containing all the interpolation nodes. The process of interpolation maps the function f to a polynomial p. This defines a mapping X from the space C([a, b]) of all continuous functions on [a, b] to itself. The map X is linear and it is a projection on the subspace  of polynomials of degree n or less.

The Lebesgue constant L is defined as the operator norm of X. One has (a special case of Lebesgue's lemma):

In other words, the interpolation polynomial is at most a factor (L + 1) worse than the best possible approximation. This suggests that we look for a set of interpolation nodes that makes L small. In particular, we have for Chebyshev nodes:

We conclude again that Chebyshev nodes are a very good choice for polynomial interpolation, as the growth in n is exponential for equidistant nodes.  However, those nodes are not optimal.

Convergence properties
It is natural to ask, for which classes of functions and for which interpolation nodes the sequence of interpolating polynomials converges to the interpolated function as ? Convergence may be understood in different ways, e.g. pointwise, uniform or in some integral norm.

The situation is rather bad for equidistant nodes, in that uniform convergence is not even guaranteed for infinitely differentiable functions. One classical example, due to Carl Runge, is the function f(x) = 1 / (1 + x2) on the interval . The interpolation error  grows without bound as . Another example is the function f(x) = |x| on the interval , for which the interpolating polynomials do not even converge pointwise except at the three points x = ±1, 0.

One might think that better convergence properties may be obtained by choosing different interpolation nodes. The following result seems to give a rather encouraging answer:

The defect of this method, however, is that interpolation nodes should be calculated anew for each new function f(x), but the algorithm is hard to be implemented numerically. Does there exist a single table of nodes for which the sequence of interpolating polynomials converge to any continuous function f(x)? The answer is unfortunately negative:

The proof essentially uses the lower bound estimation of the Lebesgue constant, which we defined above to be the operator norm of Xn (where Xn is the projection operator on Πn). Now we seek a table of nodes for which

Due to the Banach–Steinhaus theorem, this is only possible when norms of Xn are uniformly bounded, which cannot be true since we know that

For example, if equidistant points are chosen as interpolation nodes, the function from Runge's phenomenon demonstrates divergence of such interpolation. Note that this function is not only continuous but even infinitely differentiable on . For better Chebyshev nodes, however, such an example is much harder to find due to the following result:

Related concepts
Runge's phenomenon shows that for high values of , the interpolation polynomial may oscillate wildly between the data points. This problem is commonly resolved by the use of spline interpolation. Here, the interpolant is not a polynomial but a spline: a chain of several polynomials of a lower degree.

Interpolation of periodic functions by harmonic functions is accomplished by Fourier transform. This can be seen as a form of polynomial interpolation with harmonic base functions, see trigonometric interpolation and trigonometric polynomial.

Hermite interpolation problems are those where not only the values of the polynomial p at the nodes are given, but also all derivatives up to a given order. This turns out to be equivalent to a system of simultaneous polynomial congruences, and may be solved by means of the Chinese remainder theorem for polynomials. Birkhoff interpolation is a further generalization where only derivatives of some orders are prescribed, not necessarily all orders from 0 to a k.

Collocation methods for the solution of differential and integral equations are based on polynomial interpolation.

The technique of rational function modeling is a generalization that considers ratios of polynomial functions.

At last, multivariate interpolation for higher dimensions.

See also
 Newton series
 Polynomial regression

Notes

Citations

References

Further reading

External links 
 
 ALGLIB has an implementations in C++ / C#.
 GSL has a polynomial interpolation code in C
 Polynomial Interpolation demonstration.

Interpolation
Polynomials
Articles containing proofs