Repository: scipy/scipy

Issue:
More accurate roots/weights for Gauss-Hermite quadrature (Trac #1466)
_Original ticket http://projects.scipy.org/scipy/ticket/1466 on 2011-06-28 by trac user Bogdan, assigned to @pv._

System: OSX 10.6.7, Python 2.6.1 (the one provided with system), Numpy 2.0.0.dev-3071eab and SciPy 0.10.0dev (installed via SciPy Superpack)

I am using scipy.special.orthogonal.hermite and .h_roots to perform decomposition into basis of harmonic oscillator eigenfunctions (which is essentially G-H quadrature). I noticed that decomposition is not behaving well when the number of modes is big (>40 in my case). After some testing I narrowed it down to inaccurate polynomial roots provided by h_roots(). 

As far as I understand from the code, scipy implementation uses Golub-Welsch algorithm, which is general and fast because it employs linear algebra. But in our imperfect world with finite precision numbers it starts to lose stability with large matrix sizes. So I took recursive root finding algorithm from Numerical Recipes (3rd ed., section 4.6.1) and it showed much better accuracy (see attached script with comparison). 

I understand that this algorithm is less general (because it uses good guesses for first roots of Hermite polynomials) and definitely slower than G-W, but there are applications when its accuracy is necessary. I think it should be present in SciPy, at least as an alternative to the current method.



Discussion:
Attachment added by trac user Bogdan on 2011-06-28: [roots_test.py](http://projects.scipy.org/scipy/attachment/ticket/1466/roots_test.py)

_@charris wrote on 2011-06-28_

You are at about at the machine precision limit and the roots aren't symmetrical, but a single pass of Newton cleans things up nicely:

```
In [51]: from numpy.polynomial import Hermite as H

In [52]: p = H([0]*40 + [1])

In [53]: r = p.roots()

In [54]: r1 = r - p(r)/(p.deriv()(r))

In [55]: r1
Out[55]: 
array([-8.0987611392508505 , -7.41158253148546908, -6.84023730524935569,
       -6.32825535122008187, -5.85409505603039992, -5.40665424797012761,
       -4.97926097854525551, -4.56750207284439469, -4.16825706683250008,
       -3.77920675343522339, -3.3985582658596285 , -3.02487988390128448,
       -2.65699599844289569, -2.29391714187508322, -1.93479147228229587,
       -1.578869894931614  , -1.22548010904628901, -0.8740066123570881 ,
       -0.52387471383227713, -0.1745372145975824 ,  0.1745372145975824 ,
        0.52387471383227713,  0.8740066123570881 ,  1.22548010904628901,
        1.578869894931614  ,  1.93479147228229564,  2.29391714187508322,
        2.65699599844289569,  3.02487988390128448,  3.3985582658596285 ,
        3.77920675343522339,  4.16825706683250008,  4.56750207284439469,
        4.97926097854525551,  5.40665424797012761,  5.85409505603039992,
        6.32825535122008187,  6.84023730524935569,  7.41158253148546908,
        8.0987611392508505 ])









Recursive roots:
[-8.0987611392508505  -7.41158253148546908 -6.84023730524935569
 -6.32825535122008187 -5.85409505603039992 -5.40665424797012761
 -4.97926097854525551 -4.56750207284439469 -4.16825706683250008
 -3.77920675343522339 -3.3985582658596285  -3.02487988390128448
 -2.65699599844289569 -2.29391714187508367 -1.93479147228229564
 -1.578869894931614   -1.22548010904628901 -0.87400661235708799
 -0.52387471383227724 -0.1745372145975824   0.1745372145975824
  0.52387471383227724  0.87400661235708799  1.22548010904628901
  1.578869894931614    1.93479147228229564  2.29391714187508367
  2.65699599844289569  3.02487988390128448  3.3985582658596285
  3.77920675343522339  4.16825706683250008  4.56750207284439469
  4.97926097854525551  5.40665424797012761  5.85409505603039992
  6.32825535122008187  6.84023730524935569  7.41158253148546908
  8.0987611392508505 ]
```

_trac user Bogdan wrote on 2011-06-28_

Yes, this trick certainly does improve the accuracy. Although recursive algorithm is still slightly better; comparing it with this function:

```
def h_roots_numpy(n):
    p = H([0]*n + [1])
    r = p.roots()
    r1 = r - p(r)/(p.deriv()(r))
    return r1, (2 ** (n-1)) * factorial(n) * numpy.sqrt(numpy.pi) / (p.deriv()(r1) ** 2) * 4.0
```

for N=77, M=70 I get

```
Numpy roots: 0.823443522596
Recursive roots: 1.0
```

And with increasing N h_roots_numpy() just goes haywire.

_@charris wrote on 2011-07-04_

I've played with this a bit more.

1) Numpy's Hermite zero finder dies up around deg 70. I need to symmetrize the companion matrix

2) Scipy needs to use eigh instead of eig, but this doesn't help much. The eigenvectors are also off.

3) Using zeros from Scipy, improving them with Newton, and using the w = a/(H_{n}'(r)*H_{n-1}), where a is chosen so that the w sum up to sqrt(pi), works well.

Note that the routine form NR also uses Newton. I've attached a plot with the weights from the three (eig, eigh, improved), where the improved behavior for degree 100 is easy to see.

Attachment added by @charris on 2011-07-04: [gauss-jacobi-hermite.png](http://projects.scipy.org/scipy/attachment/ticket/1466/gauss-jacobi-hermite.png)

_@josef-pkt wrote on 2011-07-04_

Since I was also looking at this recently, and trying to figure out how to use it, I also found http://www.mathworks.com/matlabcentral/fileexchange/26737-legendre-laguerre-and-hermite-gauss-quadrature/content/GaussHermite.m

it uses symmetrized companion matrix (BSD license)

I don't know if that's better, since I still don't understand half of it.

_@charris wrote on 2011-07-04_

@joseph

The symmetrized matrix comes from the original paper, it is obtained by a similarity transform of the usual companion matrix.

The linked routine at mathworks is the same as the scipy version, unless matlab eig detects symmetrical matrices, in which case it would be the eigh scipy version.


Resolving patch:
diff --git a/scipy/special/orthogonal.py b/scipy/special/orthogonal.py
--- a/scipy/special/orthogonal.py
+++ b/scipy/special/orthogonal.py
@@ -87,6 +87,7 @@
 import numpy as np
 from numpy import all, any, exp, inf, pi, sqrt
 from numpy.dual import eig
+from scipy import linalg
 
 # Local imports.
 from . import _ufuncs as cephes
@@ -382,25 +383,47 @@ def laguerre(n, monic=0):
 
 
 # Hermite  1                         H_n(x)
-def h_roots(n, mu=0):
-    """[x,w] = h_roots(n)
-
-    Returns the roots (x) of the nth order Hermite polynomial,
-    H_n(x), and weights (w) to use in Gaussian Quadrature over
-    [-inf,inf] with weighting function exp(-x**2).
+def _h_gen_roots_and_weights(n, mu, factor, func):
+    """Compute the roots and weights for Gaussian-Hermite quadrature.
+    Internal function.
     """
     if n < 1:
         raise ValueError("n must be positive.")
+    
+    bn = np.sqrt(np.arange(1,n, dtype=np.float64)/factor)
+    c = np.diag(bn, -1)
+    x = linalg.eigvalsh(c, overwrite_a=True)
+
+    # improve roots by one application of Newton's method
+    dy = func(n, x)
+    df = factor*n*func(n-1, x)
+    x -= dy/df
+
+    df /= df.max()
+    w = 1 / (df * df)
+
+    # symmetrize
+    w = (w + w[::-1])/2
+    x = (x - x[::-1])/2
+
+    # scale w correctly
+    w *= np.sqrt(2.0*np.pi/factor) / w.sum()
 
-    sbn_H = lambda k: sqrt(k/2)  # from recurrence relation
-    an_H = lambda k: 0*k
-    mu0 = sqrt(pi)               # integral of weight over interval
-    val = gen_roots_and_weights(n,an_H,sbn_H,mu0)
     if mu:
-        return val + [mu0]
+        return [x, w, mu]
     else:
-        return val
+        return x, w
+ 
+
+def h_roots(n, mu=0):
+    """[x,w] = h_roots(n)
 
+    Returns the roots (x) of the nth order Hermite polynomial,
+    H_n(x), and weights (w) to use in Gaussian Quadrature over
+    [-inf,inf] with weighting function exp(-x**2).
+    """
+    return _h_gen_roots_and_weights(n, mu, 2.0, cephes.eval_hermite)
+   
 
 def hermite(n, monic=0):
     """Return the nth order Hermite polynomial, H_n(x), orthogonal over
@@ -433,17 +456,7 @@ def he_roots(n, mu=0):
     He_n(x), and weights (w) to use in Gaussian Quadrature over
     [-inf,inf] with weighting function exp(-(x/2)**2).
     """
-    if n < 1:
-        raise ValueError("n must be positive.")
-
-    sbn_He = lambda k: sqrt(k)   # from recurrence relation
-    an_He = lambda k: 0*k
-    mu0 = sqrt(2*pi)             # integral of weight over interval
-    val = gen_roots_and_weights(n,an_He,sbn_He,mu0)
-    if mu:
-        return val + [mu0]
-    else:
-        return val
+    return _h_gen_roots_and_weights(n, mu, 1.0, cephes.eval_hermitenorm)
 
 
 def hermitenorm(n, monic=0):


Test patch:
diff --git a/scipy/special/tests/test_orthogonal.py b/scipy/special/tests/test_orthogonal.py
--- a/scipy/special/tests/test_orthogonal.py
+++ b/scipy/special/tests/test_orthogonal.py
@@ -6,7 +6,7 @@
 import numpy as np
 from numpy import array, sqrt
 import scipy.special.orthogonal as orth
-from scipy.special import gamma
+from scipy.special import gamma, eval_hermite
 
 
 class TestCheby(TestCase):
@@ -134,6 +134,22 @@ def test_hermitenorm(self):
         assert_array_almost_equal(H4.c,he4.c,13)
         assert_array_almost_equal(H5.c,he5.c,13)
 
+    def test_h_roots(self):
+        # this test is copied from numpy's TestGauss in test_hermite.py
+        x, w = orth.h_roots(100)
+
+        n = np.arange(100)
+        v = eval_hermite(n[:, np.newaxis], x[np.newaxis,:])
+        vv = np.dot(v*w, v.T)
+        vd = 1 / np.sqrt(vv.diagonal())
+        vv = vd[:, np.newaxis] * vv * vd
+        assert_almost_equal(vv, np.eye(100))
+
+        # check that the integral of 1 is correct
+        assert_almost_equal(w.sum(), np.sqrt(np.pi))
+
+
+
 
 class _test_sh_legendre(TestCase):
 
diff --git a/scipy/special/tests/test_orthogonal_eval.py b/scipy/special/tests/test_orthogonal_eval.py
--- a/scipy/special/tests/test_orthogonal_eval.py
+++ b/scipy/special/tests/test_orthogonal_eval.py
@@ -1,7 +1,7 @@
 from __future__ import division, print_function, absolute_import
 
 import numpy as np
-from numpy.testing import assert_
+from numpy.testing import assert_, assert_allclose
 import scipy.special.orthogonal as orth
 
 from scipy.special._testutils import FuncData
@@ -235,3 +235,10 @@ def test_genlaguerre(self):
     def test_laguerre(self):
         self.check_poly(orth.eval_laguerre,
                    param_ranges=[], x_range=[0, 100])
+
+    def test_hermite(self):
+        v =  orth.eval_hermite(70, 1.0)
+        a = -1.457076485701412e60 
+        assert_allclose(v,a)
+
+