Repository: scipy/scipy

Issue:
Suboptimal convergence of Halley's method?
While trying out the Halley's method added in #20, I think I have found that the current implementation does not reach cubic convergence.

(_Note: This is a WIP issue, since I admit that this example is [by no means short or self-contained](http://sscce.org/), but I am willing to provide a better example and narrow down the issue as soon as I meet a couple of deadlines_)

This is the code I'm testing:

https://github.com/Juanlu001/poliastro/blob/6483c27/src/poliastro/iod/izzo.py#L174-L210

And this is the program I'm using:

``` python
from poliastro.iod import izzo

def main(M, x_ref, ll_ref):
    T_ref = izzo._tof_equation(x_ref, 0.0, ll_ref, M)
    print(M, T_ref)
    for ii in range(1, 8):
        try:
            x, _ = next(izzo._find_xy(ll_ref, T_ref, M, numiter=ii, rtol=1e-30))
        except RuntimeError as e:
            x = float(e.args[0].split()[-1])
            print(ii, x_ref, x, (x - x_ref) / x_ref)

if __name__ == '__main__':
    main(1, -0.24, 0.5)
```

When using the available SciPy implementation, these are the results. Notice that the last column is the relative error with respect to the known solution:

```
$ python convergence.py
1 5.58044804471
1 -0.24 -0.4031436981968586 0.6797654091535775
2 -0.24 -0.1769754549412486 -0.2626022710781308
3 -0.24 -0.2404985599237051 0.0020773330154380083
4 -0.24 -0.2399999996742607 -1.3572470693630953e-09
5 -0.24 -0.2400000000000002 9.251858538542972e-16
```

And with [the formula from Wikipedia](https://en.wikipedia.org/wiki/Halley's_method#Method), these are the results:

```
$ python convergence.py
1 5.58044804471
1 -0.24 -0.265113577401703 0.10463990584042918
2 -0.24 -0.240021887866687 9.1199444529159e-05
3 -0.24 -0.240000000000016 6.672902970924119e-14
4 -0.24 -0.2400000000000002 9.251858538542972e-16
```

I guess the last iteration is subject to truncation error.



Discussion:
It's a different Halley method, https://de.wikipedia.org/wiki/Halley-Verfahren

I'm not sure if you can say much about cubic convergence based on 4 data points, but otoh I did not go through the derivation in detail. The negative discriminant handling may be ad hoc by the original submitter of gh-20

In principle both are cubic, so they should exhibit similar behavior. I don't know how to obtain more information in double precision since the values go to zero very quickly, I am open to suggestions.

You can compute the Lagrange remainder and check if it is 4th order in the Power expansion: 
http://mathworld.wolfram.com/LagrangeRemainder.html

The computation maybe needs the mean value theorem used with love but at least it give you a initial idea about if the methods are converging at 3rd order. 


Resolving patch:
diff --git a/scipy/optimize/zeros.py b/scipy/optimize/zeros.py
--- a/scipy/optimize/zeros.py
+++ b/scipy/optimize/zeros.py
@@ -71,8 +71,7 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
     Find a zero of the function `func` given a nearby starting point `x0`.
     The Newton-Raphson method is used if the derivative `fprime` of `func`
     is provided, otherwise the secant method is used.  If the second order
-    derivate `fprime2` of `func` is provided, parabolic Halley's method
-    is used.
+    derivative `fprime2` of `func` is provided, then Halley's method is used.
 
     Parameters
     ----------
@@ -95,8 +94,8 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
     fprime2 : function, optional
         The second order derivative of the function when available and
         convenient. If it is None (default), then the normal Newton-Raphson
-        or the secant method is used. If it is given, parabolic Halley's
-        method is used.
+        or the secant method is used. If it is not None, then Halley's method
+        is used.
 
     Returns
     -------
@@ -131,11 +130,14 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
     
     >>> from scipy import optimize
 
-    ``fprime`` and ``fprime2`` not provided, use secant method
+    ``fprime`` not provided, use secant method
     
     >>> root = optimize.newton(f, 1.5)
     >>> root
     1.0000000000000016
+    >>> root = optimize.newton(f, 1.5, fprime2=lambda x: 6 * x)
+    >>> root
+    1.0000000000000016
 
     Only ``fprime`` provided, use Newton Raphson method
     
@@ -143,12 +145,8 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
     >>> root
     1.0
     
-    ``fprime2`` provided, ``fprime`` provided/not provided use parabolic
-    Halley's method
+    Both ``fprime2`` and ``fprime`` provided, use Halley's method
 
-    >>> root = optimize.newton(f, 1.5, fprime2=lambda x: 6 * x)
-    >>> root
-    1.0000000000000016
     >>> root = optimize.newton(f, 1.5, fprime=lambda x: 3 * x**2,
     ...                        fprime2=lambda x: 6 * x)
     >>> root
@@ -173,18 +171,14 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
                 warnings.warn(msg, RuntimeWarning)
                 return p0
             fval = func(*myargs)
-            if fprime2 is not None:
-                fder2 = fprime2(*myargs)
-            if fder2 == 0:
+            newton_step = fval / fder
+            if fprime2 is None:
                 # Newton step
-                p = p0 - fval / fder
+                p = p0 - newton_step
             else:
-                # Parabolic Halley's method
-                discr = fder ** 2 - 2 * fval * fder2
-                if discr < 0:
-                    p = p0 - fder / fder2
-                else:
-                    p = p0 - 2*fval / (fder + sign(fder) * sqrt(discr))
+                fder2 = fprime2(*myargs)
+                # Halley's method
+                p = p0 - newton_step / (1.0 - 0.5 * newton_step * fder2 / fder)
             if abs(p - p0) < tol:
                 return p
             p0 = p


Test patch:
diff --git a/scipy/optimize/tests/test_zeros.py b/scipy/optimize/tests/test_zeros.py
--- a/scipy/optimize/tests/test_zeros.py
+++ b/scipy/optimize/tests/test_zeros.py
@@ -112,3 +112,20 @@ def test_repr(self):
                          "           root: 1.0")
         assert_equal(repr(r), expected_repr)
 
+
+def test_complex_halley():
+    """Test Halley's works with complex roots"""
+    def f(x, *a):
+        return a[0] * x**2 + a[1] * x + a[2]
+
+    def f_1(x, *a):
+        return 2 * a[0] * x + a[1]
+
+    def f_2(x, *a):
+        return 2 * a[0]
+
+    z = complex(1.0, 2.0)
+    coeffs = (2.0, 3.0, 4.0)
+    y = zeros.newton(f, z, args=coeffs, fprime=f_1, fprime2=f_2, tol=1e-6)
+    # (-0.75000000000000078+1.1989578808281789j)
+    assert_allclose(f(y, *coeffs), 0, atol=1e-6)