Repository: scipy/scipy

Issue:
ENH: "vectorize" newton and other scalar methods
# Problem
Commonly we have to analyze large datasets, like timeseries. If user wants to find zeros of multiple scalar problems simultaneously, they have to use `np.vectorize`, a multidimensional solver, use multiprocessing, threading, a jitter, or custom code. There are some downsides to each of these.
* `np.vectorize` scales linearly with size of inputs, so it gets very slow for large datasets
* IMO multidimensional solvers are overkill, but I should investigate this option more. perhaps using a sparse jacobian will improve performance, but I tried `hybrj` (`root`, `fsolve`) and didn't get fast results with even small datasets compared to `brentq` or `newton`
* multiprocessing, threading, numba, etc. seem like the wrong approach here
* if I customize code, then who will maintain it?

## Proposal
The newton and bisection methods can be adapted easily to handle multiple scalar problems simultaneously. I see tremendous speed up with a simple custom newton code, with no back tracking or line-searching. This might be improved more by implementing in C, using numba, or cython maybe?

A quick comparison of the proposed custom "vectorized" newton versus using `np.vectorize` to wrap the existing one, shows a **20x speedup** for solving a [one-diode solar-cell analog using Shockley's diode equation](https://en.wikipedia.org/wiki/Theory_of_solar_cells#Characteristic_equation) with 100 simultaneous orthogonal points. Of course this proposal can be improved. But I think it would be better to have it in SciPy than many individual users' custom code.

Furthmore, if we [allow the jac to be returned with func, like some other functions already do](./issues/8345), then this halves the time, and we get 40x speedup!

### Reproducing code example:
```
def newton(func, x0, fprime, args=(), tol=1.48e-08, maxiter=50, full_output=None):
    resnorm = np.inf  # nor of residuals
    for iter in range(maxiter):
        fval = func(x0, *args)
        if fprime is True:
            fval, jval = fval
        else:
            jval = fprime(x0, *args)
        resnorm = np.sqrt(np.sum(fval**2))
        if resnorm < tol:
            break
        newton_step = fval / jval
        # do some clever damping here,
        # or use Numerical Recipes in C ch. 9.6-9.7 and do line search
        x0 -= newton_step
    if full_output:
         return x, fval, resnorm, newton_step, iter
    return x0
```

### Error message:
```
~\AppData\Local\Continuum\anaconda3\lib\site-packages\scipy\optimize\zeros.py in newton(func, x0, fprime, args, tol, maxiter, fprime2)
    138             myargs = (p0,) + args
    139             fder = fprime(*myargs)
--> 140             if fder == 0:
    141                 msg = "derivative was zero."
    142                 warnings.warn(msg, RuntimeWarning)

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

### Scipy/Numpy/Python version information:
```
import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
0.19.1 1.13.3 sys.version_info(major=3, minor=6, micro=3, releaselevel='final', serial=0)
```


ENH: "vectorize" newton and other scalar methods
# Problem
Commonly we have to analyze large datasets, like timeseries. If user wants to find zeros of multiple scalar problems simultaneously, they have to use `np.vectorize`, a multidimensional solver, use multiprocessing, threading, a jitter, or custom code. There are some downsides to each of these.
* `np.vectorize` scales linearly with size of inputs, so it gets very slow for large datasets
* IMO multidimensional solvers are overkill, but I should investigate this option more. perhaps using a sparse jacobian will improve performance, but I tried `hybrj` (`root`, `fsolve`) and didn't get fast results with even small datasets compared to `brentq` or `newton`
* multiprocessing, threading, numba, etc. seem like the wrong approach here
* if I customize code, then who will maintain it?

## Proposal
The newton and bisection methods can be adapted easily to handle multiple scalar problems simultaneously. I see tremendous speed up with a simple custom newton code, with no back tracking or line-searching. This might be improved more by implementing in C, using numba, or cython maybe?

A quick comparison of the proposed custom "vectorized" newton versus using `np.vectorize` to wrap the existing one, shows a **20x speedup** for solving a [one-diode solar-cell analog using Shockley's diode equation](https://en.wikipedia.org/wiki/Theory_of_solar_cells#Characteristic_equation) with 100 simultaneous orthogonal points. Of course this proposal can be improved. But I think it would be better to have it in SciPy than many individual users' custom code.

Furthmore, if we [allow the jac to be returned with func, like some other functions already do](./issues/8345), then this halves the time, and we get 40x speedup!

### Reproducing code example:
```
def newton(func, x0, fprime, args=(), tol=1.48e-08, maxiter=50, full_output=None):
    resnorm = np.inf  # nor of residuals
    for iter in range(maxiter):
        fval = func(x0, *args)
        if fprime is True:
            fval, jval = fval
        else:
            jval = fprime(x0, *args)
        resnorm = np.sqrt(np.sum(fval**2))
        if resnorm < tol:
            break
        newton_step = fval / jval
        # do some clever damping here,
        # or use Numerical Recipes in C ch. 9.6-9.7 and do line search
        x0 -= newton_step
    if full_output:
         return x, fval, resnorm, newton_step, iter
    return x0
```

### Error message:
```
~\AppData\Local\Continuum\anaconda3\lib\site-packages\scipy\optimize\zeros.py in newton(func, x0, fprime, args, tol, maxiter, fprime2)
    138             myargs = (p0,) + args
    139             fder = fprime(*myargs)
--> 140             if fder == 0:
    141                 msg = "derivative was zero."
    142                 warnings.warn(msg, RuntimeWarning)

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

### Scipy/Numpy/Python version information:
```
import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
0.19.1 1.13.3 sys.version_info(major=3, minor=6, micro=3, releaselevel='final', serial=0)
```


BUG: if zero derivative at root, then Newton fails with RuntimeWarning
As indicated in [this discussion](../pull/8357#discussion_r192612946) by @pvanmulbregt

>... if the derivative is 0 at the root? E.g. `f(x)=x**3 - x**2`, starting at `x0=0` (immediately at the root), ... It looks like this might be considered a failure


### Reproducing code example:
```
from scipy import optimize

# use secant
r = optimize.newton(lambda x: x**3 - x**2, x0=0)
# -1.3552527156068808e-20  close enough to zero to be the correct answer

# now try Newton
r = optimize.newton(lambda x: x**3 - x**2, x0=0, fprime=lambda x: 3*x**2 - 2*x)
# raises RuntimeWarning but solution is zero

# or Halley's
r = optimize.newton(lambda x: x**3 - x**2, x0=0, fprime=lambda x: 3*x**2 - 2*x, fprime2=lambda x: 6*x - 2)
# also raises RuntimeWarning even though solution should be zero
```

### Error message:
```
~/miniconda/lib/python3.6/site-packages/scipy/optimize/zeros.py:169: RuntimeWarning: derivative was zero.
  warnings.warn(msg, RuntimeWarning)
```

### Scipy/Numpy/Python version information:
```
1.1.0 1.14.3 sys.version_info(major=3, minor=6, micro=5, releaselevel='final', serial=0)
```




Discussion:
I will submit a PR asap
This came up before: https://github.com/scipy/scipy/issues/7242.

As you can see I was against the idea. Handling all reasonable halting conditions for a scalar solver is already a PITA, and I think the problem gets much worse for a vectorized scalar solver. IMO it is better to provide a Cython API to a scalar solver and let users handle looping over it.

Just me though, maybe others are in favor. :-)
I don't follow the argument very closely I guess but np.vectorize does not vectorize anything. It's just a convenience function to avoid for loop writing annoyance you want to apply a custom function over an array.

Also bisection methods are line searches too anyways. So I don't really get that point too. Care to elaborate?

Also halting conditions must be tested over ill-conditioned problems and I agree they are a major source of pain.

IIUC this kind of problem is normally tackled by parallel processing, using something like `multiprocessing`, `mpi4py`, or `joblib`.


I will submit a PR asap
This came up before: https://github.com/scipy/scipy/issues/7242.

As you can see I was against the idea. Handling all reasonable halting conditions for a scalar solver is already a PITA, and I think the problem gets much worse for a vectorized scalar solver. IMO it is better to provide a Cython API to a scalar solver and let users handle looping over it.

Just me though, maybe others are in favor. :-)
I don't follow the argument very closely I guess but np.vectorize does not vectorize anything. It's just a convenience function to avoid for loop writing annoyance you want to apply a custom function over an array.

Also bisection methods are line searches too anyways. So I don't really get that point too. Care to elaborate?

Also halting conditions must be tested over ill-conditioned problems and I agree they are a major source of pain.

IIUC this kind of problem is normally tackled by parallel processing, using something like `multiprocessing`, `mpi4py`, or `joblib`.




Resolving patch:
diff --git a/benchmarks/benchmarks/optimize_zeros.py b/benchmarks/benchmarks/optimize_zeros.py
--- a/benchmarks/benchmarks/optimize_zeros.py
+++ b/benchmarks/benchmarks/optimize_zeros.py
@@ -1,6 +1,7 @@
 from __future__ import division, print_function, absolute_import
 
 from math import sqrt, exp, cos, sin
+import numpy as np
 
 # Import testing parameters
 try:
@@ -56,3 +57,64 @@ def setup(self, func, meth):
 
     def time_newton(self, func, meth):
         newton(self.f, self.x0, args=(), fprime=self.f_1, fprime2=self.f_2)
+
+
+class NewtonArray(Benchmark):
+    params = [['loop', 'array'], ['newton', 'secant', 'halley']]
+    param_names = ['vectorization', 'solver']
+
+    def setup(self, vec, meth):
+        if vec == 'loop':
+            if meth == 'newton':
+                self.fvec = lambda f, x0, args, fprime, fprime2: [
+                    newton(f, x, args=(a0, a1) + args[2:], fprime=fprime)
+                    for (x, a0, a1) in zip(x0, args[0], args[1])
+                ]
+            elif meth == 'halley':
+                self.fvec = lambda f, x0, args, fprime, fprime2: [
+                    newton(
+                        f, x, args=(a0, a1) + args[2:], fprime=fprime,
+                        fprime2=fprime2
+                    ) for (x, a0, a1) in zip(x0, args[0], args[1])
+                ]
+            else:
+                self.fvec = lambda f, x0, args, fprime, fprime2: [
+                    newton(f, x, args=(a0, a1) + args[2:]) for (x, a0, a1)
+                    in zip(x0, args[0], args[1])
+                ]
+        else:
+            if meth == 'newton':
+                self.fvec = lambda f, x0, args, fprime, fprime2: newton(
+                    f, x0, args=args, fprime=fprime
+                )
+            elif meth == 'halley':
+                self.fvec = newton
+            else:
+                self.fvec = lambda f, x0, args, fprime, fprime2: newton(
+                    f, x0, args=args
+                )
+
+    def time_array_newton(self, vec, meth):
+
+        def f(x, *a):
+            b = a[0] + x * a[3]
+            return a[1] - a[2] * (np.exp(b / a[5]) - 1.0) - b / a[4] - x
+
+        def f_1(x, *a):
+            b = a[3] / a[5]
+            return -a[2] * np.exp(a[0] / a[5] + x * b) * b - a[3] / a[4] - 1
+
+        def f_2(x, *a):
+            b = a[3] / a[5]
+            return -a[2] * np.exp(a[0] / a[5] + x * b) * b ** 2
+
+        a0 = np.array([
+            5.32725221, 5.48673747, 5.49539973,
+            5.36387202, 4.80237316, 1.43764452,
+            5.23063958, 5.46094772, 5.50512718,
+            5.42046290
+        ])
+        a1 = (np.sin(range(10)) + 1.0) * 7.0
+        args = (a0, a1, 1e-09, 0.004, 10, 0.27456)
+        x0 = [7.0] * 10
+        self.fvec(f, x0, args=args, fprime=f_1, fprime2=f_2)
diff --git a/scipy/optimize/zeros.py b/scipy/optimize/zeros.py
--- a/scipy/optimize/zeros.py
+++ b/scipy/optimize/zeros.py
@@ -1,13 +1,13 @@
 from __future__ import division, print_function, absolute_import
 
 import warnings
-
+from collections import namedtuple
 from . import _zeros
-from numpy import finfo
+import numpy as np
 
 _iter = 100
 _xtol = 2e-12
-_rtol = 4 * finfo(float).eps
+_rtol = 4 * np.finfo(float).eps
 
 __all__ = ['newton', 'bisect', 'ridder', 'brentq', 'brenth']
 
@@ -36,7 +36,6 @@ class RootResults(object):
     flag : str
         Description of the cause of termination.
     """
-
     def __init__(self, root, iterations, function_calls, flag):
         self.root = root
         self.iterations = iterations
@@ -79,17 +78,27 @@ def _results_select(full_output, r):
     return x
 
 
-# Newton-Raphson method
 def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
-           fprime2=None,
-           full_output=False, disp=True):
+           fprime2=None, full_output=False, disp=True, converged=None):
     """
     Find a zero using the Newton-Raphson or secant method.
 
     Find a zero of the function `func` given a nearby starting point `x0`.
     The Newton-Raphson method is used if the derivative `fprime` of `func`
     is provided, otherwise the secant method is used.  If the second order
-    derivative `fprime2` of `func` is provided, then Halley's method is used.
+    derivative `fprime2` of `func` is also provided, then Halley's method is
+    used.
+
+    If `x0` is a sequence, then `newton` returns an array, and `func` must be
+    vectorized and return a sequence or array of the same shape as it's first
+    argument. If an optional argument, `converged`, is `True`, then the
+    return is a named tuple `(root, converged, zero_der)` in which `root` is an
+    array of the locations where `func` is zero, `converged` is an array, same
+    size as `root`, of booleans that are `True` where `func` converged, and
+    `zero_der` is another boolean array of the same size where `func` had a
+    zero derivative.
+
+    If `x0` is a sequence, then arguments `full_output` and `disp` are ignored.
 
     Parameters
     ----------
@@ -97,9 +106,10 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
         The function whose zero is wanted. It must be a function of a
         single variable of the form f(x,a,b,c...), where a,b,c... are extra
         arguments that can be passed in the `args` parameter.
-    x0 : float
+    x0 : float, sequence, or array
         An initial estimate of the zero that should be somewhere near the
-        actual zero.
+        actual zero. If not scalar, then `func` must be vectorized and return
+        a sequence or array of the same shape as it's first argument.
     fprime : function, optional
         The derivative of the function when available and convenient. If it
         is None (default), then the secant method is used.
@@ -120,14 +130,21 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
         a RootResults object.
     disp : bool, optional
         If True, display a RuntimeError if the algorithm didn't converge.
+    converged : boolean, optional
+        Only used if `x0` is a sequence. If `True` then two extras boolean
+        arrays of converged and zero derivatives are appended to the return.
 
     Returns
     -------
-    zero : float
+    root : float, sequence, or array
         Estimated location where function is zero.
     r : RootResults (present if ``full_output = True``)
         Object containing information about the convergence.  In particular,
         ``r.converged`` is True if the routine converged.
+    converged : boolean array, optional
+        For vector functions, indicates which elements converged successfully.
+    zero_der : boolean array, optional
+        For vector functions, indicates which elements had a zero derivative.
 
     See Also
     --------
@@ -149,16 +166,25 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
     sign. The brentq algorithm is recommended for general use in one
     dimensional problems when such an interval has been found.
 
+    When `newton` is used with arrays, it is best suited for the following
+    types of problems:
+    * the initial guesses, `x0`, are all relatively the same distance from the
+    roots
+    * some or all of the extra arguments, `args`, are also arrays so that a
+    class of similar problems can be solved together
+    * the size of the initial guesses, `x0`, is larger than 100 elements
+    Otherwise, a naive loop may perform better than a vector.
+
     Examples
     --------
 
+    >>> from scipy import optimize
+
     >>> def f(x):
     ...     return (x**3 - 1)  # only one real root at x = 1
-    
-    >>> from scipy import optimize
 
     ``fprime`` not provided, use secant method
-    
+
     >>> root = optimize.newton(f, 1.5)
     >>> root
     1.0000000000000016
@@ -167,11 +193,11 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
     1.0000000000000016
 
     Only ``fprime`` provided, use Newton Raphson method
-    
+
     >>> root = optimize.newton(f, 1.5, fprime=lambda x: 3 * x**2)
     >>> root
     1.0
-    
+
     Both ``fprime2`` and ``fprime`` provided, use Halley's method
 
     >>> root = optimize.newton(f, 1.5, fprime=lambda x: 3 * x**2,
@@ -179,11 +205,27 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
     >>> root
     1.0
 
+    A sequence of initial guesses and arguments is provided, use vectorized
+
+    >>> import numpy as np
+
+    >>> f = lambda x, a: x**3 - a
+    >>> fder = lambda x, a: 3 * x**2
+    >>> x = np.random.randn(100)
+    >>> a = range(-50, 50)
+    >>> vec_res = optimize.newton(f, x, fprime=fder, args=(a, ))
+
+    >>> loop_res = [optimize.newton(f, x0, fprime=fder, args=(a0,))
+    ...             for x0, a0 in zip(x, a)]
+
     """
     if tol <= 0:
         raise ValueError("tol too small (%g <= 0)" % tol)
     if maxiter < 1:
         raise ValueError("maxiter must be greater than 0")
+    if not np.isscalar(x0):
+        return _array_newton(func, x0, fprime, args, tol, maxiter, fprime2,
+                             converged)
     # Multiply by 1.0 to convert to floating point.  We don't use float(x0)
     # so it still works if x0 is complex.
     p0 = 1.0 * x0
@@ -248,6 +290,102 @@ def newton(func, x0, fprime=None, args=(), tol=1.48e-8, maxiter=50,
     return _results_select(full_output, (p, funcalls, itr + 1, _ECONVERR))
 
 
+def _array_newton(func, x0, fprime, args, tol, maxiter, fprime2,
+                  converged=False):
+    """
+    A vectorized version of Newton, Halley, and secant methods for arrays. Do
+    not use this method directly. This method is called from :func:`newton`
+    when ``np.isscalar(x0)`` is true. For docstring, see :func:`newton`.
+    """
+    try:
+        p = np.asarray(x0, dtype=float)
+    except TypeError:  # can't convert complex to float
+        p = np.asarray(x0)
+    failures = np.ones_like(p, dtype=bool)  # at start, nothing converged
+    nz_der = np.copy(failures)
+    if fprime is not None:
+        # Newton-Raphson method
+        for iteration in range(maxiter):
+            # first evaluate fval
+            fval = np.asarray(func(p, *args))
+            # If all fval are 0, all roots have been found, then terminate
+            if not fval.any():
+                failures = fval.astype(bool)
+                break
+            fder = np.asarray(fprime(p, *args))
+            nz_der = (fder != 0)
+            # stop iterating if all derivatives are zero
+            if not nz_der.any():
+                break
+            # Newton step
+            dp = fval[nz_der] / fder[nz_der]
+            if fprime2 is not None:
+                fder2 = np.asarray(fprime2(p, *args))
+                dp = dp / (1.0 - 0.5 * dp * fder2[nz_der] / fder[nz_der])
+            # only update nonzero derivatives
+            p[nz_der] -= dp
+            failures[nz_der] = np.abs(dp) >= tol  # items not yet converged
+            # stop iterating if there aren't any failures, not incl zero der
+            if not failures[nz_der].any():
+                break
+    else:
+        # Secant method
+        dx = np.finfo(float).eps**0.33
+        p1 = p * (1 + dx) + np.where(p >= 0, dx, -dx)
+        q0 = np.asarray(func(p, *args))
+        q1 = np.asarray(func(p1, *args))
+        active = np.ones_like(p, dtype=bool)
+        for iteration in range(maxiter):
+            nz_der = (q1 != q0)
+            # stop iterating if all derivatives are zero
+            if not nz_der.any():
+                p = (p1 + p) / 2.0
+                break
+            # Secant Step
+            dp = (q1 * (p1 - p))[nz_der] / (q1 - q0)[nz_der]
+            # only update nonzero derivatives
+            p[nz_der] = p1[nz_der] - dp
+            active_zero_der = ~nz_der & active
+            p[active_zero_der] = (p1 + p)[active_zero_der] / 2.0
+            active &= nz_der  # don't assign zero derivatives again
+            failures[nz_der] = np.abs(dp) >= tol  # not yet converged
+            # stop iterating if there aren't any failures, not incl zero der
+            if not failures[nz_der].any():
+                break
+            p1, p = p, p1
+            q0 = q1
+            q1 = np.asarray(func(p1, *args))
+    zero_der = ~nz_der & failures  # don't include converged with zero-ders
+    if zero_der.any():
+        # secant warnings
+        if fprime is None:
+            nonzero_dp = (p1 != p)
+            # non-zero dp, but infinite newton step
+            zero_der_nz_dp = (zero_der & nonzero_dp)
+            if zero_der_nz_dp.any():
+                rms = np.sqrt(
+                    sum((p1[zero_der_nz_dp] - p[zero_der_nz_dp]) ** 2)
+                )
+                warnings.warn('RMS of {:g} reached'.format(rms), RuntimeWarning)
+        # netwon or halley warnings
+        else:
+            all_or_some = 'all' if zero_der.all() else 'some'
+            msg = '{:s} derivatives were zero'.format(all_or_some)
+            warnings.warn(msg, RuntimeWarning)
+    elif failures.any():
+        all_or_some = 'all' if failures.all() else 'some'
+        msg = '{0:s} failed to converge after {1:d} iterations'.format(
+            all_or_some, maxiter
+        )
+        if failures.all():
+            raise RuntimeError(msg)
+        warnings.warn(msg, RuntimeWarning)
+    if converged:
+        result = namedtuple('result', ('root', 'converged', 'zero_der'))
+        p = result(p, ~failures, zero_der)
+    return p
+
+
 def bisect(f, a, b, args=(),
            xtol=_xtol, rtol=_rtol, maxiter=_iter,
            full_output=False, disp=True):


Test patch:
diff --git a/scipy/optimize/tests/test_zeros.py b/scipy/optimize/tests/test_zeros.py
--- a/scipy/optimize/tests/test_zeros.py
+++ b/scipy/optimize/tests/test_zeros.py
@@ -6,7 +6,7 @@
 from numpy.testing import (assert_warns, assert_,
                            assert_allclose,
                            assert_equal)
-from numpy import finfo
+import numpy as np
 
 from scipy.optimize import zeros as cc
 from scipy.optimize import zeros
@@ -14,12 +14,14 @@
 # Import testing parameters
 from scipy.optimize._tstutils import functions, fstrings
 
+TOL = 4*np.finfo(float).eps  # tolerance
+
+
 class TestBasic(object):
     def run_check(self, method, name):
         a = .5
         b = sqrt(3)
-        xtol = 4*finfo(float).eps
-        rtol = 4*finfo(float).eps
+        xtol = rtol = TOL
         for function, fname in zip(functions, fstrings):
             zero, r = method(function, a, b, xtol=xtol, rtol=rtol,
                              full_output=True)
@@ -56,6 +58,72 @@ def test_newton(self):
             x = zeros.newton(f, 3, fprime=f_1, fprime2=f_2, tol=1e-6)
             assert_allclose(f(x), 0, atol=1e-6)
 
+    def test_array_newton(self):
+        """test newton with array"""
+
+        def f1(x, *a):
+            b = a[0] + x * a[3]
+            return a[1] - a[2] * (np.exp(b / a[5]) - 1.0) - b / a[4] - x
+
+        def f1_1(x, *a):
+            b = a[3] / a[5]
+            return -a[2] * np.exp(a[0] / a[5] + x * b) * b - a[3] / a[4] - 1
+
+        def f1_2(x, *a):
+            b = a[3] / a[5]
+            return -a[2] * np.exp(a[0] / a[5] + x * b) * b ** 2
+
+        a0 = np.array([
+            5.32725221, 5.48673747, 5.49539973,
+            5.36387202, 4.80237316, 1.43764452,
+            5.23063958, 5.46094772, 5.50512718,
+            5.42046290
+        ])
+        a1 = (np.sin(range(10)) + 1.0) * 7.0
+        args = (a0, a1, 1e-09, 0.004, 10, 0.27456)
+        x0 = [7.0] * 10
+        x = zeros.newton(f1, x0, f1_1, args)
+        x_expected = (
+            6.17264965, 11.7702805, 12.2219954,
+            7.11017681, 1.18151293, 0.143707955,
+            4.31928228, 10.5419107, 12.7552490,
+            8.91225749
+        )
+        assert_allclose(x, x_expected)
+        # test halley's
+        x = zeros.newton(f1, x0, f1_1, args, fprime2=f1_2)
+        assert_allclose(x, x_expected)
+        # test secant
+        x = zeros.newton(f1, x0, args=args)
+        assert_allclose(x, x_expected)
+
+    def test_array_secant_active_zero_der(self):
+        """test secant doesn't continue to iterate zero derivatives"""
+        x = zeros.newton(lambda x, *a: x*x - a[0], x0=[4.123, 5],
+                         args=[np.array([17, 25])])
+        assert_allclose(x, (4.123105625617661, 5.0))
+
+    def test_array_newton_integers(self):
+        # test secant with float
+        x = zeros.newton(lambda y, z: z - y ** 2, [4.0] * 2,
+                         args=([15.0, 17.0],))
+        assert_allclose(x, (3.872983346207417, 4.123105625617661))
+        # test integer becomes float
+        x = zeros.newton(lambda y, z: z - y ** 2, [4] * 2, args=([15, 17],))
+        assert_allclose(x, (3.872983346207417, 4.123105625617661))
+
+    def test_array_newton_zero_der_failures(self):
+        # test derivative zero warning
+        assert_warns(RuntimeWarning, zeros.newton,
+                     lambda y: y**2 - 2, [0., 0.], lambda y: 2 * y)
+        # test failures and zero_der
+        with pytest.warns(RuntimeWarning):
+            results = zeros.newton(lambda y: y**2 - 2, [0., 0.],
+                                   lambda y: 2*y, converged=True)
+            assert_allclose(results.root, 0)
+            assert results.zero_der.all()
+            assert not results.converged.any()
+
     def test_newton_full_output(self):
         # Test the full_output capability, both when converging and not.
         # Use simple polynomials, to avoid hitting platform dependencies
@@ -109,8 +177,7 @@ def f(x):
         return x - root
 
     methods = [cc.bisect, cc.ridder]
-    xtol = 4*finfo(float).eps
-    rtol = 4*finfo(float).eps
+    xtol = rtol = TOL
     for method in methods:
         res = method(f, -1e8, 1e7, xtol=xtol, rtol=rtol)
         assert_allclose(root, res, atol=xtol, rtol=rtol,
@@ -134,7 +201,7 @@ def f(x):
             return x - 0.6
 
     atol = 0.51
-    rtol = 4*finfo(float).eps
+    rtol = TOL
     methods = [cc.brentq, cc.brenth]
     for method in methods:
         res = method(f, 0, 1, xtol=atol, rtol=rtol)
@@ -162,13 +229,78 @@ def f_1(x, *a):
         return 2 * a[0] * x + a[1]
 
     def f_2(x, *a):
-        return 2 * a[0]
+        retval = 2 * a[0]
+        try:
+            size = len(x)
+        except TypeError:
+            return retval
+        else:
+            return [retval] * size
 
     z = complex(1.0, 2.0)
     coeffs = (2.0, 3.0, 4.0)
     y = zeros.newton(f, z, args=coeffs, fprime=f_1, fprime2=f_2, tol=1e-6)
     # (-0.75000000000000078+1.1989578808281789j)
     assert_allclose(f(y, *coeffs), 0, atol=1e-6)
+    z = [z] * 10
+    coeffs = (2.0, 3.0, 4.0)
+    y = zeros.newton(f, z, args=coeffs, fprime=f_1, fprime2=f_2, tol=1e-6)
+    assert_allclose(f(y, *coeffs), 0, atol=1e-6)
+
+
+def test_zero_der_nz_dp():
+    """Test secant method with a non-zero dp, but an infinite newton step"""
+    # pick a symmetrical functions and choose a point on the side that with dx
+    # makes a secant that is a flat line with zero slope, EG: f = (x - 100)**2,
+    # which has a root at x = 100 and is symmetrical around the line x = 100
+    # we have to pick a really big number so that it is consistently true
+    # now find a point on each side so that the secant has a zero slope
+    dx = np.finfo(float).eps ** 0.33
+    # 100 - p0 = p1 - 100 = p0 * (1 + dx) + dx - 100
+    # -> 200 = p0 * (2 + dx) + dx
+    p0 = (200.0 - dx) / (2.0 + dx)
+    x = zeros.newton(lambda y: (y - 100.0)**2, x0=[p0] * 10)
+    assert_allclose(x, [100] * 10)
+    # test scalar cases too
+    p0 = (2.0 - 1e-4) / (2.0 + 1e-4)
+    x = zeros.newton(lambda y: (y - 1.0) ** 2, x0=p0)
+    assert_allclose(x, 1)
+    p0 = (-2.0 + 1e-4) / (2.0 + 1e-4)
+    x = zeros.newton(lambda y: (y + 1.0) ** 2, x0=p0)
+    assert_allclose(x, -1)
+
+
+def test_array_newton_failures():
+    """Test that array newton fails as expected"""
+    # p = 0.68  # [MPa]
+    # dp = -0.068 * 1e6  # [Pa]
+    # T = 323  # [K]
+    diameter = 0.10  # [m]
+    # L = 100  # [m]
+    roughness = 0.00015  # [m]
+    rho = 988.1  # [kg/m**3]
+    mu = 5.4790e-04  # [Pa*s]
+    u = 2.488  # [m/s]
+    reynolds_number = rho * u * diameter / mu  # Reynolds number
+
+    def colebrook_eqn(darcy_friction, re, dia):
+        return (1 / np.sqrt(darcy_friction) +
+                2 * np.log10(roughness / 3.7 / dia +
+                             2.51 / re / np.sqrt(darcy_friction)))
+
+    # only some failures
+    with pytest.warns(RuntimeWarning):
+        result = zeros.newton(
+            colebrook_eqn, x0=[0.01, 0.2, 0.02223, 0.3], maxiter=2,
+            args=[reynolds_number, diameter], converged=True
+        )
+        assert not result.converged.all()
+    # they all fail
+    with pytest.raises(RuntimeError):
+        result = zeros.newton(
+            colebrook_eqn, x0=[0.01] * 2, maxiter=2,
+            args=[reynolds_number, diameter], converged=True
+        )
 
 
 # this test should **not** raise a RuntimeWarning
@@ -182,16 +314,29 @@ def f_zeroder_root(x):
     # should work with secant
     r = zeros.newton(f_zeroder_root, x0=0)
     assert_allclose(r, 0, atol=zeros._xtol, rtol=zeros._rtol)
+    # test again with array
+    r = zeros.newton(f_zeroder_root, x0=[0]*10)
+    assert_allclose(r, 0, atol=zeros._xtol, rtol=zeros._rtol)
 
     # 1st derivative
     def fder(x):
         return 3 * x**2 - 2 * x
 
+    # 2nd derivative
+    def fder2(x):
+        return 6*x - 2
+
     # should work with newton and halley
     r = zeros.newton(f_zeroder_root, x0=0, fprime=fder)
     assert_allclose(r, 0, atol=zeros._xtol, rtol=zeros._rtol)
     r = zeros.newton(f_zeroder_root, x0=0, fprime=fder,
-                     fprime2=lambda x: 6*x - 2)
+                     fprime2=fder2)
+    assert_allclose(r, 0, atol=zeros._xtol, rtol=zeros._rtol)
+    # test again with array
+    r = zeros.newton(f_zeroder_root, x0=[0]*10, fprime=fder)
+    assert_allclose(r, 0, atol=zeros._xtol, rtol=zeros._rtol)
+    r = zeros.newton(f_zeroder_root, x0=[0]*10, fprime=fder,
+                     fprime2=fder2)
     assert_allclose(r, 0, atol=zeros._xtol, rtol=zeros._rtol)
 
     # also test that if a root is found we do not raise RuntimeWarning even if
@@ -201,4 +346,7 @@ def fder(x):
     # a zero derivative, so it should return the root w/o RuntimeWarning
     r = zeros.newton(f_zeroder_root, x0=0.5, fprime=fder)
     assert_allclose(r, 0, atol=zeros._xtol, rtol=zeros._rtol)
+    # test again with array
+    r = zeros.newton(f_zeroder_root, x0=[0.5]*10, fprime=fder)
+    assert_allclose(r, 0, atol=zeros._xtol, rtol=zeros._rtol)
     # doesn't apply to halley