Repository: statsmodels/statsmodels

Issue:
proportion_confint inaccurate for method="binom_test"
There is a symmetry for binomial proportion confidence interval therefore following property should holds (approximately):
proportion_confint(count, nobs, method="binom_test")[0] + proportion_confint(nobs-count, nobs, method="binom_test")[1] ==1.0
but for some cases this property does not hold, for example, the following code:
```python
from statsmodels.stats.proportion import proportion_confint
eps = 1e-8
for nobs in [47, 50]:
    for count in range(nobs+1):
        p1 = proportion_confint(count, nobs, method="binom_test")
        p2 = proportion_confint(nobs-count, nobs, method="binom_test")
        if abs(p1[0]+p2[1]-1.0)>eps:
            print(f"{nobs=} {count=} {p1[0]+p2[1]}")
```
gives ouput:
```
nobs=47 count=3 0.9899913901078568
nobs=47 count=6 0.9857792597568958
nobs=47 count=12 0.9864319871901812
nobs=47 count=16 0.9851553703015263
nobs=47 count=23 0.9802691065584829
nobs=47 count=24 0.980269106558583
nobs=47 count=31 0.978547082683319
nobs=47 count=35 0.9718877308702878
nobs=47 count=41 0.978042999703059
nobs=47 count=44 0.9680701802192785
nobs=50 count=21 0.9796386543796607
nobs=50 count=29 0.9798450168011505
```
Which indicate that there is some bug.

#### Output of ``import statsmodels.api as sm; sm.show_versions()``
<details>

INSTALLED VERSIONS
------------------
Python: 3.8.5.final.0

statsmodels
===========

Installed: 0.13.1 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\statsmodels)

Required Dependencies
=====================

cython: Not installed
numpy: 1.17.3 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\numpy)
scipy: 1.3.3 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\scipy)
pandas: 0.25.3 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas)
    dateutil: 2.8.1 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\dateutil)
patsy: 0.5.2 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\patsy)

Optional Dependencies
=====================

matplotlib: 3.2.0rc1 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\matplotlib)
    backend: TkAgg
cvxopt: Not installed
joblib: Not installed

Developer Tools
================

IPython: 7.17.0 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\IPython)
    jinja2: 2.11.2 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\jinja2)
sphinx: Not installed
    pygments: 2.6.1 (C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pygments)
pytest: Not installed
virtualenv: Not installed

</details>


Discussion:
Thanks for reporting

I can confirm this, but don't see any obvious problem.
It's either because of some discreteness in scipy.stats.binom_test or in scipy.optimize.brentq.
The differences in the confint are nontrivial, but inverting binom_test is not one of the recommended confints (both in the literature and in terms of the quickly done implementation).

But I don't have time to dig into this right now (It's too far away from what I'm currently working on.)

label `bug-wrong` because we should have consistent results for symmetry.

However my guess is that they choose different solution because of non-monotonicities or discreteness, which are often a problem with hypothesis tests for proportions.
Requirement for size is only `<= alpha` but not `== alpha` because of the discreteness of the sample space.
I have tested the current version of proportion_confint for method="binom_test" and it looks like the new version works better. But symmetry still does not holds for some cases, for example, the following code:
```python
from statsmodels.stats.proportion import proportion_confint
eps = 1e-8
for nobs in range(45, 60):
    for count in range(nobs+1):
        p1 = proportion_confint(count, nobs, method="binom_test")
        p2 = proportion_confint(nobs-count, nobs, method="binom_test")
        if abs(p1[0]+p2[1]-1.0) > eps:
            print(f"{nobs=} {count=} {p1[0]+p2[1]}")
```
gives output:
```
nobs=45 count=45 0.991280515088538
nobs=51 count=51 1.0076714652881384
nobs=53 count=53 1.0073761192248545
nobs=54 count=54 0.9927631623092528
nobs=55 count=49 0.995421075648545
nobs=55 count=51 1.0079684524905446
nobs=58 count=58 0.9932712586455265
nobs=59 count=59 0.9933873005848147
```
Also, previously noted that "Requirement for size is only <= alpha" i.e. method="binom_test" should be conservative. But the current version is not conservative. This can be seen by plotting the graph via the following code:
```python
from statsmodels.stats.proportion import proportion_confint
from scipy.special import comb
import matplotlib.pyplot as plt
import numpy as np
def plot_coverage(methods_for_test, alpha, num, nobs_list):
    # start and end points are excluded to avoid any issue with 0**0
    p_points = np.linspace(1.0/num, 1.0, num, endpoint=False)
    for nobs in nobs_list:
        # compute in advance all binomial coefficients for nobs
        bc = comb(nobs, range(nobs+1))
        for method in methods_for_test:
            # compute in advance all confidence intervals
            all_ci = [[], []]
            for count in range(nobs+1):
                ci = proportion_confint(
                    count, nobs, alpha=alpha, method=method)
                all_ci[0].append(ci[0])
                all_ci[1].append(ci[1])
            plot_data = []
            for p in p_points:
                cp = 0.0
                for count in range(nobs + 1):
                    if all_ci[0][count] < p < all_ci[1][count]:
                        cp += bc[count]*(p**count)*((1.0-p)**(nobs-count))
                plot_data.append(cp)
                if cp < 1.0-alpha:
                    print(f"{nobs=} {p=:.5f} {cp=}")
            plt.plot(p_points, plot_data, label=method + f", {nobs=}")
    plt.hlines(1.0-alpha, 0.0, 1.0, label=f"confidence line, {alpha=}")
    plt.grid(True)
    plt.legend()
    plt.xlabel('p')
    plt.ylabel(r"coverage probability")
    plt.show()


methods_for_test = ["binom_test"]
plot_coverage(methods_for_test, alpha=0.05, num=1999, nobs_list=[31, 50, 75])
```
So we can see that in some cases coverage probability is less than 1-alpha, for example:
```
nobs=31 p=0.17150 cp=0.948454483652453
nobs=31 p=0.82850 cp=0.9484545872375366
nobs=50 p=0.07450 cp=0.9486301423455323
nobs=75 p=0.84900 cp=0.9493717635429614
```
Also, proportion_confint does not work when count is array_array_like and method="binom_test", for example code:
```python
from statsmodels.stats.proportion import proportion_confint
proportion_confint(range(10), 50, method="binom_test")
```
doesn't work.

Resolving patch:
diff --git a/statsmodels/stats/proportion.py b/statsmodels/stats/proportion.py
--- a/statsmodels/stats/proportion.py
+++ b/statsmodels/stats/proportion.py
@@ -1,5 +1,6 @@
 # -*- coding: utf-8 -*-
-"""Tests and Confidence Intervals for Binomial Proportions
+"""
+Tests and Confidence Intervals for Binomial Proportions
 
 Created on Fri Mar 01 00:23:07 2013
 
@@ -8,48 +9,135 @@
 """
 
 from statsmodels.compat.python import lzip
-
+from typing import Callable, Tuple
 import numpy as np
+import pandas as pd
 from scipy import optimize, stats
 
 from statsmodels.stats.base import AllPairsResults, HolderTuple
 from statsmodels.stats.weightstats import _zstat_generic2
 from statsmodels.tools.sm_exceptions import HypothesisTestWarning
 from statsmodels.tools.testing import Holder
+from statsmodels.tools.validation import array_like
 
 FLOAT_INFO = np.finfo(float)
 
 
-def proportion_confint(count, nobs, alpha=0.05, method='normal'):
+def _bound_proportion_confint(
+    func: Callable[[float], float], qi: float, lower: bool = True
+) -> float:
+    """
+    Try hard to find a bound different from eps/1 - eps in proportion_confint
+
+    Parameters
+    ----------
+    func : callable
+        Callable function to use as the objective of the search
+    qi : float
+        The empirical success rate
+    lower : bool
+        Whether to fund a lower bound for the left side of the CI
+
+    Returns
+    -------
+    float
+        The coarse bound
+    """
+    default = FLOAT_INFO.eps if lower else 1.0 - FLOAT_INFO.eps
+
+    def step(v):
+        return v / 8 if lower else v + (1.0 - v) / 8
+
+    x = step(qi)
+    w = func(x)
+    cnt = 1
+    while w > 0 and cnt < 10:
+        x = step(x)
+        w = func(x)
+        cnt += 1
+    return x if cnt < 10 else default
+
+
+def _bisection_search_conservative(
+    func: Callable[[float], float], lb: float, ub: float, steps: int = 27
+) -> Tuple[float, float]:
+    """
+    Private function used as a fallback by proportion_confint
+
+    Used when brentq returns a non-conservative bound for the CI
+
+    Parameters
+    ----------
+    func : callable
+        Callable function to use as the objective of the search
+    lb : float
+        Lower bound
+    ub : float
+        Upper bound
+    steps : int
+        Number of steps to use in the bisection
+
+    Returns
+    -------
+    est : float
+        The estimated value.  Will always produce a negative value of func
+    func_val : float
+        The value of the function at the estimate
+    """
+    upper = func(ub)
+    lower = func(lb)
+    best = upper if upper < 0 else lower
+    best_pt = ub if upper < 0 else lb
+    if np.sign(lower) == np.sign(upper):
+        raise ValueError("problem with signs")
+    mp = (ub + lb) / 2
+    mid = func(mp)
+    if (mid < 0) and (mid > best):
+        best = mid
+        best_pt = mp
+    for _ in range(steps):
+        if np.sign(mid) == np.sign(upper):
+            ub = mp
+            upper = mid
+        else:
+            lb = mp
+        mp = (ub + lb) / 2
+        mid = func(mp)
+        if (mid < 0) and (mid > best):
+            best = mid
+            best_pt = mp
+    return best_pt, best
+
+
+def proportion_confint(count, nobs, alpha=0.05, method="normal"):
     """
     Confidence interval for a binomial proportion
 
     Parameters
     ----------
-    count : int or array_array_like
-        number of successes, can be pandas Series or DataFrame
-    nobs : int
-        total number of trials
+    count : {int, array_like}
+        number of successes, can be pandas Series or DataFrame. Arrays
+        much contain integer values.
+    nobs : {int, array_like}
+        total number of trials.  Arrays much contain integer values.
     alpha : float in (0, 1)
         significance level, default 0.05
-    method : {'normal', 'agresti_coull', 'beta', 'wilson', 'binom_test'}
-        default: 'normal'
-        method to use for confidence interval,
-        currently available methods :
+    method : {"normal", "agresti_coull", "beta", "wilson", "binom_test"}
+        default: "normal"
+        method to use for confidence interval. Supported methods:
 
          - `normal` : asymptotic normal approximation
          - `agresti_coull` : Agresti-Coull interval
          - `beta` : Clopper-Pearson interval based on Beta distribution
          - `wilson` : Wilson Score interval
          - `jeffreys` : Jeffreys Bayesian Interval
-         - `binom_test` : experimental, inversion of binom_test
+         - `binom_test` : Numerical inversion of binom_test
 
     Returns
     -------
-    ci_low, ci_upp : float, ndarray, or pandas Series or DataFrame
+    ci_low, ci_upp : {float, ndarray, Series DataFrame}
         lower and upper confidence level with coverage (approximately) 1-alpha.
-        When a pandas object is returned, then the index is taken from the
-        `count`.
+        When a pandas object is returned, then the index is taken from `count`.
 
     Notes
     -----
@@ -57,13 +145,13 @@ def proportion_confint(count, nobs, alpha=0.05, method='normal'):
     but is in general conservative. Most of the other methods have average
     coverage equal to 1-alpha, but will have smaller coverage in some cases.
 
-    The 'beta' and 'jeffreys' interval are central, they use alpha/2 in each
+    The "beta" and "jeffreys" interval are central, they use alpha/2 in each
     tail, and alpha is not adjusted at the boundaries. In the extreme case
     when `count` is zero or equal to `nobs`, then the coverage will be only
-    1 - alpha/2 in the case of 'beta'.
+    1 - alpha/2 in the case of "beta".
 
     The confidence intervals are clipped to be in the [0, 1] interval in the
-    case of 'normal' and 'agresti_coull'.
+    case of "normal" and "agresti_coull".
 
     Method "binom_test" directly inverts the binomial test in scipy.stats.
     which has discrete steps.
@@ -79,11 +167,10 @@ def proportion_confint(count, nobs, alpha=0.05, method='normal'):
         Estimation for a Binomial Proportion",
         Statistical Science 16 (2): 101–133. doi:10.1214/ss/1009213286.
     """
-
-    pd_index = getattr(count, 'index', None)
-    if pd_index is not None and callable(pd_index):
-        # this rules out lists, lists have an index method
-        pd_index = None
+    is_scalar = np.isscalar(count) and np.isscalar(nobs)
+    is_pandas = isinstance(count, (pd.Series, pd.DataFrame))
+    count_a = array_like(count, "count", optional=False, ndim=None)
+    nobs_a = array_like(nobs, "nobs", optional=False, ndim=None)
 
     def _check(x: np.ndarray, name: str) -> np.ndarray:
         if np.issubdtype(x.dtype, np.integer):
@@ -96,126 +183,131 @@ def _check(x: np.ndarray, name: str) -> np.ndarray:
             )
         return y
 
-    count = _check(np.asarray(count), "count")
-    nobs = _check(np.asarray(nobs), "nobs")
+    count_a = _check(np.asarray(count_a), "count")
+    nobs_a = _check(np.asarray(nobs_a), "count")
 
-    q_ = count / nobs
+    q_ = count_a / nobs_a
     alpha_2 = 0.5 * alpha
 
-    if method == 'normal':
-        std_ = np.sqrt(q_ * (1 - q_) / nobs)
-        dist = stats.norm.isf(alpha / 2.) * std_
+    if method == "normal":
+        std_ = np.sqrt(q_ * (1 - q_) / nobs_a)
+        dist = stats.norm.isf(alpha / 2.0) * std_
         ci_low = q_ - dist
         ci_upp = q_ + dist
-
-
-
-    elif method == 'binom_test':
+    elif method == "binom_test":
         # inverting the binomial test
-        if hasattr(stats, "binomtest"):
-            def func(qi):
-                return stats.binomtest(count, nobs, p=qi).pvalue - alpha
-        else:
-            # Remove after min SciPy >= 1.7
-            def func(qi):
-                return stats.binom_test(count, nobs, p=qi) - alpha
+        def func_factory(count: int, nobs: int) -> Callable[[float], float]:
+            if hasattr(stats, "binomtest"):
 
-        def _bound(qi, lower=True):
-            """
-            Try hard to find a bound different from eps or 1 - eps
-
-            Parameters
-            ----------
-            qi : float
-                The empirical success rate
-            lower : bool
-                Whether to fund a lower bound for the left side of the CI
-
-            Returns
-            -------
-            float
-                The coarse bound
-            """
-            default = FLOAT_INFO.eps if lower else 1.0 - FLOAT_INFO.eps
-
-            def step(v):
-                return v / 8 if lower else v + (1.0 - v) / 8
-
-            x = step(qi)
-            w = func(x)
-            cnt = 1
-            while w > 0 and cnt < 10:
-                x = step(x)
-                w = func(x)
-                cnt += 1
-            return x if cnt < 10 else default
-
-        if count == 0:
-            ci_low = 0
-        else:
-            lower_bnd = _bound(q_, lower=True)
-            ci_low = optimize.brentq(func, lower_bnd, q_)
-        if count == nobs:
-            ci_upp = 1
-        else:
-            upper_bnd = _bound(q_, lower=False)
-            ci_upp = optimize.brentq(func, q_, upper_bnd)
+                def func(qi):
+                    return stats.binomtest(count, nobs, p=qi).pvalue - alpha
 
-    elif method == 'beta':
-        ci_low = stats.beta.ppf(alpha_2, count, nobs - count + 1)
-        ci_upp = stats.beta.isf(alpha_2, count + 1, nobs - count)
+            else:
+                # Remove after min SciPy >= 1.7
+                def func(qi):
+                    return stats.binom_test(count, nobs, p=qi) - alpha
+
+            return func
+
+        bcast = np.broadcast(count_a, nobs_a)
+        ci_low = np.zeros(bcast.shape)
+        ci_upp = np.zeros(bcast.shape)
+        index = bcast.index
+        for c, n in bcast:
+            # Enforce symmetry
+            reverse = False
+            _q = q_.flat[index]
+            if c > n // 2:
+                c = n - c
+                reverse = True
+                _q = 1 - _q
+            func = func_factory(c, n)
+            if c == 0:
+                ci_low.flat[index] = 0.0
+            else:
+                lower_bnd = _bound_proportion_confint(func, _q, lower=True)
+                val, _z = optimize.brentq(
+                    func, lower_bnd, _q, full_output=True
+                )
+                if func(val) > 0:
+                    power = 10
+                    new_lb = val - (val - lower_bnd) / 2**power
+                    while func(new_lb) > 0 and power >= 0:
+                        power -= 1
+                        new_lb = val - (val - lower_bnd) / 2**power
+                    val, _ = _bisection_search_conservative(func, new_lb, _q)
+                ci_low.flat[index] = val
+            if c == n:
+                ci_upp.flat[index] = 1.0
+            else:
+                upper_bnd = _bound_proportion_confint(func, _q, lower=False)
+                val, _z = optimize.brentq(
+                    func, _q, upper_bnd, full_output=True
+                )
+                if func(val) > 0:
+                    power = 10
+                    new_ub = val + (upper_bnd - val) / 2**power
+                    while func(new_ub) > 0 and power >= 0:
+                        power -= 1
+                        new_ub = val - (upper_bnd - val) / 2**power
+                    val, _ = _bisection_search_conservative(func, _q, new_ub)
+                ci_upp.flat[index] = val
+            if reverse:
+                temp = ci_upp.flat[index]
+                ci_upp.flat[index] = 1 - ci_low.flat[index]
+                ci_low.flat[index] = 1 - temp
+            index = bcast.index
+    elif method == "beta":
+        ci_low = stats.beta.ppf(alpha_2, count_a, nobs_a - count_a + 1)
+        ci_upp = stats.beta.isf(alpha_2, count_a + 1, nobs_a - count_a)
 
         if np.ndim(ci_low) > 0:
-            ci_low[q_ == 0] = 0
-            ci_upp[q_ == 1] = 1
+            ci_low.flat[q_.flat == 0] = 0
+            ci_upp.flat[q_.flat == 1] = 1
         else:
-            ci_low = ci_low if (q_ != 0) else 0
-            ci_upp = ci_upp if (q_ != 1) else 1
-
-    elif method == 'agresti_coull':
-        crit = stats.norm.isf(alpha / 2.)
-        nobs_c = nobs + crit**2
-        q_c = (count + crit**2 / 2.) / nobs_c
-        std_c = np.sqrt(q_c * (1. - q_c) / nobs_c)
+            ci_low = 0 if q_ == 0 else ci_low
+            ci_upp = 1 if q_ == 1 else ci_upp
+    elif method == "agresti_coull":
+        crit = stats.norm.isf(alpha / 2.0)
+        nobs_c = nobs_a + crit**2
+        q_c = (count_a + crit**2 / 2.0) / nobs_c
+        std_c = np.sqrt(q_c * (1.0 - q_c) / nobs_c)
         dist = crit * std_c
         ci_low = q_c - dist
         ci_upp = q_c + dist
-
-    elif method == 'wilson':
-        crit = stats.norm.isf(alpha / 2.)
+    elif method == "wilson":
+        crit = stats.norm.isf(alpha / 2.0)
         crit2 = crit**2
-        denom = 1 + crit2 / nobs
-        center = (q_ + crit2 / (2 * nobs)) / denom
-        dist = crit * np.sqrt(q_ * (1. - q_) / nobs + crit2 / (4. * nobs**2))
+        denom = 1 + crit2 / nobs_a
+        center = (q_ + crit2 / (2 * nobs_a)) / denom
+        dist = crit * np.sqrt(
+            q_ * (1.0 - q_) / nobs_a + crit2 / (4.0 * nobs_a**2)
+        )
         dist /= denom
         ci_low = center - dist
         ci_upp = center + dist
-
     # method adjusted to be more forgiving of misspellings or incorrect option name
-    elif method[:4] == 'jeff':
-        ci_low, ci_upp = stats.beta.interval(1 - alpha, count + 0.5,
-                                             nobs - count + 0.5)
-
+    elif method[:4] == "jeff":
+        ci_low, ci_upp = stats.beta.interval(
+            1 - alpha, count_a + 0.5, nobs_a - count_a + 0.5
+        )
     else:
-        raise NotImplementedError('method "%s" is not available' % method)
-
-    if method in ['normal', 'agresti_coull']:
+        raise NotImplementedError(f"method {method} is not available")
+    if method in ["normal", "agresti_coull"]:
         ci_low = np.clip(ci_low, 0, 1)
         ci_upp = np.clip(ci_upp, 0, 1)
-    if pd_index is not None and np.ndim(ci_low) > 0:
-        import pandas as pd
-        if np.ndim(ci_low) == 1:
-            ci_low = pd.Series(ci_low, index=pd_index)
-            ci_upp = pd.Series(ci_upp, index=pd_index)
-        if np.ndim(ci_low) == 2:
-            ci_low = pd.DataFrame(ci_low, index=pd_index)
-            ci_upp = pd.DataFrame(ci_upp, index=pd_index)
-
+    if is_pandas:
+        container = pd.Series if isinstance(count, pd.Series) else pd.DataFrame
+        ci_low = container(ci_low, index=count.index)
+        ci_upp = container(ci_upp, index=count.index)
+    if is_scalar:
+        return float(ci_low), float(ci_upp)
     return ci_low, ci_upp
 
 
 def multinomial_proportions_confint(counts, alpha=0.05, method='goodman'):
-    '''Confidence intervals for multinomial proportions.
+    """
+    Confidence intervals for multinomial proportions.
 
     Parameters
     ----------
@@ -297,7 +389,7 @@ def multinomial_proportions_confint(counts, alpha=0.05, method='goodman'):
            simultaneous confidence intervals for multinomial proportions for
            small counts in a large number of cells," Journal of Statistical
            Software, Vol. 5, No. 6, 2000, pp. 1-24.
-    '''
+    """
     if alpha <= 0 or alpha >= 1:
         raise ValueError('alpha must be in (0, 1), bounds excluded')
     counts = np.array(counts, dtype=float)
@@ -316,24 +408,30 @@ def multinomial_proportions_confint(counts, alpha=0.05, method='goodman'):
     elif method[:5] == 'sison':  # We accept any name starting with 'sison'
         # Define a few functions we'll use a lot.
         def poisson_interval(interval, p):
-            """Compute P(b <= Z <= a) where Z ~ Poisson(p) and
-            `interval = (b, a)`."""
+            """
+            Compute P(b <= Z <= a) where Z ~ Poisson(p) and
+            `interval = (b, a)`.
+            """
             b, a = interval
             prob = stats.poisson.cdf(a, p) - stats.poisson.cdf(b - 1, p)
             return prob
 
         def truncated_poisson_factorial_moment(interval, r, p):
-            """Compute mu_r, the r-th factorial moment of a poisson random
-            variable of parameter `p` truncated to `interval = (b, a)`."""
+            """
+            Compute mu_r, the r-th factorial moment of a poisson random
+            variable of parameter `p` truncated to `interval = (b, a)`.
+            """
             b, a = interval
             return p ** r * (1 - ((poisson_interval((a - r + 1, a), p) -
                                    poisson_interval((b - r, b - 1), p)) /
                                   poisson_interval((b, a), p)))
 
         def edgeworth(intervals):
-            """Compute the Edgeworth expansion term of Sison & Glaz's formula
+            """
+            Compute the Edgeworth expansion term of Sison & Glaz's formula
             (1) (approximated probability for multinomial proportions in a
-            given box)."""
+            given box).
+            """
             # Compute means and central moments of the truncated poisson
             # variables.
             mu_r1, mu_r2, mu_r3, mu_r4 = [
@@ -363,8 +461,10 @@ def edgeworth(intervals):
 
 
         def approximated_multinomial_interval(intervals):
-            """Compute approximated probability for Multinomial(n, proportions)
-            to be in `intervals` (Sison & Glaz's formula (1))."""
+            """
+            Compute approximated probability for Multinomial(n, proportions)
+            to be in `intervals` (Sison & Glaz's formula (1)).
+            """
             return np.exp(
                 np.sum(np.log([poisson_interval(interval, p)
                                for (interval, p) in zip(intervals, counts)])) +
@@ -373,8 +473,10 @@ def approximated_multinomial_interval(intervals):
             )
 
         def nu(c):
-            """Compute interval coverage for a given `c` (Sison & Glaz's
-            formula (7))."""
+            """
+            Compute interval coverage for a given `c` (Sison & Glaz's
+            formula (7)).
+            """
             return approximated_multinomial_interval(
                 [(np.maximum(count - c, 0), np.minimum(count + c, n))
                  for count in counts])
@@ -404,7 +506,8 @@ def nu(c):
 
 def samplesize_confint_proportion(proportion, half_length, alpha=0.05,
                                   method='normal'):
-    '''find sample size to get desired confidence interval length
+    """
+    Find sample size to get desired confidence interval length
 
     Parameters
     ----------
@@ -429,7 +532,7 @@ def samplesize_confint_proportion(proportion, half_length, alpha=0.05,
     this is mainly to store the formula.
     possible application: number of replications in bootstrap samples
 
-    '''
+    """
     q_ = proportion
     if method == 'normal':
         n = q_ * (1 - q_) / (half_length / stats.norm.isf(alpha / 2.))**2
@@ -440,7 +543,7 @@ def samplesize_confint_proportion(proportion, half_length, alpha=0.05,
 
 
 def proportion_effectsize(prop1, prop2, method='normal'):
-    '''
+    """
     Effect size for a test comparing two proportions
 
     for use in power function
@@ -474,7 +577,7 @@ def proportion_effectsize(prop1, prop2, method='normal'):
     >>> sm.stats.proportion_effectsize([0.3, 0.4, 0.5], 0.4)
     array([-0.21015893,  0.        ,  0.20135792])
 
-    '''
+    """
     if method != 'normal':
         raise ValueError('only "normal" is implemented')
 
@@ -483,7 +586,8 @@ def proportion_effectsize(prop1, prop2, method='normal'):
 
 
 def std_prop(prop, nobs):
-    '''standard error for the estimate of a proportion
+    """
+    Standard error for the estimate of a proportion
 
     This is just ``np.sqrt(p * (1. - p) / nobs)``
 
@@ -498,7 +602,7 @@ def std_prop(prop, nobs):
     -------
     std : array_like
         standard error for a proportion of nobs independent observations
-    '''
+    """
     return np.sqrt(prop * (1. - prop) / nobs)
 
 
@@ -509,13 +613,14 @@ def _std_diff_prop(p1, p2, ratio=1):
 def _power_ztost(mean_low, var_low, mean_upp, var_upp, mean_alt, var_alt,
                  alpha=0.05, discrete=True, dist='norm', nobs=None,
                  continuity=0, critval_continuity=0):
-    '''Generic statistical power function for normal based equivalence test
+    """
+    Generic statistical power function for normal based equivalence test
 
     This includes options to adjust the normal approximation and can use
     the binomial to evaluate the probability of the rejection region
 
     see power_ztost_prob for a description of the options
-    '''
+    """
     # TODO: refactor structure, separate norm and binom better
     if not isinstance(continuity, tuple):
         continuity = (continuity, continuity)
@@ -551,7 +656,8 @@ def _power_ztost(mean_low, var_low, mean_upp, var_upp, mean_alt, var_alt,
 
 
 def binom_tost(count, nobs, low, upp):
-    '''exact TOST test for one proportion using binomial distribution
+    """
+    Exact TOST test for one proportion using binomial distribution
 
     Parameters
     ----------
@@ -569,7 +675,7 @@ def binom_tost(count, nobs, low, upp):
     pval_low, pval_upp : floats
         p-values of lower and upper one-sided tests
 
-    '''
+    """
     # binom_test_stat only returns pval
     tt1 = binom_test(count, nobs, alternative='larger', prop=low)
     tt2 = binom_test(count, nobs, alternative='smaller', prop=upp)
@@ -577,7 +683,8 @@ def binom_tost(count, nobs, low, upp):
 
 
 def binom_tost_reject_interval(low, upp, nobs, alpha=0.05):
-    '''rejection region for binomial TOST
+    """
+    Rejection region for binomial TOST
 
     The interval includes the end points,
     `reject` if and only if `r_low <= x <= r_upp`.
@@ -596,7 +703,7 @@ def binom_tost_reject_interval(low, upp, nobs, alpha=0.05):
     x_low, x_upp : float
         lower and upper bound of rejection region
 
-    '''
+    """
     x_low = stats.binom.isf(alpha, nobs, low) + 1
     x_upp = stats.binom.ppf(alpha, nobs, upp) - 1
     return x_low, x_upp
@@ -697,7 +804,8 @@ def power_binom_tost(low, upp, nobs, p_alt=None, alpha=0.05):
 def power_ztost_prop(low, upp, nobs, p_alt, alpha=0.05, dist='norm',
                      variance_prop=None, discrete=True, continuity=0,
                      critval_continuity=0):
-    '''Power of proportions equivalence test based on normal distribution
+    """
+    Power of proportions equivalence test based on normal distribution
 
     Parameters
     ----------
@@ -766,7 +874,7 @@ def power_ztost_prop(low, upp, nobs, p_alt, alpha=0.05, dist='norm',
     SAS Manual: Chapter 68: The Power Procedure, Computational Resources
     PASS Chapter 110: Equivalence Tests for One Proportion.
 
-    '''
+    """
     mean_low = low
     var_low = std_prop(low, nobs)**2
     mean_upp = upp
@@ -782,7 +890,8 @@ def power_ztost_prop(low, upp, nobs, p_alt, alpha=0.05, dist='norm',
 
 
 def _table_proportion(count, nobs):
-    '''create a k by 2 contingency table for proportion
+    """
+    Create a k by 2 contingency table for proportion
 
     helper function for proportions_chisquare
 
@@ -802,7 +911,7 @@ def _table_proportion(count, nobs):
     -----
     recent scipy has more elaborate contingency table functions
 
-    '''
+    """
     count = np.asarray(count)
     dt = np.promote_types(count.dtype, np.float64)
     count = np.asarray(count, dtype=dt)
@@ -917,7 +1026,8 @@ def proportions_ztest(count, nobs, value=None, alternative='two-sided',
 
 
 def proportions_ztost(count, nobs, low, upp, prop_var='sample'):
-    '''Equivalence test based on normal distribution
+    """
+    Equivalence test based on normal distribution
 
     Parameters
     ----------
@@ -949,7 +1059,7 @@ def proportions_ztost(count, nobs, low, upp, prop_var='sample'):
     -----
     checked only for 1 sample case
 
-    '''
+    """
     if prop_var == 'limits':
         prop_var_low = low
         prop_var_upp = upp
@@ -968,7 +1078,8 @@ def proportions_ztost(count, nobs, low, upp, prop_var='sample'):
 
 
 def proportions_chisquare(count, nobs, value=None):
-    '''test for proportions based on chisquare test
+    """
+    Test for proportions based on chisquare test
 
     Parameters
     ----------
@@ -1007,7 +1118,7 @@ def proportions_chisquare(count, nobs, value=None):
     given and count and nobs are not scalar, then the null hypothesis is
     that all samples have the same proportion.
 
-    '''
+    """
     nobs = np.atleast_1d(nobs)
     table, expected, n_rows = _table_proportion(count, nobs)
     if value is not None:
@@ -1023,7 +1134,8 @@ def proportions_chisquare(count, nobs, value=None):
 
 
 def proportions_chisquare_allpairs(count, nobs, multitest_method='hs'):
-    '''chisquare test of proportions for all pairs of k samples
+    """
+    Chisquare test of proportions for all pairs of k samples
 
     Performs a chisquare test for proportions for all pairwise comparisons.
     The alternative is two-sided
@@ -1053,7 +1165,7 @@ def proportions_chisquare_allpairs(count, nobs, multitest_method='hs'):
     Notes
     -----
     Yates continuity correction is not available.
-    '''
+    """
     #all_pairs = lmap(list, lzip(*np.triu_indices(4, 1)))
     all_pairs = lzip(*np.triu_indices(len(count), 1))
     pvals = [proportions_chisquare(count[list(pair)], nobs[list(pair)])[1]
@@ -1063,7 +1175,8 @@ def proportions_chisquare_allpairs(count, nobs, multitest_method='hs'):
 
 def proportions_chisquare_pairscontrol(count, nobs, value=None,
                                multitest_method='hs', alternative='two-sided'):
-    '''chisquare test of proportions for pairs of k samples compared to control
+    """
+    Chisquare test of proportions for pairs of k samples compared to control
 
     Performs a chisquare test for proportions for pairwise comparisons with a
     control (Dunnet's test). The control is assumed to be the first element
@@ -1102,7 +1215,7 @@ def proportions_chisquare_pairscontrol(count, nobs, value=None,
 
     ``value`` and ``alternative`` options are not yet implemented.
 
-    '''
+    """
     if (value is not None) or (alternative not in ['two-sided', '2s']):
         raise NotImplementedError
     #all_pairs = lmap(list, lzip(*np.triu_indices(4, 1)))
@@ -1116,7 +1229,8 @@ def proportions_chisquare_pairscontrol(count, nobs, value=None,
 
 def confint_proportions_2indep(count1, nobs1, count2, nobs2, method=None,
                                compare='diff', alpha=0.05, correction=True):
-    """Confidence intervals for comparing two independent proportions
+    """
+    Confidence intervals for comparing two independent proportions
 
     This assumes that we have two independent binomial samples.
 
@@ -1298,7 +1412,8 @@ def confint_proportions_2indep(count1, nobs1, count2, nobs2, method=None,
 
 def _shrink_prob(count1, nobs1, count2, nobs2, shrink_factor=2,
                  return_corr=True):
-    """shrink observed counts towards independence
+    """
+    Shrink observed counts towards independence
 
     Helper function for 'logit-smoothed' inference for the odds-ratio of two
     independent proportions.
@@ -1342,7 +1457,8 @@ def _shrink_prob(count1, nobs1, count2, nobs2, shrink_factor=2,
 def score_test_proportions_2indep(count1, nobs1, count2, nobs2, value=None,
                                   compare='diff', alternative='two-sided',
                                   correction=True, return_results=True):
-    """score_test for two independent proportions
+    """
+    Score test for two independent proportions
 
     This uses the constrained estimate of the proportions to compute
     the variance under the Null hypothesis.
@@ -1498,7 +1614,8 @@ def test_proportions_2indep(count1, nobs1, count2, nobs2, value=None,
                             method=None, compare='diff',
                             alternative='two-sided', correction=True,
                             return_results=True):
-    """Hypothesis test for comparing two independent proportions
+    """
+    Hypothesis test for comparing two independent proportions
 
     This assumes that we have two independent binomial samples.
 
@@ -1872,7 +1989,8 @@ def tost_proportions_2indep(count1, nobs1, count2, nobs2, low, upp,
 
 
 def _std_2prop_power(diff, p2, ratio=1, alpha=0.05, value=0):
-    """compute standard error under null and alternative for 2 proportions
+    """
+    Compute standard error under null and alternative for 2 proportions
 
     helper function for power and sample size computation
 
@@ -1899,7 +2017,8 @@ def _std_2prop_power(diff, p2, ratio=1, alpha=0.05, value=0):
 def power_proportions_2indep(diff, prop2, nobs1, ratio=1, alpha=0.05,
                              value=0, alternative='two-sided',
                              return_results=True):
-    """power for ztest that two independent proportions are equal
+    """
+    Power for ztest that two independent proportions are equal
 
     This assumes that the variance is based on the pooled proportion
     under the null and the non-pooled variance under the alternative
@@ -1980,7 +2099,8 @@ def power_proportions_2indep(diff, prop2, nobs1, ratio=1, alpha=0.05,
 def samplesize_proportions_2indep_onetail(diff, prop2, power, ratio=1,
                                           alpha=0.05, value=0,
                                           alternative='two-sided'):
-    """required sample size assuming normal distribution based on one tail
+    """
+    Required sample size assuming normal distribution based on one tail
 
     This uses an explicit computation for the sample size that is required
     to achieve a given power corresponding to the appropriate tails of the
@@ -2031,7 +2151,8 @@ def samplesize_proportions_2indep_onetail(diff, prop2, power, ratio=1,
 
 def _score_confint_inversion(count1, nobs1, count2, nobs2, compare='diff',
                              alpha=0.05, correction=True):
-    """Compute score confidence interval by inverting score test
+    """
+    Compute score confidence interval by inverting score test
 
     Parameters
     ----------
@@ -2104,7 +2225,8 @@ def func(v):
 
 def _confint_riskratio_koopman(count1, nobs1, count2, nobs2, alpha=0.05,
                                correction=True):
-    """score confidence interval for ratio or proportions, Koopman/Nam
+    """
+    Score confidence interval for ratio or proportions, Koopman/Nam
 
     signature not consistent with other functions
 
@@ -2139,7 +2261,8 @@ def _confint_riskratio_koopman(count1, nobs1, count2, nobs2, alpha=0.05,
 
 
 def _confint_riskratio_paired_nam(table, alpha=0.05):
-    """confidence interval for marginal risk ratio for matched pairs
+    """
+    Confidence interval for marginal risk ratio for matched pairs
 
     need full table
 


Test patch:
diff --git a/statsmodels/stats/tests/test_proportion.py b/statsmodels/stats/tests/test_proportion.py
--- a/statsmodels/stats/tests/test_proportion.py
+++ b/statsmodels/stats/tests/test_proportion.py
@@ -29,6 +29,7 @@
 )
 from statsmodels.tools.sm_exceptions import HypothesisTestWarning
 from statsmodels.tools.testing import Holder
+from statsmodels.stats.tests.results.results_proportion import res_binom, res_binom_methods
 
 probci_methods = {'agresti_coull': 'agresti-coull',
                   'normal': 'asymptotic',
@@ -37,31 +38,27 @@
                   'jeffreys': 'bayes'
                   }
 
-
-def test_confint_proportion():
-    from .results.results_proportion import res_binom, res_binom_methods
-
-
-    for case in res_binom:
-        count, nobs = case
-        for method in probci_methods:
-            idx = res_binom_methods.index(probci_methods[method])
-            res_low = res_binom[case].ci_low[idx]
-            res_upp = res_binom[case].ci_upp[idx]
-            if np.isnan(res_low) or np.isnan(res_upp):
-                continue
-            if (count == 0 or count == nobs) and method == 'jeffreys':
-                # maybe a bug or different corner case definition
-                continue
-            if method == 'jeffreys' and nobs == 30:
-                # something is strange in extreme case e.g 0/30 or 1/30
-                continue
-            ci = proportion_confint(count, nobs, alpha=0.05, method=method)
-            # we impose that confint is in [0, 1]
-            res_low = max(res_low, 0)
-            res_upp = min(res_upp, 1)
-            assert_almost_equal(ci, [res_low, res_upp], decimal=6,
-                                err_msg=repr(case) + method)
+@pytest.mark.parametrize("case",res_binom)
+@pytest.mark.parametrize("method",probci_methods)
+def test_confint_proportion(method, case):
+    count, nobs = case
+    idx = res_binom_methods.index(probci_methods[method])
+    res_low = res_binom[case].ci_low[idx]
+    res_upp = res_binom[case].ci_upp[idx]
+    if np.isnan(res_low) or np.isnan(res_upp):
+        pytest.skip("Skipping due to NaN value")
+    if (count == 0 or count == nobs) and method == 'jeffreys':
+        # maybe a bug or different corner case definition
+        pytest.skip("Skipping nobs 0 or count and jeffreys")
+    if method == 'jeffreys' and nobs == 30:
+        # something is strange in extreme case e.g 0/30 or 1/30
+        pytest.skip("Skipping nobs is 30 and jeffreys due to extreme case problem")
+    ci = proportion_confint(count, nobs, alpha=0.05, method=method)
+    # we impose that confint is in [0, 1]
+    res_low = max(res_low, 0)
+    res_upp = min(res_upp, 1)
+    assert_almost_equal(ci, [res_low, res_upp], decimal=6,
+                        err_msg=repr(case) + method)
 
 
 @pytest.mark.parametrize('method', probci_methods)
@@ -927,8 +924,12 @@ def test_power_2indep():
 
 
 @pytest.mark.parametrize("count", np.arange(10, 90, 5))
-@pytest.mark.parametrize("method", list(probci_methods.keys()) + ["binom_test"])
-def test_ci_symmetry(count, method):
+@pytest.mark.parametrize(
+    "method", list(probci_methods.keys()) + ["binom_test"]
+)
+@pytest.mark.parametrize("array_like", [False, True])
+def test_ci_symmetry(count, method, array_like):
+    _count = [count] * 3 if array_like else count
     n = 100
     a = proportion_confint(count, n, method=method)
     b = proportion_confint(n - count, n, method=method)
@@ -937,9 +938,12 @@ def test_ci_symmetry(count, method):
 
 @pytest.mark.parametrize("nobs", [47, 50])
 @pytest.mark.parametrize("count", np.arange(48))
-def test_ci_symmetry_binom_test(nobs, count):
-    a = proportion_confint(count, nobs, method="binom_test")
-    b = proportion_confint(nobs - count, nobs, method="binom_test")
+@pytest.mark.parametrize("array_like", [False, True])
+def test_ci_symmetry_binom_test(nobs, count, array_like):
+    _count = [count] * 3 if array_like else count
+    nobs_m_count = [nobs - count] * 3 if array_like else nobs - count
+    a = proportion_confint(_count, nobs, method="binom_test")
+    b = proportion_confint(nobs_m_count, nobs, method="binom_test")
     assert_allclose(np.array(a), 1.0 - np.array(b[::-1]))
 
 
@@ -955,3 +959,14 @@ def test_int_check():
     c = proportion_confint(21, 47.0, method="binom_test")
     assert_allclose(a, b)
     assert_allclose(a, c)
+
+
+@pytest.mark.parametrize("count", np.arange(10, 90, 5))
+@pytest.mark.parametrize(
+    "method", list(probci_methods.keys()) + ["binom_test"]
+)
+def test_ci_symmetry_array(count, method):
+    n = 100
+    a = proportion_confint([count, count], n, method=method)
+    b = proportion_confint([n - count, n - count], n, method=method)
+    assert_allclose(np.array(a), 1.0 - np.array(b[::-1]))