Repository: statsmodels/statsmodels

Issue:
ENH: Let users set KDE bandwidth use a user-defined bandwidth function
#### Describe the bug

Unexpected exception raised for KDE fit() when input data has exactly zero bandwidth.

#### Code Sample (copy-paste works)


```python
from statsmodels.nonparametric.kde import KDEUnivariate
import numpy as np

kde = KDEUnivariate(np.array([0.1, 0.1]))
kde.fit()
```
<details>

I did have checked wether there is a similar open bug and couldn't find one.
I did look at the source code on master, but did not run it and seems the behavior is there as well.

</details>


#### Expected Output

If running statsmodels in a pipeline then the user data might be anything, and would expect it handles this gracefully instead of raising an exception. This would allow the user to see a nice graph in eg seaborn instead of an error for a very simple case.

```python
In [1]: print(kde.density)
Out[1]: array([ ... ])
In [1]: print(kde.support)
Out[1]: array([ ... ])
```

Also behavior of `[0.1, 0.1]` should be very similar to `[0.1, 0.1000000000001]` (consider details points 1, 2, 3 below):
```python
from statsmodels.nonparametric.kde import KDEUnivariate
import numpy as np

kde = KDEUnivariate(np.array([0.1, 0.1000000000001]))
kde.fit()
```

<details>

1. The == 0 in `nonparametric/bandwidths.py:171 select_bandwidth()` comparing with a float is not kosher numerical methods practice, usually a comparison using a very small error term `eps` should be done.

2. The silverman bandwidth estimation might seem to be correct (== 0) in a situation where two observations that are the same. The bandwidth of a gaussian is actually not zero, but also not trivial to calculate.
Closest reference I could find is [this](https://math.stackexchange.com/questions/2693800/probability-that-two-consecutively-generated-integers-with-normal-distribution-a).
Net, in this case, the standard deviation estimator `std()` is not the best linear unbiased estimator for the standard deviation due to low number of samples and the uncertainty `eps`.
Based on this approach, I would suggest to use in this case `A == max(min(std(X), IQR/1.34), eps * scipy.stats.norm.ppf(q=1-eps/mu))` for input to the silverman or scott bandwidth estimators. In case `eps` is not specified just use 90% confidence directly, however the user should be allowed to specify it as well.
The logic is quite simple: the uncertainty `eps` when compared to the average `mu` gives a certain confidence level (90% in my example if we take `eps=0.01` ie one digit lower) which translates to an equivalent standard deviation based on that uncertainty ie `1.21 * eps`.
TL;DR The `eps` acts as a lower bound to standard deviation for cases with low number of observations.

3. While raising an exception in this case might seem pythonic, it's still not warranted, as one could still have a result out of this using a fitted Gaussian with `avg=0.1` and `std=0.012815515655446004` (based on point 2 above). Think about what a normal user has to do: handle exception by monkey patching the kde results so it can be used further down the code.

</details>

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

<details>

INSTALLED VERSIONS
------------------
Python: 3.8.5.final.0
OS: Linux 5.7.16-200.fc32.x86_64 #1 SMP Wed Aug 19 16:58:53 UTC 2020 x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

statsmodels
===========

Installed: 0.11.0 (/usr/lib64/python3.8/site-packages/statsmodels)

Required Dependencies
=====================

cython: Not installed
numpy: 1.18.4 (/usr/lib64/python3.8/site-packages/numpy)
scipy: 1.4.1 (/usr/lib64/python3.8/site-packages/scipy)
pandas: 0.25.3 (/usr/lib64/python3.8/site-packages/pandas)
    dateutil: 2.8.0 (/usr/lib/python3.8/site-packages/dateutil)
patsy: 0.5.1 (/usr/lib/python3.8/site-packages/patsy)

Optional Dependencies
=====================

matplotlib: 3.2.2 (/usr/lib64/python3.8/site-packages/matplotlib)
    backend: Qt5Agg 
cvxopt: Not installed
joblib: 0.13.2 (/usr/lib/python3.8/site-packages/joblib)

Developer Tools
================

IPython: 7.12.0 (/usr/lib/python3.8/site-packages/IPython)
    jinja2: 2.11.2 (/usr/lib/python3.8/site-packages/jinja2)
sphinx: 2.2.2 (/usr/lib/python3.8/site-packages/sphinx)
    pygments: 2.4.2 (/usr/lib/python3.8/site-packages/pygments)
pytest: 4.6.11 (/usr/lib/python3.8/site-packages)
virtualenv: Not installed

</details>



Discussion:
In master you get

```
RuntimeError: Selected KDE bandwidth is 0. Cannot estimate density. Either provide the bandwidth during initialization or use an alternative method.
```

This is the expected outcome. It is then up to downstream users to try catch if they think they might pass unsuitable data. 
All you have to do is 

```
kde = KDEUnivariate(np.array([0.1, 0.1]))
try:
    kde.fit()
except RuntimeError:
    kde.fit(bw=MY_PERSONAL_FAVORITE_BANDWIDTH_ESTIMATOR)
```

Thanks for taking the time to look at it. That is exactly what I am doing, however the behavior is not consistent.
Even in that case, one would raise an Exception and get the other bw estimator.
In the normal case I would go with the default bw resulting in numerically different results for numerically close inputs.

My point is `KDEUnivariate(np.array([0.1, 0.1000000000001]))` and `KDEUnivariate(np.array([0.1, 0.1]))` should provide similar results.
> Even in that case, one would raise an Exception and get the other bw estimator.

We try to not do magic things; if the function call asks for something it should either succeed, or if it cannot succeed, raise a meaningful error.  This is why we will never override the option provided.  

> My point is `KDEUnivariate(np.array([0.1, 0.1000000000001]))` and `KDEUnivariate(np.array([0.1, 0.1]))` should provide similar results.

Those arrays are numerically different within the precision of any double precision IEEE floating point compliant computer, so I don't see why they should provide the same behavior.

```
a = np.array([0.1, 0.1])
b = np.array([0.1, 0.1000000000001])

a==b
Out[5]: array([ True, False])
```

There are many other places where the output is not continuous as values approach numeric limits, for example, anything that performs a matrix rank check and raises on singular arrays will be discontinuous local to the tolerance of the array's eigenvalues.

### Ways forwards
1. IMO a better idea would be to let `bw` accept a `Callable[[ndarray], float]` so that a user could provide their favorite BW function that handles edges in the way they want rather than some ad-hoc approach. 
2. Add a new string method that implements some minimum bandwitdh. 

As to your original suggestion,

```
A = max(min(std(X), IQR/1.34), eps * scipy.stats.norm.ppf(q=1-eps/mu))
```

this is problematic if `mu` is even modestly large, in which case you have `eps*inf`. In general, it is hard to set an absolute lower bound since KDE should work reasonably well for both

```
x = np.random.standard_normal(1000)
kde = KDEUnivariate(x)
# OMG, why doesn't fit return self?!?!?
kde.fit()

eps = np.finfo(float).eps
kde2 = KDEUnivariate(eps*x)
kde2.fit()

print(kde2.bw / kde.bw)
kde2.bw / kde.bw
Out[14]: 2.220446049250313e-16

print(eps)
Out[15]: 2.220446049250313e-16
```


Excellent point with the eps / mu division.

The suggestions for 1 & 2 seem right initially, here is my input:
1. callable would be super flexible - this would be my main go-to, but again, what would prevent someone from doing even today this
```python
bw = callable(X)
kde.fit(bw=bw)
```
2. 'min_normal_reference', 'min_silverman' and 'min_scott' might be confusing to people, and not sure everyone has the need of a min-bandwith estimator. It would work though, I agree.

I did the most sensible thing in this case and got the book from Silverman and looked it up. He does mention that the approximation is only valid in asymptotic cases ref p. 103-104 (I transcribe it here under fair use) in context of Nearest-Neighbour. I beleive his statement is as equally valid for the normal "rule of thumb" case also from p. 45.

> If we presume that the likely values of f" are proportional to those of f (i.e. lower values of f" in the tails) then (5.9) suggests that the ideal local bandwidth will be proportional to f^(-1/5). This is a rather heuristic argument depending on asymptotic calculations and possibly rash assumptions.

I hope I'm not that rusty, but for Gaussian:
![formula](https://render.githubusercontent.com/render/math?math=\Large%20f''(x)=\frac{(x-\mu)^2-\sigma^2}{\sigma^4}f(x))

Considering ![formula](https://render.githubusercontent.com/render/math?math=\Large%20f~\frac{1}{\sigma}) then that would make the bandwidth:
![formula](https://render.githubusercontent.com/render/math?math=\Large%20h~\lvert%20\frac{(x-\mu)^2-\sigma^2}{\sigma^4}%20\rvert^{-\frac{2}{5}}\frac{1}{\sigma^{\frac{1}{5}}})
Note that in his p. 45 exposition he considers the integral of ```f''^2``` which is then asymptotic (tails at -inf and +inf), as such he only uses the last part of the term with sigma to the 5th power. The key is in the first part as well in my opinion.

So for asymptotic cases ```x >> mu``` (Silverman's approximation from p. 45):
![formula](https://render.githubusercontent.com/render/math?math=\Large%20h~\sigma^{\frac{7}{5}}~\sigma^1.4\approx\sigma)

For non-asymptotic cases ```x ~ mu```:
![formula](https://render.githubusercontent.com/render/math?math=\Large%20h~\sigma^{\frac{3}{5}}~\sigma^0.6)

On top, the expansion from the exponential part of ```f``` comes into play which for the non-asymptotic case we can aproximate easily around zero as ```1-0.5*(x-mu)^2/sigma^2``` so another -2 to the exponent for our initial calculation.

This would bring finally the non-asymptotic case
![formula](https://render.githubusercontent.com/render/math?math=\Large%20h~A\sigma^{-1.4}%2BB\sigma^{0.6})
which is exactly the reverse of the asymptotic case for the first case plus some term.

Net, we also need to distinguish between [0.1, 0.1] and [0.1, ... 0.1] with 200 elements. The former one would have a high bandwidth while the latter one would have a very small bandwidth but not zero.

Based on this I would propose to implement a new string bandwidth for 'non_asymptotic' as suggested in proposal 2 with the approach above. Need to figure out the constants that go in front.

What I like about the new 'non_asymptotic' case is that if you would have an array with two values which are quite far apart but you know are coming from a normal distribution then the bandwidth will be quite big, which is what one would expect instead of getting some graph with two bumps.


Resolving patch:
diff --git a/statsmodels/nonparametric/kde.py b/statsmodels/nonparametric/kde.py
--- a/statsmodels/nonparametric/kde.py
+++ b/statsmodels/nonparametric/kde.py
@@ -13,30 +13,37 @@
 """
 import numpy as np
 from scipy import integrate, stats
+
 from statsmodels.sandbox.nonparametric import kernels
 from statsmodels.tools.decorators import cache_readonly
-from statsmodels.tools.validation import array_like
+from statsmodels.tools.validation import array_like, float_like
+
 from . import bandwidths
-from .kdetools import (forrt, revrt, silverman_transform)
+from .kdetools import forrt, revrt, silverman_transform
 from .linbin import fast_linbin
 
-#### Kernels Switch for estimators ####
+# Kernels Switch for estimators
+
+kernel_switch = dict(
+    gau=kernels.Gaussian,
+    epa=kernels.Epanechnikov,
+    uni=kernels.Uniform,
+    tri=kernels.Triangular,
+    biw=kernels.Biweight,
+    triw=kernels.Triweight,
+    cos=kernels.Cosine,
+    cos2=kernels.Cosine2,
+)
 
-kernel_switch = dict(gau=kernels.Gaussian, epa=kernels.Epanechnikov,
-                    uni=kernels.Uniform, tri=kernels.Triangular,
-                    biw=kernels.Biweight, triw=kernels.Triweight,
-                    cos=kernels.Cosine, cos2=kernels.Cosine2)
 
 def _checkisfit(self):
     try:
         self.density
-    except:
+    except Exception:
         raise ValueError("Call fit to fit the density first")
 
 
-#### Kernel Density Estimator Class ###
-
-
+# Kernel Density Estimator Class
 class KDEUnivariate(object):
     """
     Univariate Kernel Density Estimator.
@@ -77,8 +84,17 @@ class KDEUnivariate(object):
     def __init__(self, endog):
         self.endog = array_like(endog, "endog", ndim=1, contiguous=True)
 
-    def fit(self, kernel="gau", bw="normal_reference", fft=True, weights=None,
-            gridsize=None, adjust=1, cut=3, clip=(-np.inf, np.inf)):
+    def fit(
+        self,
+        kernel="gau",
+        bw="normal_reference",
+        fft=True,
+        weights=None,
+        gridsize=None,
+        adjust=1,
+        cut=3,
+        clip=(-np.inf, np.inf),
+    ):
         """
         Attach the density estimate to the KDEUnivariate class.
 
@@ -95,17 +111,23 @@ def fit(self, kernel="gau", bw="normal_reference", fft=True, weights=None,
             - "triw" for triweight
             - "uni" for uniform
 
-        bw : str, float
+        bw : str, float, callable
             The bandwidth to use. Choices are:
 
             - "scott" - 1.059 * A * nobs ** (-1/5.), where A is
-              `min(std(X),IQR/1.34)`
+              `min(std(x),IQR/1.34)`
             - "silverman" - .9 * A * nobs ** (-1/5.), where A is
-              `min(std(X),IQR/1.34)`
+              `min(std(x),IQR/1.34)`
             - "normal_reference" - C * A * nobs ** (-1/5.), where C is
               calculated from the kernel. Equivalent (up to 2 dp) to the
               "scott" bandwidth for gaussian kernels. See bandwidths.py
-            - If a float is given, it is the bandwidth.
+            - If a float is given, its value is used as the bandwidth.
+            - If a callable is given, it's return value is used.
+              The callable should take exactly two parameters, i.e.,
+              fn(x, kern), and return a float, where:
+
+              * x - the clipped input data
+              * kern - the kernel instance used
 
         fft : bool
             Whether or not to use FFT. FFT implementation is more
@@ -113,11 +135,11 @@ def fit(self, kernel="gau", bw="normal_reference", fft=True, weights=None,
             is implemented. If FFT is False, then a 'nobs' x 'gridsize'
             intermediate array is created.
         gridsize : int
-            If gridsize is None, max(len(X), 50) is used.
+            If gridsize is None, max(len(x), 50) is used.
         cut : float
             Defines the length of the grid past the lowest and highest values
-            of X so that the kernel goes to zero. The end points are
-            -/+ cut*bw*{min(X) or max(X)}
+            of x so that the kernel goes to zero. The end points are
+            -/+ cut*bw*{min(x) or max(x)}
         adjust : float
             An adjustment factor for the bw. Bandwidth becomes bw * adjust.
 
@@ -126,11 +148,13 @@ def fit(self, kernel="gau", bw="normal_reference", fft=True, weights=None,
         KDEUnivariate
             The instance fit,
         """
-        try:
-            bw = float(bw)
-            self.bw_method = "user-given"
-        except:
+        if isinstance(bw, str):
             self.bw_method = bw
+        else:
+            self.bw_method = "user-given"
+            if not callable(bw):
+                bw = float_like(bw, "bw")
+
         endog = self.endog
 
         if fft:
@@ -140,18 +164,32 @@ def fit(self, kernel="gau", bw="normal_reference", fft=True, weights=None,
             if weights is not None:
                 msg = "Weights are not implemented for fft"
                 raise NotImplementedError(msg)
-            density, grid, bw = kdensityfft(endog, kernel=kernel, bw=bw,
-                    adjust=adjust, weights=weights, gridsize=gridsize,
-                    clip=clip, cut=cut)
+            density, grid, bw = kdensityfft(
+                endog,
+                kernel=kernel,
+                bw=bw,
+                adjust=adjust,
+                weights=weights,
+                gridsize=gridsize,
+                clip=clip,
+                cut=cut,
+            )
         else:
-            density, grid, bw = kdensity(endog, kernel=kernel, bw=bw,
-                    adjust=adjust, weights=weights, gridsize=gridsize,
-                    clip=clip, cut=cut)
+            density, grid, bw = kdensity(
+                endog,
+                kernel=kernel,
+                bw=bw,
+                adjust=adjust,
+                weights=weights,
+                gridsize=gridsize,
+                clip=clip,
+                cut=cut,
+            )
         self.density = density
         self.support = grid
         self.bw = bw
-        self.kernel = kernel_switch[kernel](h=bw) # we instantiate twice,
-                                                # should this passed to funcs?
+        self.kernel = kernel_switch[kernel](h=bw)  # we instantiate twice,
+        # should this passed to funcs?
         # put here to ensure empty cache after re-fit with new options
         self.kernel.weights = weights
         if weights is not None:
@@ -170,18 +208,22 @@ def cdf(self):
         """
         _checkisfit(self)
         kern = self.kernel
-        if kern.domain is None: # TODO: test for grid point at domain bound
-            a,b = -np.inf,np.inf
+        if kern.domain is None:  # TODO: test for grid point at domain bound
+            a, b = -np.inf, np.inf
         else:
-            a,b = kern.domain
-        func = lambda x,s: kern.density(s,x)
+            a, b = kern.domain
+
+        def func(x, s):
+            return kern.density(s, x)
 
         support = self.support
-        support = np.r_[a,support]
+        support = np.r_[a, support]
         gridsize = len(support)
         endog = self.endog
-        probs = [integrate.quad(func, support[i - 1], support[i],
-                    args=endog)[0] for i in range(1, gridsize)]
+        probs = [
+            integrate.quad(func, support[i - 1], support[i], args=endog)[0]
+            for i in range(1, gridsize)
+        ]
         return np.cumsum(probs)
 
     @cache_readonly
@@ -220,9 +262,9 @@ def entropy(self):
         """
         _checkisfit(self)
 
-        def entr(x,s):
-            pdf = kern.density(s,x)
-            return pdf*np.log(pdf+1e-12)
+        def entr(x, s):
+            pdf = kern.density(s, x)
+            return pdf * np.log(pdf + 1e-12)
 
         kern = self.kernel
 
@@ -231,7 +273,7 @@ def entr(x,s):
         else:
             a, b = -np.inf, np.inf
         endog = self.endog
-        #TODO: below could run into integr problems, cf. stats.dist._entropy
+        # TODO: below could run into integr problems, cf. stats.dist._entropy
         return -integrate.quad(entr, a, b, args=(endog,))[0]
 
     @cache_readonly
@@ -261,16 +303,24 @@ def evaluate(self, point):
         return self.kernel.density(self.endog, point)
 
 
-#### Kernel Density Estimator Functions ####
-
-def kdensity(X, kernel="gau", bw="normal_reference", weights=None, gridsize=None,
-             adjust=1, clip=(-np.inf, np.inf), cut=3, retgrid=True):
+# Kernel Density Estimator Functions
+def kdensity(
+    x,
+    kernel="gau",
+    bw="normal_reference",
+    weights=None,
+    gridsize=None,
+    adjust=1,
+    clip=(-np.inf, np.inf),
+    cut=3,
+    retgrid=True,
+):
     """
     Rosenblatt-Parzen univariate kernel density estimator.
 
     Parameters
     ----------
-    X : array_like
+    x : array_like
         The variable for which the density estimate is desired.
     kernel : str
         The Kernel to be used. Choices are
@@ -281,24 +331,38 @@ def kdensity(X, kernel="gau", bw="normal_reference", weights=None, gridsize=None
         - "tri" for triangular
         - "triw" for triweight
         - "uni" for uniform
-    bw : str, float
-        "scott" - 1.059 * A * nobs ** (-1/5.), where A is min(std(X),IQR/1.34)
-        "silverman" - .9 * A * nobs ** (-1/5.), where A is min(std(X),IQR/1.34)
-        If a float is given, it is the bandwidth.
+    bw : str, float, callable
+        The bandwidth to use. Choices are:
+
+        - "scott" - 1.059 * A * nobs ** (-1/5.), where A is
+          `min(std(x),IQR/1.34)`
+        - "silverman" - .9 * A * nobs ** (-1/5.), where A is
+          `min(std(x),IQR/1.34)`
+        - "normal_reference" - C * A * nobs ** (-1/5.), where C is
+          calculated from the kernel. Equivalent (up to 2 dp) to the
+          "scott" bandwidth for gaussian kernels. See bandwidths.py
+        - If a float is given, its value is used as the bandwidth.
+        - If a callable is given, it's return value is used.
+          The callable should take exactly two parameters, i.e.,
+          fn(x, kern), and return a float, where:
+
+          * x - the clipped input data
+          * kern - the kernel instance used
+
     weights : array or None
-        Optional  weights. If the X value is clipped, then this weight is
+        Optional  weights. If the x value is clipped, then this weight is
         also dropped.
     gridsize : int
-        If gridsize is None, max(len(X), 50) is used.
+        If gridsize is None, max(len(x), 50) is used.
     adjust : float
         An adjustment factor for the bw. Bandwidth becomes bw * adjust.
     clip : tuple
-        Observations in X that are outside of the range given by clip are
-        dropped. The number of observations in X is then shortened.
+        Observations in x that are outside of the range given by clip are
+        dropped. The number of observations in x is then shortened.
     cut : float
-        Defines the length of the grid past the lowest and highest values of X
+        Defines the length of the grid past the lowest and highest values of x
         so that the kernel goes to zero. The end points are
-        -/+ cut*bw*{min(X) or max(X)}
+        -/+ cut*bw*{min(x) or max(x)}
     retgrid : bool
         Whether or not to return the grid over which the density is estimated.
 
@@ -314,16 +378,16 @@ def kdensity(X, kernel="gau", bw="normal_reference", weights=None, gridsize=None
     Creates an intermediate (`gridsize` x `nobs`) array. Use FFT for a more
     computationally efficient version.
     """
-    X = np.asarray(X)
-    if X.ndim == 1:
-        X = X[:, None]
-    clip_x = np.logical_and(X > clip[0], X < clip[1])
-    X = X[clip_x]
+    x = np.asarray(x)
+    if x.ndim == 1:
+        x = x[:, None]
+    clip_x = np.logical_and(x > clip[0], x < clip[1])
+    x = x[clip_x]
 
-    nobs = len(X) # after trim
+    nobs = len(x)  # after trim
 
     if gridsize is None:
-        gridsize = max(nobs,50) # do not need to resize if no FFT
+        gridsize = max(nobs, 50)  # do not need to resize if no FFT
 
         # handle weights
     if weights is None:
@@ -334,7 +398,7 @@ def kdensity(X, kernel="gau", bw="normal_reference", weights=None, gridsize=None
         weights = np.asarray(weights)
 
         if len(weights) != len(clip_x):
-            msg = "The length of the weights must be the same as the given X."
+            msg = "The length of the weights must be the same as the given x."
             raise ValueError(msg)
         weights = weights[clip_x.squeeze()]
         q = weights.sum()
@@ -342,48 +406,66 @@ def kdensity(X, kernel="gau", bw="normal_reference", weights=None, gridsize=None
     # Get kernel object corresponding to selection
     kern = kernel_switch[kernel]()
 
-    # if bw is None, select optimal bandwidth for kernel
-    try:
-        bw = float(bw)
-    except:
-        bw = bandwidths.select_bandwidth(X, bw, kern)
+    if callable(bw):
+        bw = float(bw(x, kern))
+        # user passed a callable custom bandwidth function
+    elif isinstance(bw, str):
+        bw = bandwidths.select_bandwidth(x, bw, kern)
+        # will cross-val fit this pattern?
+    else:
+        bw = float_like(bw, "bw")
+
     bw *= adjust
 
-    a = np.min(X, axis=0) - cut * bw
-    b = np.max(X, axis=0) + cut * bw
+    a = np.min(x, axis=0) - cut * bw
+    b = np.max(x, axis=0) + cut * bw
     grid = np.linspace(a, b, gridsize)
 
-    k = (X.T - grid[:, None])/bw  # uses broadcasting to make a gridsize x nobs
+    k = (
+        x.T - grid[:, None]
+    ) / bw  # uses broadcasting to make a gridsize x nobs
 
     # set kernel bandwidth
     kern.seth(bw)
 
     # truncate to domain
-    if kern.domain is not None: # will not work for piecewise kernels like parzen
+    if (
+        kern.domain is not None
+    ):  # will not work for piecewise kernels like parzen
         z_lo, z_high = kern.domain
         domain_mask = (k < z_lo) | (k > z_high)
-        k = kern(k) # estimate density
+        k = kern(k)  # estimate density
         k[domain_mask] = 0
     else:
-        k = kern(k) # estimate density
+        k = kern(k)  # estimate density
 
-    k[k < 0] = 0 # get rid of any negative values, do we need this?
+    k[k < 0] = 0  # get rid of any negative values, do we need this?
 
-    dens = np.dot(k, weights)/(q*bw)
+    dens = np.dot(k, weights) / (q * bw)
 
     if retgrid:
         return dens, grid, bw
     else:
         return dens, bw
 
-def kdensityfft(X, kernel="gau", bw="normal_reference", weights=None, gridsize=None,
-                adjust=1, clip=(-np.inf, np.inf), cut=3, retgrid=True):
+
+def kdensityfft(
+    x,
+    kernel="gau",
+    bw="normal_reference",
+    weights=None,
+    gridsize=None,
+    adjust=1,
+    clip=(-np.inf, np.inf),
+    cut=3,
+    retgrid=True,
+):
     """
     Rosenblatt-Parzen univariate kernel density estimator
 
     Parameters
     ----------
-    X : array_like
+    x : array_like
         The variable for which the density estimate is desired.
     kernel : str
         ONLY GAUSSIAN IS CURRENTLY IMPLEMENTED.
@@ -395,26 +477,40 @@ def kdensityfft(X, kernel="gau", bw="normal_reference", weights=None, gridsize=N
         "par" for Parzen
         "rect" for rectangular
         "tri" for triangular
-    bw : str, float
-        "scott" - 1.059 * A * nobs ** (-1/5.), where A is min(std(X),IQR/1.34)
-        "silverman" - .9 * A * nobs ** (-1/5.), where A is min(std(X),IQR/1.34)
-        If a float is given, it is the bandwidth.
+    bw : str, float, callable
+        The bandwidth to use. Choices are:
+
+        - "scott" - 1.059 * A * nobs ** (-1/5.), where A is
+          `min(std(x),IQR/1.34)`
+        - "silverman" - .9 * A * nobs ** (-1/5.), where A is
+          `min(std(x),IQR/1.34)`
+        - "normal_reference" - C * A * nobs ** (-1/5.), where C is
+          calculated from the kernel. Equivalent (up to 2 dp) to the
+          "scott" bandwidth for gaussian kernels. See bandwidths.py
+        - If a float is given, its value is used as the bandwidth.
+        - If a callable is given, it's return value is used.
+          The callable should take exactly two parameters, i.e.,
+          fn(x, kern), and return a float, where:
+
+          * x - the clipped input data
+          * kern - the kernel instance used
+
     weights : array or None
         WEIGHTS ARE NOT CURRENTLY IMPLEMENTED.
-        Optional  weights. If the X value is clipped, then this weight is
+        Optional  weights. If the x value is clipped, then this weight is
         also dropped.
     gridsize : int
-        If gridsize is None, min(len(X), 512) is used. Note that the provided
+        If gridsize is None, min(len(x), 512) is used. Note that the provided
         number is rounded up to the next highest power of 2.
     adjust : float
         An adjustment factor for the bw. Bandwidth becomes bw * adjust.
         clip : tuple
-        Observations in X that are outside of the range given by clip are
-        dropped. The number of observations in X is then shortened.
+        Observations in x that are outside of the range given by clip are
+        dropped. The number of observations in x is then shortened.
     cut : float
-        Defines the length of the grid past the lowest and highest values of X
+        Defines the length of the grid past the lowest and highest values of x
         so that the kernel goes to zero. The end points are
-        -/+ cut*bw*{X.min() or X.max()}
+        -/+ cut*bw*{x.min() or x.max()}
     retgrid : bool
         Whether or not to return the grid over which the density is estimated.
 
@@ -446,48 +542,54 @@ def kdensityfft(X, kernel="gau", bw="normal_reference", weights=None, gridsize=N
         the Fast Fourier Transform. Journal of the Royal Statistical Society.
         Series C. 31.2, 93-9.
     """
-    X = np.asarray(X)
-    X = X[np.logical_and(X > clip[0], X < clip[1])] # will not work for two columns.
-                                                # will affect underlying data?
+    x = np.asarray(x)
+    # will not work for two columns.
+    x = x[np.logical_and(x > clip[0], x < clip[1])]
 
     # Get kernel object corresponding to selection
     kern = kernel_switch[kernel]()
 
-    try:
-        bw = float(bw)
-    except:
-        bw = bandwidths.select_bandwidth(X, bw, kern) # will cross-val fit this pattern?
+    if callable(bw):
+        bw = float(bw(x, kern))
+        # user passed a callable custom bandwidth function
+    elif isinstance(bw, str):
+        # if bw is None, select optimal bandwidth for kernel
+        bw = bandwidths.select_bandwidth(x, bw, kern)
+        # will cross-val fit this pattern?
+    else:
+        bw = float_like(bw, "bw")
+
     bw *= adjust
 
-    nobs = len(X) # after trim
+    nobs = len(x)  # after trim
 
     # 1 Make grid and discretize the data
     if gridsize is None:
-        gridsize = np.max((nobs, 512.))
-    gridsize = 2**np.ceil(np.log2(gridsize)) # round to next power of 2
+        gridsize = np.max((nobs, 512.0))
+    gridsize = 2 ** np.ceil(np.log2(gridsize))  # round to next power of 2
 
-    a = np.min(X) - cut * bw
-    b = np.max(X) + cut * bw
-    grid,delta = np.linspace(a, b, int(gridsize), retstep=True)
+    a = np.min(x) - cut * bw
+    b = np.max(x) + cut * bw
+    grid, delta = np.linspace(a, b, int(gridsize), retstep=True)
     RANGE = b - a
 
-#TODO: Fix this?
-# This is the Silverman binning function, but I believe it's buggy (SS)
-# weighting according to Silverman
-#    count = counts(X,grid)
-#    binned = np.zeros_like(grid)    #xi_{k} in Silverman
-#    j = 0
-#    for k in range(int(gridsize-1)):
-#        if count[k]>0: # there are points of X in the grid here
-#            Xingrid = X[j:j+count[k]] # get all these points
-#            # get weights at grid[k],grid[k+1]
-#            binned[k] += np.sum(grid[k+1]-Xingrid)
-#            binned[k+1] += np.sum(Xingrid-grid[k])
-#            j += count[k]
-#    binned /= (nobs)*delta**2 # normalize binned to sum to 1/delta
-
-#NOTE: THE ABOVE IS WRONG, JUST TRY WITH LINEAR BINNING
-    binned = fast_linbin(X, a, b, gridsize) / (delta * nobs)
+    # TODO: Fix this?
+    # This is the Silverman binning function, but I believe it's buggy (SS)
+    # weighting according to Silverman
+    #    count = counts(x,grid)
+    #    binned = np.zeros_like(grid)    #xi_{k} in Silverman
+    #    j = 0
+    #    for k in range(int(gridsize-1)):
+    #        if count[k]>0: # there are points of x in the grid here
+    #            Xingrid = x[j:j+count[k]] # get all these points
+    #            # get weights at grid[k],grid[k+1]
+    #            binned[k] += np.sum(grid[k+1]-Xingrid)
+    #            binned[k+1] += np.sum(Xingrid-grid[k])
+    #            j += count[k]
+    #    binned /= (nobs)*delta**2 # normalize binned to sum to 1/delta
+
+    # NOTE: THE ABOVE IS WRONG, JUST TRY WITH LINEAR BINNING
+    binned = fast_linbin(x, a, b, gridsize) / (delta * nobs)
 
     # step 2 compute FFT of the weights, using Munro (1976) FFT convention
     y = forrt(binned)
@@ -495,10 +597,11 @@ def kdensityfft(X, kernel="gau", bw="normal_reference", weights=None, gridsize=N
     # step 3 and 4 for optimal bw compute zstar and the density estimate f
     # do not have to redo the above if just changing bw, ie., for cross val
 
-#NOTE: silverman_transform is the closed form solution of the FFT of the
-#gaussian kernel. Not yet sure how to generalize it.
-    zstar = silverman_transform(bw, gridsize, RANGE)*y # 3.49 in Silverman
-                                                    # 3.50 w Gaussian kernel
+    # NOTE: silverman_transform is the closed form solution of the FFT of the
+    # gaussian kernel. Not yet sure how to generalize it.
+    zstar = silverman_transform(bw, gridsize, RANGE) * y
+    # 3.49 in Silverman
+    # 3.50 w Gaussian kernel
     f = revrt(zstar)
     if retgrid:
         return f, grid, bw


Test patch:
diff --git a/statsmodels/nonparametric/tests/test_kde.py b/statsmodels/nonparametric/tests/test_kde.py
--- a/statsmodels/nonparametric/tests/test_kde.py
+++ b/statsmodels/nonparametric/tests/test_kde.py
@@ -8,6 +8,7 @@
 from statsmodels.distributions.mixture_rvs import mixture_rvs
 from statsmodels.nonparametric.kde import KDEUnivariate as KDE
 import statsmodels.sandbox.nonparametric.kernels as kernels
+import statsmodels.nonparametric.bandwidths as bandwidths
 
 # get results from Stata
 
@@ -348,3 +349,42 @@ def test_fit_self(reset_randomstate):
     kde = KDE(x)
     assert isinstance(kde, KDE)
     assert isinstance(kde.fit(), KDE)
+
+
+class TestKDECustomBandwidth(object):
+    decimal_density = 7
+
+    @classmethod
+    def setup_class(cls):
+        cls.kde = KDE(Xi)
+        cls.weights_200 = np.linspace(1, 100, 200)
+        cls.weights_100 = np.linspace(1, 100, 100)
+
+    def test_check_is_fit_ok_with_custom_bandwidth(self):
+        def custom_bw(X, kern):
+            return np.std(X) * len(X)
+        kde = self.kde.fit(bw=custom_bw)
+        assert isinstance(kde, KDE)
+
+    def test_check_is_fit_ok_with_standard_custom_bandwidth(self):
+        # Note, we are passing the function, not the string - this is intended
+        kde = self.kde.fit(bw=bandwidths.bw_silverman)
+        s1 = kde.support.copy()
+        d1 = kde.density.copy()
+
+        kde = self.kde.fit(bw='silverman')
+
+        npt.assert_almost_equal(s1, kde.support, self.decimal_density)
+        npt.assert_almost_equal(d1, kde.density, self.decimal_density)
+
+    @pytest.mark.parametrize("fft", [True, False])
+    def test_check_is_fit_ok_with_float_bandwidth(self, fft):
+        # Note, we are passing the function, not the string - this is intended
+        kde = self.kde.fit(bw=bandwidths.bw_silverman, fft=fft)
+        s1 = kde.support.copy()
+        d1 = kde.density.copy()
+
+        kde = self.kde.fit(bw=kde.bw, fft=fft)
+
+        npt.assert_almost_equal(s1, kde.support, self.decimal_density)
+        npt.assert_almost_equal(d1, kde.density, self.decimal_density)