Repository: kornia/kornia

Issue:
[Feat] Discrete Gaussian kernel (gaussian) should be adapted for discrete filtering
## 🐛 Bug
The [current Gaussian kernel](https://github.com/kornia/kornia/blob/9a49b3c473fbbef6df1e0f245f8e99e36f6fec6a/kornia/filters/kernels.py#L19) simply samples the continuous Gaussian function:
```python
def gaussian(window_size, sigma):
    x = torch.arange(window_size).float() - window_size // 2
    if window_size % 2 == 0:
        x = x + 0.5
    gauss = torch.exp((-x.pow(2.0) / float(2 * sigma ** 2)))
    return gauss / gauss.sum()
```

## Expected behavior
Discrete Gaussian filtering should be compatible with scale-space theory and rely on modified Bessel function for its kernel weights. See e.g.:
https://en.wikipedia.org/wiki/Scale_space_implementation#The_discrete_Gaussian_kernel

## Additional context
Note that this is a similar issue to that raised in MONAI:
https://github.com/Project-MONAI/MONAI/issues/363

I came here while looking for a PyTorch implementation of a Discrete Gaussian filter but realised this one was also sampling the continuous Gaussian function.

ITK implements such a behaviour in itk::GaussianOperator:
https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Core/Common/include/itkGaussianOperator.hxx

[Wikipedia](https://commons.wikimedia.org/wiki/File:Discrete_Gaussian_kernel.svg) also provides a basic python implementation (albeit relying on scipy and without normalisation):
```python
from scipy.special import iv
import numpy as np

def discrete_gaussian_kernel(n, t):
    T = np.exp(-t) * iv(n, t)
    return T
```


Discussion:
@tvercaut thanks for pointing this out. I wasn't aware of this problem. How will this affect to the actual implementation ? From what I understand while reading in wikipedia seems that methods using discrete kernel can be more robust during differentiation. If that's true, why not supporting it.

To implement this is in pytorch we need to double check that we can reproduce [`scipy.iv`](https://github.com/kornia/kornia/issues/559 ) as well. After a quick search I found some related comments on that already from pytorch community showing already some ideas. [link1](https://discuss.pytorch.org/t/modified-bessel-function-of-order-0/18609) [link2](https://github.com/nicola-decao/s-vae-pytorch/blob/671c46c0772a2c92a602252311864c48c816bc44/hyperspherical_vae/ops/ive.py). Just found that also JAX implements [Modified Bessel kernel](https://jax.readthedocs.io/en/latest/_autosummary/jax.scipy.special.i1e.html#jax.scipy.special.i1e)

Besides, we were discussing with @wyli to potentially merge some functionalities under a common API. Probably. this could one first step for this initiative.
I will check the actual difference. In case of we are doing the change, we also would need to modify our spatial_gradient kernels. 
And it should help with local features localization precision as well.
As shown in the wikipedia image, the difference is subtle but noticeable especially for small sigmas.

Thinking a bit further, I realise some use cases might need to get the  gradient with respect to sigma. In that scenario, I assume a full pytorch implementation (not relying on scipy) is important. Also, it would be important to check the numerical stability of such gradients as the involved operations become more convoluted.

Maybe having both the sampled Gausian kernel and the proper discrete Gaussian kernel makes sense.
Regardless of the gradients, we cannot rely on scipy anyway - our policy is to be PyTorch our only dependency. 
Great, then porting a set of modified Bessel functions would be the way to go. The code in ITK for these is self contained and could thus be a good start:
https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Core/Common/include/itkGaussianOperator.h (for documentation and default parameters)
https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Core/Common/include/itkGaussianOperator.hxx (for actual implementation)
@tvercaut great ! how does compare scipy vs itk implementation ? I would target scipy behavior since seems to be a more popular package.
> @tvercaut great ! how does compare scipy vs itk implementation ? I would target scipy behavior since seems to be a more popular package.

`scipy.special.iv` is also interesting https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.iv.html
https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.ive.html
https://github.com/scipy/scipy/issues/1030
@wyli @tvercaut do you think you might have some time to implement this feature. We can give support for integration
/cc @ducha-aiki check: https://github.com/Project-MONAI/MONAI/pull/1020
> do you think you might have some time to implement this feature. We can give support for integration

Hi @edgarriba I'm pushing a refined gaussian1d to monai,
but this implementation is not differentiable https://github.com/Project-MONAI/MONAI/pull/1097


@wyli I don't see why the implementation is not differentiable. I mean, kernel generation does not need to be differentiable, we just need weight to do convolution then. Being differentiable w.r.t. sigma is nice, but not must
> @wyli I don't see why the implementation is not differentiable. I mean, kernel generation does not to be differentiable, we just need weight to do convolution then.

Yes, it is differentiable w.r.t sigma :) there were some errors in the recursion, the latest commit fixes it and seems to work fine. I'm adding some unit tests about the gradient to confirm it. 
@wyli @tvercaut do you already have any working real example showing the difference for using continuous and discrete kernels ?
@edgarriba If you only care about smoothing an image somewhat, any bell shaped kernel will do the trick. For many (most?) use cases, I assume this won't make any noticeable difference.

If you need more fine-grained scale-space analysis, the difference between the sampled Gaussian and the discrete Gaussian however can start to matter. For example applying N times a Gaussian with standard deviation σ may not be the same as aplying a Gaussian of standard deviation  √N·σ with the sampled Gaussian. It becomes also important when looking at the stability under a derivative operators for example (typically Laplacian). If you are intersted, I encourage you to read the Lindeberg paper for a good coverage: https://doi.org/10.1109/34.49051
@tvercaut I'll checkout thanks :)
@wyli we can try to generalize the functionalities and have `kornia.filter.gaussian` and `kornia.filter.gaussian_discrete` ? We are releasing in a month or v0.5. We could introduce this as part of kornia by that time in case you consider that can be useful.
> @tvercaut I'll checkout thanks :)
> @wyli we can try to generalize the functionalities and have `kornia.filter.gaussian` and `kornia.filter.gaussian_discrete` ? We are releasing in a month or v0.5. We could introduce this as part of kornia by that time in case you consider that can be useful.

sure, I can try to create a PR as a non-breaking new feature to kornia, or help review a PR if someone is already working on this ticket
@wyli that sounds good we could try to target to include this feature for v0.5

Resolving patch:
diff --git a/kornia/__init__.py b/kornia/__init__.py
--- a/kornia/__init__.py
+++ b/kornia/__init__.py
@@ -71,6 +71,8 @@
 )
 from kornia.filters import (
     get_gaussian_kernel1d,
+    get_gaussian_erf_kernel1d,
+    get_gaussian_discrete_kernel1d,
     get_gaussian_kernel2d,
     get_laplacian_kernel1d,
     get_laplacian_kernel2d,
diff --git a/kornia/filters/__init__.py b/kornia/filters/__init__.py
--- a/kornia/filters/__init__.py
+++ b/kornia/filters/__init__.py
@@ -15,6 +15,8 @@
     get_diff_kernel2d,
     get_spatial_gradient_kernel2d,
     get_gaussian_kernel1d,
+    get_gaussian_discrete_kernel1d,
+    get_gaussian_erf_kernel1d,
     get_gaussian_kernel2d,
     get_laplacian_kernel1d,
     get_laplacian_kernel2d,
@@ -24,6 +26,8 @@
 
 __all__ = [
     "get_gaussian_kernel1d",
+    "get_gaussian_discrete_kernel1d",
+    "get_gaussian_erf_kernel1d",
     "get_gaussian_kernel2d",
     "get_laplacian_kernel1d",
     "get_laplacian_kernel2d",
diff --git a/kornia/filters/kernels.py b/kornia/filters/kernels.py
--- a/kornia/filters/kernels.py
+++ b/kornia/filters/kernels.py
@@ -1,7 +1,7 @@
-from typing import Tuple, List, Union, cast
+from typing import Tuple, List, Union, cast, Optional
 
 import torch
-import torch.nn as nn
+from math import sqrt
 
 from kornia.geometry.transform.affwarp import rotate, rotate3d
 
@@ -24,6 +24,99 @@ def gaussian(window_size, sigma):
     return gauss / gauss.sum()
 
 
+def gaussian_discrete_erf(window_size, sigma):
+    r"""Discrete Gaussian by interpolating the error function. Adapted from:
+    https://github.com/Project-MONAI/MONAI/blob/master/monai/networks/layers/convutils.py
+    """
+    device = sigma.device if isinstance(sigma, torch.Tensor) else None
+    sigma = torch.as_tensor(sigma, dtype=torch.float, device=device)
+    x = torch.arange(window_size).float() - window_size // 2
+    t = 0.70710678 / torch.abs(sigma)
+    gauss = 0.5 * ((t * (x + 0.5)).erf() - (t * (x - 0.5)).erf())
+    gauss = gauss.clamp(min=0)
+    return gauss / gauss.sum()
+
+
+def _modified_bessel_0(x: torch.Tensor) -> torch.Tensor:
+    r"""Adapted from:
+    https://github.com/Project-MONAI/MONAI/blob/master/monai/networks/layers/convutils.py
+    """
+    if torch.abs(x) < 3.75:
+        y = (x / 3.75) * (x / 3.75)
+        return 1.0 + y * (
+            3.5156229 + y * (3.0899424 + y * (1.2067492 + y * (0.2659732 + y * (0.360768e-1 + y * 0.45813e-2))))
+        )
+    ax = torch.abs(x)
+    y = 3.75 / ax
+    ans = 0.916281e-2 + y * (-0.2057706e-1 + y * (0.2635537e-1 + y * (-0.1647633e-1 + y * 0.392377e-2)))
+    return (torch.exp(ax) / torch.sqrt(ax)) * (
+        0.39894228 + y * (0.1328592e-1 + y * (0.225319e-2 + y * (-0.157565e-2 + y * ans)))
+    )
+
+
+def _modified_bessel_1(x: torch.Tensor) -> torch.Tensor:
+    r"""adapted from:
+    https://github.com/Project-MONAI/MONAI/blob/master/monai/networks/layers/convutils.py
+    """
+    if torch.abs(x) < 3.75:
+        y = (x / 3.75) * (x / 3.75)
+        ans = 0.51498869 + y * (0.15084934 + y * (0.2658733e-1 + y * (0.301532e-2 + y * 0.32411e-3)))
+        return torch.abs(x) * (0.5 + y * (0.87890594 + y * ans))
+    ax = torch.abs(x)
+    y = 3.75 / ax
+    ans = 0.2282967e-1 + y * (-0.2895312e-1 + y * (0.1787654e-1 - y * 0.420059e-2))
+    ans = 0.39894228 + y * (-0.3988024e-1 + y * (-0.362018e-2 + y * (0.163801e-2 + y * (-0.1031555e-1 + y * ans))))
+    ans = ans * torch.exp(ax) / torch.sqrt(ax)
+    return -ans if x < 0.0 else ans
+
+
+def _modified_bessel_i(n: int, x: torch.Tensor) -> torch.Tensor:
+    r"""adapted from:
+    https://github.com/Project-MONAI/MONAI/blob/master/monai/networks/layers/convutils.py
+    """
+    if n < 2:
+        raise ValueError("n must be greater than 1.")
+    if x == 0.0:
+        return x
+    device = x.device
+    tox = 2.0 / torch.abs(x)
+    ans = torch.tensor(0.0, device=device)
+    bip = torch.tensor(0.0, device=device)
+    bi = torch.tensor(1.0, device=device)
+    m = int(2 * (n + int(sqrt(40.0 * n))))
+    for j in range(m, 0, -1):
+        bim = bip + float(j) * tox * bi
+        bip = bi
+        bi = bim
+        if abs(bi) > 1.0e10:
+            ans = ans * 1.0e-10
+            bi = bi * 1.0e-10
+            bip = bip * 1.0e-10
+        if j == n:
+            ans = bip
+    ans = ans * _modified_bessel_0(x) / bi
+    return -ans if x < 0.0 and (n % 2) == 1 else ans
+
+
+def gaussian_discrete(window_size, sigma) -> torch.Tensor:
+    r"""Discrete Gaussian kernel based on the modified Bessel functions. Adapted from:
+    https://github.com/Project-MONAI/MONAI/blob/master/monai/networks/layers/convutils.py
+    """
+    device = sigma.device if isinstance(sigma, torch.Tensor) else None
+    sigma = torch.as_tensor(sigma, dtype=torch.float, device=device)
+    sigma2 = sigma * sigma
+    tail = int(window_size // 2)
+    out_pos: List[Optional[torch.Tensor]] = [None] * (tail + 1)
+    out_pos[0] = _modified_bessel_0(sigma2)
+    out_pos[1] = _modified_bessel_1(sigma2)
+    for k in range(2, len(out_pos)):
+        out_pos[k] = _modified_bessel_i(k, sigma2)
+    out = out_pos[:0:-1]
+    out.extend(out_pos)
+    out = torch.stack(out) * torch.exp(sigma2)  # type: ignore
+    return out / out.sum()  # type: ignore
+
+
 def laplacian_1d(window_size) -> torch.Tensor:
     r"""One could also use the Laplacian of Gaussian formula
         to design the filter.
@@ -309,10 +402,10 @@ def get_gaussian_kernel1d(kernel_size: int,
 
     Examples::
 
-        >>> kornia.image.get_gaussian_kernel(3, 2.5)
+        >>> kornia.filters.get_gaussian_kernel1d(3, 2.5)
         tensor([0.3243, 0.3513, 0.3243])
 
-        >>> kornia.image.get_gaussian_kernel(5, 1.5)
+        >>> kornia.filters.get_gaussian_kernel1d(5, 1.5)
         tensor([0.1201, 0.2339, 0.2921, 0.2339, 0.1201])
     """
     if (not isinstance(kernel_size, int) or (
@@ -326,6 +419,80 @@ def get_gaussian_kernel1d(kernel_size: int,
     return window_1d
 
 
+def get_gaussian_discrete_kernel1d(kernel_size: int,
+                                   sigma: float,
+                                   force_even: bool = False) -> torch.Tensor:
+    r"""Function that returns Gaussian filter coefficients
+    based on the modified Bessel functions. Adapted from:
+    https://github.com/Project-MONAI/MONAI/blob/master/monai/networks/layers/convutils.py
+
+    Args:
+        kernel_size (int): filter size. It should be odd and positive.
+        sigma (float): gaussian standard deviation.
+        force_even (bool): overrides requirement for odd kernel size.
+
+    Returns:
+        Tensor: 1D tensor with gaussian filter coefficients.
+
+    Shape:
+        - Output: :math:`(\text{kernel_size})`
+
+    Examples::
+
+        >>> kornia.filters.get_gaussian_discrete_kernel1d(3, 2.5)
+        tensor([0.3235, 0.3531, 0.3235])
+
+        >>> kornia.filters.get_gaussian_discrete_kernel1d(5, 1.5)
+        tensor([0.1096, 0.2323, 0.3161, 0.2323, 0.1096])
+    """
+    if (not isinstance(kernel_size, int) or (
+            (kernel_size % 2 == 0) and not force_even) or (
+            kernel_size <= 0)):
+        raise TypeError(
+            "kernel_size must be an odd positive integer. "
+            "Got {}".format(kernel_size)
+        )
+    window_1d = gaussian_discrete(kernel_size, sigma)
+    return window_1d
+
+
+def get_gaussian_erf_kernel1d(kernel_size: int,
+                              sigma: float,
+                              force_even: bool = False) -> torch.Tensor:
+    r"""Function that returns Gaussian filter coefficients by interpolating the error fucntion,
+    adapted from:
+    https://github.com/Project-MONAI/MONAI/blob/master/monai/networks/layers/convutils.py
+
+    Args:
+        kernel_size (int): filter size. It should be odd and positive.
+        sigma (float): gaussian standard deviation.
+        force_even (bool): overrides requirement for odd kernel size.
+
+    Returns:
+        Tensor: 1D tensor with gaussian filter coefficients.
+
+    Shape:
+        - Output: :math:`(\text{kernel_size})`
+
+    Examples::
+
+        >>> kornia.filters.get_gaussian_erf_kernel1d(3, 2.5)
+        tensor([0.3245, 0.3511, 0.3245])
+
+        >>> kornia.filters.get_gaussian_erf_kernel1d(5, 1.5)
+        tensor([0.1226, 0.2331, 0.2887, 0.2331, 0.1226])
+    """
+    if (not isinstance(kernel_size, int) or (
+            (kernel_size % 2 == 0) and not force_even) or (
+            kernel_size <= 0)):
+        raise TypeError(
+            "kernel_size must be an odd positive integer. "
+            "Got {}".format(kernel_size)
+        )
+    window_1d = gaussian_discrete_erf(kernel_size, sigma)
+    return window_1d
+
+
 def get_gaussian_kernel2d(
         kernel_size: Tuple[int, int],
         sigma: Tuple[float, float],
@@ -347,12 +514,12 @@ def get_gaussian_kernel2d(
 
     Examples::
 
-        >>> kornia.image.get_gaussian_kernel2d((3, 3), (1.5, 1.5))
+        >>> kornia.filters.get_gaussian_kernel2d((3, 3), (1.5, 1.5))
         tensor([[0.0947, 0.1183, 0.0947],
                 [0.1183, 0.1478, 0.1183],
                 [0.0947, 0.1183, 0.0947]])
 
-        >>> kornia.image.get_gaussian_kernel2d((3, 5), (1.5, 1.5))
+        >>> kornia.filters.get_gaussian_kernel2d((3, 5), (1.5, 1.5))
         tensor([[0.0370, 0.0720, 0.0899, 0.0720, 0.0370],
                 [0.0462, 0.0899, 0.1123, 0.0899, 0.0462],
                 [0.0370, 0.0720, 0.0899, 0.0720, 0.0370]])
@@ -390,10 +557,10 @@ def get_laplacian_kernel1d(kernel_size: int) -> torch.Tensor:
         - Output: math:`(\text{kernel_size})`
 
     Examples::
-        >>> kornia.image.get_laplacian_kernel(3)
+        >>> kornia.filters.get_laplacian_kernel(3)
         tensor([ 1., -2.,  1.])
 
-        >>> kornia.image.get_laplacian_kernel(5)
+        >>> kornia.filters.get_laplacian_kernel(5)
         tensor([ 1.,  1., -4.,  1.,  1.])
 
     """
@@ -419,12 +586,12 @@ def get_laplacian_kernel2d(kernel_size: int) -> torch.Tensor:
 
     Examples::
 
-        >>> kornia.image.get_laplacian_kernel2d(3)
+        >>> kornia.filters.get_laplacian_kernel2d(3)
         tensor([[ 1.,  1.,  1.],
                 [ 1., -8.,  1.],
                 [ 1.,  1.,  1.]])
 
-        >>> kornia.image.get_laplacian_kernel2d(5)
+        >>> kornia.filters.get_laplacian_kernel2d(5)
         tensor([[  1.,   1.,   1.,   1.,   1.],
                 [  1.,   1.,   1.,   1.,   1.],
                 [  1.,   1., -24.,   1.,   1.],


Test patch:
diff --git a/test/filters/test_gaussian.py b/test/filters/test_gaussian.py
--- a/test/filters/test_gaussian.py
+++ b/test/filters/test_gaussian.py
@@ -16,6 +16,22 @@ def test_get_gaussian_kernel(window_size, sigma):
     assert kernel.sum().item() == pytest.approx(1.0)
 
 
+@pytest.mark.parametrize("window_size", [5, 11])
+@pytest.mark.parametrize("sigma", [1.5, 5.0])
+def test_get_gaussian_discrete_kernel(window_size, sigma):
+    kernel = kornia.get_gaussian_discrete_kernel1d(window_size, sigma)
+    assert kernel.shape == (window_size,)
+    assert kernel.sum().item() == pytest.approx(1.0)
+
+
+@pytest.mark.parametrize("window_size", [5, 11])
+@pytest.mark.parametrize("sigma", [1.5, 5.0])
+def test_get_gaussian_erf_kernel(window_size, sigma):
+    kernel = kornia.get_gaussian_erf_kernel1d(window_size, sigma)
+    assert kernel.shape == (window_size,)
+    assert kernel.sum().item() == pytest.approx(1.0)
+
+
 @pytest.mark.parametrize("ksize_x", [5, 11])
 @pytest.mark.parametrize("ksize_y", [3, 7])
 @pytest.mark.parametrize("sigma", [1.5, 2.1])