Repository: kornia/kornia
Pull request: Add more border_type options for filter2D
#### Changes

Fixes #973 

Added valid padding option as requested in #973.
Added tests for the same.

#### Type of change
<!-- Please delete options that are not relevant. -->
- [x] 🧪 Tests Cases
- [x] 🔬 New feature (non-breaking change which adds functionality)
- [x] 📝 This change requires a documentation update


Problem statement:
Add more border_type options for filter2D
## 🚀 Feature
1. For filter2D, add border_types that are provided by **filter2** of **Matlab**.
    For example, border_type of 'valid':
      ```
      %     'valid' - returns only those parts of the correlation
      %               that are computed without the zero-padded
      %               edges, size(Y) < size(X).
      ```
2. If possible, add a border_type option for some loss functions (e.g. ssim_loss). This option is used to control the behavior of filter2D in ssim_loss.

## Motivation
In my field, some metrics were written by Matlab, and I have to migrate these codes into PyTorch. The issue is, the Matlab code used a border_type(i.e., "valid") that is not supported by Kornia, resulting in different scores between Matlab-version and PyTorch-version.

## Pitch
filter2D function will support "valid" border type.

## Alternatives
Don't think of any alternative solutions.

## Additional context

Hints from issue discussion:
If I understand it right, 'valid' should not be a ```border_type``` here but a padding strategy similar to tensorflow's ```valid padding``` and ```same padding```. Meanwhile, what has been implemented in kornia is the ```same padding```? If everything above is correct, I think the ```valid padding``` could be a good feature to add.
@tlsshh @shijianjian go for it, please provide a concise api for that
@tlsshh Are you interested in sending a PR regarding the ```valid``` padding?
> @tlsshh Are you interested in sending a PR regarding the `valid` padding?

I would like to ... but I am not clear about what I should do, and I never programmed anything like that before ... 
@tlsshh join our slack - we'll guide you
> @tlsshh join our slack - we'll guide you

Ok. I joined the slack. What should I do next?
join #mentoring channel and let's discuss there

Commit message: Implemented feature requested in issue #973
Added tests for the same

Patch to review:
diff --git a/kornia/filters/filter.py b/kornia/filters/filter.py
index 15d2f2e530..c5aac40b83 100644
--- a/kornia/filters/filter.py
+++ b/kornia/filters/filter.py
@@ -31,7 +31,8 @@ def _compute_padding(kernel_size: List[int]) -> List[int]:
 
 
 def filter2d(
-    input: torch.Tensor, kernel: torch.Tensor, border_type: str = 'reflect', normalized: bool = False
+    input: torch.Tensor, kernel: torch.Tensor, border_type: str = 'reflect', normalized: bool = False,
+    padding: str = 'same'
 ) -> torch.Tensor:
     r"""Convolve a tensor with a 2d kernel.
 
@@ -49,6 +50,8 @@ def filter2d(
           The expected modes are: ``'constant'``, ``'reflect'``,
           ``'replicate'`` or ``'circular'``.
         normalized: If True, kernel will be L1 normalized.
+        padding: This defines the type of padding.
+          2 modes available ``'same'`` or ``'valid'``
 
     Return:
         torch.Tensor: the convolved tensor of same size and numbers of channels
@@ -62,7 +65,7 @@ def filter2d(
         ...    [0., 0., 0., 0., 0.],
         ...    [0., 0., 0., 0., 0.],]]])
         >>> kernel = torch.ones(1, 3, 3)
-        >>> filter2d(input, kernel)
+        >>> filter2d(input, kernel, padding='same')
         tensor([[[[0., 0., 0., 0., 0.],
                   [0., 5., 5., 5., 0.],
                   [0., 5., 5., 5., 0.],
@@ -78,6 +81,9 @@ def filter2d(
     if not isinstance(border_type, str):
         raise TypeError(f"Input border_type is not string. Got {type(kernel)}")
 
+    if not isinstance(padding, str):
+        raise TypeError(f"Input padding is not string. Got {type(padding)}")
+
     if not len(input.shape) == 4:
         raise ValueError(f"Invalid input shape, we expect BxCxHxW. Got: {input.shape}")
 
@@ -93,19 +99,24 @@ def filter2d(
 
     tmp_kernel = tmp_kernel.expand(-1, c, -1, -1)
 
-    # pad the input tensor
     height, width = tmp_kernel.shape[-2:]
-    padding_shape: List[int] = _compute_padding([height, width])
-    input_pad: torch.Tensor = F.pad(input, padding_shape, mode=border_type)
+
+    # pad the input tensor
+    if padding == 'same':
+        padding_shape: List[int] = _compute_padding([height, width])
+        input = F.pad(input, padding_shape, mode=border_type)
 
     # kernel and input tensor reshape to align element-wise or batch-wise params
     tmp_kernel = tmp_kernel.reshape(-1, 1, height, width)
-    input_pad = input_pad.view(-1, tmp_kernel.size(0), input_pad.size(-2), input_pad.size(-1))
+    input = input.view(-1, tmp_kernel.size(0), input.size(-2), input.size(-1))
 
     # convolve the tensor with the kernel.
-    output = F.conv2d(input_pad, tmp_kernel, groups=tmp_kernel.size(0), padding=0, stride=1)
+    output = F.conv2d(input, tmp_kernel, groups=tmp_kernel.size(0), padding=0, stride=1)
 
-    return output.view(b, c, h, w)
+    if padding == 'same':
+        return output.view(b, c, h, w)
+    else:
+        return output.view(b, c, h - height + 1, w - width + 1)
 
 
 def filter3d(
diff --git a/test/filters/test_filters.py b/test/filters/test_filters.py
index 23bdb35654..fb23d26785 100644
--- a/test/filters/test_filters.py
+++ b/test/filters/test_filters.py
@@ -8,19 +8,34 @@
 
 
 class TestFilter2D:
-    def test_smoke(self, device, dtype):
+    @pytest.mark.parametrize("padding", ["same", "valid"])
+    def test_smoke(self, padding, device, dtype):
         kernel = torch.rand(1, 3, 3, device=device, dtype=dtype)
+        _, height, width = kernel.shape
         input = torch.ones(1, 1, 7, 8, device=device, dtype=dtype)
-        assert kornia.filter2d(input, kernel).shape == input.shape
+        b, c, h, w = input.shape
+        if padding == 'same':
+            assert kornia.filter2d(input, kernel, padding=padding).shape == (b, c, h, w)
+        else:
+            assert kornia.filter2d(
+                input, kernel, padding=padding).shape == (b, c, h - ((height // 2) * 2), w - ((width // 2) * 2))
 
     @pytest.mark.parametrize("batch_size", [2, 3, 6, 8])
-    def test_batch(self, batch_size, device, dtype):
+    @pytest.mark.parametrize("padding", ["same", "valid"])
+    def test_batch(self, batch_size, padding, device, dtype):
         B: int = batch_size
         kernel = torch.rand(1, 3, 3, device=device, dtype=dtype)
+        _, height, width = kernel.shape
         input = torch.ones(B, 3, 7, 8, device=device, dtype=dtype)
-        assert kornia.filter2d(input, kernel).shape == input.shape
-
-    def test_mean_filter(self, device, dtype):
+        b, c, h, w = input.shape
+        if padding == 'same':
+            assert kornia.filter2d(input, kernel, padding=padding).shape == (b, c, h, w)
+        else:
+            assert kornia.filter2d(
+                input, kernel, padding=padding).shape == (b, c, h - ((height // 2) * 2), w - ((width // 2) * 2))
+
+    @pytest.mark.parametrize("padding", ["same", "valid"])
+    def test_mean_filter(self, padding, device, dtype):
         kernel = torch.ones(1, 3, 3, device=device, dtype=dtype)
         input = torch.tensor(
             [
@@ -37,7 +52,7 @@ def test_mean_filter(self, device, dtype):
             device=device,
             dtype=dtype,
         )
-        expected = torch.tensor(
+        expected_same = torch.tensor(
             [
                 [
                     [
@@ -53,10 +68,28 @@ def test_mean_filter(self, device, dtype):
             dtype=dtype,
         )
 
-        actual = kornia.filter2d(input, kernel)
-        assert_close(actual, expected)
+        expected_valid = torch.tensor(
+            [
+                [
+                    [
+                        [5.0, 5.0, 5.0],
+                        [5.0, 5.0, 5.0],
+                        [5.0, 5.0, 5.0],
+                    ]
+                ]
+            ],
+            device=device,
+            dtype=dtype,
+        )
 
-    def test_mean_filter_2batch_2ch(self, device, dtype):
+        actual = kornia.filter2d(input, kernel, padding=padding)
+        if padding == 'same':
+            assert_close(actual, expected_same)
+        else:
+            assert_close(actual, expected_valid)
+
+    @pytest.mark.parametrize("padding", ["same", "valid"])
+    def test_mean_filter_2batch_2ch(self, padding, device, dtype):
         kernel = torch.ones(1, 3, 3, device=device, dtype=dtype)
         input = torch.tensor(
             [
@@ -74,7 +107,7 @@ def test_mean_filter_2batch_2ch(self, device, dtype):
             dtype=dtype,
         ).expand(2, 2, -1, -1)
 
-        expected = torch.tensor(
+        expected_same = torch.tensor(
             [
                 [
                     [
@@ -90,10 +123,28 @@ def test_mean_filter_2batch_2ch(self, device, dtype):
             dtype=dtype,
         ).expand(2, 2, -1, -1)
 
-        actual = kornia.filter2d(input, kernel)
-        assert_close(actual, expected)
+        expected_valid = torch.tensor(
+            [
+                [
+                    [
+                        [5.0, 5.0, 5.0],
+                        [5.0, 5.0, 5.0],
+                        [5.0, 5.0, 5.0],
+                    ]
+                ]
+            ],
+            device=device,
+            dtype=dtype,
+        ).expand(2, 2, -1, -1)
 
-    def test_normalized_mean_filter(self, device, dtype):
+        actual = kornia.filter2d(input, kernel, padding=padding)
+        if padding == 'same':
+            assert_close(actual, expected_same)
+        else:
+            assert_close(actual, expected_valid)
+
+    @pytest.mark.parametrize("padding", ["same", "valid"])
+    def test_normalized_mean_filter(self, padding, device, dtype):
         kernel = torch.ones(1, 3, 3).to(device)
         input = torch.tensor(
             [
@@ -112,7 +163,7 @@ def test_normalized_mean_filter(self, device, dtype):
         ).expand(2, 2, -1, -1)
 
         nv: float = 5.0 / 9  # normalization value
-        expected = torch.tensor(
+        expected_same = torch.tensor(
             [
                 [
                     [
@@ -128,12 +179,30 @@ def test_normalized_mean_filter(self, device, dtype):
             dtype=dtype,
         ).expand(2, 2, -1, -1)
 
-        actual = kornia.filter2d(input, kernel, normalized=True)
+        expected_valid = torch.tensor(
+            [
+                [
+                    [
+                        [nv, nv, nv],
+                        [nv, nv, nv],
+                        [nv, nv, nv],
+                    ]
+                ]
+            ],
+            device=device,
+            dtype=dtype,
+        ).expand(2, 2, -1, -1)
+
+        actual = kornia.filter2d(input, kernel, normalized=True, padding=padding)
 
         tol_val: float = utils._get_precision_by_name(device, 'xla', 1e-1, 1e-4)
-        assert_close(actual, expected, rtol=tol_val, atol=tol_val)
+        if padding == 'same':
+            assert_close(actual, expected_same, rtol=tol_val, atol=tol_val)
+        else:
+            assert_close(actual, expected_valid, rtol=tol_val, atol=tol_val)
 
-    def test_even_sized_filter(self, device, dtype):
+    @pytest.mark.parametrize("padding", ["same", "valid"])
+    def test_even_sized_filter(self, padding, device, dtype):
         kernel = torch.ones(1, 2, 2, device=device, dtype=dtype)
         input = torch.tensor(
             [
@@ -151,7 +220,7 @@ def test_even_sized_filter(self, device, dtype):
             dtype=dtype,
         )
 
-        expected = torch.tensor(
+        expected_same = torch.tensor(
             [
                 [
                     [
@@ -167,15 +236,34 @@ def test_even_sized_filter(self, device, dtype):
             dtype=dtype,
         )
 
-        actual = kornia.filter2d(input, kernel)
-        assert_close(actual, expected)
+        expected_valid = torch.tensor(
+            [
+                [
+                    [
+                        [0.0, 0.0, 0.0, 0.0],
+                        [0.0, 5.0, 5.0, 0.0],
+                        [0.0, 5.0, 5.0, 0.0],
+                        [0.0, 0.0, 0.0, 0.0],
+                    ]
+                ]
+            ],
+            device=device,
+            dtype=dtype,
+        )
 
-    def test_noncontiguous(self, device, dtype):
+        actual = kornia.filter2d(input, kernel, padding=padding)
+        if padding == 'same':
+            assert_close(actual, expected_same)
+        else:
+            assert_close(actual, expected_valid)
+
+    @pytest.mark.parametrize("padding", ["same", "valid"])
+    def test_noncontiguous(self, padding, device, dtype):
         batch_size = 3
         inp = torch.rand(3, 5, 5, device=device, dtype=dtype).expand(batch_size, -1, -1, -1)
         kernel = torch.ones(1, 2, 2, device=device, dtype=dtype)
 
-        actual = kornia.filter2d(inp, kernel)
+        actual = kornia.filter2d(inp, kernel, padding=padding)
         assert_close(actual, actual)
 
     def test_gradcheck(self, device):
@@ -187,14 +275,15 @@ def test_gradcheck(self, device):
         kernel = utils.tensor_to_gradcheck_var(kernel)  # to var
         assert gradcheck(kornia.filter2d, (input, kernel), raise_exception=True)
 
-    def test_jit(self, device, dtype):
+    @pytest.mark.parametrize("padding", ["same", "valid"])
+    def test_jit(self, padding, device, dtype):
         op = kornia.filter2d
         op_script = torch.jit.script(op)
 
         kernel = torch.rand(1, 3, 3, device=device, dtype=dtype)
         input = torch.ones(1, 1, 7, 8, device=device, dtype=dtype)
-        expected = op(input, kernel)
-        actual = op_script(input, kernel)
+        expected = op(input, kernel, padding=padding)
+        actual = op_script(input, kernel, padding=padding)
         assert_close(actual, expected)
 
 


Code review comments:
diff_hunk: @@ -93,19 +99,24 @@ def filter2d(
 
     tmp_kernel = tmp_kernel.expand(-1, c, -1, -1)
 
-    # pad the input tensor
     height, width = tmp_kernel.shape[-2:]
-    padding_shape: List[int] = _compute_padding([height, width])
-    input_pad: torch.Tensor = F.pad(input, padding_shape, mode=border_type)
+
+    # pad the input tensor
+    if padding == 'same':
+        padding_shape: List[int] = _compute_padding([height, width])
+        input = F.pad(input, padding_shape, mode=border_type)
 
     # kernel and input tensor reshape to align element-wise or batch-wise params
     tmp_kernel = tmp_kernel.reshape(-1, 1, height, width)
-    input_pad = input_pad.view(-1, tmp_kernel.size(0), input_pad.size(-2), input_pad.size(-1))
+    input = input.view(-1, tmp_kernel.size(0), input.size(-2), input.size(-1))
 
     # convolve the tensor with the kernel.
-    output = F.conv2d(input_pad, tmp_kernel, groups=tmp_kernel.size(0), padding=0, stride=1)
+    output = F.conv2d(input, tmp_kernel, groups=tmp_kernel.size(0), padding=0, stride=1)
 
-    return output.view(b, c, h, w)
+    if padding == 'same':
+        return output.view(b, c, h, w)
path: kornia/filters/filter.py
text: @user1:
```suggestion
        out = output.view(b, c, h, w)
```

diff_hunk: @@ -93,19 +99,24 @@ def filter2d(
 
     tmp_kernel = tmp_kernel.expand(-1, c, -1, -1)
 
-    # pad the input tensor
     height, width = tmp_kernel.shape[-2:]
-    padding_shape: List[int] = _compute_padding([height, width])
-    input_pad: torch.Tensor = F.pad(input, padding_shape, mode=border_type)
+
+    # pad the input tensor
+    if padding == 'same':
+        padding_shape: List[int] = _compute_padding([height, width])
+        input = F.pad(input, padding_shape, mode=border_type)
 
     # kernel and input tensor reshape to align element-wise or batch-wise params
     tmp_kernel = tmp_kernel.reshape(-1, 1, height, width)
-    input_pad = input_pad.view(-1, tmp_kernel.size(0), input_pad.size(-2), input_pad.size(-1))
+    input = input.view(-1, tmp_kernel.size(0), input.size(-2), input.size(-1))
 
     # convolve the tensor with the kernel.
-    output = F.conv2d(input_pad, tmp_kernel, groups=tmp_kernel.size(0), padding=0, stride=1)
+    output = F.conv2d(input, tmp_kernel, groups=tmp_kernel.size(0), padding=0, stride=1)
 
-    return output.view(b, c, h, w)
+    if padding == 'same':
+        return output.view(b, c, h, w)
+    else:
+        return output.view(b, c, h - height + 1, w - width + 1)
path: kornia/filters/filter.py
text: @user1:
```suggestion
        out = output.view(b, c, h - height + 1, w - width + 1)
```
And return out

diff_hunk: @@ -49,6 +50,8 @@ def filter2d(
           The expected modes are: ``'constant'``, ``'reflect'``,
           ``'replicate'`` or ``'circular'``.
         normalized: If True, kernel will be L1 normalized.
+        padding: This defines the type of padding.
+          2 modes available ``'same'`` or ``'valid'``
path: kornia/filters/filter.py
text: @user1:
```suggestion
          2 modes available ``'same'`` or ``'valid'``.
```

@user1:
We don’t do any checking on `valid` later