Repository: pytorch/vision
Pull request: Add Conv2dNormActivation and Conv3dNormActivation Blocks
Closes #5430 

Simple code to try this. It worked fine.

```
class SimpleModel(torch.nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.layer1 = Conv3dNormActivation(3, 15, stride=2, padding=2,
                                           activation_layer=torch.nn.LeakyReLU,
                                           inplace=True,
                                           bias=True)
    
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = self.layer1(x)
        return x

class EasyModel(torch.nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.layer1 = Conv2dNormActivation(3, 15, stride=2, padding=2,
                                           activation_layer=torch.nn.LeakyReLU,
                                           inplace=True,
                                           bias=True)
    
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = self.layer1(x)
        return x



if __name__ == "__main__":

    image_tensor = torch.rand((5, 3, 224, 224))
    model = EasyModel()
    out = model(image_tensor)
    print(out.size())  # torch.Size([5, 15, 113, 113])

    video_tensor = torch.rand((5, 3, 10, 224, 224))
    model = SimpleModel()
    out = model(video_tensor)
    print(out.size())  # torch.Size([5, 15, 6, 113, 113]

```


Problem statement:
ConvNormActivation block for 3D convolutions
### 🚀 The feature

I suggest we add a `Conv3d` equivalent for the  `ConvNormActivation` block

### Motivation, pitch

This will be used in the upcoming ResNext3d implementation. 

### Alternatives


I see two possible alternatives:

1. We can re-purpose the existing method in a BC way  by adding an extra named parameter `conv layer` with default value of `Conv2D`. 

2. We can explicitly add a new class `Conv3DNormActivation`, which will be very similar to the existing one.

### Additional context

_No response_

Hints from issue discussion:
In my opinion 2nd one is better. 

I can give a shot at this :) 
Thanks for the feedback @oke-aditya!

I was thinking and discussing with @datumbox offline, and I'm inclined to the following solution:

1. Create an Alias for the existing `ConvNormActivation` and call it `Conv2dNormActivation`. (this will be done in a separate PR. I would also add a deprecation warning for the ConvNormActivation in this case.
2. Create a new method for 3D convolutions named `Conv3dNormActivation`, which seems to be your preferred method too. 

The advantages of this approach is that we have a clear naming convention in line with the `nn.Conv2d` and `nn.Conv3d`  from pytorch, and we wouldn't have to validate invalid combinations of `norm_layer` and  the new `conv_layer` paramter in `ConvNormActivation`.

The major disadvantage is that the current `ConvNormActivation` is already used in [9 models](https://github.com/pytorch/vision/search?q=ConvNormActivation) inside TorchVision and [numerous places](https://github.com/search?p=3&q=ConvNormActivation&type=Code) externally but I think this can be mitigated with the deprecation warning and fazing out in future releases. 
Also, thank you for offering to give it a shot, let's just wait for some more feedback and then would be great if you wish to work on it!
The approach sounds good to me. If you bring a separate PR, that does step 1, we should replace all existing uses in TorchVision and also do the same on the next fbsync for all FBcode internal cases.
@oke-aditya I kicked off by renaming the `ConvNormActivation` as we all seem to be aligned. Looking forward to your contribution for step 2.
Nice will do this over the weekend :smile:

Commit message: Add ops.conv3d

Patch to review:
diff --git a/docs/source/ops.rst b/docs/source/ops.rst
index 2a960474205..19ca9761186 100644
--- a/docs/source/ops.rst
+++ b/docs/source/ops.rst
@@ -46,4 +46,5 @@ Operators
     StochasticDepth
     FrozenBatchNorm2d
     ConvNormActivation
+    Conv3dNormActivation
     SqueezeExcitation
diff --git a/torchvision/ops/__init__.py b/torchvision/ops/__init__.py
index 8ba10080c1f..e0685e009c8 100644
--- a/torchvision/ops/__init__.py
+++ b/torchvision/ops/__init__.py
@@ -14,7 +14,7 @@
 from .feature_pyramid_network import FeaturePyramidNetwork
 from .focal_loss import sigmoid_focal_loss
 from .giou_loss import generalized_box_iou_loss
-from .misc import FrozenBatchNorm2d, ConvNormActivation, SqueezeExcitation
+from .misc import FrozenBatchNorm2d, ConvNormActivation, Conv3dNormActivation, SqueezeExcitation
 from .poolers import MultiScaleRoIAlign
 from .ps_roi_align import ps_roi_align, PSRoIAlign
 from .ps_roi_pool import ps_roi_pool, PSRoIPool
@@ -52,6 +52,7 @@
     "StochasticDepth",
     "FrozenBatchNorm2d",
     "ConvNormActivation",
+    "Conv3dNormActivation",
     "SqueezeExcitation",
     "generalized_box_iou_loss",
 ]
diff --git a/torchvision/ops/misc.py b/torchvision/ops/misc.py
index 268962e204c..c424f03fa14 100644
--- a/torchvision/ops/misc.py
+++ b/torchvision/ops/misc.py
@@ -161,3 +161,62 @@ def _scale(self, input: Tensor) -> Tensor:
     def forward(self, input: Tensor) -> Tensor:
         scale = self._scale(input)
         return scale * input
+
+
+class Conv3dNormActivation(torch.nn.Sequential):
+    """
+    Configurable block used for Convolution-Normalzation-Activation blocks.
+
+    Args:
+        in_channels (int): Number of channels in the input video.
+        out_channels (int): Number of channels produced by the Convolution-Normalzation-Activation block
+        kernel_size: (int, optional): Size of the convolving kernel. Default: 3
+        stride (int, optional): Stride of the convolution. Default: 1
+        padding (int, tuple or str, optional): Padding added to all four sides of the input. Default: None, in wich case it will calculated as ``padding = (kernel_size - 1) // 2 * dilation``
+        groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1
+        norm_layer (Callable[..., torch.nn.Module], optional): Norm layer that will be stacked on top of the convolution layer. If ``None`` this layer wont be used. Default: ``torch.nn.BatchNorm3d``
+        activation_layer (Callable[..., torch.nn.Module], optinal): Activation function which will be stacked on top of the normalization layer (if not None), otherwise on top of the conv layer. If ``None`` this layer wont be used. Default: ``torch.nn.ReLU``
+        dilation (int): Spacing between kernel elements. Default: 1
+        inplace (bool): Parameter for the activation layer, which can optionally do the operation in-place. Default ``True``
+        bias (bool, optional): Whether to use bias in the convolution layer. By default, biases are included if ``norm_layer is None``.
+    """
+
+    def __init__(
+        self,
+        in_channels: int,
+        out_channels: int,
+        kernel_size: int = 3,
+        stride: int = 1,
+        padding: Optional[int] = None,
+        groups: int = 1,
+        norm_layer: Optional[Callable[..., torch.nn.Module]] = torch.nn.BatchNorm3d,
+        activation_layer: Optional[Callable[..., torch.nn.Module]] = torch.nn.ReLU,
+        dilation: int = 1,
+        inplace: Optional[bool] = True,
+        bias: Optional[bool] = None,
+    ) -> None:
+
+        if padding is None:
+            padding = (kernel_size - 1) // 2 * dilation
+        if bias is None:
+            bias = norm_layer is None
+        layers = [
+            torch.nn.Conv3d(
+                in_channels,
+                out_channels,
+                kernel_size,
+                stride,
+                padding,
+                dilation=dilation,
+                groups=groups,
+                bias=bias,
+            )
+        ]
+        if norm_layer is not None:
+            layers.append(norm_layer(out_channels))
+        if activation_layer is not None:
+            params = {} if inplace is None else {"inplace": inplace}
+            layers.append(activation_layer(**params))
+        super().__init__(*layers)
+        _log_api_usage_once(self)
+        self.out_channels = out_channels


Code review comments:
diff_hunk: @@ -161,3 +161,62 @@ def _scale(self, input: Tensor) -> Tensor:
     def forward(self, input: Tensor) -> Tensor:
         scale = self._scale(input)
         return scale * input
+
+
+class Conv3dNormActivation(torch.nn.Sequential):
+    """
+    Configurable block used for Convolution-Normalzation-Activation blocks.
+
+    Args:
+        in_channels (int): Number of channels in the input video.
+        out_channels (int): Number of channels produced by the Convolution-Normalzation-Activation block
+        kernel_size: (int, optional): Size of the convolving kernel. Default: 3
+        stride (int, optional): Stride of the convolution. Default: 1
+        padding (int, tuple or str, optional): Padding added to all four sides of the input. Default: None, in wich case it will calculated as ``padding = (kernel_size - 1) // 2 * dilation``
+        groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1
+        norm_layer (Callable[..., torch.nn.Module], optional): Norm layer that will be stacked on top of the convolution layer. If ``None`` this layer wont be used. Default: ``torch.nn.BatchNorm3d``
+        activation_layer (Callable[..., torch.nn.Module], optinal): Activation function which will be stacked on top of the normalization layer (if not None), otherwise on top of the conv layer. If ``None`` this layer wont be used. Default: ``torch.nn.ReLU``
+        dilation (int): Spacing between kernel elements. Default: 1
+        inplace (bool): Parameter for the activation layer, which can optionally do the operation in-place. Default ``True``
+        bias (bool, optional): Whether to use bias in the convolution layer. By default, biases are included if ``norm_layer is None``.
+    """
+
+    def __init__(
+        self,
+        in_channels: int,
+        out_channels: int,
+        kernel_size: int = 3,
+        stride: int = 1,
+        padding: Optional[int] = None,
+        groups: int = 1,
+        norm_layer: Optional[Callable[..., torch.nn.Module]] = torch.nn.BatchNorm3d,
+        activation_layer: Optional[Callable[..., torch.nn.Module]] = torch.nn.ReLU,
+        dilation: int = 1,
+        inplace: Optional[bool] = True,
+        bias: Optional[bool] = None,
+    ) -> None:
+
+        if padding is None:
path: torchvision/ops/misc.py
text: @user2:
There is lots of duplicate code with 2d. Should we have a base class to avoid it and both implementation call the same base? PyTorch dos something similar with [BatchNorm](https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/batchnorm.py#L338).

@user1 thoughts?

@author:
I thought the same. There is duplication. We can probably just create a base class convbatchNorm and then extend that for 2d and 3d ?

@user1:
I agree, please see my proposal here: https://github.com/pytorch/vision/pull/5440