# Masked Convolutions Within Skip Connections for Video Anomaly Detection
## 3 Methodology
### Rationale Behind the Architecture  
In our UNet architecture, the encoder operates as a spatio-feature extractor for each frame, unearthing a fresh layer of features at every ConvBlock. The MaskedConv3D layers, situated strategically between these ConvBlocks, serve a unique and vital role. As the features extracted by the encoder are passed through the skip connections, the MaskedConv3D layers replace these features, generating new ones by interpreting the context provided by the existing surrounding features. This stage is where the actual inference takes place and is also the primary site of novelty in our architecture.  
Specifically, these MaskedConv3D layers contribute the key capability to detect anomalies--they do this by failing to accurately reconstruct the features from the surrounding context when presented with anomalous data. This failure is what signals the presence of an anomaly and differentiates the anomalous data from the normal.  
Figure 5: This figure illustrates the architecture of the Encoder ConvBlock used in the proposed MaskedSkipUNet framework. The ConvBlock consists of three Conv2D layers with kernel sizes of 3, 5, and 7, respectively. The outputs of these layers are concatenated along the channel dimension and passed through batch normalization and ReLU activation. The resulting feature maps are then downsampled, followed by another Conv2D layer and additional batch normalization and ReLU layers.  
Following this feature extraction and inference process, the decoder assumes its role in the framework, which is singularly focused on assembling these inferred features back together, ultimately yielding a reconstructed output.  
In contrast to \"Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection\" [14], our proposed MaskedSkipUNet architecture uses MaskedConv3D layers not just as feature extractors, but as a mechanism to infer reconstruction based on surrounding spatio-temporal hierarchical features.  
The MaskedSkipUNet architecture improves upon methods like \"Superpixel Masking and Inpainting\" [12]. By focusing the masking operation on the convolutional kernels rather than the frames themselves, MaskedSkipUNet sidesteps the computational burden of generating multiple masked copies of each frame. This kernel-based masking is not only computationally more efficient but also retains critical temporal and spatial information that could be lost when masking frames directly. Furthermore, the architecture incorporates a temporal dimension through its MaskedConv3D layers, an aspect that traditional inpainting methods are currently unable to address.  
The MaskedConv3D layers in our model enable the model to leverage the temporal and spatial context surrounding anomalies, facilitating accurate detection and preserving normal patterns. By excluding the anomalies from direct reconstruction, the MaskedSkipUNet effectively distinguishes anomalies from normal frames, resulting in significantly improved anomaly detection accuracy. This difference in the application of masked convolutional layers underlines the unique approach of our proposed MaskedSkipUNet architecture in the field of video anomaly detection.