# An Efficient Anomaly Detection System for Crowded Scenes Using Variational Autoencoders
## 3 Method
### Convolutional VAE Architecture and Feature Extraction
The first part could be considered as the reconstruction loss to make the generated frame series \(\hat{\mathcal{I}}_{t}\) close to the original series \(\mathcal{I}_{t}\), thus the loss function can be defined as follows:  
\[\mathcal{L}_{con}:=\|\mathcal{I}_{t}-\hat{\mathcal{I}}_{t}\|^{2} \tag{3}\]  
In addition, as we assume the distribution of each feature representation that described by a mean map and a standard deviation map takes on an approximate Gaussian form with an approximate diagonal covariance, the second part is the Kullback-Leibler (KL) divergence, which measures the difference between two probability distributions. \[\mathcal{L}_{KL}:=D_{KL}(\mathcal{N}(\mathbf{\mu}_{t},\mathbf{\sigma}_{t}^{2})\| \mathcal{N}(\mathbf{0},\mathbf{I})) \tag{4}\]  
Specifically, the KL divergence can be computed and differentiated without estimation:  
\[D_{KL}(\mathcal{N}(\mathbf{\mu}_{t},\mathbf{\sigma}_{t}^{2})\|\mathcal{N}(\mathbf{0},\mathbf{ I}))=-\frac{1}{2}\sum_{k=1}^{K}(1+\log(\sigma_{tk}^{2})-(\mu_{tk})^{2}-\sigma_{tk} ^{2}) \tag{5}\]  
where \(K=h_{l}\times w_{l}\times c_{l}\) is the total number of the pixels in the mean tensor (also the standard deviation tensor). Finally, the total loss function is as follows. \[\mathcal{L}=\mathcal{L}_{KL}+\mathcal{L}_{con} \tag{6}\]  
In the training processing, [PERSON] [43] based Stochastic Gradient Descent method is used for parameter optimization and the learning rate is set to 0.0001. As a fully convolutional network, the resolutions of input and output are the same. Consequently, there is no need to crop or resize the input video frames. Different from ordinary auto-encoder, the variational auto-encoder aims to learn the posterior distributions of feature representations, in the form of parameters mean \(\mu\) and standard deviation \(\sigma\). From the view of numerical simulation, the mean \(\mu\) can be treated as a statistical representation of the input data and the standard deviation \(\sigma\) as a noise intensity regulator. Therefore, for the input series \(\mathcal{I}_{t}\), we define the mean tensor \(\mathbf{\mu}_{t}\) from the encoder as the feature representation \(\mathbf{f}_{t}\) of the appearance and motion patterns. Suppose we have the mean tensor \(\mathbf{\mu}_{t}\) with size \(h_{l}\times w_{l}\times c_{l}\) from the encoder at the \(l\)th layer, the feature representation \(\mathbf{f}_{t}\) is considered to consist of \(h_{l}\times w_{l}\) feature vectors with \(c_{l}\) dimensions. Specifically, for each position \((i,j)\), where \(i\in[1,h_{l}],\ j\in[1,w_{l}]\), the feature vectors can be written as:\[\mathbf{f}_{t}(i,j)=[f_{t}(i,j,1),f_{t}(i,j,2),\ldots,f_{t}(i,j,c_{l})]^{\top} \tag{7}\]  
According to the architecture of the conventional neural network, each feature vector \(\mathbf{f}_{t}(i,j)\) is derived from a specific receptive field, which is a sub-region of the original sequence \(\mathcal{I}_{t}\). In other words, instead of cropping the sequence into patches and extracting the features one by one, we divide the sequence \(\mathcal{I}_{t}\) into overlapping patches containing appearance and motion information densely and extract features \(\mathbf{f}_{t}=\{f_{t}(i,j)|i=1,2,\ldots,h;j=1,2,\ldots,w\}\), from all the receptive fields at one time. In general, the features learned by our convolutional VAE are the further statistical representations of that learned by an ordinary convolutional auto-encoder, which means our features can represent the appearance and motion patterns that the receptive fields belong to rather than just some specific receptive fields samples. On the other hand, according to the assumption and the corresponding constraint condition of VAE, the elements of our feature vector tend to be independent and decoupled, comparing with the feature learned by the conventional auto-encoder.