# A Review of Abnormal Crowd Behavior Recognition Technology Based on Computer Vision
## 4 Research Status of Abnormal Crowd Behavior Recognition
### Deep Learning-Based Methods
#### 4.2.2 Methods Based on Autoencoders  
Autoencoders (AEs) (2019) constitute an unsupervised learning method. The basic architecture consists of two parts, the encoder and the decoder. It reconstructs the original input by learning the effective compressed representation of the data. The structure is shown in Figure 7. In anomaly detection, AEs are trained to reconstruct normal behavior data. Thus, for those inputs significantly different from the training data (i.e., abnormal behaviors), their reconstruction errors will increase. During this process, the Mean Square Error (MSE) (2019) is commonly used as the loss function. MSE is a statistical indicator that measures the difference between the predicted value and the true value, and the expression is shown in Equation (1), as follows:  
\[MSE=\frac{1}{n}\sum_{i=1}^{n}(\hat{y_{i}}-y_{i})^{2} \tag{1}\]  
Figure 5: The structure of TS-CNN. Among them, \(\hat{y}_{i}\) is the predicted value of the model for the \(i\)th sample, \(y_{i}\) is the true value of this sample, and \(\hat{y}_{i}-y_{i}\) is the residual (the difference between the predicted value and the true value). The smaller the value of MSE, the smaller the average gap between the predicted value and the true value of the model, and the higher the fitting degree of the model. In recent years, Autoencoders have been widely used for abnormal crowd behavior recognition. These methods are mainly divided into two categories [59]: methods based on similarity measurement and methods based on hidden feature representation learning. (1) The method based on similarity measurement. This is a technique used to evaluate and quantify the degree of similarity between two data objects, samples, sets or vectors. In the field of abnormal crowd behavior recognition, abnormal behavior recognition is carried out by comparing the similarity between the original input image and the reconstructed image by the Convolutional Autoencoder (CAE), and the process is shown in Figure 8. The similarity between the original image and the reconstructed image is compared through similarity measurement techniques (such as Euclidean Distance [60], Pearson Correlation Coefficient [61], etc.). If the similarity is low, it indicates that there is abnormal behavior in the original image. Figure 6: The structure of LDA-Net. Note, the red rectangles are the detected boundary of the moving objects. The yellow rectangle is the YOLO5 model, in which the blue ones are convolutional layers and are used for extracting features, the pink ones are bottleneck structure modules and are used for reducing network parameters, and the orange ones are Concat modules and are used for connecting feature maps of different layers. Figure 7: The structure of an AutoEncoder. The original video sequence contains the appearance features of pedestrians, and the use of optical flow analysis can describe the dynamic trajectories of foreground targets. Information fusion can be carried out through the Convolutional Autoencoder (CAE) to extract the appearance attributes and dynamic features of pedestrians. [PERSON] et al. [62] designed a CNN combining CAE and U-Net, which share the same encoder. CAE is responsible for learning the normal appearance structure, while U-Net attempts to associate these structures with the corresponding motion templates. This design aims to jointly improve the detection ability of abnormal frames through two complementary streams (one dealing with appearance and the other dealing with motion) and the model supports end-to-end training. In addition to the fusion of optical flow analysis, some recent studies have utilized the processing ability of the Convolutional LSTM network (ConvLSTM) [63] for time series and combined it with the Convolutional Autoencoder (CAE), significantly improving the extraction ability of spatiotemporal features in videos. [PERSON] et al.