# A Review of Abnormal Crowd Behavior Recognition Technology Based on Computer Vision
## 4 Research Status of Abnormal Crowd Behavior Recognition
### Deep Learning-Based Methods
#### 4.2.4 Methods Based on Long Short-Term Memory Network  
Compared with feedforward neural networks, the Recurrent Neural Network (RNN) shows significant advantages in processing sequence data and is particularly good at understanding and processing time series information. The Long Short-Term Memory (LSTM) network, as an improvement and extension of RNN, is mainly designed to solve the problems of gradient vanishing and gradient explosion encountered in the training process of long sequence data. LSTM captures and analyzes the changing patterns of behaviors over time and automatically identifies abnormal sequences that do not conform to it by learning the patterns of normal behavior sequences. LSTM introduces a gate mechanism for controlling the flow and loss of features, including three gating mechanisms: the input gate, forget gate, and output gate. The structure is shown in Figure 13.  
Figure 12: Enhanced reconstruction process combined with autoencoders. As shown in the figure above, \(X_{t}\) represents the input at the current moment, \(h_{t-1}\) and \(h_{t}\) represent the output of the previous unit and the current unit of LSTM, \(C_{t-1}\) and \(C_{t}\) represent the state of the previous unit and the current unit, \(\sigma\) represents the sigmoid activation function, and \(tanh\) represents the hyperbolic tangent activation function. (1) The forget gate. This determines the information to be retained. Based on \(h_{t-1}\) and \(X_{t}\), the forget gate outputs a number between 0 and 1 for the state \(C_{t-1}\). A value of 0 means elimination and 1 means retention. The expression is shown in the Equation (2):  
\[f_{t}=\sigma(W_{f}\bullet[h_{t-1},x_{t}]+b_{f}) \tag{2}\]  
(2) The input gate. This determines the newly incoming information. Composed of the sigmoid function and the tanh function, the result of multiplying the output values is used to update the state. The expression is shown in the Equations (3)-(5):  
\[i_{t}=\sigma(W_{i}\bullet[h_{t-1},x_{t}]+b_{i}) \tag{3}\]  
\[\widetilde{C}_{t}=\tanh(W_{c}\bullet[h_{t-1},x_{t}]+b_{c})) \tag{4}\]  
\[C_{t}=f_{t}*C_{t-1}+i_{t}*C_{t} \tag{5}\]  
(3) The output gate. This determines the output information. The output state is determined through the sigmoid layer, and the output result of the state through the tanh layer is multiplied by the output result of the sigmoid layer. The expression is shown in the Equations (6) and (7), as follows:  
\[O_{t}=\sigma(W_{o}\bullet[h_{t-1},x_{t}]+b_{o}) \tag{6}\]  
\[H_{t}=O_{t}*\tanh(C_{t}) \tag{7}\]  
In the field of abnormal crowd behavior recognition, researchers usually combine LSTM with other technologies to improve the recognition rate and processing speed of crowd abnormal behaviors. [PERSON] et al. [77] integrated the focal loss function into the LSTM algorithm, optimized the model's attention to abnormal samples, and reduced the model loss. This improvement not only enhanced the sensitivity of the LSTM algorithm to abnormal behaviors but also effectively improved the detection accuracy. At the same time, the author also improved the ViBe image foreground extraction method by initializing the background model and randomly selecting the neighborhood sample set to accurately extract the foreground target, thereby simplifying the scene complexity. [PERSON] et al. [78] combined the FCN with strong spatial feature extraction ability and the LSTM with excellent time series modeling ability. The structure is shown in Figure 14. FCN is responsible for extracting high-level semantic features from video frames, while LSTM is used to capture the changing patterns of these features over time. By simultaneously learning the spatial and temporal information in the video, it effectively distinguishes normal and abnormal behaviors in the video. [PERSON] et al. [79] proposed an improved end-to-end supervised learning method that combines the Long Short-Term Memory Network  
Figure 13: The structure of LSTM. (LSTM) and the Convolutional Neural Network (CNN), especially in processing optical flow features.