# A Review of Abnormal Crowd Behavior Recognition Technology Based on Computer Vision
## 4 Research Status of Abnormal Crowd Behavior Recognition
### Deep Learning-Based Methods
#### 4.2.5 Methods Based on the Self-Attention Mechanism  
In the field of deep learning, the attention mechanism (Self-Attention SA) [80] can be interpreted as a transformation process involving the mapping between a query vector and a series of key-value pairs, thereby generating an output vector. This output vector is obtained by performing a weighted sum of each Value vector, where the weight attached to each Value vector is determined based on the quantitative evaluation of the correlation between its corresponding Key vector and the main query vector. The calculation of the attention mechanism can be divided into three stages, as shown in Figure 15.  
Stage 1 is to calculate the similarity between Query and Key. Common methods include the vector dot product, Cosine similarity, and MLP network. The expression is shown in the Equations (8) and (9), as follows:  
Vector dot product \[Sim(Query,key_{i})=Query\bullet Key_{i}\] (8)  
(1) Cosine similarity \[Sim(Query,key_{i})=\frac{Query\bullet Key_{i}}{\|Query\|\bullet\|Key_{i}\|}\] (9)  
Figure 14: The structure of FCN-LSTM. Figure 15: The process of attention mechanism computation. 2. MLP network \[Sim(Query,key_{i})=MLP(Query\bullet Key_{i})\] (10) Stage 2 introduces _SoftMax_ for normalization to sort out the probability distribution of all of the elements. The expression is shown in the Equation (11), as follows: \[a_{i}=SoftMax(\text{Sim}_{i})=\frac{e^{Sim_{i}}}{\sum\limits_{j=1}^{n}e^{Sim_{i}}}\] (11) Stage 3 performs weighted summation to obtain the _Attention_ value. The expression is shown in Equation (12), as follows: \[Attention(Q,K,\text{V})=SoftMax(\frac{QK^{T}}{\sqrt{\text{d}_{k}}})V\] (12) Self-attention is a special form of the attention mechanism. Its uniqueness lies in that, when calculating the attention weight, the query, key, and value all come from different parts of the same input sequence. It was first widely used in the \"Transformer\" model, which greatly promoted the development of the field of natural language processing, and is currently also widely used in crowd sequence modeling and abnormal behavior recognition tasks. Self-attention can not only capture the long-distance dependency within the sequence but also remain efficient in parallel computing, thereby overcoming some limitations of the Recurrent Neural Network (RNN) model. [PERSON] et al. [81] designed a Self-Attention Feature Aggregation (SAFA) module. The structure is shown in Figure 16. This module regenerates the feature map by aggregating the embedding representations with similar information according to the attention map. At the same time, the prior bias on normal data is minimized through the self-suppression strategy, and it is used in combination with the pixel-level frame prediction error to jointly detect abnormal frames, enhancing the ability to recognize abnormalities. [PERSON] et al. [82] proposed an autoencoder model that integrates the global self-attention mechanism to capture the interaction information between the overall features of the image, facilitating the model to understand the context in which the behavior occurs and improving the accuracy of abnormal behavior determination. At the same time, a memory module is embedded in the bottleneck layer of the autoencoder to constrain the model's over-generalization of normal behaviors. [PERSON] et al. [83] fused the attention mechanism and the bidirectional Long Short-Term Memory Autoencoder Network (SABiAE). The structure is shown in Figure 17. The encoder with the self-attention mechanism captures global appearance features, and the self-attention bidirectional LSTM network is used to reduce the loss of target features, thereby extracting the information between video frames globally. The model reconstructs the frame through the decoding process and determines abnormal behaviors based on the reconstruction error. \[Sim(Query,key_{i})=Query\bullet Key_{i}\]0 The structure of SAFA. [PERSON] et al.