# FTGM: Fast Transformer-Based Global Matching for Particle Image Velocimetry
## 3 Approach
### Feature Extraction and Enhancement  
Given two frames \(I_{1}\) and \(I_{2}\), FTGM first employs a feature extraction backbone composed of a three-layer residual network with weight-sharing to obtain downsampled initial features \(F_{1}\), \(F_{2}\). The essence of global matching fundamentally lies in identifying corresponding pixels between frames, with the key challenge being the enhancement of feature distinctiveness to facilitate the matching. To account for their interdependencies more comprehensively, the Transformer is a natural choice. It excels at capturing the reciprocal relationships between two sets through its attention mechanism. Position Embedding. Since the features \(F_{1}\) and \(F_{2}\) have no notion of the spatial position, it is imperative to inject positional information for each pixel. In this work, we employ fixed 2D sine and cosine functions as positional encodings of different frequencies (following DETR [37]):  
\[\left\{ \begin{array}{l}PE_{(pos,2i)}=\sin(pos/10,000^{2i/d_{model}})\\ PE_{(pos,2i+1)}=\cos(pos/10,000^{2i+1/d_{model}})\end{array} \right., \tag{1}\]  
where _pos_ is the position of the pixel, \(i\) represents the dimensional information, and for any offset \(k\), the positional encoding \(PE_{pos+k}\) can be expressed as a linear function of \(PE_{pos}\). The incorporation of positional information not only considers the similarity of features but also their spatial distances, which aids in resolving ambiguities and enhancing the performance. Customized Transformer. Considering the fundamental differences between PIV data and the optical flow datasets from natural scenes, we define a Transformer block to improve the quality of the initial features (Figure 2). Features \(F_{1}\) and \(F_{2}\) are concatenated and symmetrically processed to serve as the source and target, respectively. The source is fed into the self-attention layer. Unlike the standard Transformer, we omit the feed-forward network in the self-attention layer to ensure the \"purity\" of the model's feature representation based on the input sequence's inherent structure and internal dependencies, a concept akin to the functionality of a context encoder. In the cross-attention layer, the output of the self-attention layer interacts with the original target features. In the FFN layer, the concatenated result of the cross-attention layer's output and the source undergoes dimension expansion and non-linear activation, allowing the FFN to fuse information from both the input sequence and the encoder output, thereby generating richer feature representations. Residual connections and layer normalization maintain numerical stability and effective information propagation. Attention Mechanism. The attention mechanism, central to the Transformer architecture, is capable of capturing dependencies between any two positions within the input image. Given query vectors \(Q\in\mathbb{R}^{d_{q}\times d_{m}}\), key vectors \(K\in\mathbb{R}^{d_{q}\times d_{m}}\), and value vectors \(V\in\mathbb{R}^{d_{q}\times d_{m}}\), where \(d_{m}\) is the feature dimension, the attention module computes the similarity between the \(Q\) and \(K\) and then applies a softmax function to obtain the weights on the value, and the attention is calculated as  
\[Attention(Q,K,V)=softmax(QK^{T}/\sqrt{d_{m}})\cdot V. \tag{2}\]The attention weights determine the proportion of each value vector in the final output so that the model can focus on salient features for the current task and disregard irrelevant portions. For more details about attention, please refer to [15]. Shifted Local Window Attention. In the context of Transformer architectures, the computational expense of the attention mechanism is considerable, stemming from its quadratic computational complexity when dealing with input features. To enhance computational efficiency, we implement a strategy that employs a fixed count of local-window attentions.