# Scene Text Detection Based on Multi-Headed Self-Attention Using Shifted Windows
## 3 Method
### Multi-Headed Self-Attention Based on Shifted Windows  
Multiheaded self-attention first appears in the Transformer model from Natural Language Processing (NLP). For an input image of size \(M\times M\) and input dimension \(C\), when the number of multi-heads was h and the probability mapping function was _SoftMax_, the specific calculations are shown in Equations (11) and (12):  
\[Attention(Q,K,V)=SoftMax(\frac{QK^{T}}{\sqrt{d}}+B)V, \tag{11}\]  
\[MultiHead(Q,K,V)=concat(head_{1},\dots,head_{h})W^{o}, \tag{12}\]  
where \(Q,K,V\in R^{M^{2}\times d}\) are the query, key, and value matrices, respectively, which are obtained by multiplying the pixel matrix of size \(M^{2}\times C\) by the matrices \(W_{q}\), \(W_{k}\), and \(W_{o}\) of size \(C\times d\) with random initialization and learnable parameters, respectively; d is the dimension of the query and key matrices, also equal to \(C/h\); B \(\in R^{M^{2}\times M^{2}}\) is the relative position bias; concat is the feature association operation; \(head_{i}\) is a one-time self-attention calculation, \(1\leq i\leq h\), and \(i\in Z\); \(W^{o}\in R^{h\times C}\) is the fusion matrix, which is used to reduce the feature matrix to \(M^{2}\times C\). ViT (Vision of Transformer) [23] formally introduces the Transformer into the field of computer vision. By slicing the input 2D image into properly sized patches, each patch is flattened into a 1D vector, and then position encoding is added and then fed into the Transformer encoder. To reduce the computational cost of computing the global self-attention of an image, Swin Transformer [12] improves on ViT. After slicing the input image into patches, the whole input image is then divided into multiple windows of fixed size that do not overlap, and then the self-attention is computed separately inside each window. The idea of shifted windows is proposed to increase the connection across windows, and the self-attention is calculated once more after shifted windows, which is used to obtain the relationship between windows. At the same time, for the convenience of computation, cyclic shift and mask operations are used to reduce the computation and ensure the non-interference between each window. In this paper, we borrowed the latter idea in calculating the self-attention of fused features \(F\). The complete multi-headed self-attention calculation process is shown in Figure 3.  
The input fused feature map \(F\) was first divided into a window size of \(M\times M\), and then the self-attention inside each window was calculated using Equation (11). The multi-headed self-attention was calculated using Equation (12), which is called Windows Multi-headed Self Attention (W-MSA) inside the window. At this point, the computed multi-headed self-attention was only within each window and lacked connections across windows on the feature map. To increase the mutual interaction between windows, the divided windows were shifted from the upper left corner of the feature map in the [M/2, M/2] steps to the lower right window. The new generated windows crossed the previous window boundaries, providing a link between them. Another multi-headed self-attention calculation was then performed, and this attention calculation was called Shifted Windows Multi-headed Self Attention (SW-MSA). However, the number of windows was larger than that previously, and the computation volume increased. The size of each window was not fixed, so it could not be computed  
Figure 3: Multi-headed self-attention based on shifted windows. in parallel. In order to solve this problem, the window was fully filled to its original size by using cyclic shift, which can be seen in Figure 3. However, cyclic shift will cause some windows to be composed of some actual non-adjacent sub-windows inside, which need to be masked to ensure that they do not interfere with each other when calculating the multi-headed self-attention. Finally, each pixel in the feature map was then reset to its original position by the reverse cyclic shift operation to obtain the self-attention feature \(S\).