# Building Damage Assessment Based on Siamese Hierarchical Transformer Framework
## 2 Methods
### Spatial Fusion Module  
In Swin Transformer, the computation of self-attention is limited in each local non-overlapping window, resulting in linear computational cost in relation to image size and reduced memory overhead. However, this mechanism cuts down the ability of global modeling compared to Transformer [4]. Moreover, additional spatial knowledge helps eliminate the blurred contours of the post-disaster building. Therefore, the Spatial Fusion (SF) module is proposed to allow for more spatial knowledge learned by the encoder and boost the information exchange over the W-Trans block and SW-Trans block.  
By adding the weighted feature to the output of the W-Trans block, the SF module emphasizes spatial information from neighboring pixels of the same class label as the central pixel, and it suppresses pixels of different class labels. The structure of the SF module is shown in Figure 5. In phase \(n\), let the input feature of W-Trans block \(z^{l-1}\in\mathbb{R}^{(h\times w)\times c_{1}}\) reshape into \(s\in\mathbb{R}^{h\times w\times c_{1}}\), where \(c_{1}=2^{n-1}C\), \(w=\frac{W}{2^{n+1}}\) and \(h=\frac{H}{2^{n+1}}\). The \(s\) is sent into a \(3\times 3\) dilated convolution layer of dilation rate 2.  
A simple convolution layer can be formed as:  
\[\hat{s}(x,y)=\sum_{d^{\prime}=1}^{c_{1}}\left(\sum_{-L\leq i,j\leq L}k_{d}\left( i,j,d^{\prime}\right)\cdot F_{d^{\prime}}(x-i,y-j)\right) \tag{12}\]  
where \(x\), \(y\) are integers for which \(0\leq x\leq w,0\leq x\leq h\), and \(1\leq d^{\prime}\leq c_{1}\). \(K_{d}\) stands for coefficients for all pairs \((i,j,d^{\prime})\) for which \(|i|,|j|\leq L\). \(F_{d^{\prime}}\) is the input and \(\hat{s}\) is the output of the layer. It can be inferred that \(\hat{s}\) corresponding to the point \((x,y)\) is determined by \(F_{d^{\prime}}(x-i,y-j)\) at points \((x-i,y-j)\) corresponding to \(|i|,|j|\leq L\).  
Compared with convolution, dilated convolution makes a larger receptive field. A dilated convolution layer is formed as:  
\[\hat{s}(x,y)=\sum_{d^{\prime}=1}^{c_{1}}\left(\sum_{-L\leq i,j\leq L}k_{d} \left(i,j,d^{\prime}\right)\cdot F_{d^{\prime}}(x-\ell\cdot i,y-\ell\cdot j)\right) \tag{13}\]  
where \(\ell\) refers to the dilation rate. In this case, convolution is performed on \((x-\ell\cdot i,y-\ell\cdot j)\) instead of \((x-i,y-j)\). Dilated convolution is conducted within more distant pixels. This approach expands the receptive field and thereby enhances the perception of global information.  
Then, the channel number is reduced to \(c_{1}/2\), and the feature passes through the global average pooling layer in the horizontal and vertical directions, respectively, to get the feature map statistics of spatial information. The formula of the above two directions is represented as:  
\[v_{w_{j}}^{i}=\frac{1}{h}\sum_{j=0}^{h-1}\hat{s}^{i}(j,k), \tag{14}\]  
\[v_{h_{k}}^{i}=\frac{1}{w}\sum_{j=0}^{w-1}\hat{s}^{i}(j,k), \tag{15}\]  
where \(i\), \(j\), and \(k\) are the indexes of the channel and the horizontal and vertical directions, respectively, and \(0\leq k\leq h\), \(0\leq j\leq w\), \(0\leq i<c_{1}/2\). The tensor \(v_{w}\) contains the attention weights in the horizontal direction and \(v_{h}\) in the vertical direction. Then, the \(v_{w}\) and \(v_{h}\) obtained from (14) and (15) multiply to obtain the attention feature map on spatial, \(M\in R^{h\times w\times\frac{1}{2}}\), following a convolution layer to double the dimension. The final feature  
Figure 5: The structure of SF module.  
map \(F\) output from SF is generated by adding \(M\) and \(s^{l+1}\) yielded from the SW-Trans block. The feature \(F\in R^{l\times w\times c_{1}}\) is formulated as  
\[M=v_{w}\otimes v_{h} \tag{16}\]  
\[F=z^{l+1}\oplus\zeta(Ge(M)) \tag{17}\]  
where \(\oplus\) refers to element-level addition, and \(\otimes\) refers to matrix multiplication. \(\zeta(\cdot)\) and \(Ge(\cdot)\) stand for the \(1\times 1\) convolutional layer and GELU activation function, respectively.