## 3 Materials and Methods
### Network Encoder
Therefore, the W-MSA module and the SW-MSA module appear together in pairs. In order to efficiently calculate the shifted window mechanism, the self-attention calculation formula is as follows:  
\[\rm{Attention(Q,K,V)=Softmax(\frac{QK^{T}}{\sqrt{d}}+B)V} \tag{5}\]  
where Q, K, and V represent the query, key, and value matrices, respectively. \(\rm{B}\in\rm{R}^{M^{2}\times M^{2}}\) is the relative position offset, used to represent the relative position information between patches. **DualTrans Positional Encoding. ** Since position information is not taken into account in the calculation of multi-head attention, the position order of the patch has no influence on the results of self-attention calculation, while position information is crucial for capturing spatial structure or shape in visual tasks, which will greatly reduce the accuracy of brain-tumor segmentation. Therefore, researchers are increasingly adding the location information back; for example, [18] further included the location information into the calculation of the multi-head attention mechanism and used trainable parameters to further optimize the relative position information. First, the relative position distance between each position abc \(\epsilon\)\(\text{H}_{\text{m}\times\text{m}\times\text{m}}\) and the center point was defined as ijk in the window. Relative distances are decomposed across dimensions, so each element in abc \(\epsilon\)\(\text{H}_{\text{m}\times\text{m}\times\text{m}}\) accepts distances in three dimensions, namely the X-axis offset \(\text{r}_{\text{a}-\text{i}}\), the Y-axis offset \(\text{r}_{\text{b}-\text{j}}\), and the Z-axis offset \(\text{r}_{\text{c}-\text{k}}\). The X, Y, and Z axis offset is embedded and connects together to form \(\text{r}_{\text{a}-\text{i}\text{b}-\text{j}\text{c}-\text{k}}\), and the relative attention of this spatial structure can be defined as:  
\[Y_{\text{ijk}-\sum\limits_{\text{a},\text{b},\text{b},\text{c} }\text{H}_{\text{0},\text{(ijk)}}\text{softmax}_{\text{abc}}(\text{Q}_{\text {ijk}}^{\text{r}}\text{K}_{\text{abc}}+\text{Q}_{\text{ijk}}^{\text{r}}\text{ r}_{\text{a}-\text{i}\text{b}-\text{j}\text{c}-\text{k}}^{\text{r}})\text{V}_{\text{abc}}} \tag{6}\]  
Experiments have proven that this method can effectively improve the classification accuracy of visual tasks. However, we noticed that the position deviation proposed by this method only depends on the query pixel \(\text{Q}_{\text{ijk}}\) rather than key pixel \(\text{V}_{\text{abc}}\), but the corresponding position information should also be paid attention to in key pixel \(\text{V}_{\text{abc}}\). Therefore, we propose attention based on three-dimensional relative positional embeddings. First, we define the relative distance from ijk to each element abc \(\epsilon\)\(\text{H}_{\text{m}\times\text{m}\times\text{m}}\), where the relative distance is decomposed into three dimensions. On this basis, we add a positional bias term \(\text{r}_{\text{a}-\text{i}\text{b}-\text{j}\text{c}-\text{k}}^{\text{v}}\) related to the key pixel, with the specific formula being:  
\[Y_{\text{ijk}-\sum\limits_{\text{a},\text{b},\text{b},\text{c} }\text{H}_{\text{0},\text{(ijk)}}\text{softmax}_{\text{abc}}(\text{Q}_{ \text{ijk}}^{\text{r}}\text{K}_{\text{abc}}+\text{Q}_{\text{ijk}}^{\text{r}} \text{r}_{\text{a}-\text{i}\text{b}-\text{j}\text{c}-\text{k}}^{\text{d}})( \text{V}_{\text{abc}}+\text{r}_{\text{a}-\text{i}\text{b}-\text{j}\text{c}- \text{k}}^{\text{v}})} \tag{7}\]  
where \(\text{r}_{\text{a}-\text{i}\text{b}-\text{j}\text{c}-\text{k}}^{\text{d}}\) is the position code of learnable value. Compared with Formula 6, the increase in vector \(\text{r}_{\text{a}-\text{i}\text{b}-\text{j}\text{c}-\text{k}}^{\text{v}}\) does not introduce more parameters because they are shared among the attention heads between each layer, while the increase in \(\text{r}_{\text{a}-\text{i}\text{b}-\text{j}\text{c}-\text{k}}^{\text{v}}\) can obtain more position information, which helps to improve the segmentation accuracy of brain tumors.