## 2 Related Works
### Multi-Head Attention Mechanism  
Attention mechanisms have multiple applications in computer vision as crucial components of neural networks [19; 20]. When integrated into the liver tumor segmentation process, attention mechanisms enable the model to adaptively extract lesion features while suppressing irrelevant regions. This ensures that the network focuses on pertinent information for a specific segmentation task.  
Figure 1: The structure of atrous spatial pyramid pooling.  
A generalized attention mechanism can be defined as a method for mapping a query (Q) to a set of keys (K) and values (V). In contrast, the multi-head attention mechanism (MHA) [21] implements a method for mapping a query to multiple key-value pairs. Figure 2 illustrates the structure of the multi-head attention mechanism.  
In the multi-head attention mechanism, the input data consist of \(Q\), \(K\), and \(V\) matrices. They are mapped to different subspaces by linear transformation to obtain new matrices: \(Q_{i}\in R^{m\times d_{k}},K_{i}\in R^{m\times d_{k}}\), and \(V_{i}\in R^{m\times d_{V}}\). This transformation is achieved by multiplying them with a learnable weight matrix, as shown in Equation (1).  
\[Q_{i},K_{i},V_{i}=QW_{i}^{Q},KW_{i}^{K},VV_{i}^{V} \tag{1}\]  
where \(W_{i}^{Q}\), \(W_{i}^{K}\) and \(W_{i}^{V}\) denote the learnable weights of the corresponding \(Q\), \(K\), and \(V\), respectively.  
Then, scaled dot-product attention is executed for each attention head. This operation is used to compute the attention weights, as shown in Equation (2). It calculates the dot production of \(Q\) and \(K^{T}\) to determine the degree of the relationship between \(Q\) and \(K\). Subsequently, the outcome is rescaled, and the similarity scores undergo normalization via the softmax function. This process guarantees that the sum of attention weights across all positions equals 1. These weights are employed in a multiplication operation with V to derive the output of the respective attention head.  
\[head=Attention(Q,K,V)=softmax\Big{(}\Big{(}QK^{T}\Big{)}/\Big{(}\sqrt{d_{k}} \Big{)}\Big{)}V \tag{2}\]  
where \(\sqrt{d_{k}}\) represents the scaling factor, which is designed to prevent gradient explosion in the similarity score matrix caused by excessive dimensionality.  
Finally, the outputs of each attention head are concatenated and mapped again to obtain the final attention output, as shown in Equation (3).  
\[MultiHead(Q,K,V)=(Concatenate(head_{1}\cdots head_{h}))W^{o} \tag{3}\]  
where \(head_{i}\) represents the \(i\)th attention head, and the inclusion of multiple attention heads enables the model to concurrently focus on various pieces of subspace information from distinct locations. Additionally, \(W^{o}\) signifies the trainable weight matrix used for linear mapping.  
Incorporating multi-head attention into liver and liver tumor segmentation enables the model to selectively extract pertinent features while concurrently attenuating superfluous regions. This strategy guarantees the network's concentration on pertinent information for a given segmentation task, thereby mitigating segmentation errors induced by noisy signals. Furthermore, leveraging multi-head attention empowers the model to enhance its spatial perception, subsequently elevating segmentation accuracy.  
Figure 2: The structure of multi-head attention and scaled dot-product attention.