# PAM-UNet: Enhanced Retinal Vessel Segmentation Using a Novel Plenary Attention Mechanism
## 2 Methods
### Plenary Attention Mechanism  
In order to better learn and attend to vessel structures of different shapes and sizes, we propose a novel method called the Plenary Attention Mechanism (PAM). The design of PAM is inspired by existing attention mechanisms, such as Squeeze-and-Excitation (SE) [36] and Coordinate Attention (CA) [37].  
The plenary attention mechanism first focuses on those meaningful channels of the feature map, and then focuses on the meaningful information on those meaningful channels along the X- and Y-directions of the feature map. This operation is exactly the same as the method of reading in real life--finding the important pages first, and then reading these important pages in detail.  
The PAM can be divided into two parts: channel attention and positional embedding. In convolutional networks, many channels are redundant, and highlighting meaningful channels enables the network to capture more important feature information. However, using only channel attention may overlook positional information. After applying channel attention to the feature maps, average-pooling is performed along the X- and Y-directions of the feature maps separately. The resulting feature maps are then fed through a shared MLP (multi-layer perceptron) [38] for learning. This process helps preserve accurate spatial positional information in one direction, aiding the network in more accurately localizing the target of interest. A schematic of the plenary attention mechanism is shown in Figure 2.  
In implementing channel attention, the input feature map \(F\) is first passed through a global average-pooling operation to generate \(F_{avg}\) (\(\mathbb{C}\times 1\times 1\)). Then, \(F_{avg}\) is passed through an MLP containing a hidden layer. To reduce the computational cost, the number of channels will be compressed to \(C/r\) in the hidden layer, with \(r\) being the reduction rate. Finally, the channel weights are obtained by sigmoiding the feature vectors through the MLP. The obtained weights are multiplied with \(F\) to obtain \(F_{c}\). \(F_{c}\) will highlight important channels. In summary, the channel attention is calculated as in Equation (1).  
\[F_{c}=\sigma(MLP(AvgPool(F)))\cdot F=\sigma(W_{1}(W_{0}(F_{avg})))\cdot F \tag{1}\]  
Here, \(\sigma\) is the sigmoid function with \(W_{0}\in\mathbb{R}^{\frac{C}{r}\times\mathbb{C}}\), \(W_{1}\in\mathbb{R}^{\mathbb{C}\times\frac{C}{r}}\). \(\cdot\) stands for element-by-element multiplication.  
Figure 1: PAM-UNet network architecture diagram.  
To implement positional embedding, average-pooling is performed along the X-, Y-direction of \(F_{c}\) to obtain \(F_{x}\) and \(F_{y}\). Then, \(F_{x}\) and \(F_{y}\) are passed through a shared MLP containing a hidden layer. The reduction rate of MLP is the same as that in channel attention. Finally the feature vectors from the MLP are sigmoided to obtain the weights in the X-, Y-direction. The obtained weights are multiplied with F to obtain \(F^{\prime}\). \(F^{\prime}\) not only highlights the important channels but also adds the position information. The positional embedding is calculated in Equation (2).  
\[\begin{split} F^{\prime}&=\sigma(MLP(X_{AvgPool}(F_ {c})))\cdot\sigma(MLP(Y_{AvgPool}(F_{c})))\cdot F\\ &=\sigma(W_{2}(W_{3}(F_{x})))\cdot\sigma(W_{2}(W_{3}(F_{y}))) \cdot F\end{split} \tag{2}\]  
Here, \(\sigma\) is the sigmoid function with \(W_{2}\in\mathrm{R}^{\frac{\zeta}{\tau}\times C}\), \(W_{3}\in\mathrm{R}^{C\times\frac{\zeta}{\tau}}\). \(\cdot\) stands for element-by-element multiplication.