## 3 PUB-SalNet
### U-SalNet Architecture  
Our U-SalNet model includes two attentional mechanisms: global attention and local attention, which are integrated into the U-Net architecture, as is shown in the purple dashed-line box in Figure 2b. U-SalNet aims to selectively segment salient objects from the background by generating an attention map at each pixel. We apply the two modes of attention to refining salient regions in biomedical images. The detailed U-SalNet architecture is shown in Figure 3. We first upsample each level of feature maps in the decoder branch and concatenate them with their corresponding levels of features in the encoder branch. After concatenating the outputs from encoder and decoder, we get a feature map \(\Gamma\in\mathbb{R}^{C\times W\times H}\) as the input to the attention module, where \(C\), \(W\), \(H\) denote the channels, width, and height, respectively. For global attention, as is shown in Figure 3a, to generate attention over the whole feature map \(\Gamma\) for each pixel, we first apply four ReNet models [23] to sweep across a feature volume both horizontally and vertically along two directions to concentrate global information. Next, a convolution operation is performed to transform this feature map to \(D_{G}\) channels, where \(D_{G}=W\times H\). At the same time,  
Figure 2: The overview framework of our proposed method. The processing pipeline consists of three main steps: (**a**) pre-training on the SalSeg-CECT data set; (**b**) prediction using the U-SalNet model; (**c**) unsupervised attentional backpropagation iterating on single images. The Enc and Dec stand for the encoder and decoder. The \(\otimes\), \(\otimes\), and \(\otimes\) denote the global attention mechanism, local attention mechanism and convolutional decoding, respectively. the feature vector \(x^{w,h}\) at each pixel \((w,h)\) is normalized via softmax to obtain global attention weights \(\Phi^{w,h}\), as is shown in Equation (1). Here, \(i,j\in\{1,...,D_{G}\}\). In order to generate the global attention feature \(\Gamma_{att}\), as is shown in Equations (2) and (3), the features at all locations in the whole feature map \(\Gamma\) are summed with weights according to \(\Phi_{i}^{w,h}\). \(\Phi_{i}^{w,h}\) refers to the salient correlation between the object pixel \((w,h)\) and the pixel at the \(i^{th}\) location \((w_{i},h_{i})\). \(Conv_{i}\in\mathbb{R}^{C}\) is the Conv feature at \((w_{i},h_{i})\) in \(\Gamma\). \(\Gamma_{att}\) has the same size with \(\Gamma\). For the local attention, as is shown in Figure 2(b), we consider a local feature cube \(\Gamma^{w,h}\in\mathbb{R}^{W^{\prime}\times H^{\prime}\times C}\) centered at \((w,h)\) with width \(W^{\prime}\) and height \(H^{\prime}\). \(\Phi^{\prime}\) is derived from convolution layers with a reception field of \(W^{\prime}\times H^{\prime}\) in the original feature map for each location. Similar to global attention, the features in \(\Gamma^{w,h}\) are weightedly summed by \(\Phi^{w,h}\) to construct \(\Gamma_{att}^{\prime}\), as is shown in Equation (4):  
\[\Phi_{i}^{w,h} =\text{softmax}\left(x^{w,h}\right) \tag{1}\] \[=\frac{\exp\left(x_{i}^{w,h}\right)}{\sum_{j=1}^{D_{G}}\exp\left( x_{j}^{w,h}\right)},\]  
\[\Gamma_{att}^{w,h}=\sum_{i=1}^{D_{G}}\Phi_{i}^{w,h}Conv_{i}, \tag{2}\]  
\[Conv_{i}=\int_{\mathbb{R}^{C}}f(\tau)g(t-\tau)d\tau \tag{3}\]  
\[\Gamma_{att}^{w,h}=\sum_{i=1}^{D_{i}}\Phi_{i}^{w,h}Conv_{i}^{w,h}, \tag{4}\]  
\[Loss=-(y_{t}\log(y_{p})+(1-y_{t})\log(1-y_{p})). \tag{5}\]  
It is worth noticing that, although our attention mechanism is similar to [24]'s work, they only consider the serial combination of global and local attention. Our U-SalNet focuses more on fusing two attention modules through convolutional decoding. In addition, their model adopts deep supervision,  
Figure 3: The architecture of our U-SalNet model. **(a)** Global Attention and **(b)** Local Attention corresponds to \(\otimes\) and \(\otimes\) from Figure 1(b), respectively. GA and LA stand for Global Attention and Local Attention. Conv means the convolution operation.