# HPS-Net: Multi-Task Network for Medical Image Segmentation with Predictable Performance
## 3 The Proposed HPS-Net
### Network Structure  
HPS-Net is a multi-task network. The first task of HPS-Net is to generate realistic and diverse segmentation hypotheses, which is the same as PHiSeg and other related methods. The second task is to provide the measure predictions including precision, accuracy, the true positive rate (TPR), the true negative rate (TNR), or other measures. HPS-Net includes four sub-networks, namely, the posterior network, the prior network, the likelihood network, and the measure network. The posterior network, the prior network, and the likelihood network are for medical image segmentation, the structures of which are similar to PHiSeg [10]. The measure network is for predicting the different measurement values. Figure 4 illustrates the detailed network structure of the proposed HPS-Net for three latent levels and four resolution levels. The inputs of the posterior network are the medical image and the corresponding ground-truth segmentation, while the input of the prior network is only the medical image. The goal of these two sub-networks is to learn the latent spaces corresponding to the posterior probability distribution and the prior probability distribution, respectively, and maximize the similarity of the learned latent spaces of these two sub-networks. From Figure 4, we can see that the posterior network and the prior network are sym  
Figure 4: Network structure of the proposed HPS-Net for three latent levels and four resolution levels. HPS-Net includes four sub-networks, namely, the posterior network, the prior network, the likelihood network, and the measure network. metrical. Such a symmetric network structure allows us to learn the reasonable latent spaces in a confrontational way. With the learned latent spaces, random segmentation variants can be generated, which are then to be input to the likelihood network. With the random segmentation variants as the input, the likelihood network is trained to generate diverse segmentation hypotheses at multiple resolution levels. It should be noted that the resolution levels need at least one more level than the latent levels, and whether each resolution level contains a latent level is optional. The example in Figure 4 has a total of four resolution levels and three latent levels. Figure 5 shows the example of five resolution levels and two latent levels. We obtained the best results in the experiments with seven resolution levels and five latent levels. The final and the most important sub-network is the measure network, which enables HPS-Net to generate the predicted measurement values. The measure network takes the medical image and its corresponding segmentation hypotheses as the input. To fuse the information, the segmentation hypotheses at different levels are upsampled to the same size of the medical image and then fused with the medical image by channel concatenation. The resulting multi-channel image is then input into the backbone network. Here, we used the SE-Inception-V4 (the code and the details of SE-Inception-V4 can be found in [[https://github.com/taki0112/SENet-Tensorflow](https://github.com/taki0112/SENet-Tensorflow)]([https://github.com/taki0112/SENet-Tensorflow](https://github.com/taki0112/SENet-Tensorflow)), accessed on 27 October 2021) as the backbone network, which is a combination of Inception-v4 [17] and the squeeze-and-excitation (SE) block [18]. As a convolutional neural network, Inception-v4 has an excellent performance in the field of image recognition. The SE block can adaptively adjust the relationship between different channels by explicitly modeling interdependencies between channels. As the input of the measure network is a multi-channel image and the importance of each channel is quite different, applying the SE block to Inception-v4 can further improve the performance at a slight additional computational cost. After the combination, SE-Inception-V4 processes the image by convolution, pooling, residual connection, squeeze, excitation, and other operations, breaking down the image into features.