# Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness
## 4 Experiments and Results
### Ablation Study  
Binary Cross-Entropy (BCE), Mean Square Error (MSE), and Hinge loss are popular loss functions for classification. Cross-entropy calculated a score that summarizes the average difference between the actual and predicted probability distributions for predicting class 1. The score is minimized, and the perfect cross-entropy value is 0. The range of BCE output is from 0 to 1. MSE computes the sum of squared distances between the ground truth value and the predicted value. The Hinge loss function emphasizes examples to have the correct sign, adding more error when there is a difference in the sign between the ground truth and the predicted value. Figure 5 shows qualitative comparison inpainting results between using binary cross entropy loss, L2 loss and Hinge loss in discriminators. Table 4 compares the results from different types of loss functions used in discriminators with the property of square-shaped and arbitrary-shaped regions. Our results outperformed others when we chose Hinge loss for our discriminators during training the model. We also validate the effectiveness of SRM in our model. Table 5 shows the quantitative results with SRM and without using it, with the property of square-shaped and arbitrary-shaped regions. The results show that generated images by the model with SRM are of better performance, which proved the positive effect of SRM in making the model generate features with useful details. Tables 6 and 7 show the quantitative comparison of PSNR/SSIM/MSE/UQI between the multi-task and mono-task framework in the property of arbitrary-shaped and square-shaped regions. Although we integrated the edge learning task or organ boundary task, the results are slightly increased in the PSNR metric. Our method outperformed the rest of the methods when we used a multi-task learning framework based on both edge and organ boundary learning. It does prove the positive value of adding the auxiliary task into the network. Figure 6 presents qualitative comparison inpainting results between using the mono-task framework, multi-task with organ boundary knowledge framework, multi-task with edge knowledge framework and multi-task with boundary combined with edge knowledge framework.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Layer** & **Kernel Size** & **Stride** \\ \hline \multirow{8}{*}{SRM} & Conv2d + PReLU & [6,6] & [1,1] \\  & Conv2d + PReLU & [1,1] & [1,1] \\  & Conv2d & [3,3] & [1,1] \\  & Conv2d & [3,3] & [1,1] \\  & Conv2d + PReLU & [3,3] & [1,1] \\  & Conv2d + PReLU & [1,1] & [1,1] \\  & ConvTransposed + IN +ReLU & [9,9] & [2,2] \\ \hline \multirow{4}{*}{Deep Block} & Conv2d + IN & [8,8] & [1,1] \\  & Conv2d + IN & [8,8] & [1,1] \\  & Conv2d & [1,1] & [1,1] \\  & Conv2d & [1,1] & [1,1] \\ \hline \multirow{8}{*}{SRM} & Conv2d + PReLU & [6,6] & [1,1] \\  & Conv2d + PReLU & [1,1] & [1,1] \\  & Conv2d & [3,3] & [1,1] \\  & Conv2d & [3,3] & [1,1] \\  & Conv2d + PReLU & [3,3] & [1,1] \\  & Conv2d + PReLU & [1,1] & [1,1] \\  & ConvTransposed + IN + ReLU & [9,9] & [2,2] \\ \hline \multirow{4}{*}{Deep Block} & Conv2d + IN & [8,8] & [1,1] \\  & Conv2d + IN & [8,8] & [1,1] \\ \cline{1-1}  & Conv2d & [1,1] & [1,1] \\ \hline \multirow{4}{*}{Deep Block} & Conv2d + IN & [8,8] & [1,1] \\ \cline{1-1}  & Conv2d + IN & [8,8] & [1,1] \\ \cline{1-1}  & Conv2d & [1,1] & [1,1] \\ \hline \multirow{4}{*}{Deep Block} & ReflectionPad2d & \multirow{4}{*}{[7,7]} & \multirow{4}{*}{[1,1]} \\ \cline{1-1}  & Conv2d & [7,7] & [1,1] \\ \hline \hline \end{tabular}
\end{table}
Table 12: Architecture of decoding part 3 in our network.