# Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness
## 1 Introduction  
Computed Tomography (CT) has been one of the essential medical imaging systems and utilized for expert diagnoses. However, the CT images are often distorted by reflection from metallic implants or foreign objects such as pacemakers, catheters, and drainage tubes. Moreover, medical images are sometimes degraded due to the sudden movements of the patient during the scanning phase. Many approaches have been proposed for the restoration of deformed images, which include research results on noise reduction, image translation, or inpainting. Among these methods, inpainting has emerged as a reasonably effective and popular method today. Several studies on medical image inpainting have been proposed, including the technique of handling damaged square-shaped regions [1, 2]. However, in the real situation, the defects are mostly not of the squares, but of arbitrary-shaped regions, which launched the study of medical image inpainting with any damaged forms [3], resulting in the restoration of practical failures with any deformation. These techniques still suffer from incomplete restorations, such as blurred boundaries and loss ofthe organ structures inside the deformed part. To overcome such problems of restoration failures, structural information has been exploited, where the information of edges is the main tool for implementing the learning process [4; 5; 6; 7; 8]. EdgeConnect [4] is a two-stage adversarial model that comprises an edge generator followed by an image completion network. The edge generator hallucinates the edges in the missing area (either square-shaped or arbitrary-shaped). The image completion network fills in the distorted regions using hallucinated edges as a priori for the inpainting. Edge structures and color-aware maps are fused in a two-stage generative adversarial network (GAN) [5]. In the first stage, edges with the missing regions are used to train an edge structure generator. Meanwhile, distorted input images with the missing part are transformed into a global color feature map by the content-aware fill algorithm. In the second stage, the edge map and the colormap are fused to generate the refined image. The authors in [6] proposed a foreground-aware image inpainting system that explicitly disentangles structure inference and content completion. The foreground contours are predicted first and then the inpainting is performed using the predicted contours as a guidance. The Edge-Guided GAN [7] method is an edge-guided generative adversarial network to restore brain MRI images which have distortion or missing parts. A multi-task learning framework with auxiliary tasks of edge finding and gradient map prediction is used to incorporate the knowledge of the image structure to assist inpainting [8]. Even though the edges in the image provide a part of structural information, there are several drawbacks in such methods. First, the edges are obtained from any objects in the image and they do not represent the spatial structures of the organs of interest. Second, the edges are more complex than the specific organ descriptions and this could make it harder for the model to understand the structure of organs. Therefore, we believe that the edges alone cannot provide sufficient knowledge of organ structures in the body, resulting in the still poor restoration quality. The proposed method is trained and predicted in an end-to-end framework. Our contributions can be summarized as follows:  
* We propose a framework based on edge and organ boundary awareness to reconstruct deformed regions in CT images. * We newly introduced the use of organ boundaries in addition to edges to establish enough structural knowledge for the inpainting of damaged regions, including the part of the organs. Specifically, multi-task learning is employed to train the network simultaneously for the prediction of edges and organ boundaries. The use of organ boundaries for the learning of structural information in medical image inpainting has never been tried before, and it is adopted for the first time in the literature.