# ColorMedGAN: A Semantic Colorization Framework for Medical Images
## 1 Introduction  
Colorization has always been a popular topic in computer vision research [1] and there have been many challenges in general colorization research to date. Human organ and tissue images are often in grayscale, which creates a big challenge when applying colorization methods using supervised learning with paired data. However, color images of medical images can be better visualized [2; 3] and can play a special role in aiding diagnosis and treatment for both professionals and non-professionals. For instance, color Doppler ultrasound measures blood flow velocity through the Doppler effect and displays it in the image through colorization [4]. In addition, modern medicine often employs three-dimensional reconstruction technology to observe human organs, and through colorization and three-dimensional display, their appearance can be made more realistic and intuitive, leading to better observation results [5]. Early image colorization algorithms have traditionally focused on modeling the correspondence between grayscale and color [6; 7]. In 2001, [PERSON] et al. [8] proposed a novel colorization concept based on the observation that channels in the CIELAB(L*a*b* color model developed by The International Commission on Illumination) color space are uncorrelated. They developed a set of color transfer formulas that can be applied to each color component, yielding promising results. Alternatively, some researchers have explored the diffusion of color from local to global regions. [PERSON] and [PERSON] [9] leveraged the local Markov property of the image to minimize the color difference between neighboring pixels and achieve colorization, while [PERSON] et al. [10] implemented a user-guidedapproach with colored sketches to minimize user interaction. However, these approaches either heavily relied on user interaction or resulted in desaturated colorizations, limiting their practicality and applicability [11]. In recent years, the development of deep learning has led to impressive performance in colorization methods [12]. One of the pioneering approaches, Colorful Image Colorization, was initially proposed by [PERSON] et al. [11] and treats colorization as a classification problem. Several studies [12; 13; 14] have employed GAN (generative adversarial network) [15] for image colorization, which has produced impressive results but has not matched the performance of the method proposed by [PERSON] et al. [11]. The Pix2 pix model [12] based on GAN has also achieved excellent performance in colorization tasks. To achieve the colorization of real-world objects, ColorGAN(Image Colorization with Generative Adversarial Networks) [16] was introduced, which utilizes conditional GANs. While methods such as Pix2 Pix have shown promising results, they rely on paired datasets where inputs and labels are paired one-to-one. Architectures like CycleGAN [17], DualGAN(Unsupervised Dual Learning for Image-to-Image Translation) [18], and DiscoGAN(generative adversarial networks that learn to discover relations between different domains) [19] leverage two GANs to map features from one set to another, which is beneficial for datasets with unpaired images (a dataset with non-corresponding input and label data). Recently, Transformer architecture has gained attention in the field of image colorization. [PERSON] et al. [20] proposed a grayscale colorization network (Colorization Transformer), called Colorization Transformer, based on Transformer blocks, which enables the matching of color information to input grayscale images at low resolution. However, these methods have their own limitations. Architectures like Transformer require a large amount of data and computing resources, while self-supervised methods like CycleGAN are prone to producing artifacts during colorization, which may result in the loss of structural information and contours. Low-level semantic information such as contours and geometric shapes plays a crucial role in the colorization process.