# ColorMedGAN: A Semantic Colorization Framework for Medical Images
## 3 Experiment and Result
### Comparisons with State-of-the-Art Networks  
Colorization of medical images presents unique challenges due to the lack of paired black-and-white and color data, making it difficult to use supervised learning methods. Furthermore, current methods tend to suffer from color bleeding in medical images with texture details, as shown in Figures 8 and 9. In particular, Figure 8 displays the results of [PERSON]'s method [11], which exhibits color bleeding and semantic discontinuity. Moreover, Figure 9 further shows the results of more advanced methods, all of which are affected by unnatural phenomena such as color bleeding, loss of structure and contour information, and discontinuity of color semantics. In addition, due to the lack of semantic information for guiding the color of MRI tissues, the color information in the image cannot reflect the relationship between tissues, thus leading to color inconsistency between different slices of the same organ. In Figure 8a \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **SSMI \(\uparrow\)** & **PSNR \(\uparrow\)** & **FID \(\downarrow\)** \\ \hline Multi modal:1 & **0.885** & 33.40 & **223.14** \\
**Multi modal:2** & 0.884 & **35.95** & **102.84** \\ Multi modal:3 & 0.871 & 28.34 & 215.95 \\ w/o multi modal & 0.875 & 28.01 & 156.72 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of different multi-modal modes in ColorMedGAN. Multi modal:1 denotes element-wise addition after element-wise multiplication and Sigmoid activation function; Multi modal:2 denotes concatenate directly; Multi modal:3 denotes element-wise multiplication and Sigmoid activation function. Figure 8: This is the result of [PERSON]’s method being replicated in our dataset. (**a–d**) are the results of different slices of a patient’s data, and it can be seen that the colors between the slices are not continuous and the colors lose their semantics. Figure 7: The results and details of comparison of the influence of multi-modal discriminator. (**a**) Origin; (**b**) Without; (**c**) With. depict different slices of an organ, and as a result, the same tissue regions in these slices will have different colors. However, different regions may also have the same color. When comparing the four images horizontally, it can be observed that the colors of the tissues in different slices lack consistency. Such colorization is meaningless and fails to reflect the relationship between tissues. We compared several unsupervised coloring models, which were also trained using the unpaired dataset constructed by OASIS1 [38] and HUMAN2 [39]. We selected 1000 oasis MR grayscale images and 610 images from 800 frozen anatomical images composed of HUMAN2 and Korean human datasets [40] for training. As shown in Figure 9, due to the small dataset and lack of intensity constraints, the result of Baseline CycleGAN [17] is far from expectation, the structure of generated images is easily migrated to the modality of frozen anatomical images, edges and contours are completely distorted, textures are almost lost, and the generated images are meaningless; the result of colorCycleGAN [37] shows that the model can ensure the consistency of intensity values, the structure of images can be retained, but due to the lack of global edge constraints, the generated images are obviously stained with colors, and the generated colors are semantically meaningless; [PERSON]'s method [11] generates images with no semantic colors and serious color bleeding, which cannot learn the semantics of structures in medical images; the method of ChromaGAN [52], a generative network is used to infer the chromaticity of a given grayscale image conditioned to semantic clues, is similar to [PERSON]'s, and sometimes the color generation will fail, resulting in poor quality of generated images;From the results obtained by our method, it can be seen that the generated image is more semantically continuous, the texture details and contour edges are better maintained, the color contrast between the tissues is more realistic in the sense.