# An Integration Framework for the Inpainting and Colorization of Arbitrary Masked Grayscale Images
## 4 Experiments
### Results and Discussion  
We conducted extensive experiments to evaluate our integrated framework's performance in handling arbitrary masked grayscale images. Our evaluation encompasses both qualitative and quantitative analyses across various scenarios and comparison with state-of-the-art methods. To the best of our knowledge, there are no existing studies that simultaneously perform both inpainting and colorization in a fully integrated manner. Most prior works address these tasks separately, either focusing solely on inpainting or assuming fully reconstructed grayscale images for colorization. The models we selected as baselines were UNet+PatchGAN [42; 43] and Latent Code [44]. We directly combined UNet and PatchGAN to create a single model capable of jointly learning both inpainting and colorization tasks. The Latent Code model was used for inpainting, while the colorization process employed the same model as ours, DDColor. Figure 2 illustrates the qualitative results produced by our proposed framework. Each row showcases an example of the input and output images, providing a step-by-step visualization of the processing pipeline. In the first column, the grayscale input images with masked regions are displayed, representing the initial input to the system. The second column highlights the detected masks that delineate the areas to be reconstructed. The third column shows the results of the inpainting process, where missing regions are seamlessly reconstructed with high structural fidelity. Finally, the last column presents the colorized output, where the grayscale images are enhanced with realistic and contextually appropriate colorization. This sequence demonstrates the framework's ability to consistently preserve structural integrity while delivering natural color distributions. The quantitative evaluation results presented in Table 2 demonstrate the effectiveness of our proposed framework on the DACON dataset. Using our comprehensive metric \(\text{Score}_{\text{final}}\), which combines structural similarity, masked region quality, and color histogram correlation, our method achieves a score of 0.607, outperforming both the UNet+PatchGAN baseline (0.588) and the recent Latent Code approach (0.547). This superior performance can be attributed to the synergistic integration of our three core components: SAM's precise mask generation, MAT's robust inpainting capabilities, and DDColor's sophisticated colorization approach. The improvement is particularly notable given the challenging nature of the DACON dataset, which includes diverse scenarios and complex masked regions. The results on the Places365-Standard dataset (Figure 3) further demonstrate our framework's robustness across a wide range of real-world scenarios. The framework shows consistent performance across various scene types, including both indoor and outdoor environments, while maintaining realistic textural details and natural color distributions. \begin{table}
\begin{tabular}{l c} \hline \hline
**Method** & \(\mathbf{Score_{final}}\) \\ \hline UNet+PatchGAN [42; 43] & 0.588 \\ Latent Code [44] & 0.547 \\ Ours & 0.607 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Quantitative comparison with state-of-the-art methods on the DACON dataset. The final score (\(\mathbf{Score_{final}}\)) is computed as defined in Equation (5). Figure 2: Qualitative results of the proposed integrated approach. From left to right: the original grayscale input image with arbitrary masked regions, the detected mask and subsequent inpainted image, and the final colorized output produced by the DDColor model. Figure 3: Additional results from the Places365-Standard dataset, showcasing our framework’s performance across diverse scene types. From left to right: masked input, inpainting result, and final colorized output. In comparison with existing methods (Figure 4), our approach demonstrates several key advantages. First, it shows more coherent structural continuation in inpainted regions, which is particularly evident in architectural elements.