MultiColor: Image Colorization by Learning from Multiple Color Spaces

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-02 08:19:54

Abstract: Deep networks have shown impressive performance in the image restoration tasks, such as image colorization. However, we find that previous approaches rely on the digital representation from single color model with a specific mapping function, a.k.a., color space, during the colorization pipeline. In this paper, we first investigate the modeling of different color spaces, and find each of them exhibiting distinctive characteristics with unique distribution of colors. The complementarity among multiple color spaces leads to benefits for the image colorization task.

We present MultiColor, a new learning-based approach to automatically colorize grayscale images that combines clues from multiple color spaces. Specifically, we employ a set of dedicated colorization modules for individual color space. Within each module, a transformer decoder is first employed to refine color query embeddings and then a color mapper produces color channel prediction using the embeddings and semantic features. With these predicted color channels representing various color spaces, a complementary network is designed to exploit the complementarity and generate pleasing and reasonable colorized images. We conduct extensive experiments on real-world datasets, and the results demonstrate superior performance over the state-of-the-arts. The code will be available.

Main paper: 
 introduction: 
 Image colorization is an artful process that seeks to infuse grayscale images with color in a remarkably realistic manner [5, 8, 40, 45] . This technology is pivotal in various fields, including digital art enhancement [33] and legacy photos/videos restoration [41, 42, 47] . The core challenge is how to predict the missing color channels that achieve satisfactory image colorization. Overcoming the challenge requires leveraging potential models that can resolve the interactions between color channels and grayscale.Over the last few years, deep learning techniques have yielded significant improvements in image colorization. Early methods [8, 36, 46] employ convolutional networks to predict the per-pixel color distributions, while recent works [18, 21, 43, 44] take advantage of pre-trained generative adversarial networks [10] or transformer [38] to pursue the vividness and fidelity of colorized images. The pixels within the color image can be represented in various color spaces, such as RGB, HSV, and CIE-Lab. Existing image colorization approaches usually train the model under a specific color space. As shown in Figure 1(b, c ), we observe that they may suffered colorized results bias, especially for the contents with complex color interactions. The color spaces are designed to support the reproducible representations of color with unique properties, and incorporating the properties across different color spaces is helpful to generate pleasing and reasonable colorized images (Figure 1(d) ).In this paper, we develop a learning-based approach, namely MultiColor, that utilizes the complementarity from multiple color spaces for image colorization. Starting from an encoder that extracts multi-scale feature maps, we employ a set of dedicated colorization modules for individual color space. Inspired by the success of query-based methods [4, 6, 7, 18] , the colorization module for a specific color space also employs the powerful attention mechanism combined with a sequence of learnable queries. Specifically, a transformer decoder is first employed to refine color query embeddings, and then a color mapper produces color channel prediction using the embeddings and semantic features. In order to fully tap the potential of multiple color spaces, we further introduce the color space complementary network. The contributions of this paper include the following:• Unlike the existing works based on single color space, the proposed MultiColor employs learnable color queries to modeling various color channels of multiple color spaces, which are utilized to generate visually reasonable colorized images.To our knowledge, this is the first work to image colorization with multiple color spaces. • We design a simple yet effective color space complementary network to combine various color information from multiple color spaces, which helps maintain color balance and consistency for the overall colorized image.the ImageNet [34] colorization benchmark, and demonstrate that MultiColor outperforms recent state-of-the-arts. Furthermore, we test our model on two additional datasets (COCO-Stuff [3] and ADE20K [48] ) without finetuning, and our approach shows very competitive results with multiple color spaces.The remainder of this paper is organized as follows. Section 2 discusses related works and Section 3 briefly reviews the background of our work. Our proposed MultiColor is described in Section 4 and extensively evaluated in image colorization experiments in Section 5. Finally, Section 6 concludes this paper. 
related work: 
 Image Colorization aims to add color information to a grayscale image in a realistic way. Cheng et al. [8] propose the first deep learning based image colorization method. Zhang et al. [46] takes grayscale image as input and predicts the corresponding ab color channels of the image in the CIE-Lab color space. InstColor [36] presents an approach to colorizing grayscale images that takes into account the objects and their semantic context in the image. Some researchers introduce rich representations from pre-trained GAN [2, 19] into the colorization task. Besides, an adversarial learning colorization approach is employed to infer the chromaticity of a given grayscale image conditioned to semantic clues [39] . GCPColor [44] produces vivid and diverse colorization results by leveraging multi-resolution GAN features. BigColor [21] utilizes color prior for images with complex structures. Despite the expansive representation space afforded by GANs, these methods encounter several constraints, particularly when dealing with images exhibiting complex structures, resulting in inconsistent results. Recently, Transformer [38] has been extended to image colorization task [17, 18, 24, 43] . ColorFormer [17] utilizes a Color Memory Assisted Hybrid-Attention Transformer (CMHAT) to generate highquality colorized images. The key innovation of CT2 [43] is using color tokens, which are special tokens that are used to provide information about the color palette of the image. DDColor [18] includes a multi-scale image decoder and a transformer-based color decoder. The two decoder aims to learn semantic-aware color embedding and optimize color queries.Current image colorization techniques are primarily based on single color space which suffers from poor generalization ability. For instance, the models of [17, 18, 43, 44] are built upon CIE-Lab color space, and [8] employ the YUV color space. Although single color space has the ability to represent image color information, there are limitations imposed by the complexity of real-world scenarios, where color distribution can be more intricate and varied. To address the limitations, this work aims to integrate multiple color spaces, which can provide a more comprehensive representation of color information, adaptable to different contexts. By leveraging the diversity and richness afforded by multiple color spaces, we can effectively broaden color scope, offering more adaptable and flexible strategy for the colorization process. Color Space Combination. The combined use of multiple color spaces has garnered significant attention in the computer vision field due to its ability to capture features at various levels. Color-Net [11] is introduced to demonstrate the significant impact of color spaces on image classification accuracy. Kumar et al. [23] proposed a method for enhancing foggy images by fusing modifications of image histograms in the RGB and HSV color space which significantly improves image contrast and reduces noise. Ucolor [26] enriches the diversity of feature representations by incorporating the characteristics of different color spaces into a unified structure. Peng et al. [27] fused multiple algorithms in RGB and HSV color spaces to improve brightness, contrast, and preserve rich details. Wan et al. [40] leveraged the RGB color space for the initial colorization of super-pixels and subsequently utilized the YUV color space for color propagation. This approach struck a harmonious balance between efficiency and effectiveness in the image processing. Mast [25] investigated loss designation in different color spaces, revealing that the decorrelated color space can force models to learn more robust features. DucoNet [37] explored image harmonization in dual color spaces, supplementing entangled RGB features with disentangled L, a, b channel feature to alleviate the workload in the harmonization process. 
background: 
 Given a grayscale image 𝐼 𝑔 ∈ R 𝐻 ×𝑊 ×1 with height 𝐻 and width 𝑊 , the colorization model aims to generate colorized image Î 𝑐 ∈ R 𝐻 ×𝑊 ×3 . One popular paradigm is to predict missing color channels ŷ ∈ R 𝐻 ×𝑊 ×2 and then compute Î 𝑐 through specific color space mapping function F , i.e.,EQUATION)Here ŷ can be 𝑎𝑏 color channels from CIE-Lab, 𝐻𝑆 from HSV, or channels in other color space that represent color. Given the color space to learn missing colors, the transform function F is fixed. Thus, it can not be optimized along with the colorization network during training. Color space refers to specific organization of colors which can be represented as tuples of numbers. Each of them offers a special perspective on color representation [9, 20, 31] . For instance, the CIE-Lab color space is designed to approximate human vision and perception which is device-independent and perceptually uniform, allowing for accurate color reproduction across different mediums and devices. The HSV color space is a cylindrical color model that represents colors based on hue, saturation, and value components representing image colors according to perceptual attributes. And the YUV color space separates the brightness from the color information components and is widely applied in video systems.For the image colorization task, there are only the brightness values for each pixel in the input grayscale image. As various color spaces are utilized to define and reproduce colors in different digital imaging systems, they have with own distinct characteristics and limitations. The brightness can be mapped to one channel of the color spaces, such as the L channel in CIE-Lab and the V channel in HSV. Here we visualize the gamut of different color spaces at a specific brightness value in Figure 2 . We can observe that the illustrated colors in different color spaces are inconsistent, and colors missing from one color space may appear in another color space. For example, the bright green color in the upper left corner of the YUV color space disappears in the other color spaces of the corresponding brightness values.We believe that colorization learning in different color spaces will have different preferences. The channel values in different color spaces can be complementary, and utilizing information from multiple color spaces can boost the quality of generated colorized images. A variety of color spaces facilitate a more nuanced and comprehensive understanding of the color representation. The strategy of multiple color spaces provides a wider range of color representations to accurately convey the subtle changes and complexity of colors in images. Combining the information from different spaces has the potential to overcome learning bias in single space. Through the utilization of multiple color spaces, we can harness their complementary attributes, facilitating a more comprehensive and nuanced depiction of color information. In the following section, we will present our approach to leverage the potential of multiple color spaces to colorize the grayscale images. It contains three components. The encoder aims to extract multiscale feature maps that represent image contents. The colorization modules operate on the learned color queries and generate color channels for individual color space. Finally, the color space complementary network is utilized to combine the color channels of multiple color spaces to generate colored images. We describe below each individual component in detail. 
encoder: 
 The encoder takes the grayscale image 𝐼 𝑔 as input and estimates multi-scale semantic features, which are the essential foundation for the subsequent operations. We follow [18] and adopt ConvNeXt [28] for the feature extractor. The upsampling operation comprises four sequential stages which incrementally enlarge the spatial resolution of the features, and each stage is made up of an upsampling layer and a concatenation layer. Specifically, the upsampling layer is implemented through convolution and pixel-shuffle, while the concatenation layer also incorporates a convolution, integrating features from corresponding stages of the feature extractor by shortcut connections. The encoder generates 4 intermediate feature maps 𝐹 = {𝐹 1 , 𝐹 2 , 𝐹 3 , 𝐹 4 } with resolutions of 𝐻 16 × 𝑊 16 , 𝐻 8 × 𝑊 8 , 𝐻 4 × 𝑊 4 and 𝐻 × 𝑊 . Thanks to the downsampling and upsampling operations, our method can capture a complete feature pyramid. These multiscale features are used as the input of the color space modeling stage to guide the optimization of the color query embeddings and color channels.As shown in the left of Figure 3 , the colorization module contains two key parts, i.e., the transformer decoder and the color mapper. The transformer decoder aims to refine 𝑁 learnable color queries through the multi-scale features. The color mapper predicts different color channels by receiving the refined color embedding and the feature map from the encoder. The whole pipeline follows the meta-architecture of Mask2Former [6] . Transformer Decoder. Standard transformer decoder [22] as follows:EQUATIONEQUATIONEQUATIONwhere X 𝑙 𝑐 is color query embeddings at the 𝑙 th layer, 𝐹 𝑗 is the 𝑗 th intermediate feature map. CA(•), MHSA(•), FFN(•), LN(•) indicate crossattention, multi-head self-attention, feed-forward network, layer normalization; 𝑓 𝑄 , 𝑓 𝐾 and 𝑓 𝑉 are linear transformations. We perform these sets of alternate operations 𝐿 3 times in the 3-layer transformer decoder. Specifically, the first three layers receive feature maps 𝐹 1 , 𝐹 2 and 𝐹 3 , respectively. This pattern is repeated for all following layers. Color Mapper. The color mapper combines the feature map of the last upsampling layer in the encoder and the refined color embedding of the transformer decoder to produce color channels. The structure is shown in Figure 4 , and it can be formulated as:EQUATIONColor queriesH × W × C × H × W × NColor channels where 𝐹 4 ∈ R 𝐻 ×𝑊 ×𝐶 is the feature map, X 𝐿 𝑐 ∈ R 𝑁 ×𝐶 is the refined color embedding from the transformer decoder, Conv is 1 × 1 convolution layer, and ŷ ∈ R 𝐻 ×𝑊 ×2 is the predicted color channels from individual color space.… Conv F 4Since there are 𝑛 𝑐 colorization modules representing multiple color spaces, we denote the predictions as { ŷ1 , ŷ2 , • • • , ŷ𝑛 𝑐 }.The goal of CSCNet is to produce colorized image according to the brightness value and color channels. A straightforward approach = 1. Block1 Block2 Block3 Block4 Block5         conv 1 , 32 BN+ReLU conv 2 , 32 BN+ReLU         × 𝐿 1         conv 1 , 64 BN+ReLU conv 2 , 64 BN+ReLU         × 𝐿 2         conv 1 , 128 BN+ReLU conv 2 , 128 BN+ReLU         × 𝐿 3         conv 1 , 64 BN+ReLU conv 2 , 64 BN+ReLU         × 𝐿 4         conv 1 , 32 BN+ReLU conv 2 , 3 BN+ReLU        is to employ a transform function like Eq. ( 1). Here we employ an alternative approach by using a transform network Φ(•), i.e.,EQUATIONThere are two benefits for this design. First, using the differentiable transform network keeps the model training in an end-to-end manner. Different from the fixed mapping function F , the whole process with Φ is parameter-optimizable. Second, it is easier to extend under the multiple color spaces scenarios, as there exists no direct mapping function that converts multiple color space information into a single color space.Given the predicted color channels { ŷ1 , ŷ2 , • • • , ŷ𝑛 𝑐 } from the colorization modules, the color space complementary network Φ 𝑐𝑠𝑐 learn the color mapping from multi-color channels so that more color representation can be better explored for image colorization. The complementary network based on multiple color spaces can be formulated as:EQUATIONThe detailed structure of the CSCNet is listed in Table 1 . The network is lightweight and consists of five blocks. Each block contains the convolution operation which is followed by batchnorm (BN) layer and rectified linear unit (ReLU) layer. Different settings of conv 1 and conv 2 will be examined in the ablation study. The scale of the output in each block is the same as the input feature. Note that, the output of block5 is 3, corresponding to the 3 channels of produced colorized image.We adopt the following losses to train our image colorization network, including color channel loss, perceptual loss, adversarial loss, and colorfulness loss.Here 𝐿 1 loss is applied to perform color channel level supervision in predicted color channels and groundtruth color channels, i.e.,EQUATION)where ŷ𝑖 (𝑦 𝑖 ) and 𝜆 𝑖 𝑐 are the predicted (groundtruth) color channels and weight for the 𝑖 th color space. Perceptual Loss. We adopt perceptual loss to capture high-level semantics and try to simulate human perception of image quality. We adopt a pre-trained VGG-16 [35] to extract features from both the colorized image Î 𝑐 and the groundtruth color image 𝐼 𝑐 and compute the loss as follow,EQUATION)where 𝜙 𝑗 (.) denotes the first convolutional layer of 𝑗 th block of VGG-16 (𝑗 = 1, 2, 3, 4, 5). 𝜆 𝑗 is the weight of the corresponding layer. Adversarial Loss. We also exploit the difference on the whole image level with the Adversarial Loss [10] . Specifically, we use the popular PatchGAN discriminator 𝐷 [16] , and the adversarial loss L 𝑎𝑑𝑣 is defined as follow:EQUATIONColorfulness Loss. Following [18] , we also introduce colorfulness loss L 𝑐 to generate more colorful and visually pleasing images: 11) where 𝜎 𝑟𝑔𝑦𝑏 (•) and 𝜇 𝑟𝑔𝑦𝑏 (•) denote the standard deviation and mean value on the sRGB color space [12] . Full Objective. Therefore the full objective L for the image colorization network is formed as:EQUATIONEQUATIONwhere 𝜆 𝑐𝑐 , 𝜆 𝑝𝑒𝑟 , 𝜆 𝑎𝑑𝑣 , and 𝜆 𝑐 are the trade-off parameters for different terms. 
experiments 5.1 experimental setting: 
 Datasets. We conduct experiments on three datasets: ImageNet [34] , COCO-Stuff [3] and ADE20K [48] . We use the training part of Im-ageNet to train our method and evaluate it on the validation part (val50k). ImageNet (val5k) is the first 5k images of the validation set. Besides, in order to show the generalization of our method, we test on COCO-Stuff and ADE20K validation sets without any fine-tuning. Evaluation Metrics. To comprehensively evaluate the performance of our method, we use Fréchet inception distance (FID) [14] and colorfulness score (CF) [12] . FID measures the distribution similarity between generated images and groundtruth images, and CF reflects the vividness of generated images. It is worth noting that a high colorfulness score does not always mean good visual quality, because it encourages rare colors, leading to unreal colorization results. Therefore, we provide the absolute CF difference (ΔCF) between the colorized images and the groundtruth images. Besides, we provide Peak Signal-to-Noise Ratio (PSNR) [15] for reference, although it is a widely held view that the pixellevel metrics may not well reflect the actual colorization performance [13, 17, 30, 36, 39, 44] . Implementation Details. In order to keep fair comparison with the sota methods, we follow some experimental settings with previous methods [17, 18] . Our method is implemented using Py-Torch [32] . The network are trained from scrach with AdamW [29] optimizer and set 𝛽 1 = 0.9, 𝛽 2 = 0.99, weight decay = 0.01. For the upsampling stages in encoder, the feature dimensions are 512, 512, 256, and 256, respectively. We empirically set 𝑁 =100 for the color queries in colorization modules. The hyper-parameter 𝜆 𝑖 𝑐 controls the color information weight of different color spaces. We set them to 0.1 for CIE-Lab, 10 for HSV and YUV. Besides, we set 𝜆 𝑗 to 0.0625, 0.125, 0.25, 0.5 and 1 when 𝑗 = 1, 2, 3, 4, 5, respectively. We set 𝜆 𝑐𝑐 to 1.0, 𝜆 𝑝𝑒𝑟 to 5.0, 𝜆 𝑎𝑑𝑣 to 1.0, and 𝜆 𝑐 to 0.5. The whole network is trained in an end-to-end self-supervised manner for 400,000 iterations with batch size of 16. The learning rate is initialized to 1𝑒 -4 , which is decayed by 0.5 at 80,000 iterations and every 40,000 iterations thereafter. All the images are resized to 256 × 256. During training, we adopt color augmentation [18, 21] to real color images. The experiments are conducted on 4 Tesla A100 GPUs.To evaluate the performance of our method, we compare our results with previous state-of-the-art image automatic colorization, including CNN-based methods (CIC [46] , InstColor [36] ), GAN-based methods (GCPColor [44] , BigColor [21] ), and transformer-based methods (ColTran [24] , CT2 [43] , ColorFormer [17] , DDColor [18] ). Quantitative Comparison. Table 2 shows the quantitative comparisons. For the ImageNet [34] , COCO-Stuff [3] and ADE20K [48] datasets, the results of previous methods are reported by [17, 18] . For the missing results of GCPColor [44] , CT2 [43] and BigColor [21] in COCO-Stuff dataset, we show the results by running their official codes. On the ImageNet, our method achieves the lowest FID value, indicating its capability to produce high-quality and realistic colorization results. Besides, our method gains the lowest FID on the COCO-Stuff and ADE20K datasets, demonstrating its strong generalization ability. The lower ΔCF indicates more precise colorization results, and we achieve the lowest ΔCF across all datasets, suggesting its effectiveness in achieving natural and lifelike colorization results. Furthermore, the best scores on PSNR demonstrate MultiColor colorizes images with plausible colors. MultiColor outperforms all previous work with significant margins. Qualitative Comparison. Figure 5 presents visualization of image colorization results. We display comparisons of images in different scenes from the ImageNet validation dataset. Note that the GT images are provided for reference only but the evaluation criterion should not be color similarity. A noticeable trend is that our results exhibit a more vivid appearance. We can see that the ball colorization (Row 1) of previous methods looks unnatural and introduces color bleeding effects in contrast to our consistent dark blue. Meanwhile, our method produces saturated results for the hue of the background (lawn). InstColor [36] employs a pre-trained detector to detect objects and cannot color the whole image well (Column 2). GCPColor [44] and ColorFormer [17] usually lead to incorrect semantic colors and low color richness. DDColor [18] may produce rare colors. Instead, our method maintains the consistent color and captures the details as shown in row 2, 3 and 4 of Figure 5 . Furthermore, our method can yield more diverse and lively colors for whole image as shown in the last row of Figure 5 .We now perform a series of ablation studies to analyze MultiColor. We first verify the effectiveness of modeling multiple color spaces strategy, and then study the effect of some other settings about the CSCNet. Finally, we investigate the effect of the feature scales. All ablation results are evaluated on the ImageNet val-5k dataset. Color Space. Table 3 shows the effectiveness of the strategy for modeling multiple color spaces. We can observe that the incorporation of multiple color spaces significantly facilitates colorization performance. Additionally, we visualize the colorized results of the ablation experiments in Figure 6 . In single color space, the overall color of the colorized images may be incorrect semantic colors and low color richness. The combination of multiple color spaces makes the restored colors more realistic and higher saturation. For example, the color of the chicken is more consistent with human perception (Row 1). Such a remarkable promotion benefits from complementarity among multiple color spaces, which can capture not only saturation of hues but also color contrast. We also analyze the influence of the kernel size of the blocks. As shown in Table 5 , we can see that convolutional kernels of different sizes have an effect on performance. We choose the convolutional operation of 1 × 1 and 3 × 3 as it achieves the best results. Feature Scales. Recall that the multi-scale features are generated from the upsampling operations of encoder. As shown in Table 6 (a), we list the metrics generated by performing the framework on a single-scale feature only. We can observe that a feature with higher resolution tends to boost performance. Table 6(b) shows the results of our approach from multi-scale feature maps. It verifies that the multi-scale features for modeling color space is beneficial for image colorization. Figure 7 shows the visualization of features scale. Number of Transformer Decoder Operation. We vary the number of transformer decoder operation 𝐿 3 to evaluate its effectiveness. The number is set to 3 in our default settings. Other operation number are also tried and the results are shown in Table 7 . With the increase of the number, FID decreases from 4.22 to 2.17, and the PSNR increases from 23.77 to 24.69 dB.  
conclusion: 
 In this paper, we propose a novel framework for image colorization by learning from multiple color spaces. MultiColor combines various color information of multiple color spaces, for powerful and robust representation competence. Specifically, Transformer decoder progressively refines color query embeddings by leveraging multi-scale image features of encoder, followed by producing color channels of various color spaces through color mapper. Furthermore, we propose color space complementary network to combine color channels and achieve internal color space information complementation. Extensive experiments indicate that MultiColor achieve satisfactory colorization results.  
multicolor: image colorization by learning from multiple color spaces: 
 Anonymous Authors) may be bias, and our approach learns from multiple color spaces to obtain colorized image with more detailed color information (d).Deep networks have shown impressive performance in the image restoration tasks, such as image colorization. However, we find that previous approaches rely on the digital representation from single color model with a specific mapping function, a.k.a., color space, during the colorization pipeline. In this paper, we first investigate the modeling of different color spaces, and find each of them exhibiting distinctive characteristics with unique distribution of colors. The complementarity among multiple color spaces leads to benefits for the image colorization task. We present MultiColor, a new learning-based approach to automatically colorize grayscale images that combines clues from multiple color spaces. Specifically, we employ a set of dedicated colorization modules for individual color space. Within each module, a transformer decoder is first employed to refine color query embeddings and then a color mapper produces color channel prediction using the embeddings and semantic features. With these predicted color channels representing various color spaces, a complementary network is designed to exploit the complementarity and generate pleasing and reasonable colorized images. We conduct extensive experiments on real-world datasets, and the results demonstrate superior performance over the state-of-the-arts. The code will be available. 
 
 Appendix:  


Decision: this paper investigates the colorization of grayscale images. specifically, it learns from multiple complementary color spaces using a transformer architecture and a complementary color module. 

initially, the paper receives ratings of BA, R, BA, BA, with praises on the experimental results, the method being simple and effective, and the design being intuitive and well motivated. there are also concerns on the failure cases, reasoning behind system designs, lack of theoretical analysis, need for further clarification, and need for more discussion on the working mechanism. during rebuttal, the authors successfully addressed some concerns, and the final ratings improved to WA, R, BA, BA. 

given the circumstances, the AC recommends Accept. although theoretical analysis would certainly help the readership, as the current manuscript has introduced its approach in a relatively clear manner and has successfully verified its effectiveness via experiments, the AC believes this is worth sharing with the community. with that said, the authors are strongly encouraged to include the content requested by reviewers (e.g. failure cases, discussion on the working mechanism) in their final version.

Reviews: review: paper_summary: paper_summary: This paper proposes to colorize the grayscale image by learning the clues from multiple color spaces. The transformer decoder and the color mapper modules are employed for each color space to extract the color channel prediction. The complementary network is used to generate the final colorized images. The results and the comparisons demonstrate the good performance of the proposed method.
strength_weakness: strength_weakness: The proposed framework extracts the various color channels of multiple color spaces and generates visually reasonable colorized images, which achieve better colorization results than previous methods, qualitatively and quantitatively.
limitations: limitations: 1. What’s a failure case? I did not see any discussion about the failure or bad colorized results. 
2. What's the inference time comparison, training cost comparison, or computation flops comparison? If the proposed model has a much larger complexity than previous methods, it may be unfair to compare the performance with previous methods. 
3. Why not add a branch for RGB color space? 
4. What is the ground truth image for Figure 1?
paper_summary: paper_summary: This paper introduces a method called MultiColor, which utilizes the complementarity of multiple color spaces for image colorization. Initially, ConvNeXt networks are employed to obtain four different-sized feature maps. Subsequently, specialized colorization modules are used for each color space. Inspired by the success of query-based methods, the colorization modules for specific color spaces also incorporate powerful attention mechanisms combined with a series of learnable queries. Finally, a Color Space Complementary Network is introduced to integrate various color information from multiple color spaces.
strength_weakness: strength_weakness: A simple yet effective Color Space Complementary Network is designed to integrate various color information from multiple color spaces, enabling the acquisition of more color information from grayscale images in multiple color spaces.
limitations: limitations: 1. Although the results of the ablation experiments indicate some complementarity among multiple color spaces, the paper lacks detailed theoretical analysis regarding the complementarity of different color spaces.

2. The Colorfulness metric results are not satisfactory, and the visualization results are average.

3. In the Color Space ablation experiments, there are doubts about whether the model was retrained when using only one or two channels. If the model was not retrained, the results in Table 3 cannot conclusively demonstrate the effectiveness of the color space model. (Because originally the model takes input from three color spaces for colorization, now with one color space missing, the colors may become less vibrant.)

4. The description of the content in the paper is not clear enough and requires further elucidation. For example, in Section 4.3, how are the results obtained from multiple color spaces {y_1, y_2, ..., y_n} fused? Is it acceptable to have a variable number of inputs (n)?
paper_summary: paper_summary: This paper focuses on the image colorization task and propose MultiColor, a learning-based approach to colorize grayscale images that combines clues from multiple color spaces. Both quantitative evaluation and visual samples are provided.
strength_weakness: strength_weakness: - The proposed method is intuitive and reasonable.
- The evaluation is sufficient and convincing.
- The implementation details are provided and the ablation experiments are well conducted.
limitations: limitations: - It would be better if failure cases could be provided.
- The method proposed in this paper reconstructs color images directly from monochrome images without receiving any additional guidance. This greatly limits the application of the method in this paper, making it difficult to migrate to tasks such as image color editing.
- Comparisons with generative model-based approaches, such as L-CAD, are missing
paper_summary: paper_summary: The paper proposes a new approach called MultiColor for image colorization that leverages complementary information from multiple color spaces. It contains an encoder to extract multi-scale features, multiple colorization modules to model different color spaces separately, and a color space complementary network to fuse color channels from different spaces. The colorization modules employ a transformer decoder to refine learnable color queries and predict color channels, while the CSCNet generates the final colorized image.
strength_weakness: strength_weakness: 1.	The use of multiple color spaces for image colorization seems novel and well-motivated.
2.	The modular design with dedicated colorization blocks for different color spaces is clear and reasonable.
3.	The transformer-based architecture enables modeling long-range dependencies in the colorization task.
limitations: limitations: 1.	The motivation for using multiple color spaces could be strengthened with more qualitative/quantitative analysis of their individual characteristics and complementarity. It is somewhat surprising that adopting such a simple design can bring so much improvement. More insights into how the multi-color space strategy helps could be provided, such as visualizing intermediate results or predicted channels.
2.	The design of CSCNet is simple but its effectiveness needs more discussion and analysis through ablation studies.
3.	How do individual colorization modules handle color space transformations internally? More details are needed.
4.	Will the performance improve or degrade with increasing number of color spaces? An analysis is required. It seems that HSV space is the most useful one.
5.	For each color channel module, is the supervision information separate?
6.	Since this method adopts multiple color spaces with multiple networks, the complexity and parameters may be larger than other methods. Detailed complexity comparison is needed.
7.	More qualitative results and user study are recommended.