Generalizing ISP Model by Unsupervised Raw-to-raw Mapping

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-09 09:54:28

Abstract: ISP (Image Signal Processor) serves as a pipeline converting unprocessed raw images to sRGB images, positioned before nearly all visual tasks. Due to the varying spectral sensitivities of cameras, raw images captured by different cameras exist in different color spaces, making it challenging to deploy ISP across cameras with consistent performance. To address this challenge, it is intuitively to incorporate a raw-to-raw mapping (mapping raw images across camera color spaces) module into the ISP. However, the lack of paired data (i.e., images of the same scene captured by different cameras) makes it difficult to train a raw-to-raw model using supervised learning methods. In this paper, we aim to achieve ISP generalization by proposing the first unsupervised raw-to-raw model. To be specific, we propose a CSTPP (Color Space Transformation Parameters Predictor) module to predict the space transformation parameters in a patch-wise manner, which can accurately perform color space transformation and flexibly manage complex lighting conditions. Additionally, we design a CycleGAN-style training framework to realize unsupervised learning, overcoming the deficiency of paired data. Our proposed unsupervised model achieved performance comparable to that of the state-of-the-art semi-supervised method in raw-to-raw task. Furthermore, to assess its ability to generalize the ISP model across different cameras, we for the first formulated cross-camera ISP task and demonstrated the performance of our method through extensive experiments. Codes will be publicly available.

Main paper: 
 introduction: 
 ISP refers to an image processing system aiming to transform the unprocessed raw images captured by camera into visually pleasing sRGB images for human perception, while serving for various downstream visual tasks [24, 26, 27, 32, 33, 35, 37, 39] , such as object detection, semantic segmentation, and instance segmentation. The system of a traditional ISP includes white balance module, denoising module, demosaicing module etc. Since the features are manually designed, traditional ISP suffers from problems such as Top shows the challenge of ISP's cross-camera deployment, middle shows the application of raw-to-raw in the test phase (i.e., domain adaptation), bottom shows the application of raw-to-raw in the train phase (i.e., data augmentation).tedious parameter tuning, low flexibility and limited adaptability [12, 18] . In recent years, with the rapid advancement of deep learning technology, AI-ISP has emerged as a more effective approach for raw image processing. Current AI-ISP methods either replace sub-modules in the traditional ISP system [1, 3, 5, 6, 11] or the entire ISP [15] [16] [17] [18] with an end-to-end deep neural network. By training on raw images that are paired with expert fine-tuned or professional camera rendered sRGB images, AI-ISP achieves much higher image quality than a traditional ISP.However, owing to the varying spectral sensitivities of cameras, raw images captured by different cameras exist in different color spaces, caused AI-ISP exhibit poor performance when deployed across different cameras. Fig. 1: (a) presents an example to illustrate it, when deployed on a blue camera, the yellow ISP fails to render the sRGB image accurately. While some cross-camera methods and multi-camera datasets have been developed for tasks such as white balance [1, 3, 4, 7] and denoising [8, 14, 38] , research on the cross-camera deployment of entire ISP models remains limited.Intuitively, a raw-to-raw mapping module, designed to convert raw images across camera color spaces, can be placed before the ISP to ensure consistency in the raw data from different cameras, as illustrated in Fig. 1 : (b) (c). Previous raw-to-raw methods [2, 28] require physical access to cameras and generate pixel-levelpaired images from different cameras for training, which is a tedious and difficult process. In addition to this, their methods assume uniform illumination condition, which may not suitable in scenes with complex lighting.Unpaired image translation provides an alternative solution to the raw-to-raw task. However, the inherent domain gaps present in unpaired images taken by different cameras, stemming from variations in noise, blur, and camera physical characteristics, pose a significant challenge for raw-to-raw tasks under unsupervised settings as illustrated in Fig. 2 . Traditional unsupervised image translation methods may struggle when faced with uncertain domain gaps, potentially resulting in erroneous mappings.To address the above-mentioned challenges, we propose a novel unsupervised raw-to-raw model which is trained based on the Cy-cleGAN framework [41] . A key component of our model named CSTPP (Color Space Transformation Parameters Predictor) predicts the color space transformation parameters and applies them in a patch-wise manner, rather than performing per-pixel image translation. This parameterized method allows our model to focus on the domain gap of the color space while ignoring others it cannot map, thereby enabling correct color space transformation. Our proposed model serves dual purposes: it can function as a domain adaptation module during the testing phase of the ISP model, and it can also be used as a data augmentation method during the training phase as illustrated in Fig. 1 : (b) (c), respectively. In summary, the contributions of this paper are as follows:• We propose an unsupervised raw-to-raw model for the first time. In the raw-to-raw task, our unsupervised method performs equally well as Afifi et al.'s semi-supervised method [2] , surpassing significantly the results of other traditional unsupervised methods. • We have extended the raw-to-raw task to multi-illumination scenes and achieved the best results in corresponding quantitative evaluation. • In the proposed cross-camera ISP task, we conducted extensive quantitative and qualitative evaluations. Despite the absence of paired training data from target camera, our method still enables the ISP model to produce visually pleasing results. 
related work 2.1 isp: 
 Traditional ISP typically consists of modules for denoising, demosaicing, white balancing, color toning etc. Experts need to spend considerable time carefully tuning parameters for each module.With the development of deep learning, end-to-end networks have been introduced to directly reconstruct sRGB images from raw images. Ignatov et al. [18] first proposed a high-quality raw-rgb dataset, consisting of pairs of raw images captured by a mobile phone camera and sRGB images rendered by a professional camera. They accomplished this challenging reconstruction task using a pyramid model that trained hierarchically. With the demand for deploying AI-ISP into mobile devices, current state-of-the-art AI-ISP models, such as syenet [15] and microISP [17] , not only achieve high-quality results but also have few parameters.In addition to serving human vision, some researchers focus on the impact of ISP on downstream vision tasks [24, 26, 27, 32, 33, 35, 37, 39] such as object detection, semantic segmentation, and instance segmentation. Mosleh et al. [27] differentiated the ISP modules and deployed them in a series of downstream tasks. They optimized the parameters of the ISP module end-to-end by incorporating the loss from downstream tasks. Sun et al. [35] introduced reinforcement learning into ISP task, they dynamically adjust the parameters of the ISP module based on the performance of downstream tasks. Compared to traditional ISP designed for human vision, their ISP for machine perception can achieve better performance on a variety of visual tasks.While ISP has found success in various imaging systems, the majority of current ISP algorithms are customized for specific cameras. Significant domain gaps between raw images captured by different cameras pose challenges for deploying ISP across various cameras [10] . In general, each camera requires a separate dataset for ISP model training (learning-based ISP) or extensive parameter tuning (traditional ISP). Additionally, training ISP using joint multi-camera raw data is challenging due to the lack of paired data and the difficulty for models to simultaneously map from multiple camera-specific color spaces to sRGB.The objective of the raw-to-raw task is to establish a mapping function 𝑓 capable of accurately transforming raw images from Camera A's color space to Camera B's color space, accommodating diverse scenes and lighting conditions. In brief, the mapping function can be expressed as:EQUATIONwhere 𝐼 𝐴 and 𝐼 𝐵 denote the raw images captured by camera A and camera B, respectively, under identical lighting and scene conditions.From [20, 28] , the raw-to-raw mapping can be approximated as a channel-wise color transformation 𝑇 (𝐿) ∈ R 3×3 . Under illumination condition 𝐿, the packed raw image 𝐼 𝐴 = [𝑟, 𝑔, 𝑏] captured by camera A can be transformed to the color space of camera B approximately as follows:EQUATIONNguyen et al. [28] proposed that by employing irreversible quadratic transformation, equation 2 can be further approximated by Eq. 3:EQUATIONhere, 𝐼 𝐴 𝑞𝑡 = [𝑟 2 , 𝑔 2 , 𝑏 2 , 𝑟 × 𝑔, 𝑔 × 𝑏, 𝑟 × 𝑏, 𝑟, 𝑔, 𝑏], 𝑇 𝑞𝑡 ∈ R 9×3 . Correspondingly, for raw images with four channels [𝑟, 𝑔 𝑟 , 𝑔 𝑏 , 𝑔] rather than three, it can simply generalize that𝐼 𝐴 𝑞𝑡 = [𝑟 2 , 𝑔 2 𝑟 , • • • , 𝑔 𝑏 , 𝑏] and 𝑇 𝑞𝑡 ∈ R 14×4 .Koskinen et al. [22] and C5 [3] attempted to enhance the crosscamera generalization of the white balance module using raw-toraw mapping. However, their methods require ground truth illumination and extensive meta data. Afifi et al. [2] were the first to introduce neural networks into the raw-to-raw task, they placed a standard color chart in the scene and captured raw images using two cameras. Subsequently, they estimated the parameters of the quadratic transformation 𝑇 𝑞𝑡 by minimizing the color differences between the two color charts. They applied this transformation to the raw images and obtaining pixel-level-paried data. They then utilized a Unet-like [34] model for semi-supervised training. Although their trained model can accomplish raw-to-raw mapping for other scenes without having to again capture paired images, access to actual devices to create paired dataset before training is still inevitable. Moreover, this approach is only suitable for ideal scenes with uniform lighting. In general scenes with non-uniform lighting, the transformation parameters vary significantly across different regions of the image, making it impossible to produce high-quality supervised data and conduct supervised training.The goal of the UNIT (Unpaired Image Translation) task is to map images from domain 𝜒 𝐴 to domain 𝜒 𝐵 without ground truth, such as from sunny to rainy or from male to female. Zhu et al. [41] proposed CycleGAN for the UNIT task, utilizing cycle-consistency loss and identity loss to bridge domain gaps. Cut [29] introduced a contrastive learning approach to image translation task and achieved good results in cases where only one-way mappings exist. UVC-GAN [36] introduced a pixel-wise transformer into the CycleGAN framework, while Swin-UNIT [23] addressed performance issues caused by high-resolution images by introducing swin-transformer block which has a linear complexity.Unlike the per-pixel translation methods mentioned above, Chai et al [9] . proposed a parameterized network for predicting color enhancement parameters. Their method achieved excellent visual results on the MIT-5K color enhancement dataset. However, their method applies a uniform transformation across the entire image, resulting in limited flexibility in tasks calling for local transformation. 
method 3.1 patch-wise color space transformation: 
 In this section, we discuss patch-wise color space transformation. It is worth noting that the color space transformation mentioned here refers to the direct conversion from the color space of one camera to the color space of another camera. The pixel values of raw images can be directly modeled as follows [28] :EQUATIONhere, [𝑎, 𝑏] represents the visible light wavelength range 380nm-720nm, 𝑅(𝜆) represents the spectral reflectance property at wavelength 𝜆, 𝐿(𝜆) denotes the spectral intensity at wavelength 𝜆, and 𝐶 𝑐𝑎𝑚 (𝜆) denotes the camera sensor sensitivity at wavelength 𝜆.By discretizing the integral form of Eq. 4, an equivalent matrix representation can be obtained, as shown in Eq. 5:EQUATIONhere, 𝑛 represents the discretized resolution. 𝑅 is a vector composed of the reflectance at various wavelengths. 𝐿 is a diagonal matrix where the elements on its diagonal represent the spectral power of the scene illumination at each wavelengths. Each columns of 𝐶 𝑐𝑎𝑚 represents the sensitivity curve of the red, green, and blue filters respectively (we approximate filters within a Bayer unit to be located at the same position). For a raw image with m pixels, when applying a single transformation, the raw-to-raw task can be seen as solving the following matrix equation for the unknown matrix 𝑇 3×3 :EQUATIONSince both 𝐶 𝐴 and 𝐶 𝐵 are not invertible square matrices, there is no exact solution 𝑇 that holds for any 𝑅 and 𝐿 [20] . The method proposed by Nguyen et al. [28] essentially involves finding an approximate solution 𝑇 (𝐿) for each lighting condition 𝐿, assuming 𝑅 1 , 𝑅 2 , • • • , 𝑅 𝑚 are standard color chart reflectances. While this approach has achieved extremely low color chart errors, it still faces the following problems: Firstly, attempting to approximate multiple color mappings with single 𝑇 (𝐿) may lead to significant discrepancies in the elements of 𝑇 (𝐿), making it more likely to become an ill-conditioned matrix. This, in turn, can result in certain parts of the image being mapped outside the color gamut [2] . Secondly, applying the same transformation 𝑇 (𝐿) (i.e., Global calibration mentioned in Table 1 ) to multiple raw images captured under different lighting conditions yields poor results. From this, we can infer that globally applying the same transformation 𝑇 (𝐿) in a non-uniform lighting scene could degrade performance. Additionally, for neural networks, learning the mapping from images to ill-conditioned matrices is much more challenging compared to learning the mapping from images to regular matrices.To address these issues, we propose predicting transformation parameters in a patch-wise manner. Within a small patch, reflectance and lighting condition are roughly consistent, implying that Eq. 6 only involves a single row. Patch-wise color space transformation can be seen as solving multiple independent single-row equations as illustrated in Eq. 7:                   𝑅 1 𝐿 1 𝐶 𝐴 𝑇 1 = 𝑅 1 𝐿 1 𝐶 𝐵 𝑅 2 𝐿 2 𝐶 𝐴 𝑇 2 = 𝑅 2 𝐿 2 𝐶 𝐵 . . . 𝑅 𝑚 𝐿 𝑚 𝐶 𝐴 𝑇 𝑚 = 𝑅 𝑚 𝐿 𝑚 𝐶 𝐵 . (7)Solving these equations for an approximate solution is much easier for neural networks, and it is also less likely to result in mapping outside the color gamut.Our proposed model is training based on the CycleGAN architecture [41] . CycleGAN is a general unsupervised image-to-image translation framework that learns mutual mappings between domain 𝜒 𝐴 and domain 𝜒 𝐵 by jointly training two pairs of generators and discriminators. The overall framework is illustrated in Fig. 3 . In the raw-to-raw task, 𝜒 𝐴 represents the color space of camera A, while 𝜒 𝐵 represents the color space of camera B. Generators are trained to map raw images between color spaces A and B while preserving other information such as texture, scene and lighting information. Discriminators play a role in discerning between generated images and real ones, guiding generators to produce more authentic outputs. For the discriminator, we employ the same structure as the discriminator in [41] . For the generator, we propose a novel structure based on the CSTPP module, which will be discussed in Sec. 3.3.During training, the total loss function for the two generators 𝐿𝑜𝑠𝑠 𝑔𝑒𝑛 consists of three components. The cycle consistency loss, as shown in Eq. 8, is employed to maintain the structural integrity of the raw image during the raw-to-raw mapping:EQUATIONhere, 𝐼 𝐴 represents the raw image captured by camera A, 𝐼 𝐴𝐵 denotes the output when 𝐼 𝐴 is attempted to be transformed to the color space of camera B by the generator, and 𝐼 𝐴𝐵𝐴 represents the result when this output is fed into another generator, attempting to transform it back to the color space of camera A. Since raw images are always in a specific camera color space, we apply the identity loss as shown in Eq. 9, to encourage the generator to discern the color space of the raw images and to perform an identity mapping on raw images that belong to the same color space:EQUATIONThe adversarial loss, as shown in Eq. 10, is employed to optimize the generators under the guidance of discriminators. It's worth noting that we utilize the lsgan loss [25] instead of the cross-entropy adversarial loss, which facilitates more stable training:EQUATION)here, 𝐷𝑖𝑠 ( * ) represents the probability output by the discriminator. The total loss function for the generator is depicted in Eq. 11:EQUATIONhere, 𝛼 and 𝛽 represent hyperparameters, which are set to 10 and 5 in the following experiments, respectively. The total loss function for the discriminator is given by Eq. 12:EQUATION)In recent years, models based on ViT [13] have seen widespread adoption in the field of computer vision. Benefiting from their powerful capability to extract global information, ViT-based models have achieved remarkable success in many visual tasks. As a variant of ViT, Conformer [31] includes both transformer and convolution branch. While the transformer branch extracts global features, the convolution branch effectively captures local features. Moreover, the two branches exchange information at various levels through upsampling and downsampling mechanisms.As described in Sec. 3.1, our proposed model needs to predict the color space transformation parameters for each patch based on both reflectance and lighting information. Since RGB values are determined by both lighting and object reflectance, the same RGB value may correspond to multiple combinations of lighting and reflectance. Therefore, relying solely on information within the patch cannot separate the two, transformation parameters heavily reliant on global information. Based on this, we use a conformer as the backbone of our generator to extract local and global features corresponding to each patch, which are then mapped to polynomial transformation parameters by the CSTPP module.In the situation that the input raw image has four channels (packed rggb) and applying quadratic transformation, raw image 𝐼 𝐴 captured by camera A undergoes downsampling by a factor of 𝑀. Subsequently, 𝐼 𝐴 will be input into the convolution branch, at the same time, 𝐼 𝐴 will be splited into several patches 𝐼 𝐴 (𝑖) of size 𝑃𝑠×𝑃𝑠, and soon be fed into the transformer branch. Conformer acting as a feature extractor, processes each 𝐼 𝐴 (𝑖) into feature vectors 𝑉 𝐴 (𝑖) ∈ R 𝑑𝑖𝑚 containing both local and global information, where 𝐷𝑖𝑚 is the embedding dimension of transformer branch. After that, the CSTPP module processes the image as follows:EQUATIONhere, 𝑃𝑇 refers to Polynomial Transform, and 𝑈 𝑝 denotes upsampling with bilinear interpolation applied to each parameter matrix 𝑇 𝑞𝑡 (𝑖). The interpolation occurs between adjacent patch to prevent block boundary effects. Subsequently, 𝐼 𝐴 will undergo pixel-wise matrix product with the upsampled parameter matrix, and the resulting raw image 𝐼 𝐴𝐵 in camera B's color space will be the output.The CSTPP module limits the operations that the model can perform on raw image to channel-level polynomial combination, which typically only adjust the color of the image. This transformation ignores domain gaps caused by factors such as scenes, noise, blur, and physical properties of the camera, while effectively addressing what caused by the camera's spectral sensitivity(i.e. color space) as illustated in Fig. 2 . Additionally, since we apply patch-wise transformation rather than global transformation, our model retains considerable flexibility. Even for scenes with complex lighting, our model can effectively perform raw-to-raw task. 
experiment: 
 In subsequent experiments, we use the same model architecture. For the transformer branch, we set the embedding dimension to 256, patch size to 16, head to 4, and depth to 6. For the convolution branch, we set the base channels to 32 and gradually increase to 256 with the increase of depth. At the same time, we apply the same parameter sharing mechanism as Chai et al. [9] to stabilize the training process, this means that our generator A and generator B will share all parameters in the conformer module except for batch normalization layer.We will conduct experiments on the raw-to-raw task under both single illumination and multi illumination conditions, respectively, to demonstrate the effectiveness of our parameterized method and the patch-wise strategy. We will also conduct experiments on the proposed cross camera ISP task to demonstrate the benefits of our method for ISP generalization.Dataset. For the evaluation of single illumination raw-to-raw task, we use the dataset proposed by Afifi et al. [2] . This dataset consists of 392 unpaired images captured by Samsung-s9 and iPhone-x, along with 137 pairs of pixel-level paired images created using the method described in Sec. Experiment Settings. For this part of the experiment, the batch size of our model is set to 4. We use Adam optimizer [21] with betas set to 0.5 and 0.999. The learning rate for the generators are set to 1e-5, and for the discriminators are set to 2e-5. Other methods are trained using the same settings as described in their respective papers or codes. The dataset is split into 256 × 256 patches as mentioned in [2] . Random flipping and rotation are applied for data augmentation during training for 50 epochs. Except for the Afifi et al.'s model, which uses an additional 22 pixel-level paired images for training, all other models only use the 392 unpaired images for training. All models are tested on 115 pairs of full solution pixel-level paired images.Testing Results. The quantitative results in terms of PSNR, SSIM, and MAE (Mean Absolute Error) metrics are shown in Table 1 . Our unsupervised model achieves performance that comparable to Afifi et al. 's semi-supervised model across all metrics. By applying Chai et al. 's method, which is a parameterized model for color enhancement tasks, to the raw-to-raw task, it outperforms several other non-parameterized unsupervised models in SSIM metric, indicating that parameterized models can work better in preserving structural information in images. Raw-to-raw tasks precede ISP or other raw-inputting tasks, so introducing noise, deformation, or artifacts that may harm these tasks is detrimental. Our parameterized model effectively ensures that these issues will not occur, this can be reaffirmed by the qualitative results depicted in Fig. 4 , where we achieved results closest to the ground truth, while non-parameterized unsupervised methods like Swin-UNIT (second column) and UVCGAN (fourth column) show varying degrees of degradation in the information present in the input raw images.Dataset. We utilize the LSMI (Large Scale Multi Illumination) dataset [19] to assess the model's raw-to-raw mapping capability under non-uniform lighting conditions. The LSMI dataset comprises over 7486 raw images captured in more than 2700 scenes with multiple illuminations by three different cameras: Sony, Galaxy, and Nikon. Each image contains three standard color charts placed at different positions. Because there are overlapping scenes captured by both Sony and Galaxy, we utilized a total of 232 pairs of such scene-level paired images for testing, the remaining 3416 unpaired images are employed for training.Experiment Settings. In this part of the experiment, the images are preprocessed following [19] , other training settings are identical to those in Sec. 4.1.Since the test set matches at the scene level rather than the pixel level, we use the KL Divergence Distance of the image pixel value distribution as the metric following [28] . This metric is used to evaluate the consistency of the overall color distribution of the image as illustrated in Eq. 14: EQUATIONExperiment Design. To evaluate the ability of the raw-to-raw model to generalize the ISP model across different cameras, we design the following experiment which treat raw-to-raw model as In this experiment, we apply raw-to-raw models trained in Sec. 4.1, including UVCGAN, Afifi et al. 's, and our model, before various Deep ISP models [15] [16] [17] [18] in their training or testing phase. We denote the raw images captured by iPhone-x and their rendered sRGB images as image pair 𝐴 → 𝐴, and similarly, those captured and rendered by Samsung-s9 as 𝐵 → 𝐵. [9] , our method, and ground truth, respectively. The 1/1.6 gamma correction is also applied for visualization.Specifically, for the evaluation of the data augmentation step, we use the raw-to-raw model to convert the raw images captured by iPhone-x to the color space of Samsung-s9, while keeping the sRGB images unchanged, resulting in a new image pair 𝐴𝐵 → 𝐴. We train the Deep ISP model using this image pair and test it on 𝐵 → 𝐴. It's worth noting that we don't test on 𝐵 → 𝐵 because sRGB images rendered by different ISP also exhibit significant differences. For the evaluation of the adaptation module, we first train the Deep ISP model using image pair 𝐴 → 𝐴 and then testing by 𝐵𝐴 → 𝐴. We then swap two cameras and repeat the experiments. In this experiment, there is no overlap between the training set of the raw-to-raw model and the test set of the ISP task.Due to the fact that 𝐴 → 𝐵 and 𝐵 → 𝐴 are scene-level paired rather than pixel-level paired, the KL Divergence Distance (see Eq. 14) is also used as a metric. Additionally, we employ the FID [30] (Fréchet Inception Distance) metric which commonly used in generative adversarial networks evaluation, to measure the consistency between the generated image set and the ground truth set in high-dimensional space.Testing Results. Cross Camera ISP is a highly challenging task, even minor differences in the raw domain can become noticeable due to the non-linear operations of the ISP. The quantitative results presented in Table 5 demonstrate that our proposed model can effectively enhance the performance of Deep ISP, whether when it be deployed in training or testing phase. Compared with the Afifi et al.'s semi-supervised method, our unsupervised method performs better in FID metric and comparable in KL metric. As shown in in Fig. 6 , our visualization result surpass the other methods significantly. Overall, our results exhibit closer proximity to the ground truth and exhibit fewer instances of local color loss, this further underscores the effectiveness of our parameterired model and patch-wise transform strategy. 
ablation study: 
 We demonstrate the effectiveness of the CSTPP module in raw-toraw task through following ablation experiment. Since the backbone of our proposed model is not a generative model, we use the UVCGAN [36] for this experiment. Specifically, we replace the decoder part of UVCGAN with the CSTPP module. Subsequently, we conduct the same experiment as mentioned in Sec. 4.1, and the quantitative results are shown in Table 3 . The inclusion of the CSTPP module significantly improves the performance of uvcgan, especially a notable increase in the SSIM metric. This indicates that the channel-level combination performed by the CSTPP module can effectively preserve the structural information of the image, without causing local deformation or introducing artifacts. Additionally, since our proposed model is parameterized, we can significantly reduce computational complexity by downsampling input images multiple times. We conducted downsampling from 2 to 16 times during the testing phase, and the quantified results are shown in Table 4 . Even after reducing the pixel count of input image by hurdreds times (16× downsample), our model still maintains good performance.  
conclusion: 
 In this paper, we propose an unsupervised raw-to-raw model for the first time and introduce it to address the generalization issue of ISP. Our parameterized model directly predicts color space transformation parameters in a patch-wise manner, enabling accurate and flexible handling of the raw-to-raw task in general scenes. Extensive experimental results demonstrate that our model excels not only in the raw-to-raw task but also in achieving cross-camera deployment for ISP. Considering the simplicity and effectiveness of our proposed method, it has great potential to be directly deployed into exisintg AI-ISP platforms, which is one of our ongoing work.  
 
 Appendix:  


Decision: The paper received positive ratings (BA, WA, WA) from the reviewers pre-rebuttal. The reviewers raised a few concerns and suggestions, which were duly addresses in the rebuttal. Two reviewers further raised their ratings post rebuttal, thereby resulting in final ratings of WA, WA and A respectively. The AC agrees with the reviewers' positive evaluation of this work and recommends acceptance.

Reviews: review: paper_summary: paper_summary: The paper is well-written and presents a significant contribution to the field of image processing. The proposed method shows promise, particularly in its ability to handle the raw-to-raw mapping task in an unsupervised manner. However, the paper could be improved by addressing the potential weaknesses, such as providing a more thorough comparison with existing methods, discussing the model's limitations, and demonstrating its practical utility in real-world applications. Despite these areas for improvement, the paper is a valuable addition to the literature and offers a solid foundation for future research in this area.
strength_weakness: strength_weakness: 1.The introduction of an unsupervised raw-to-raw model and the CSTPP module is a significant advancement in the ISP field.
2.The model demonstrates comparable performance to state-of-the-art semi-supervised methods in both raw-to-raw and cross-camera ISP tasks.
3.The method has potential for direct deployment in existing AI-ISP platforms, highlighting its real-world applicability.
limitations: limitations: 1. More detailed descriptions of the CSTPP module and the CycleGAN-style training framework are needed to facilitate replication and deeper comprehension. Specifically, it would be beneficial to provide the motivations behind these components.
2. Please include the corresponding comparison mentioned in related work in the experimental results . Additionally, the paper could benefit from a more detailed comparison with state-of-the-art methods, particularly regarding computational efficiency and practical applicability.
3. The paper cites the Conformer model developed by others; please describe it in more detail so that the reader can easily understand its function and relevance.
4. The ablation experiments need to be refined. For instance, Please conduct an ablation experiment on the Conformer. It is important to demonstrate that the validity of the method stems from your proposed module and not solely from the Conformer.
5. Pay attention to the formatting and writting..
paper_summary: paper_summary: The paper proposes an approach to address the challenge of Image Signal Processor (ISP) generalization across different cameras by introducing an unsupervised raw-to-raw mapping model. The authors develop a Color Space Transformation Parameters Predictor (CSTPP) module that predicts color space transformation parameters in a patch-wise manner and integrates this with a CycleGAN-style training framework. The results demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. The design is reasonable, predicting a patch-wise mapping matrix between two domains.
2. The performance appears promising, showing better results than a semi-supervised method published in BMVC 2021.
limitations: limitations: 1. The main issue of the paper is the writing quality in my opinion.
2. The writing of the paper needs improvement. The introduction of the significance of this task is not very clear, especially for those unfamiliar with it. For instance, one might wonder why not use the color matrix saved in metadata to obtain a relatively camera-independent color space first. Providing a discussion supported by experiments would be better.-
3. Many figures are unclear. For example, Fig. 2 seems meaningless and lacks sufficient explanation. It appears that the mainstream of existing methods also involves learning the mapping matrix instead of direct image-to-image mapping.
4. Eq. 7 is somewhat confusing. What is the meaning of “implying that Eq. 6 only involves a single row” in L327?
paper_summary: paper_summary: This paper proposes a CycleGAN-based framework for unsupervised RAW-to-RAW mappings, where a color space transformation parameters predictor module is proposed for accurate color mapping. Experiments demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: (1). The overall framework is well-designed and the proposed ideas are well-motivated.

(2). The overall writing is good.

(3). Experiments and analyses are thorough throughout the paper.
limitations: limitations: (1). I notice that the input images are not fully aligned with the corresponding reference images in Fig 4 and Fig. 5, so do the authors calculate PSNR, SSIM, and MAE directly using the reference image or after aligning the reference image with the input image?

(2). The authors should compare the computational efficiency of the proposed method with other methods in terms of inference speed, model size, and memory consumption.