Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizer
Authors: Yecong Wan
Mingwen Shao
Renlong Wu
Wangmeng Zuo
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Gaussian Splatting
3D Editing
3D Colorization
3D Generation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose a unified and versatile 3D colorization framework, termed Color3D, which enables user-guided colorization of both static and dynamic scenes.

Abstract: In this work, we present Color3D, a highly adaptable framework for colorizing both static and dynamic 3D scenes from monochromatic inputs, delivering visually diverse and chromatically vibrant reconstructions with flexible user-guided control. In contrast to existing methods that focus solely on static scenarios and enforce multi-view consistency by averaging color variations which inevitably sacrifice both chromatic richness and controllability, our approach is able to preserve color diversity and steerability while ensuring cross-view and cross-time consistency. In particular, the core insight of our method is to colorize only a single key view and then fine-tune a personalized colorizer to propagate its color to novel views and time steps. Through personalization, the colorizer learns a scene-specific deterministic color mapping underlying the reference view, enabling it to consistently project corresponding colors to the content in novel views and video frames via its inherent inductive bias. Once trained, the personalized colorizer can be applied to infer consistent chrominance for all other images, enabling direct reconstruction of colorful 3D scenes with a dedicated Lab color space Gaussian splatting representation. The proposed framework ingeniously recasts complicated 3D colorization as a more tractable single image paradigm, allowing seamless integration of arbitrary image colorization models with enhanced flexibility and controllability. Extensive experiments across diverse static and dynamic 3D colorization benchmarks substantiate that our method can deliver more consistent and chromatically rich renderings with precise user control. Project Page: https://yecongwan.github.io/Color3D/.

Published as a conference paper at ICLR 2026
COLOR3D: CONTROLLABLE AND CONSISTENT 3D COLORIZATION WITH PERSONALIZED COLORIZER
Yecong Wan 12 Mingwen Shao Renlong Wu', Wangmeng Zuo
Faculty of Computing, Harbin Institute of Technology Harbin Institute of Technology Zhengzhou Research Institute
3 Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology (yecongwan, hi mirenoongwu @ @gmaill 1. com, mw27 MaNN278811 (22126..0 6 com, wmzuo@hi t. edu, en
Viewpoint
Viewpoint
light blue flower"
Color
"A man in gellow room"
Vienpoint
Vieupoint
Exemplary Visual Results of Color3D. Color3D is unified controllable 3D colorization framework for both static and dynamic scenes, producing vivid and chromatically rich renderings with strong cross-vies view and cross-time consistency. Our method supports diverse colorization con- trols, including language-guin guided (left), automatic inference (middle), and reference- based (right), showcasing its versatility and practical value.
Figure 1:
ABSTRACT
In this work, we present Color3D, a highly adaptable framework for colorizing both static and dynamic 3D scenes from monochromatic inputs, delivering visu- ally diverse and chromatically vibrant reconstructions with flexible user-guided control. In contrast to existing methods that focus solely on static scenarios and enforce multi view consistency by averaging color variations which inevitably sacrifice both chromatic richness and controllability, serve color diversity and steerability while ensuring cross-view view and cross-ting time consistency. In particular, the core insight of our method is to colorize only single key view and then fine-tune a personalized colorizer to propagate its color novel views and time steps. Through personalization, the colorizer learns scene-specific deterministic color mapping underlying the reference view, enabling it to consistently project corresponding colors the content in novel views and video frames via its inherent inductive bias. Once trained, the personalized col- orizer can be applied infer consistent chrominance for all other images, enabling direct reconstruction of colorful 3D scenes with dedicated Lab color space Gaussian splatting representation. The proposed framework ingeniously recasts
approach is able to pre-
"Corresponiing Author

---

Published as a conference paper at ICLR 2026
complicated 3D colorization as more tractable single image paradigm, allowing seamless integration of arbitrary image colorization models with enhanced flexibil- ity and controllability. Extensive experiments across diverse static and dynamic 3D colorization benchmarks substantiate that our method can deliver more con- sistent and chromatically rich renderings with precise user control. Project Page: www.ggtthhh... TLtthub.io/Cooor3)
INTRODUCTION
The emerging 3D reconstruction techniques, such as Neural Radiance Fields (NeRF) Mildenhall al. 2222)) and 31) Gaussian Splatting (3DGS) Kerbl et al. (2023), have catalyzed high-fidelity
Mooel-View synthesis from the observed color images. These methods evolves rapidly CChen et al.
(2022): - et al. (2022): Garbin et al. (2021): y Müller al. (2022): : Yu al. (2024);
Cheng et al. (2024a): Wu et al. (2024);
; Lu et al. (2024), and are extended to dynamic scenes Liu et al. (2023b); Guo et al. y Shao et al. (2023); Duan et al. (2024) : [i et al. ( Yang et al. (2023):
Wu et al. (2024)) Yang et al. (2024c) with object motion modeling. Despite these advances, there is still a compelling challenge, i.e., reconstructing colorful 3D scenes from monochromatic inputs. This can significantly enhance the visual realism and expression of 3D reconstructions, unlocking new applications across digital art, artistic creation, and cultural heritage preservation.
In light of this, straightforward way is to train 3D colorization model on massive collections of 3D scene data. However, the formidable complexity of 3D geometry Lu et al. (2024), and the temporal intricacies in dynamic scenes Yan et al. (2024), making it prohibitively infeasible in practice. An alternative solution is to first colorize multi-view monochromatic images with 2D colorization models and then reconstruct the 3D content. Nevertheless, its naive implementation leads to severe cross-view color shift, due to latest 2D colorization models Kang et al. (2023): Yang et al. (0024a) struggling colorize multi-view images consistently (See Fig. 18).
To mitigate the color inconsistency, a few attempts Dhiman et al. (2023): Cheng et al. (2024b) aim average multi-view color variations via distillation Dhiman et al. (2023) or dynamic color injection Cheng et al. (022bb). While partially effective, they inevitably dilute the palette richness, producing desaturated and tone-flattened results. Furthermore, smoothing color variations makes the colorized results unpredictable, sacrificing user-controlled colorization ability. Moreover, existing studies only focus on static scenes, while controllable colorization of dynamic ones remains an open and unexplored problem, where color consistency in spatial and temporal dimensions should be maintained.
In this work, we suggest novel paradigm for 3D colorization, i.e., first colorizing a key image and then propagating the color information to novel views and time steps by fine-tuning personalized colorizer. The benefits are three-fold. First, it transforms the complex 3D colorization into simpler color information propagation task, enabling more consistent results. Second, since only a key image needs to be controlled, it is easier to achieve controllable 3D colorization. Third, it provides a chance for unifying 3D colorization of both static and dynamic scenes.
Driven by the motivations, we propose Color3D, unified controllable 3D colorization framework for both static and dynamic scenes (Fig. [T. where a personalized colorizer is optimized for each scene to enable consistent color propagation. Specifically, Color3D first selects the most informative view from the monochromatic images, and then colorizes it with desirable off-the-shelf image colorization model. Next, to colorize other views or timesteps consistently, Color3D personalizes a scene-specific colorizer to learn the deterministic color mapping underlying this colorized view. In which a single view augmentation strategy is devised to expand the sample space, thereby enhancing the colorizer's generalization and its capacity to handle unseen visual content. Finally, the personalized colorizer employed to infer consistent chromatic content of remaining views or frames, followed by direct reconstruction of colorful static dynamic 3D scenes via a dedicated Lab color space Gaussian splatting representation, where luminance and chrominance components are optimized separately. Extensive experiments on public static and dynamic 3D datasets substantiate that our Color3D can produce colorful 3D content that aligns with user intentions, outperforming alternatives quantitatively and qualitatively. Additionally, Color3D also achieves visually promising results on real-world in-the-wild scenes, further demonstrating the practicality of our method in legacy restoration.
In conclusion, the main contributions are summarized as follows:
2

---

Published as a conference paper at ICLR 2026
We propose unified and versatile 3D colorization framework, termed Color3D, which enables user-guided colorization of both static and dynamic scenes by tuning a personalized colorizer for each scene, advancing the controllability and interactivity of 3D colorization.
To facilitate personalized colorizer tuning, we propose a customized key view selection strategy along with single view augmentation scheme to enhance coloring richness and generalization. Furthermore, we introduce Lab Gaussian representation to improve color reconstruction fidelity and better preserve scene structures.
Our Color3D delivers vibrant. controllable, consistent results that faithfully align with user intent, significantly outperforming alternatives on variety of benchmarks.
2 RELATED WORK
Radiance Fields for Static and Dynamic Scenes. Radiance field representations Mildenhall et al.
(2021): Kerbl et al. (2023) have driven paradigm shift in novel view synthesis. Neural Radiance
Fields (NeRF) Mildenhal et al. (202T) pioneered implicit volumetric modeling with coordinate-
based neural networks. inspiring numerous extensions (Brrron et al. (2021: 2022 )023); Verbin et al.
(2022); Chen et al. (2022); Fridovich-Kei et al. (2022); Garbin et al.
(2021 Müller et al. (2022) and
subsequent work on dynamic scenes Park et al. 2222aa bb: Wang et a.. (2023); p Fang et al. (2022) Liu
et al. 2023b): Guo et al. (2023): Shao et al. (2023). However, NeRF-based methods remain hindered by costly training and slow rendering. To overcome this, 3D Gaussian Splatting (3DGS) Kerbl et al.)
(2023) introduced explicit Gaussian representation that enables real-time, photorealistic rendering. Recent advances further extend Gaussian splatting to dynamic reconstruction Duan et al. (2024);
Li et al. (2024b); Yang et al. (2023); Wu al. (2024); Yang et al. (202cc), combining efficiency and fidelity. Motivated by these developments, we employ 3DGS and 4DGS as canonical static and dynamic representations, building unified framework for controllable 3D colorization with strong spatial-temporal consistency.
From 2D Colorization to 3D. Image colorization aims recover plausible chromatic content from grayscale inputs, with decades of progress improving realism and controllability Zhang et al. (2017); Wu al. (202T); Kim et al. (2022); Kang al. (2023); et al. (2022). Beyond automatic approaches, user-guided methods leverage reference images He et al. (2018); Huang et al. (2022); Zhao et al. (2021) or language prompts Weng et al. (2023); Zabari et al. (2023); Weng et al. (2022): Chang it al. (203)); Liang et al. (2024). Extending to videos introduces temporal consistency challenges, tackled by matching Yang et al.] (2024b), palette transfer Wang et al.] (2025), ) attention Li al.) (2024a), and memory propagation Yang et al. (2024a). Yet these remain limited to 2D domains, lacking cross-view consistency. Efforts on 3D scene colorization are still nascent. GBC Liao Lia et al.
(2024) exploits video models for continuous inputs, while ChromaDistill Dhiman et al. (2023) and ColorNeRF Cheng et al. (2024b) transfer knowledge from pretrained colorizers by averaging inconsistent predictions, often sacrificing controllability and vividness. Moreover, colorizing dynamic consistency remains unaddressed. Our work bridges this gap with a unified framework for both static and dynamic settings, achieving visually rich, consistent, and user-controllable 3D colorization.
3D scenes while ensuring
3 METHODOLOGY
Our core idea is to fine-tune personalized colorizer for each scene using a single colorized key view and then employ it to propagate the color information to novel views and time steps. By specializing in the scene-specific deterministic color mapping derived from this reference view, the colorizer is able to project the same content in novel views to the corresponding colors in the reference view through the inherent inductive bias capability, thereby elegantly addressing the challenge of color consistency across both static and dynamic 3D scenes.
3.1 OVERVIEW OF COLOR3D
The schematic illustration of the proposed Color3D is depicted in Fig, 2 Color3D is primarily composed two phases, i.e., the personalized colorizer training stage and the 3D scene colorization stage. In the first stage, we begin by selecting single key view from the inputs and apply off- the-shelf image colorization model to generate a colorized reference view. With the augmented data from customized single view augmentation scheme, we then fine-tune a personalized colorizer to learn the definite color mapping underlying this reference view. In the second stage, the personalized colorizer is employed to infer consistent chromatic content for the remaining views or frames, which are directly leveraged to optimize the vibrant 3D scene with a tailored Lab Gaussian representation.
3

---

Published as a conference paper at ICLR 2026
Stage 1: Personalized Colorizer Training
Stage 2: 3D Scene Colorization
Key View Selection
Longuege
2D Colorization Methods
"Ann
L-CAD
Colorful Single View
wear
Single View Aug Augmentation
Automatic
Lab 3DGS
Personalized Colorizee
Personaliedd Colorizer
Endropy
DDCalor
Reference
RGB
Channel
Uniioooor
Lb
Figure 2: The overall pipeline of Color3D. Our framework comprises two primary stages. In the first stage, initially identify the most informative key view from the given monochromatic images and video frames, and employ an off-the-shelf image colorization model to generate a colorized single view. Then, a single view augmentation scheme is elaborated to amplify the data, and the augmented samples are subsequently used to fine-tune a per-scene personalized colorizer. In the second stage, this personalized colorizer is utilized infer consistent chromatic content of the remaining views or frames, and directly reconstruct the colorful 3D scene with Lab color space 3DGS or 4DGS.
3.2 STAGE 1: PERSONALIZED COLORIZER TRAINING
In the initial stage, we aim fine-tune a personalized colorizer using a single colorized reference view, which enables consistent color propagation to novel views and frames. In contrast to training a standard image colorization model on diverse samples with inherent one-to-many color mappings. which results in inconsistent color predictions under minor contextual changes, our personalized colorizer is explicitly trained learn the scene-specific one-to-one color mapping derived from a single image. This allows for consistent chromatic outputs across both spatial and temporal variations.
Key View Selection. Since our goal is to fine-tune a personalized colorizer using a single view, this view should capture the widest range of visual information while minimizing redundancy, which is crucial to ensure that the colorizer can be more robust to handle other views of the scene. To achieve this, we propose heuristic strategy that selects the most informative and representative view by comparing feature similarity entropies across all candidates.
Specifically, given set of N candidate monochromatic inputs, first extract their single dimension feature representations using CLIP Radford et al. (2021) and the concated feature matrix can be denoted as F E I NxD, where D is the dimension of the feature space. To ensure that similarity comparisons reflect angular distances, we normalize each feature vector to unit length F; = TF.l for i 1,2, N. Then, we compute the pairwise cosine similarity matrix measure the relationships between different views For each view i, we define a probability distribution Púj over its similarity with all other views, and use this distribution to compute the information entropy (I:)) associated with view
F where S.y represents the similarity between view and view j.
H(I.) -, Pog Poj
exp(Sit)
Pi
Ee cxp(Sit)
higher entropy value indicates that the view is more evenly related to all other views, meaning encapsulates a broader and more diverse set of visual information. We define the optimal key view as the one that maximizes the entropy:
arg max
Key View Colorization. After obtaining the key view, users can apply any off-the-shelf image colorization model perform colorization on the single view. Such a single-view-basd based strategy allows for the seamless integration of various mature techniques, including language- tagguage-guided Weng al. (2023); Chang et al. (2023); Liang et al. (2024). reference-based He et al. (2018); [Huang et al. (2022), and automatic Kang et al. (2023); Ji et al. (2022) image colorization methods without any additional tuning or modification, thereby offering more flexible and convince controllability over both static and dynamic 3D scene colorization.
Single View Augmentation. In order to avoid overfitting and more robustly generalize to novel views and video frames, we propose single view augmentation scheme that combines generative augmentations and traditional augmentations to generate more samples in which the same content consistently retains its color across all instances. As illustrated in Fig. Aa). the proposed scheme
4

---

Published as a conference paper at ICLR 2026
Generative Aug.
Traditional
Augmented Imoges
LLVVA
Orbit
DDColor
CNN Decoder
Encoder
Adapler
Outpainting Model
12V Model
Novel View Model
Rotation
Flip
RGD
Details Colorizer
Warteeup Herrtionn
f
q
C
SHA
SHL
STL
Grld Shuffle
Elastic
Outpainting
Novel Views
Transform
Channel
ab Channel
α
SH,
SH.
(a) Single View Augmentation
Lab Gaussian
Figure 3: (a): Illustration of the proposed single view augmentation scheme that combines generative augmentations and traditional augmentations to enrich the single colored view with consistent color distribution. (b): Architecture of the colorizer consists of a frozen DDColor encoder alongside trainable adapter and CNN decoder. (c): Lab Gaussian that first warms up with three L channels and then switches to full Lab channels for color optimization.
first leverages generative models to produce plausible scene content beyond the provided single view, while ensuring chromatic coherence with the input to enrich the diversity colorization supervision. Specifically, given single colored view, incorporate three complementary generative strategies: (1) Outpainting: we divide the image into a x 2 grid and apply Stable Diffusion Rombach et al. (2022) to each of the four regions individually imagine plausible scene content beyond their outer boundaries. Meanwhile, we employ LLaVA Liu et al.] (2023a) to generate a descriptive caption based on the image, which serves as the text prompt guide the diffusion process. (2) Image-to- Video: to simulate newly appearing objects and motions in dynamic scenes, we generate continuous video frames conditioned on single image using Stable Video Diffusion (affuusionBBattmaann Blattmaan al. (2023).
This approach provides coherent object motions and gradual changes in viewpoint, enabling the personalized colorizer to learn more robust color assignments for content that emerges over time. (3) Novel View: to simulate consistent novel viewpoints, we employ Stable Virtual Camera Zhou al. 2025) generate views along a predefined orbital trajectory with consistent content across perspectives. Notably, we do not require the generated content to be completely the same as the scene to be colored; instead, we emphasize preserving consistent chromatic styles with the input view, which is sufficient to improve coloring robustness and enhance coloring richness for scene content not included the single view.
Afterward, we apply series of traditional augmentations further enhance the samples, including rotation, flip. grid shuffle that divides the image into grids and randomly disrupts them, and elastic transform (Simard al. (2003) that introduces smooth deformations simulate realistic variations in object shape and structure. This can facilitate the learning of more robust and variation-agnostic color mapping.
Finally, these augmented images are converted from RGB to the CIE Lab color space lizuka et al. (2016), where the channel is used the input the colorizer and the ab channels serve the target for color prediction. The Lab color space not only decouples luminance from chromatic components but also offers a perceptually uniform representation that better aligns with human visual perception.
Colorizer Tuning. With the augmented samples, we can fine-tune personalized colorizer to learn the scene-specific, aggnatio deterministic color mapping underlying the single view. The structure of the colorizer is illustrated in Fig. b). We adopt an encoder from the pre-trained image colorization model DDColor Kang et al. (2023) and freeze its parameters to preserve its strong capability in extracting ligh-level color-relevant features. This enables our model to learn semantic-level color mappings from limited samples, which are inherently more robust to viewpoint and motion variations than low-level luminance-to-cole correspondences. On the other hand, deliberately avoid using a pre-trained decoder, whose built-in color priors are a major source of multi-view color inconsistency. Instead, employ lightweight, learnable CNN--ased decoder initialized from scratch to serve as a clean and unbiased slate for learning personalized color mappings. Furthermore, we equip the encoder with additional learnable adapters Chen et al. (2024a) to better adapt to the current scenario while preserving its original pre-trained priors. The personalized colorizer is fine-ture finettuned using a simple L1 loss:
Lcoloriser pab mda ||
(3)
5

---

Published as a conference paper at ICLR 2026
where poh and G denote the predicted ab channels by the personalized colorizer and the ground- truth ab channels, respectively. Please refer to Appendix for additional network details and a more in-depth explanation of why the personalized colorizer can achieve consistent color propagation.
3.3 STAGE 2: 3D SCENE COLORIZATION
With the personalized colorizer, in this stage, aim to generate consistent colors for other views and video frames, which are then directly utilized to reconstruct colorful static and dynamic 3D scenes. Considering that the luminance information of the scene is already known and it can serve reliably scene structure constraint, we advocate optimizing in CIE Lab color space lliuka et al.) (001)). By decoupling the known luminance (L. channel) from the predicted chrominance (ab channels), our approach can constrain them separately to more stably optimize the 3D scene, attenuating the effect of perturbations in the predicted chromatic information. To achieve this, we deploy 3DG et al. (203)) and 4DGS Wu et al. (2024) to model static and dynamic 3D scenes and propose a Lab Gaussian representation for rendering images in Lab color space and facilitating separate optimization of luminance and chrominance.
Lab Gaussian. The vanilla Gaussian attributes including position (n), rotation (q), scaling (s), opacity (at), and Spherical Harmonies (SH) coefficients (, S.HH,HHH,S SHa, BB) for modeling RGB color channels. To better deouple luminance and chrominance, we retain the same architecture but reformulate the three sets of SH coefficients to the Lab color space as {S SHo), }. which separately parameterize the luminance (L) and chrominance (a and b) channels. Following the original splatting pipeline Kerbl et al. (2023). : we can render images with Lab color space representation directly from these SH coefficients. This strategy allows for fully leveraging the high- fidelity luminance information to provide more stable optimization signals for Gaussian primitives such as position and shape, thereby improving training stability and scene convergence.
Due to the differing numerical ranges between I. and ab channels in the Lab color space, directly modeling chrominance and luminance jointly with the same Gaussian primitives can lead to unstable optimization and convergence. To mitigate this issue, we normalize rendered Lab channels into unified range of [0, IJ. Specifically, the luminance channel L, originally in [0, 100), is scaled by a factor of 1/ //000. The chrominance channels a and b originally spanning [- 128, 127]. are first shifted by 128 and then scaled by 1/255 accordingly. The normalization process can be formulated as:
100* a' = a + 128
L
+ 128
L'
6'
(4)
255
255
Optimization objectives. We deploy the same loss functions for both static and dynamic scenarios and otherwise remain identical with the original 3DGS and 4DGS. For the rendered image in the Lab color space, we optimize the L and ab channels separately. Since the L channel contains the core structural and high-frequency detail information of the scene, we deploy an edge loss Lodge to encourage the Gaussian primitives to better capture fine textures and structural details.
(5) where v denotes the Laplacian operator, and Rd, G are the rendered and ground-trun truth luminance channels, respectively. In addition, we also deploy the L1 and LD-SSIM from 3DGS and the overall objective for the L channel can be written as:
Lodgo = Σ VV (V(B)) - (G4))² + 2
(O(R-)-((GY))
C (1-119)44 + BLDDSSIM where (3 is set to 0.2 similar to the original 3DGS Kerbl et al. (2023). For the ab channels, which primarily contain low- low-frequency chromatic information, we exclude the edge loss term and utilize only the following objective:
+ Ledge:
(6)
Las [1=3
Warm-Up. To facilitate more accurate 3D structural and temporal motion modeling, we propose to first optimize the structure and deformation of the scene with only luminance supervision for warm-up and then co-optimize the scene with color constraints. More concretely, as illustrated in Fig. 3(c), during the first half of training iterations in 3DGS or 4DGS, we allocate all three sets of SH coefficients to represent the L channel and render three-channel luminance image, supervised by the luminance loss L, In the latter half of the training, we then reassign two of the SH coefficient sets to represent the and b chrominance channels for color modeling, and optimize them following the above- above-present presented objectives. This warm-up up scheme stabilizes carly-stage geometry learning while providing strong initialization for color modeling, ultimately yielding more robust colorization.

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative comparisons on static (DL3DV-140, LLFF, and Mip-NeRF 360) 3D scene colorization benchmarks. The top-performing results are highlighted with color.
Method
Language
Automatic
Reference
FIDL CLIP Scoret
MEL
TCL RID, Colorfult MEL TCL FIDJ ReF-LPIPS), MEJ TCP
DL3DV-140 (140 Static 3D Scenes)
Colooiiae
63.56
28.15
0.146
0.038
3DGS+ VidcoColorizer
77.89
22.38
0.128 0.031
Color3D (Ours)
37.48
32.65
0.084 0.017
LLFF (Static 3D Scenes)
GaussianEditor
136.49
0.6346
0.093 0.017
GenN2N
52.63
25.24
0.078
0.015
Ref-NPR
111.93
0.7958
0.112
0.018
ColorNeRI
138. 62
0.6388
0.089
0.018
58.26
21.78
0.062
0.012
103.68
0.7891
0.086
0.013
3DGS +maaeeClorizzrr
87.45
0.6415
0.178
0.035
40.25
34.07
0.084
0.017
67.48
0.7366
0.131 0.023
3DGS+ Voollliidenollrrizrr
25
0.6397
0.092
0.017
51.37
26.68
0.069
0.015
88.19
0.7493
0.095 0.015
Color3D (Ours)
62.84
0.6544
0.051 0.009
35.10
33.99
0.056
0.007
41.36
0.6823
0.055
0.008
Mip-NeRF 360 (Static 3D Scenes)
GaussianEditor
123.71
0.6045
0.088 0.020
GenN2N
83.69
23.44
0.137 0.032
Ref-NPR
146.32
0.7928
0.143 0.026
ColorNeRF
163.12
0.6075
0.093
0.020
87.42
18.90
0.093
0.021
121.57
0.7852
0.106 0.021
3DGSSImageColooizz
112.33
0.6148
0.202
0.042
62.32
29.32
0.135
0.027
86.25
0.7521
0.178 0.032
3DGS+ Cideoooriizr
119.24
0.6122
0.104 0.023
82.17
23.69
0.099
0.022
92.47
0.7582
0.122 0.024
Color3D (Ours) 68.23
0.6246
0.058 0.012
39.03
33,36
0.082 0.016
48.62
0.7028
0.079 0.015
Table 2: Quantitative comparisons on dynamic (DyNeRF and HyperNeRF) 3D scene colorization benchmarks. The top-performing results are highlighted with color.
Method
Language
Automatic
Reference
FIDI CLIP Scoret
MEL
TC
FID, Colorfult MEL TC, FIDI Ref-LPIPS,, MEL TCT
DyNeRF (Dynamic 3D Scenes)
Instruct 4D-to-4D
112.35
0.6082
0.062 0.011
4DGSSImageColoofe
89.39
0.6159
0.124 0.025
39.17
29.45
0.080
0.018
84.65
0.7446
0.128 0.024
IDDIS-VideoColoriaa
88.55
0.6199
0.071 0.014
42.36
23.42
0.073
0.014
91.22
0.7598
0.098 0.018
Color3D (Ours)
58.62
0.6271
0.041 0.007
37.28
32.75
0.062
0.009
52.77
0.6717
0.052 0.008
HyperNeRF (Dynamic 3D Scenes)
Instruct 4D-1o-4D
118.62
0.6053
0.065 0.012
MMGSsImageColoriia.
105.23
0.6175
0.132 0.029
40.32
29,6 66.66
0.090
0.017
76.28
0.7422
0.105
0.018
4DGS + VideoCollrizer
110.46
0.6128
0.095
0.017
41.85
24.68
0.084
0.015
89.33
0.7482
0.085 0.014
Color3D (Ours)
63.28
0.6257
0.045 0.008
37.99
33.68
0.067
0.010
58.63
0.6610
0.055 0.008
4 EXPERIMENTS AND ANALYSIS
4.1 EXPERIMENTAL SETTINGS
Implementation details. All experiments are conducted using the PyTorch framework on NVIDIA RTX A6000 GPUs We evaluate our method three static datasets (DL3DDV-140 Ling al. 2024). LLFF MiFdeeildlahal et al. (2019) and Mip-NeRF 360 Barron et al. (2022) and two dynamic datasets (DyNeRF Li et al. (022) and HyerNeRF Park Park et (202Ib)), all of which are converted to monochrome images for colorization. Structure-from-Motion (SIM) is employed to initialize Gaussian points from monochrome inputs in all experiments. Without loss of generality, both our method and competing approaches deploy the same set of image colorization models for fair comparison: ControlColor Liang et al. (2024) for language-guide (2023) for automatic colorization, and UniColor Huang et al. (2022) for reference-based colorization. Apart from our proposed Lab Gaussian representation and optimization objectives, all other settings kept identical to the original 3DGS and 4DGS frameworks. Notably, our entire pipeline incurs only about eight additional minutes for personalized colorizer tuning, which is considered acceptable for chromatic 3D scene reconstruction.
colorization, DDColor Kang et al.)
Metrics. We adopt CLIP score Radford et al. (2021) for language-guit -uidedd colorization to measure text-image alignment, Colorful Score Hasler & Suesstrunk (2003) for automatic colorization to assess color vividness, and Ref-LPIPS Zhang et al. (2023) for reference-bart
colorization to evaluate perceptual similarity. Fréchet Inception Distance (FI1)) Heusel et al. (007) is employed standard
7

---

Published as a conference paper at ICLR 2026
"A purple tree"
Automatic
ColorNeRF
ColorNeRF
ColorNeRF
3DGS ControlColor
3DGS DDColor
3DGS + UniColor
Color3D (Ours)
Color3D (Ours)
Color3D (Ours)
Figure 4:
more Qualitative comparisons on static 3D scene colorization benchmarks. Our method produces
and color-rich results while maintaining multi-view consistency.
metric for assessing the rendering quality of all colorization tasks. Additionally, to evaluate the consistency of colorization across views and time, we propose a metric termed Matching Error (ME). It utilizes dense image matching model Shenn al. (2024) to identify pixel correspondences between ground-truth color images from different views or time steps, and computes the average color difference at these matched locations in the rendered colorized results. Additionally, compute consistency metrics Dhiman et al.] (2023) based on optical along temporal sequences to more comprehensively assess color consistency across different methods. We report the average of the short-term and long-term consistency scores, which we denote as TC.
correspondences across views and
4.2 RESULTS ON STATIC 3D SCENE COLORIZATTIO
The experimental results for quantitative comparisons in static 3D scenes are shown in the middle and upper portions of Tab. Apart from image colorization baselines, we also include a statc-of-the-ar video colorization model, ColorMNet Yang al. (2024a), reference for more comprehensive evaluations. is observed that our Color3I) delivers remarkable performance gains and outperforms all competitive methods significantly across diverse colorization tasks. A direct combination of 3DGS and image colorization models leads to significantly higher Matching Error (ME), indicating severe multi-view inconsistency. While ColorNeRF Cheng al. (2024b) improves 3D color consistency by averaging inconsistent 2D colorizations, this strategy inevitably sacrifices color richness and controllability. resulting in noticeable degradation FID scores and other task-specific evaluation metrics. We also demonstrate visual comparisons in Fig. 4 As suggested, our method effectively accommodates various forms of user control and accurately generates the desired colorized scenes while preserving strong multi-view consistency. In contrast, direct integration of image colorizers introduces obvious view inconsistency and visual artifacts, while ColorNeRF produces overly uniform color tones with limited color diversity and controllability.
4.3 RESULTS ON DYNAMIC 3D SCENE COLORIZATION
The quantitative and qualitative experimental results for the dynamic 3D scenarios are exhibited in the bottom portion Tab. [ and Fig. 5 It can observed that our method significantly outperforms the combination of 4DGS and image colorizers, particularly under manguage-guide and reference-based settings, achieving better FID scores along with substantial reductions in Matching Error (ME) by 0.83 and 0.70, respectively. Moreover, the visual results further demonstrate that method offers more precise controllability and superior consistency both spatially and temporally.
8

---

Published as a conference paper at ICLR 2026
"A Peach blackout curtain"
Automatic
4DGS ControlColar
4DGS DDColor
4DGS UniCalor
Color3D (Ours)
Color3D (Ours)
Color3D (Ours)
Figure 5: Qualitative comparisons on dynamic 3D scene colorization benchmarks. Our method
consistently yields spatial-tempore coherent results with vivid and perceptually realistic color.
Figure 6: Left: Novel views of static 3D scene colorization from in mo-the-wild wild monochrome multi-vier images. Right: Novel views of dynamic 3D scene colorization from a historical monochrome video.
4.4 RESULTS ON WORLD APPLICATIONS
To more rigorously validate the effectiveness of our proposed Color3D, we also conduct experiments on collected in-the-wile monochrome multi-view images and old movies. As illustrated in Fig. o our method is capable of producing realistic and vivid colorizations while maintaining impressive color consistency across viewpoints and time, demonstrating the effectiveness our approach in revitalizing historical or legacy visual content.
4.5 DIAGNOSTIC ANALYSIS COLOR CONSISTENCY
To better understand the internal behavior of our personalized colorizer and further validate the mechanism behind its strong cross-view consistency, we conducted diagnostic experiment focusing on the feature representations the trained model.
Monochrome images
Feature maps after PCA clustering
Colorized Images
Figure Visualization of cross-view feature consistency and color consistency.
Specifically, we analyze whether colorizer produces consistent and stable features across different viewpoints of the same 3D scene content crucial prerequisite for achieving view-consistent color predictions. To this end, we extract features from two distinct camera viewpoints and project them into a -iwdemenniionll space using PCA for visualization. The reduced features then clustered and codor-coded such that similar features share the same color, enabling intuitive comparison across views. As illustrated in Fig. 7 for the same content in 3D space such flower clusters, pillars, or stone paths the extracted features remain consistent and stable across different viewpoints. This indicates that the trained colorizer maintains highly stable feature embeddings for the same 3D content despite changes in camera perspective. Such feature-level consistency directly aligns with the model's final color outputs and provides strong empirical evidence supporting the design rationale of our personalized colorizer. Please refer to Appendix A.3 for more rigorously theoretical analysis.
4.6 ABLATION STUDIES
In Tab. 3 we conduct ablation experiments on the dedicated components introduced in Color3D. The effectiveness of each proposed component in Color3D is evaluated by gradually integrating them into

---

Published as a conference paper at ICLR 2026
the model, revealing their individual contributions the overall performance. More detailed analyses and visual demonstrations are as below.
Table 3: Ablation studies on Eanguage-guidess colorization on the Mip-NeRF 360 dataset.
Variant
FID
CLIP Scoret
MEJ
TCL
Bascline
115.66
0.6145
0.078
0.018
+ Key View Selection
100.28 (+15.38)
0.6175 (+0.0030)
0.074 (+0.004)
0.017 (+0.001)
+ Single View Augmentation
85,25 (+15.03)
0.6196 (+0.0021)
0.069 (+0.005)
0.016 (+0.001)
+ Fine- Tame-TTning-BBee -Baeed Colorizer
75.48 (+9.77)
0.6225 (+0.0029)
0.064 (+0.005)
0.013 (+0.003)
+ Lab Gaussian
68.23 (47.25)
0.6246 (+0.0021)
0.058 (+0.006)
0.012 (+0.001)
Effort of key view selection. As shown in Fig. aa). randomly selected key view often fail to sufficiently capture the full content distribution of the scene, leading to fine-tuned colorizers exhibiting limited color richness when generalized to unseen regions of the scene. In contrast, our key view selection scheme identifies more informative and representative key view, thereby enabling the propagation of richer and more diverse colorizations across novel views.
Effort of single view augmentation. Fig. B(b) demonstrates that the proposed single view aug- mentation strategy can effectively extrapolate potential visual content beyond the selected view and expand the sample space, thereby facilitating the colorizer to generate more plausible and enriched colorizations for objects originally visible in the key view and better preserve scene saturation.
Effort of personalized colorizer. As illustrated in Fig. R(c), a randomly initialized colorizer, due to its suboptimal feature extraction capabilities, tends cause color drifting and miscolorization when generalized to novel views. On the contrary, our personalized colorizer fine-tunes a pre-trained DDColor encoder to learn a high-level semantically aware color mapping, ensuring more accurate and consistent colorization across novel views in the scene.
Effort of Lab Gaussian. As depicted in Fig. 8 d), the vanilla RGB Gaussian representation, which entangles luminance and chrominance, tends to introduce blurring artifacts and ghosting since color perturbations will simultaneously affect all three channels. In contrast, our proposed Lab Gaussian representation decouples luminance from predicted chrominance for independent optimization, resulting in significantly sharper textures and more faithful structural details.
Random View
Colored Novel View
Single Colorful View
Sampled Augmented
Randomly Initialized
Vanilla
RGB
Selected Key View
Colored Novel View
Novel View w/o Aug.
Novel View w Aug.
Our Colorizer
Lab Gaussian
(a)
(b)
Figure 8: Visual ablation study illustrating the impact of (a) key view selection, (b) single view augmentation, (c) personalized colorizer design, and (d) Lab Gaussian representation.
CONCLUDING REMARKS
In this work, we propose Color3D, an innovative framework that can efficiently reconstruct colorful static and dynamic 3D scenes from monochromatic inputs with desirable controllability and consis- tency. In contrast to existing methods that rely color averaging strategies enforce multi-view consistency, which often leads to monotonous hues and uncontrollable results, our approach elegantly tackles cross-view and cross-time consistency by fine-tuning personalized colorizer using single colorized view. By specializing in the deterministic color mapping underlying this reference view, the colorizer can consistently propagate the intended colors to novel views and frames, preserving color consistency without sacrificing vividness and diversity. Such a design also empowers users control the overall scene via controlling only view, enabling flexible and concise controlled colorization. Extensive experiments on various static and dynamic 3D scene colorization benchmarks manifest the effectiveness, superiority, and controllability of our method.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo BaarinnBruuall,, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5855-5864, 2021.
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5470-5479, 2022.
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 19697-19705, 2023.
Peter Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv: 006601261, 2018.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXi:2211.15122,, 2023.
Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18392-18402, 2023.
Zheng Chang, Shuchen Weng, Peixuan Zhaang, Yu Yu Si Li, and Boxin Shi. L-coins: Language- baased colorization with instance awareness. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19221-19230, 2023.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European conference on computer vision. pp. 333-350. Springer, 2022.
Hao Chen, Ran Tao, Han Zhang, Yidong Wang, Xiang Li, Wei Ye, Jindong Wang, Guosheng Hu, and Marios Savvides. Conv-adapter: Exploring parameter efficient transfer learning for convnets. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1551-1561, 2024a.
Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang. Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In Proceedings of the IEFE/CVF conference on computer vision and pattern recognition, pp. 21476-21485, 2024b.
Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, and Xuejin Chen. Gaussianpro: 3d gaussian splatting with progressive propagation. In Forty-first International Conference on Machine Learning, 2024a.
Yean Cheng, Renjie Wan, Shuchen Weng, Chengxuan Zhu, Yakun Chang, and Boxin Shi. Colorizing monochromatic radiance fields. Proceedings of the AAAI Conference on Artificial Intelligence. volume 38, pp. 1317-1325, 2024b.
Ankit Dhiman, R Srinath, Srinjay Sarkar, Lokesh R Boregowda, and R Venkatesh Babu. Corf: Colorizing radiance fields using knowledge distillation. arXiv preprint arXiv:2309.07668, 2023.
Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, and Baoquan Chen. 4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024.
Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nießner, and Qi Tian. Fast dynamic radiance fields with time-awar aware neural voxels. In SIGGRAPH Asia 2022 Conference Papers, pp. 1-9, 2022.
11

---

Published as a conference paper at ICLR 2026
Sara Siidovich-Keil. Alex Yu. Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5501-5510, 2022.
Stephan J Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, and Julien Valentin. Fastnerf: High-fidelity neural rendering at 200fps. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11346-143555 2021.
Xiang Guo, Jiadai Sun, Yuchao Dai, Guanying Chen, Xiaoqing Ye, Xiao Tan, Errui Ding, Yumeng Zhang, and Jingdong Wang. Forward flow for novel view synthesis of dynamic scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16022-16033, 2023.
David Hasler and Sabine E Suesstrunk. Measuring colorfulness in natural images. In Human vision and electronic imaging VIII, volume 5007, pp- 87-95. SPIE, 2003.
Mingming He, Dongdong Chen, Jing Liao, Pedro V Sander, and Lu Yuan. Deep exemplar-based colorization. ACM Transactions on Graphics (TOG), 37(4):1-16, 2018.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Zhitong Huang, Nanxuan Zhao, and Jing Liao. Unicolor: A unified framework for multi-modal colorization with transformer. ACM Transactions on Graphics (TOG), (1(6):1 2022.
Satoshi lizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Let there be color! joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Transactions on Graphics (ToG), 35(4):1-11, 2016.
Xiaozhong Ji, Boyuan Jiang. Donghao Luo, Guangpin Tao, Wenqing Chu, Zhifeng Xie, Chengjie Wang, and Ying Tai. Colorformer: Image colorization via color memory assisted hybrid-attention transformer. In European Conference Computer Vision, pp. 20-36. Springer, 2022.
Lihan Jiang, Yucheng Mao, Linning Xu, Tao Lu, Kerui Ren, Yichen Jin, Xudong Xu, Mulin Yu, Jiangmiao Pang, Feng Zhao, al. Anysplat: Feed-forward 3d gaussian splatting from unconstrained views. arXiv preprini arXi:22505.23716. 2025.
Xiaoyang Kang, Tao Yang, Wenqi Ouyang, Peiran Ren, Lingzhi Li, and Xuansong Xie. Ddcolor: Towards photo-realistic image colorization via dual decoders. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 328-338, 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139-1, 2023.
Geonung Kim, Kyoungkook Kang, Scongtae Kim, Hwayoon Lee, Schoon Kim, Jonghyun Kim, Seung-Hwan Baek, and Sunghyun Cho. Bigcolor: Colorization using generative color prior for natural images. In European Conference on Computer Vision, pp. 350-366. Springer, 2022.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloc Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference computer vision, pp. 4015 2023.
Jaxing Li, Hongbo Zhao, Yijun Wang, and Jianxin Lin. Towards photorealistic video colorization via gated color- guided image diffusion models. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 10891-10900, 2024a.
Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, al. Neural 3d video synthesis from multi-view vidco. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5521-5531, 2022.
12

---

Published as a conference paper at ICLR 2026
Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time time dynamic view synthesis. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pattern Recognition, pp. 1508-85200 2024b.
Zhexin Liang. Zhaochen Li, Shangchen Zhou, Chongyi Li, and Chen Change Loy. Control color: Multimodal diffusion- interactive image colorization. arXiv preprint arXiv:2402. 2024.
Jiecheng Liao, Shi He, Yichen Yuan, and Hui Zhang. Gbe: Gaussian-based colorization and super- resolution for 3d reconstruction. In The 19th ACM SIGGRAPH International Conference on Virlual-Reality Continuum and its Applications in Industry, pp. 1-9, 2024.
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, I Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. D13dy-10k: A large-scale scene dataset for deep learning- baseg 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22160 2024.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023a.
Yu-Lun Liu, Chen Gao, Andreas Meuleman, Hung- Yu Tseng, Ayush Saraf, Changil Kim, Yung-Wi Yu Chuang, Johannes Kopf, and Jia-Bin Huang. Robust dynamic radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13-23, 2023b.
Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Ben Mildenhall, Pratul P Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ra- mamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Transactions on Graphics (ToG), 38(4):1-14, 2019.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106 2021.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with multiresolution hash encoding. ACM transactions graphics (TOG), 41)) 1-15, 2022.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generaliza- tion in deep learning. Advances in neural information processing systems, 30, 2017.
Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualle Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF international conference computer vision, pp. 5865-5874, 2021a.
Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualle and Steven Seitz. Hypernerf: higher-dimensionall representation for topologically varying neural radiance fields. arXiv preprint 1rXiv:2106.13228, 2021b.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Ruizhi Shao, Zerong Zheng, Hanzhang Tu, Boning Liu, Hongwen Zhang, and Yebin Liu. Tensor4d: Efficient neural 4d decomposition for high-fidelity dynamic reconstruction and rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16632-16642, 2023.
13

---

Published as a conference paper at ICLR 2026
Xuelun Shen, Zhipeng Cai, Wei Yin, Matthias Müller, Zijun Li, Kaixuan Wang, Xiaozhi Chen, and Cheng Wang. Gim: Learning generalizable image matcher from internet videos. arXiv preprint arXiv:2402.11095, 2024.
Patrice Y Simard, David Steinkraus, and John C Platt. Best practices for convolutional neural networks applied to visual document analysis. In ICDAR. IEEE, 2003.
Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields. IEEE Transactions Visualization and Computer Graphics, 2965):7332-274)) 2023.
Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srinivasan. Ref-nerf: Structured view-dependent appearance for neural radiance fields. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5481-5490. IEEE, 2022.
Feng Wang. Zilong Chen, Guokang Wang, Yafei Song, and Huaping Liu. Masked space-time hash encoding for efficient dynamic scene reconstruction. Advances in neural information processing systems, 36:70497-70510, 2023.
Han Wang. Yuang Zhang, Yuhong Zhang, Lingxiao Lu, and Li Song. Consistent video colorization via palette guidance. arXiv preprint arXiv:2501. 19331, 2025.
Shuchen Weng, Hao Wu, Zheng Chang, Jiajun Tang, Si Li, and Boxin Shi. L-code: Language-based colorization using color-object decoupled conditions. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 2677-2684, 2022.
Shuchen Weng, Peixuan Zhang, Yu Li, Si Li, Boxin Shi, et al. L-cad: Language-based colorization with any-level descriptions using diffusion priors. Advances in Neural Information Processing Systems, 36:77174-7771 2023.
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang- 4d gaussian splatting for real-time dynamic scene rendering. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 20310-20320, 2024.
Yanze Wu, Xintao Wang, Yu Li, Honglun Zhang, Xun Zhao, and Ying Shan. Towards vivid and diverse image colorization with generative color prior. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 14377-14386, 2021.
Jinbo Yan, Rui Peng. Luyang Tang, and Ronggang Wang. 4d gaussian splatting with scale-aware residual field and adaptive optimization for real-time rendering of temporally complex dynamic scenes. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 7871-7880, 2024.
Yixin Yang, Jiangxin Dong, Jinhui Tang, and Jinshan Pan. Colormnet: A memory-based deep spatial-temporal feature propagation network for video colorization. In European Conference on Computer Vision, pp. 336-352. Springer, 2024a.
Yixin Yang, Jinshan Pan, Zhongzheng Peng, Xiaoyu Du, Zhulin Tao, and Jinhui Tang. Bistnet: Semantic image prior guided bidirectional temporal feature fusion for deep exemplar-based video colorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024b.
Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene represen- tation and rendering with 4d gaussian splatting, arXiv preprint arXiv:2310. 10642, 2023.
Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20331-20341 2024c.
Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting; Alias-free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19447-19456, 2024.
14

---

Published as a conference paper at ICLR 2026
Nir Zabari, Aharon Azulay, Alexey Gorkor, Tavi Halperin, and Ohad Fried. Diffusing colors: Image colorization with text guided diffusion. In SIGGRAPH Asia 2023 Conference Papers, pp. 1-11, 2023.
Richard Zhang, Jun-Yan Zhu, Phillip Isola, Xinyang Geng, Angela Lin, Tianhe Yu, and Alexei A Efros. Real-time user-guided image colorization with learned deep priors. arXiv preprint (ttp::00500999 2017.
Yuechen Zhang, Zexin He, Jinbo Xing. Xufeng Yao, and Jiaya Jia. Ref-npr: Reference-baie -based non- photorealistic radiance fields for controllable scene stylization. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 4242-4251, 2023.
Hengyuan Zhao, Wenhao Wu, Yihao Liu, and Dongliang He. Color2embed: Fast exemplar-bar image colorization using color embeddings. arXiv preprint arXiv:2/06.080// 2021.
Jensen (Jinghao) Zhou, Hang Gao, Vikram Voleti, Aaryaman Vasishta, Chun-Han Yao. Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models. arXiv preprint arXiv:2503. 14489, 2025.
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1
3DGS 4DGS
3D Gaussian Splatting (3DGS) is an emerging and highly effective representation for novel view synthesis, which models 3D scene set of G anisotropic Gaussians. Each Gaussian gi defines spatial density function:
(i(x) exp (-) mi)
(8)
where 1 i S G, 14 E R denotes the mean (i.e., the Gaussian center), and Σ; E R is the covariance matrix encoding its anisotropic shape. In practice, Σ; parameterized by a positive scaling vector R and unit quaternion qi I+ representing its orientation. Each Gaussian is further associated with an opacity value oę E IR,, and a set of spherical harmonics (SH) coefficients for view-dependent RGB color modeling.
4D Gaussian Splatting (4DGS) extends 3DGS by introducing a learnable deformation field, allowing dynamic scenes be modeled via canonical 3D Gaussians. Temporal dynamics are captured by predicting time-dependent offsets in position, rotation, and scale, formulated as:
9: (14, + 6p4, Q: + 8g., 8, + 88,, 04 (,)) (ōtt )g,, F(u4!);
where FF indicates deformation prediction module.
A.2 MORE METHOD DETAILS AND ANALYSIS
Structure details pensorlized colorizer. As shown in Fig. [),, our personalized colorizer adopts encoder-dccoder structure, augmented with lightweight adapter modules to enable efficient per- scene fine-tuning. The encoder comprises several pre-trained Conv NeXt blocks, each equipped with inserted adapter. During fine-tuning, the backhone ConvNeXt blocks from pre-trained DDColor Kang et al. (2023) are frozen, and only the adapters and decoder are updated. Each adapter (Fig. 9))) follows bottleneck design consisting of depth-wise convolution, nonlinearity, and point-wise convolution. Given an input feature map, the adapter first reduces its channel dimension, applies nonlinear transformation (ReLU), and projects back to the original size. residual connection modulated by an attention weight further refines the adaptation. The decoder is composed of stack of simple and lightweight CNN blocks, each consisting of two convolutional layers, one instance normalization layer, and LeakyRel.U serves activation function. Our structure design effectively allows the colorizer to specialize in learning scene-speeific color mappings from given views while maintaining the semantic representation capacity gained from the pre-trained image colorization model.
Cm x H W
down
Depth-wise Conu.
ConuNex: Block
ConvNeXt Block
ConvNext Block
ConvNeXt Block
Adapter
Adapter
Adapter
CNN Block
Up Sample
Up Sample
Nonlinearity
Gen
X W y
Up
wise Conv.
Cout x x W
(a) Network Structure of Personalized Colorizer
(b) Structure of Adapter
Figure 9: (a) Overall architecture of the proposed personalized colorizer. Only the adapters and decoder are updated during fine-tuning (b) Structure of the adapter module Chen et al. (2024a). which efficiently injects scene-specific colorization capabilities.
16

---

Published as a conference paper at ICLR 2026
Why can the personalized colorizer achieve consistent color propagation? Here, intuitively explain why a colorizer trained on augmented samples from a single colorized view can maintain color consistency when generalized to novel viewpoints and video framess We illustrate this by comparing a standard image colorization model with our per-scene personalized colorizer.
Standard image colorization model. Typically, training general-purpose image colorization model requires large-scale luminance-color image pairs. However, the same semantic object may appear in different colors across images, for example, door might be white in one image but black in another. Consequently, the model learns an uncertain one-to-many color mapping and tend to infer colors based on contextual cues rather than object identity, making it prone to inconsistent color predictions when facing slight viewpoint or motion changes.
Our per-scene personalized colorizer. augmented samples derived from single key view, where the same objects maintain consistent colors across all variations. These augmentations simulate changes in deformation, context, viewpoint, motion, unseen content, etc. This setup encourages the model to learn scene- specific, variation-agnosticc, one-to-one color mapping. Leveraging the inherent inductive bias of deep neural networks Neyshabur et al.) (2017); Battaglia et al. (2018), the trained colorizer can predict consistent colors for novel views and video frames by mapping the same content to the corresponding colors it learned during training.
contrast, our personalized colorizer trained on
A.3 THEORETICAL ANALYSIS OF WIYY THE LEARNED MAPPING REMAINS CONSISTENT UNDER LARGE VIEW CHANGES.
To help readers gain deeper understanding of how the proposed personalized colorizer achieves multi-view consistent color propagation, this section provides rigorous convergence guarantees that theoretically establish its effectiveness.
Notation, Let S denote the scene surface (luminance). For a 3D surface point X E S and a camera/view v V, let (x) denote the image patch (or local crop) containing the projection of X. Let ø I + Rd be the pretrained encoder (frozen in our pipeline), and v : R2 - IR be the lightweight adapter color decoder (which we optimize per scene). Define the composite F v For two views v, o' of the same 3D point X we define the color discrepancy
A.(X;u, v') = -1
('(ww(()))
High-level goal. Show (under explicit assumptions) (i) a decomposed upper bound on A, that isolates encoder and adapter/decoder contributions, and (ii) how single-view augmentation con- sistency training reduces the bound's dominant terms. Provide (iii) convergence statements for adapter/decoder optimization under clearly stated assumptions.
A.3.1 ASSUMPTIONS
We explicitly state the assumptions used below. These are empirical/practical but necessary for rigorous statements.
A1 (Encoder local Lipschitzness) There exists Lg > 0 and 2 0 such that for any patches T,
(6(7)=((77)) < T. I |- I']l + cs.
The term es models finite-capacity non-smooth residual errors and is assumed small when
1, are semantically similar.
A2 (Approximate viewpoint equivariance) For typical geometric viewpoint transform T relating views of the same 3D point,
[T 6(T[))| 8eq.
where deg is small under moderate viewpoint changes (this expresses encoder's empirical multi-view invariance).
A3 (Adapter local Lipschitzness) The adapter &b is locally Lipschitz on the relevant feature domain: there exists Ly > 0 such that for features 2, 2,
[[x(()) <>'')l| < Løllz - -
In practice, Tag be controlled via architecture size and weight regularization.
17

---

Published as a conference paper at ICLR 2026
A4 (Augmentation consistency objective) Training uses empirical loss combining super- vised reference fit (the augmented images) and a consistency term:
(()) Ex Σ ((
+ X
Σ (NN Kep
where are adapter parameters, Tk are single-view augmentations, and A 2 0.
A.3.2 LEMMA 1 (FEATURE DIFFERENCE UPPER BOUND).
Under AI-A2, for any two views v, v' of the same 3D point X,
[I)(T.(X)) (X))l I - (X) - Io [X)]] co deq.
Proof. By triangle inequality, write I, ~ T[..] + where models sampling/noisseoocchsiiin residual. Then [T.) - (f - (l) - (T.III) + ( (I.
By A2 the first term is S 5eq. For the second term apply A1 with /' I, and = T(I,), noting 1| - Torl || ||vl|: absorbing ||z]|| into 6 yields the stated bound.
[T[[]] w
A.3.3 LEMMA 2 (COLOR DIFFERENCE VIA FEATURE DIFFERENCE).
Under A3, for any two views v, v',
Ax(x,,, S Lov (I)(1.(X)) (X)((
Proof. Immediate from the Lipschitz property of v/.
A.3.4 PROPOSITION 1 (DECOMPOSED COLOR DISCREPANCY BOUND).
Combining Lemmas A.3.2 and A.3.3 for any 3D point X and views u, v',
A s Le (LolI I + cs + Seg
Proof. Direct composition of the two lemmas.
The bound decomposes color difference into: (i) image-level discrepancy [ T (sampling. occlusion), (ii) encoder approximation residual eg, and (iii) encoder equivariance error deq Reducing Ly (smaller adapter sensitivity) or reducing the effective ||4, - L, |-sensitivvtyy (via consistency training that shrinks feature/ color variance under T3) will reduce A
Interpretation.
A.3.5 How CONSISTENCY TRAINING COMPRESSES THE BOUND (INFORMAL BUT QUANTITATIVE SKETCH).
We formalize the effect of the consistency term by considering the second moment (variance) of color outputs across augmentations for fixed 3D point:
VP (X) == Er [[((((((((())))) - m. (X)])),
(X) = E, ([(((((((())))))
The consistency penalty directly bounds Vi (X). Under standard first-order optimality conditions (or gradient flow dynamics), increasing À reduces the equilibrium value of ViJ (X). Using the Lipschitz relation in Proposition 1 and Jensen, an upper bound on expected squared color discrepancy across sampled view pairs (v, o') can be written
Ex,,, [ (Tug T.p EEI T 113] + (Iap) ).
The consistency term reduces the first RHS term by effectively shrinking the empirical [[[],, Ter 1-- sensitivity through learned invariance in sb. We denote this shrinkage by factor ()) E (0, 1] which empirically decreases increases. Thus
F & (7A) (I. T.s) - f.r 1 [ (I.)) + )eg)
precise closed-form for (A) depends on dymamics/arcciitcctrr:; we treat (7) as observable and empirically verifiable (see diagnostics below).
18

---

Published as a conference paper at ICLR 2026
A.3.6 JACOBIAN / SENSITIVITY PERSPECTIVE (FORMALIZATION).
Let (1) = V (1) denote the Jacobian of F' u wrt image pixels (vectorized), By first-order Taylor expansion, for small image perturbations 8I,
(F((+61)) F() & [1) |
By chain rule Jr(!) Jo (o( and hence || < | || Consistency training
implicitly penalizes || (or its expected norm) because reducing output variance across known transforms forces smaller directional derivatives along cenfffor--nnueedd directions; therefore the product || is reduced, decreasing sensitivity viewpoint siewpoint-inuuced induced perturbations.
A.3.7 CONVERGENCE RESULTS FOR ADAPTER/DECODE OPTIMIZATION
We give convergence statements: Assume by is L-smooth parameters (i.e., VoL is Lipschitz with constant L) and L is lower bounded. Then gradient descent (or stochastic gradient descent under standard assumptions) converges to a first-order stationary point: lim;- [Vv][(0)]]
0. we further assume the Polyak Lojjsieee (PL) condition holds locally (i.e., there exists >> 0 with [()] 2 ((((0) - )), then gradient descent converges linearly to a global minimizer with rate 1 ax..
Concluding statement. The preceding derivation has provided (i) decomposed upper bound on cross-view color discrepancy, (ii) a Jacobian-based mechanism describing how consistency training reduces sensitivity to viewpoint-induced perturbations, and (iii) standard stationary-poin results for smooth nonconvex adapter/decoder. The above analysis theoretically substantiates that the proposed personalized colorizer, when optimized with consistency constraints, is guaranteed to produce view-invariant color predictions across varying viewpoints.
A.4 METHOD EXTENSION
All-in-one training scheme for large-scale datasets. Essentially, our personalized colorizer only includes spene-specific adapter and a lightweight decoder, while the large encoder is shared across all scenes. This design enables us to jointly train a large number of scenes using a routing-based strategy. The semantic illustration of this mechanism is provided in Fig. TO Concretely, images from different scenes are stacked along the batch dimension, and each sample is routed through its corresponding adapter and decoder based the scene ID, which enables efficient all-in-one training. By applying this all-in-one personalized colorizer training scheme, we are able to train colorizers for all 140 scenes in the DL3DV- 140 dataset within only 55 minutes. This experiment further demonstrates the practicality and efficiency of our approach for large-scale real-world deployment.
DDColor Encoder
Decoder Scene
Adapter for Scene
Decoder for Scene
Adapter for Scar
Adapter Scere
Decoder for Scenne
Figure 10: Illustration of the proposed all-in-one training scheme for large-scale datasets.
Generalized to feed-forward style paradigm. To further explore the applicability of the proposed framework beyond per-scene optimization, we investigate its extension to feed-forward 3D gen- eration paradigm. As illustrated in Fig. [1] we first apply our proposed single-view generative augmentation strategy on large-scale image dataset to synthesize substantial spatial and viewpoint variations. Using only one colored image as the key view, we train generalizable personalized colorizer based on a transformer network that learns to colorize all augmented views in manner. At inference time, follow the exact pipeline described in the paper: colorize single input view and then propagate its chroma to other viewpoints using the generalizable personalized colorizer, without any per-scene optimization. The resulting set of multi-vien colorized images can
19

---

Published as a conference paper at ICLR 2026
One-Time Personalized Colorizer Training
Feed-Forward 3D Scene Colorization
Key View
Language
Personalized Calorizer
"An. owl uear
LOOAD
Colortal Single
Generailzabie Pesenalibed Colorize
Food Facd fonword 30 Generation Model
Generalimble
20 Colobattin
Automatic GDCala
Siggle View
Augmentizion Ausmentrrioo
Enntrapy nfflnnl.
from Lergo scale Patasets
Reference
RGB
Figure 11: Illustration of the proposed feed-forwart style 3D colorization framework.
then be fed into a feed-forward 3D reconstruction model-such as AnySplat Jiang et al. (2025) to directly obtain a fully colored Gaussian representation. We trained the generalizable personalized colorizer on the COCO dataset and validated the feed-forward 3D colorization pipeline on the 1------0-Beechaatt demonstrating the feasibility of this extension.
As shown in Tab. and Fig. 12 our experiments indicate that the proposed framework already exhibits promising corwad 3D colorization potential. While fully feed-forward solution indeed appealing, our observations show that a globally generalized colorizer still falls short of per-scene adapter in both chromatic richness and multi-view consistency, and suffers from significant resolution degradation. Given the large-baseline viewpoints and high-resolution reconstruction settings considered in our work, the per-scene optimization route remains the most practical and reliable choice at present. Nonetheless, this preliminary investigation highlights a viable direction for future research.
Table 4: Quantitative comparisons of automatic colorization the DL3DV-140 dataset.
Method
FID,
Colorfult
MEJ
TCJ
Generalizable Colorizer + AnySplat
82.36
24.37
0.142
0,030
Per-Scene Colorizer + AnySplat
66.78
29.39
0.109
0.025
Generalizable Personalized Colorizer AnySplat
Per-Scene Personalized Colorizer AnySplat
Visual comparison of generalizable personalized colorizer and per-scene personalized colorizer for feed-forward 3D colorization.
Figure 12:
A.5 MORE EXPERIMENTS
Dataset details.
(0-Benchmark Ling et al. (2024) consists of 140 large-scale 360 real-world scenes. All DI.3V- 1EADD-10-BBR 10-
images are processed at resolution of 960 x 540 for our experiments.
LLFF Dataset Mildenhall al. (2019) consists of 8 aerward-facine real-world scenes. Following prior work, we select every eighth image for testing and use the remaining images for training. All images processed at resolution of 1008 756 for our experiments.
Mip-NeRF 360 Dataser Barron et al. (022) comprises 9 worldworld scenes, including outdoor and 4 indoor environments. Following the same protocol LLFF, we one-eighth of the views for testing and the remaining views for training. All experiments are conducted on images downsampled by factor of 4 (approximately 1K resolution).
DyNeRF Dataser Li et al. (2022) includes six 10-second video sequences captured at 30 fps by 15 to 20 cameras with face forward perspective, involving extended periods and intricate camera motions.
HyperNeRF Park et al. (20216) is another dynamic dataset captured using one or two cameras following relatively straightforward camera trajectories, but with more intricate camera motions.
20

---

Published as a conference paper at ICLR 2026
More implementation details of pensorlized colorizer tuning. For the single view augmentation strategy, we first generate nine augmented samples based on the original view. Specifically, four samples are produced using the outpainting model Stable Rombacch et al. (2022), one sample is extracted from the final frame generated by the image-to-video model Stable Video Diffusion Blatiman et al. (2023), and three samples are obtained by sparsely sampling along an orbit using the novel view synthesis model Stable Virtual Camera Zhou et al. (2025), together with the original view itself. The nine samples were then randomized to apply traditional augmentation operations. During training, the samples input to the personalized colorizer are randomly cropped to 320 x 320. We train the personalized colorizer for 4K iterations using the Adam optimizer. During training, only one sample is used per iteration, i.e., batchsize = 1, which allows the model to focus on capturing the most distinctive and rich color information from the given sample. The learning rate is initialized to x 10" which is steadily decreased to x 10 using the cosine annealing strategy.
Quantitative comparison with ChromaDistill. Additionally, following ChromaDistill's experimen- tal setup, we conduct automatic image colorization experiments using BigColor Kim et al. (2022) across four scenes: Cake, Pasta, Buddha, and Leaves. The experimental results are depicted in Tab. b It is observed that method achieves substantially higher consistency than ChromaDistill in both short-term and long-ten consistency. Notably, on the Cake scene, our approach improves long-term consistency by 0.023, further demonstrating the superiority of the proposed personalized colorizer in preserving cross-view coherence and validating the effectiveness of our overall framework.
Table 5: Quantitative comparisons with ChromaDistill on cross-view short-term and long-term consistency,
Short-Term Consistency 4
Long-Term Consistency
Method
Cake
Pasta
Buddha
Leaves
Cake
Pasta
Buddha
Leaves
ChromaDistill (BigColor)
0.019
0.015
0.015
0.008
0.033
0.025
0.023
0.013
Color3D (BigColor)
0.008
0.009
0.009
0.007
0.010
0.018
0.012
0.011
Single view or more? Due to the inherent instability of 2D colorization models, even small perturba- tions in the input image can lead to noticeably different color predictions for both individual objects and the overall scene style. When two or more key views are used for training the personalized colorizer, these inconsistencies across views introduce conflicting supervision, which often causes the model to converge to color averaging or to produce view riew-dependent color shifts during 3D colorization. As shown in Fig. [3] the results obtained by training with two views clearly exhibit such inconsistencies, making it difficult for the personalized colorizer to maintain coherent colors across views. In addition, the quantitative comparison in Tab. shoows that using single key view achieves better consistency and avoids both color drifting and color averaging, leading to superior FID and Colorful scores.
Single Key Views
Key
Colorization Results
Colorization Results
Figure 13:
Visual comparison of personalized colorizer training with single vs. two key views.
Robustness to key view selection. Our framework demonstrates strong robustness with respect to the choice of the key view. As illustrated Fig. 14 varying the selected key view consistently leads to coherent and chromatically vibrant 3D colorization results. Even when the first key view contains almost no grass, our approach can still produce reasonable and vibrant colors. Moreover, for distant buildings that do not appear in the key view all, our method can still infer plausible colors for them.
21

---

Published as a conference paper at ICLR 2026
Table 6: Quantitative comparison of using different numbers of key views for automatic colorization on the Mip-NeRF-360 dataset.
Number of Key Views
FID|
Colorful
MEJ
TCL
Three Views
52.38
25.26
0.098
0.022
Two Views
45.62
27.36
0.092
0.018
One View
39.03
33.36
0.082
0.016
These can be primarily attributed the generative data augmentation strategy and the colorizer's ability to interpolate colors for unseen content, which together enable stable colorization even when the selected key view is not ideal.
)
Content
Content
Key
Reconstruction
Figure 14: Visual comparison of the colorization results with different key views.
What is the color of semantically similar but visually different objects? To further evaluate the generalization capability our personalized colorizer, we conduct an experiment on scenes containing semantically similar yet visually distinct objects. As shown in Fig. 15] our method consistently assigns plausible and stable colors even when object appearances the target views deviate from those in the key view. This behavior arises from two key design choices. First, the frozen encoder, pre-train pre-trained on large-scale datasets, provides robust and viewpoint-invarre features. During optimization, the encoder yields consistent representations for the same object across different viewpoints while still capturing feature differences for objects that are semantically similar but visually distinct. Second, the decoder, trained jointly the key view and our generatively augmented images, learns broader color mapping space. This enables it to perform effective color interpolation when encountering unseen visual features.
visual
Flowerpot
Trunk
Door or Window
Key
Figure 15: Visual comparison the colorization results on objects that share semantic similarity but differ in visual appearance. More views can be found in Fig. 23
Performance under challenging lighting conditions. We also demonstrate the performance of our method under challenging lighting conditions in Fig. [6 Specifically, we evaluate scenes with complex lighting, including lamps, reflections, shadows, and highly specular materials. Our results show that the proposed method effectively handles these challenging scenarios and produces consistent and reasonable colorization even in regions affected by shadows and projections. These findings further validate the reliability and effectiveness of our approach in practical applications.
Optimization efficiency, In Tab. 7 we compare the optimization time across different methods. While Stage 1 introduces extra 8 minutes, the Stage 2 optimization is significantly faster than that of 3DGS-ImageColorizer and the overall time is even shorter than reconstructing a 3D scene from normal colored images using 3DGS. This efficiency is mainly due proposed
22

---

Published as a conference paper at ICLR 2026
Projection
Lamp
Reflection
Reflective Material
Figure 16: Visual results of our method under challenging lighting conditions.
Lab-space optimization strategy, which first optimizes the I. channel to capture structural information and then learns color modeling. This strategy substantially reduces redundant Gaussian clones and splits, allowing fewer Gaussian primitives to model the scene more effectively. In contrast, SDGS+ImageColorizz splits a large number of Gaussian primitives to model multi-view inconsistent colors, resulting in longer optimization times than standard 3DGS reconstruction. ColorNeRF requires an even longer 16 hours to reconstruct a single scene. These results demonstrate that our method is more efficient than baseline reconstruction approaches, and that per-scene optimization is acceptable in practice.
Table 7: Average optimization time on Mip-NeRF-360 (static) and DyNeRF (dynamic) datasets.
Dataset
Ours
3D/4DGS
ColorNeRE
Stage
Stage
ImageColorizer
Original
Static (Mip-NeRF-360)
8 mins
31 mins
50 mins
42 mins
16
Dynamic (DyNeRIF)
mins
38 mins
54 mins
mins
Generalization on dissimilar scenes. As discussed in Section 4.5 and illustrated in Fig. 7, our personalized colorizer can transferred to similar scenes without retraining, which can reduce the optimization burden for large-scale similar scenes. Here, we further evaluate the performance of the proposed method on more challenging, substantially different scenes. Specifically, we train the colorizer on the "Garden" scene from Mip-NeRF 360 and transfer it to the "Room" scene. As shown in Fig. 17 our method achieves suboptimal performance when the target scene is very different from the training scene. For example, some sofa regions inherit green tones from the source scene, and the overall color richness is reduced. Nevertheless, the colorizer still produces overall consistent and stable results and, in partieular, performs reasonably well for elements that share similar semantic properties, such as the vegetation outside the window. In summary, the personalized colorizer demonstrates potential generalization to similar scenes, reducing the need for retraining across many visually similar scenes in large-scale datasets. However, we still advocate scene-specific fine-tuning achieve optimal 3D colorization results, which remains our recommended practice in this work.
Figure 17: Visualization of the generalization performance dissimilar scene.
Image and video colorizers vs. per-scene colorizer. We evaluate the effectiveness of our per-scene colorizer design by comparing with two representative baselines: the image colorization model DDColor Kang et al. (2023) and the video colorization method ColorMNet Yang et al. (2024a), as illustrated in Fig. 18 can be observed that DDColor suffers from severe cross-view color inconsistencies due to its per-image one-to-many color mapping nature. Treating multi-view images as a video sequence and applying video colorization similarly fails to resolve this issue, since the viewpoint changes inherent in 3D capture far exceed the temporal variations commonly seen in video data. In contrast, our personalized colorizer learns a -peeceip,,,,ic,, one-to-one color mapping from the reference view and leverages its inherent inductive bias to consistently project colors to
23

---

Published as a conference paper at ICLR 2026
corresponding content in novel views and time steps. This design effectively addresses the challenge of color consistency in both static and dynamic 3D scenes.
AI
#1
Image Colorization Method
Per-Scene (Ours)
Figure 18: Visual ablation study illustrating the effectiveness of the per-scene colorizer design.
Effort 3D representations. We further investigate how different 3D representations impact colorization performance. As shown in Tab compared to 3DGS, combining NeRI[Mildenhal)] iet al. (202I) or NeRFPlayer Song et al. (2023) with image colorizers yields suboptimal consistency, reflected by increase of 0.023 in ME. This mainly because the explicit modeling of 3DGS gives an inherent resistance and geometric priors, which can naturally alleviate color inconsistency to some extent. In addition,
replace the Lab Gaussian representation in Color3I) with NeRF and NeRFPlayer, respectively. The results underscore the effectiveness of our proposed Lab Gaussian representation while also demonstrating that the Color3D framework can be adapted to various 3D reconstruction backbones with reasonable performance.
Table Quantitative comparisons language-guided 3D scene colorization on Mip-NeRF 360 and DyNeRF datasets.
Language- Guided Colorization
Method
FID
CLIP Scoret
ME
TCJ
Mip-NeRF 360 (Static 3D Scenes)
104.85
0.6153
0,225
0.046
3DGSsImageColloizer
112.33
0,6148
0.202
0.042
Color3D-NeRI
76.68
0.6213
0.065
0.014
Color3D (Ours)
68.23
0.6246
0.058
0.012
DyNeRF (Dynamic 3D Scenes)
NeRTPPaayr--mmggeColoieee
86.77
0.6148
0.133
0.027
4DGS+imaagCoolrrizrr Color3D-NeRFPlayy NeRFPlayer
89,39
0.6159
0.124
0.025
63.29
0.6243
0.046
0.009
Color3D (Ours)
58.62
0.6271
0.041
0.008
Can editing or stylization methods work for colorization? In Tab. of of the main text, we present quantitative comparisons with representative 3D) editing method (GaussianEditor Chen al. (2024b)) and 3D stylization method (Ref-NPR Zhang et al. (2023)), both of which yield suboptimal performance on colorization tasks. To demonstrate their visual limitations, we further provide
qualitative comparisons in Fig. 19
is observed that GaussianEditor struggles comply with color editing instructions when applied to monochromatic radiation fields, primarily prioritizing semantic fidelity over chromatic accuracy. As result, the edited outputs, although containing colors, often lack chromatic realism and diversity, and may also suffer from structural artifacts. Similarly, Ref-NPR primarily focuses transferring global appearance by relying on perceptual constraints, but fails to accurately capture local color values. In contrast, colorization aims to produce plausible, detailed, and spatially coherent colors that align with the underlying scene content. Consequently, editing and stylization methods are inherently ill-suited for faithful 3D scene colorization. Moreover, their controllability remains limited: editing is typically driven by text prompts, while stylization relies on reference images to guide appearance transfer.
Universal color manipulation framework. Beyond colorizing 3D scenes from monochromatic inputs, our proposed scheme can also be naturally extended to 3D scene recoloring. Specifically, we apply InstructPix2Pix Brooks al. (2023) to edit the colors of single key view and leverage SAM
24

---

Published as a conference paper at ICLR 2026
Eausoi
Color3D (Ours)
Ref-NPR
Color3D (Ours)
Figure 19: Visual comparisons with the 3D editing method GaussianEditor Chen et al. 2024b), and the 3D stylization method Ref-NPR Zhang et al. 2023).
Kirillov al. (2023) to constrain the editing within specified regions. For the personalized colorizer,
the input is the original image, and the output is its recolored version. As illustrated in Fig. [20 our
method not only enables colorization from monochromatic inputs, but also supports high-fidelity and consistent color manipulations, demonstrating its robustness and versatility.
Original
Novel View of Recoloring
Figure 20: Visual results of 3D recoloring.
User study. To further assess the perceptual quality of the generated results, conduct compre- hensive user study, acknowledging that subjective human perception remains a critical benchmark in design 5 evaluation questions covering multiple aspects,
the field of colorization. Specifically, including color richness, consistency, aesthetic preference, image quality, and the fidelity of alignment between the synthesized views and the provided conditions (text, exemplars). Participants are asked to rate each sample on a scale of to 5 for each aspect. For a fair comparison, we recruit 30 participants, each evaluating 10 sets of comparisons across different methods. Aggregated results are shown in Fig. 21 The naive combination of 3DGS with off- off-the-shelf image colorizers yields reasonably rich colors but suffers from significant inconsistency and reduced visual quality. While ColorNeRI demonstrates better consistency, it often lacks color diversity and visual appeal. In contrast, our method consistently achieves higher user ratings across aspects, highlighting its advantage in terms of perceptual quality and user preference.
More visual results. We further present additional colorization renderings of our Color3D Fig. 22 Fig. 23 and Fig. 24] Our approach exhibits strong consistency across both spatial and temporal dimensions, delivering vibrant and semantically coherent colorizations that faithfully adhere to user-specified intent.
A.6
LIMITATIONS AND FUTURE WORK
One potential limitation, shared with other single-view methods, is that our approach tends to favor assigning plausible colors consistent with the observed input rather than hallucinating entirely novel chromatic content when confronted with highly out-of-domain views (e.g., reconstructing large room with multiple unseen bedrooms). future work, we plan to further explore the integration of generative priors to enrich color diversity under drastically unseen viewpoints while maintaining color consistency. Moreover, the per-scene personalization strategy introduced in this work is highly general and carries substantial potential beyond colorization. Extending this paradigm to
25

---

Published as a conference paper at ICLR 2026
Color Richness
3.5
Color3D
Image Quality
Consistency
3DGS+2DColorizer
ColorNeRF
Aligament
Aesthetic
Figure 21: User study results. Our method demonstrates superior performance across all evaluated aspects.
other scene attributes- such as illumination enhancement, white balance adjustment, and style rrapeser-representts a promising avenue toward broader and more versatile scene-level editing capabilities.
26

---

Published as a conference paper at ICLR 2026
"A yellow horn"
"A green room"
"A man with blue clothes
Figure 22: Visual results of language-guided 3D scene colorization. From top bottom are samples from the LLFF, Mip-NeRF 360, and DyNeRF datasets.
27

---

Published as a conference paper at ICLR 2026
Figure 23: Visual results of automatic 3D scene colorization. From top to bottom are samples from the LLFF, Mip-NeRF 360, and DyNeRF datasets.
28

---

Published as a conference paper at ICLR 2026
Figure 24: Visual results of reference- based 3D scene colorization. From top bottom samples from the LLFF, Mip-NeRF 360, and DyNeRF datasets.
29


Reviews:
[{"review_id": "ju3tRjBijV", "reviewer": "Reviewer_uPuV", "summary": "The paper presents Color3D, a unified framework for colorizing both static and dynamic 3D scenes reconstructed from monochrome inputs. Rather than colorizing multiple views independently, which causes cross-view inconsistency, the method colorizes one key view using any off-the-shelf 2D colorization model, then fine-tunes a per-scene personalized colorizer to propagate the learned color mapping to all other views or time steps.E xperiments on LLFF, Mip-NeRF 360, and DyNeRF datasets demonstrate consistent and controllable colorization across viewpoints and time, with improvements in FID, CLIP score, and Matching Error compared to baselines.", "questions": "1. How stable is the personalized colorizer’s output under large camera baselines (e.g., >60° change)? If still good, what are the fundamental factors that may contribute to this effect?\n\n2. How sensitive is performance to errors in key-view selection?\n\n3. How could this appoarch be generalized to feed-forward style 3D generation models? This means per-scene optimizations are removed.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "**High controllability and flexibility.**\nThe system supports multiple control modalities: reference-based colorization, language-conditioned colorization, and automatic default color prediction.\n\n**Computational practicality.**\nDespite using fine-tuning, the reported per-scene personalization time (~8 minutes) is relatively efficient compared to retraining full colorization networks. The framework does not require full 3D model retraining, and adapters make it lightweight enough for scene-level deployment.", "weaknesses": "**Limited generalization beyond scene-specific fine-tuning.**\nThe reliance on per-scene personalized colorizer tuning implies that a new model must be trained for each scene. This restricts scalability for large datasets or interactive applications. The approach is elegant but computationally heavy when many scenes must be processed.\n\n**Inductive bias assumption not rigorously examined.**\nThe claim that a single-view-trained colorizer generalizes to novel viewpoints via inductive bias remains empirical. The paper could benefit from deeper theoretical or diagnostic analysis to substantiate why the learned mapping remains consistent under large view changes.\n\n**Limited evaluation.**\nThe datasets being evaluated on are LLFF (static), Mip-NeRF-360 (static), and DyNeRF(dynamic). Although promising results are shown, the scale of evaluation is still limited. Also, for dynamic scenes, there are more dimensions for evaluation like temporal color-consistency, which is missing in the current experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LQL4JGuCaB", "reviewer": "Reviewer_Rt9r", "summary": "This paper introduces Color3D, a framework for colorizing both static and dynamic 3D scenes (represented by 3D/4D Gaussian Splatting) from grayscale inputs. Its core idea is to avoid the multi-view inconsistency of applying 2D colorizers independently by instead personalizing a single colorization model per scene. This is done by selecting and colorizing one key view, then fine-tuning a pre-trained colorizer (using adapters) on augmented versions of this single view to learn a scene-specific, deterministic color mapping. This personalized colorizer is then used to colorize all other views/frames consistently. A dedicated Lab color space Gaussian representation with a warm-up strategy is used to improve reconstruction fidelity.", "questions": "The paper shows that the personalized colorizer is robust to viewpoint changes from the key view. How does it handle novel views containing objects or textures that are semantically similar but visually different (e.g., another type of chair)? Would it incorrectly transfer learned colors or struggle to color them plausibly?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow. The proposed method can be adapted to various colorization models, making it highly practical and valuable for real-world applications, especially in AR/VR scenarios.\n\n2. The experimental results on LLFF and Mip-NeRF 360 and D-NeRF colorization setting have shown the Color3D's good performance, not only on statistic 3d scenes but also on dynamic 4d scenes. Also, the method demonstrably outperforms existing alternatives across multiple  metrics (FID, CLIP Score, Matching Error), showing superior consistency, color vividness, and alignment with user intent.", "weaknesses": "1. It seem like the requirement to fine-tune a personalized colorizer for every new scene adds a non-trivial computational cost (∼8 minutes per scene) compared to a generic, one-time-trained model, limiting its scalability for large-scale applications.\n2. The entire color propagation relies on a single key view. If this view is unrepresentative or lacks critical scene elements, the colorizer's generalization may be hampered, potentially leading to incomplete or less vibrant colorization in occluded regions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2cH37clt6w", "reviewer": "Reviewer_Rsix", "summary": "1. This paper focuses on the task of controllable and consistent 3D scene editing. It aims to address the limitations of previous methods that often lack precise controllability and multi-view color consistency.\n\n2. The authors propose Color3D, a two-stage framework. In the first stage, a text-to-image diffusion model is used to modify the reference view’s color according to user input. In the second stage, Score Distillation Sampling (SDS) is applied to enforce 3D consistency. Unlike prior work, the method performs Gaussian Splatting optimization in the LAB color space, which helps maintain color consistency across different viewpoints.\n\n3. Experiments on Color-Edit-3D dataset show that Color3D achieves state-of-the-art results. The ablation study highlights the importance of the Color Consistency Regularization (CCR) module, which provides a clear performance gain when included.", "questions": "Please address the issues listed in the weaknesses section.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The method is technically sound and well-motivated.\n\n2. The experiments are thorough and provide strong empirical validation.", "weaknesses": "1. The computational cost is unclear — how long does the optimization take during the second stage?\n\n2. The main novelty seems to lie in optimizing in the LAB color space instead of RGB. While this is a reasonable choice for editing tasks, the contribution may be somewhat limited in scope.\n\n3. It would be interesting to see how the approach performs under challenging lighting conditions (e.g., scenes with lamps or strong reflections).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I2fvjk8qc7", "reviewer": "Reviewer_TQsU", "summary": "Color3D proposes a method for colorizing monochromatic images in 3D representations for static and dynamic scenes. Naive implementations, like colorizing 2D multi-view images and reconstructing the 3D scene lead to severe cross-view inconsistencies. Recent methods that distil the color information to a 3D representation sacrifice controllability and often have desaturated colors in the final output.\n\nKey contributions of the proposed methodology are as follows:\n\n-   The key idea is to colorize (automatic/reference-based/prompt-based)  a single \"key\" view which is the most informative one and then fine-tune a scene-specific colorization network on that view.\n-   This scene-specific colorizer learns a deterministic color mapping for the scene and is then applied to all other views/frames, enforcing cross-view and cross-time color  consistency .\n-   Finally, the colorized views (with known luminance and predicted chrominance) are fused into a Gaussian Splatting representation in Lab color space.\n\nExperiments on standard benchmarks (LLFF, Mip-NeRF 360, DyNeRF) and \"in-the-wild\" legacy videos show that Color3D produces vivid and consistent colorizations.", "questions": "-   **[Q1]** Have the authors tried fine-tuning on _more_ than one view? Using two or three colorized views might improve coverage for large scenes. Is there a reason the approach is limited to one view?\n\n-   **[Q2]** How does the proposed method handle motion for dynamic scenes? In case the object that was not in the key view appears in the scene, how is its colour determined? Does the generative augmentation simulate such cases? This is not clear in the current manuscript.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- **[S1] Technical Novelty:** The per-scene colorization for a single view is a novel idea which achieves consistent colorization across views. The robust technical pipeline achieves this consistency through key design choices. Specifically, utilizing a pre-trained 2D colorization encoder (DDColor) preserves the generalization capability of the model. The use of the Lab color space leads to more stable results. Finally, warm-up training of the 3D representation on the Luma ($\\text{L}$) channel first ensures the model establishes a strong geometric structure before introducing complex color information.\n\n- **[S2] Practical Applications:** The authors show results on \"in-the-wild\" multi-view images and historical video (Fig.6), producing vivid and plausible colors while maintaining consistency.\n\n- **[S3] Controllability**: The proposed method allows users to control the colorization using text descriptions or reference images. Earlier methods did not offer this type of user control.\n\n- **[S4]** Thorough Experimentation for Key-View selection: The authors performed detailed experiments for \"Key-View Selection\" module. Results in Fig. 7 demonstrate that this module is critical for better colorization.", "weaknesses": "- **[W1] Limited Comparison to Recent Methods:** The authors mention \"ChromaDistill\" in Related Work, but do not perform a quantitative comparison. For complete experimentation, it is also necessary to quantitatively compare the results against a video colorization baseline.\n\n- **[W2] Cross-View Consistency**: ChromaDistill utilized a  long-term and short-term view consistency method for measuring the geometric consistency. However, the authors do not show this metric. The manuscript will benefit from this metric.\n\n- **[W3]** As the method utilizes training for each scene, it has an extra training overhead.\n\n\n**Typos:**\n\n-   L74: It should be \"aim\" instead of \"aims\".\n-   L857: It should be \"suffers\" instead of \"suffer\".\n-   L347: It should be \"entire\" instead of \"entir\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)