Title: KaRF: Weakly-Supervised Kolmogorov-Arnold Networks-based Radiance Fields for Local Color Editing
Venue: NeurIPS 2025
Accepted: yes
Keywords: Neural Radiance Fields

Local Color Editing

Kolmogorov-Arnold Networks
Abstract: Recent advancements have suggested that neural radiance fields (NeRFs) show great potential in color editing within the 3D domain. However, most existing NeRF-based editing methods continue to face significant challenges in local region editing, which usually lead to imprecise local object boundaries, difficulties in maintaining multi-view consistency, and over-reliance on annotated data. To address these limitations, in this paper, we propose a novel weakly-supervised method called KaRF for local color editing, which facilitates high-fidelity and realistic appearance edits in arbitrary regions of 3D scenes. At the core of the proposed KaRF approach is a unified two-stage Kolmogorov-Arnold Networks (KANs)-based radiance fields framework, comprising a segmentation stage followed by a local recoloring stage. This architecture seamlessly integrates geometric priors from NeRF to achieve weakly-supervised learning, leading to superior performance. More specifically, we propose a residual adaptive gating KAN structure, which integrates KAN with residual connections, adaptive parameters, and gating mechanisms to effectively enhance segmentation accuracy and refine specific editing effects. Additionally, we propose a palette-adaptive reconstruction loss, which can enhance the accuracy of additive mixing results. Extensive experiments demonstrate that the proposed KaRF algorithm significantly outperforms many state-of-the-art methods both qualitatively and quantitatively. Our code and more results are available at: https://github.com/PaiDii/KARF.git.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the task of color editing in neural radiance fields (NeRFs), with a specific focus on combining local editing and palette-based editing. Traditional palette-based approaches typically lack spatial precision unless paired with segmentation masks. The authors overcome this limitation by using SAM-generated masks to supervise the learning of a local 3D segmentation field, which is then used to perform palette-based decomposition in localized regions.\n\nStarting from a pre-trained NeRF, the method leverages its multi-channel features to train a per-point segmentation based on KAN (upgraded to GRBFKAN), followed by palette decomposition on the segmented regions. The key contribution lies in enabling localized palette decomposition, which allows for richer and more controllable local edits.\n\nThe authors demonstrate improved local editing capabilities compared to a basic KAN-based baseline.\n\nSTRENGTHS AND WEAKNESSES:\nSupervision Level and Use of SAM Masks:\nThe method relies on segmentation masks generated by SAM on multiple views to train a 3D segmentation field via a KAN-based radiance field. While the authors frame this as “weak supervision,” the requirement of 2D masks from multiple viewpoints constitutes a considerably stronger form of supervision than prior methods. ICE-NeRF uses sparse scribbles on a single image and IReNe relies only on a single segmentation mask. In contrast, KaRF appears to need multiple SAM masks to function effectively. Additionally, the paper does not clearly state how many views are used — Figure 4 shows two, but it's unclear if that's just for illustration. Calling this “weak supervision” is, in my view, somewhat misleading.\n\nLack of Justification for Using KANs:\nThe paper introduces using KANs, specifically GRBF-KANs, as an architectural innovation for both segmentation and recoloring. While ablations show that their residual adaptive gating improves over basic KANs and B-spline variants, there is no direct numerical comparison with standard MLPs, despite MLPs being used in all previous NeRF-based editing frameworks. Since the architectural shift to KANs is the major contribution, this omission weakens the justification for their use. \n\nViewdep Inconsistency:\nThe paper states that segmentation is view-invariant (Eq. 7), but Figure 3 shows the view direction as an input to the segmentation module, creating an inconsistency. This should be clarified, as it raises doubts about whether the segmentation field truly disentangles geometry from appearance.\n\nMore importantly, I’m confused by the treatment of view direction in the palette weight prediction. In the text, the palette weights W are defined as a function of view direction (Eq. 10). That’s acceptable, although in previous SOTA methods, like PaletteNeRF, the palette weights were view-invariant. In other papers, like IReNe, there was a focus on explicitly dividing specular and diffuse components. Now, if there’s a need to make the weights not be affected too much by view-dependent color, I would have assumed Equation 11 to be used with view-invariant palette weights W — to say: \"learn as much of the color as you can through W, so the view direction can then learn only the residual.\" But that doesn’t make sense if W is not direction-invariant.\nIf W is view-dependent, then Equation 11 is effectively a standard RGB reconstruction loss — so why not call it that? I believe a proper RGB reconstruction loss is actually missing in KaRF (if Equation 11 is not intended for that) to fully reconstruct the scene. I would have expected both a photometric RGB loss and a loss that encourages W to learn as much of the diffuse color as possible.\nAs written, Equation 11 seems like a standard RGB reconstruction loss, and Equation 13 is just to push W to learn the main color components. If that’s the case, I wouldn’t call Equation 11 \"L_palette\" — it’s more accurate to call it \"PhotometricLoss\".\nI would like the authors to clarify this point.\n\nLimited Scope of Ablation Studies:\nMost ablation studies are conducted on single scenes only — e.g., LPalette is tested only on Lego, GRBF-KAN only on Kitchen, and time comparison only on Fern. This narrow scope makes it difficult to assess the generalizability of the architectural improvements or the claimed efficiency. Broader evaluation would strengthen the evidence for the proposed choices.\n\nLimited Qualitative Evaluation on Challenging Scenes:\nI would have appreciated more qualitative results, especially full 360° visualizations on the Mip-NeRF 360 dataset. Since reflections and complex lighting effects are critical challenges in recoloring, it is important to see how well the method performs in those settings. Static frame comparisons are insufficient to assess view consistency in reflective surfaces. I find it suspicious that no complete 360° video of Mip-NeRF 360, or even images showing significant viewpoint changes, were included either in the main paper or the supplementary material. This omission makes me think the method may not generalize well in 360° cases.\n\nQUESTIONS:\nHow many SAM masks are actually needed for training?\n\nHow is SAM being prompted?\n\nWould it be possible to visualize or quantify the separation between what is modeled as view-independent (via W) and view-dependent (residuals)?\n\nClarification is needed on the input-output relationship between the segmentation map and the palette weights W — specifically, how (and whether) view direction influences both.\n\nIt is unclear whether the parts of the scene not belonging to the segmented object (e.g., background or other classes) are also palette-decomposed. If they are not palette-decomposed, what does KaRF learn for those regions? Just a single base color per pixel/point?\nOr does KaRF learn separate palette decompositions for each segmentation class (e.g., one palette for object 1, another for object 0)?\n\n\nI would be open to accepting this paper if the following conditions are addressed:\n\n-The authors demonstrate that the segmentation module can operate effectively with only a single input mask or single user interaction, and provide both quantitative and qualitative evidence of its generalization capabilities.\n\n-There is clear empirical justification—both qualitative and quantitative—for using KAN or GRBF-KAN over standard MLPs. It should be demonstrated that the improved performance and enhanced local editability are due to the KAN architecture itself, and not simply the result of increased supervision(SAM) compared to other methods.\n\n-The paper includes more comprehensive 360° visual evidence, particularly on the Mip-NeRF 360 dataset, showing that the method maintains multi-view consistency and handles well view-dependent effects.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-13"}

comments: SUMMARY:
This paper introduces an alternative backbone for NeRF by replacing the MLP with a KAN-based network for local color editing. Given the flexible activation prediction of KAN the authors aim to improve segmentation with finer edges and more precise color editing than MLP based NeRF. The method however added architectural complexity significantly, and the authors did not specify the runtime of the model but simply state certain design to faciliate the computational overhead. Additionally, the method lacks rigorous analysis of scalability and training stability beyond the shown dataset examples.

STRENGTHS AND WEAKNESSES:
The given implementation details are not sufficient to reproduce the method without the release of the codebase. The author also did not specify whether they plan to release the code. 
The authors did not specify how many datasets were used for experiments, but merely explained the number of types of scenes. The authors also did not specify the change in implementation for different types of NeRF scenes. 
The runtime comparison is not extensive and significant enough through simply providing "roughly 10 min" without specifying the type of scenes. 
The use of limited cross-dimensional interaction by KAN can create interdependencies between view directions and 3d locations compared to fully connected MLPs. How does the model handle the view-dependent effects such as specularity or transparency in editing? How would the model behave for scenes with complex lighting? 
Overall, the reason to adapt KAN for the specific task of localized NeRF recoloring is not well conveyed through the writing of this paper, and the authors have not shown enough results to demonstrate superiority of the method.

QUESTIONS:
See weakness.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-25

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces KaRF (Kolmogorov-Arnold Networks-based Radiance Fields), a novel weakly-supervised method for local color editing of neural radiance fields (NeRF).\n\nThe first major contribution is the overall KaRF framework architecture. It introduces a two-stage approach that combines segmentation and local recoloring while requiring minimal guiding information for selective recoloring of arbitrary regions. This design achieves multi-view consistency and precise boundary control, addressing key limitations of existing approaches.\n\nOn the technical side, the paper makes several innovative contributions. It proposes a novel \"residual adaptive gating KAN structure\" that enhances segmentation accuracy and editing refinement. The authors introduce a key modification to vanilla KAN by replacing B-spline functions with Gaussian Radial Basis Functions (GRBFs), which better handle high-frequency spatial details and sharp color transitions. They also introduce a palette-based color-adaptive loss to improve color mixing accuracy.\n\nIn terms of performance, KaRF demonstrates superior results compared to state-of-the-art methods both qualitatively and quantitatively. The approach shows better handling of color bleeding issues and boundary precision compared to existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nTechnical Innovation and Soundness:\nThe paper introduces several novel technical contributions that build meaningfully on prior work. The substitution of B-splines with GRBFs in the KAN framework is well-justified, as it better handles high-frequency spatial details and sharp color transitions. The residual adaptive gating KAN structure shows thoughtful design in addressing training stability while maintaining expressive power.\n\nClear Problem Motivation:\nThe authors clearly articulate the limitations of existing NeRF-based editing approaches, particularly around boundary precision, multi-view consistency, and color bleeding. The proposed solution directly addresses these identified gaps in current methods.\n\nComprehensive Evaluation:\nThe evaluation is thorough, testing the method across multiple datasets (NeRF-Synthetic, Mip-NeRF 360, LLFF) and comparing against relevant state-of-the-art methods. The inclusion of both qualitative and quantitative metrics, along with user studies, strengthens the validation of their claims.  \n\nLimitations\n\nLimited Analysis of KAN Modifications:\nOne of the paper's most significant contributions is the novel use of KAN for NeRF editing and its modification to incorporate Gaussian RBFs instead of B-splines. However, the analysis of this crucial modification is insufficiently thorough. The main paper only provides one anecdotal comparison in Figure 7, while the supplementary material's Table 6 only shows results for a single scene (kitchen). A more rigorous analysis across multiple scenes and different types of editing operations would be valuable to understand when and why GRBFs outperform B-splines. This would help readers better understand the trade-offs and benefits of this fundamental architectural choice.\n\nComputational Efficiency:\nThe method requires approximately 10 minutes for processing, which is almost twice as slow as LAENeRF and significantly slower than other recent approaches. For instance, IReNE (Mazzucchelli et al., 2024) achieves similar editing capabilities in about 5 seconds. This performance gap could limit the method's practical applicability, particularly in interactive editing scenarios where quick feedback is essential. The paper would benefit from a discussion of potential optimizations or trade-offs between quality and speed.\n\nWhile the paper claims superior segmentation performance, the actual improvements over existing methods are relatively modest. As shown in Table 1, KaRF achieves an IoU of 94.7% and accuracy of 99.5%, which represents only a small improvement over SA3D (93.7% IoU, 99.2% accuracy) and MVSeg (90.7% IoU, 98.8% accuracy).\n\nQUESTIONS:\nPlease comment on the limitations raised above.\n\nLIMITATIONS:\nLimitations are discussed in the supplementary materials. The reviewer wonders if this is an acceptable practice, as key limitations should be presented in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

comments: SUMMARY:
The paper introduces a new method for palette-based recoloring of NeRFs. The authors introduce the KAN structure to refine the initial semantic segmentations obtained from SAM, obtaining segmentations of high quality. The resulting regions are used to build a hierarchy of masks, palettes and mixing weights following the LoCoPalettes paper. Results show better consistency and user control than previous works.

STRENGTHS AND WEAKNESSES:
- Strengths:
- New method for intuitive recoloring of NeRFs, with results of better quality than previous works.
- Introduction of the KAN network for the mask refinement and recoloring task.
- Clarity of exposition and evaluation.

- Weaknesses:
- Smart combination of existing work, but with great results.
- Processing times seem too high for becoming practical.

QUESTIONS:
The method is sound, presented and evaluated properly. More than a question, it is good to see existing methods for 2D to be repurposed for NeRFs in such a clean and successful way.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01