Title: DreamLight: Towards Harmonious and Consistent Image Relighting
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image Composition

Image Relighting

Image Harmonization
Abstract: We introduce a model named DreamLight for universal image relighting in this work, which can seamlessly composite subjects into a new background while maintaining aesthetic uniformity in terms of lighting and color tone. The background can be specified by natural images (image-based relighting) or generated from unlimited text prompts (text-based relighting). Existing studies primarily focus on image-based relighting, while with scant exploration into text-based scenarios. Some works employ intricate disentanglement pipeline designs relying on environment maps to provide relevant information, which grapples with the expensive data cost required for intrinsic decomposition and light source. Other methods take this task as an image translation problem and perform pixel-level transformation with autoencoder architecture. While these methods have achieved decent harmonization effects, they struggle to generate realistic and natural light interaction effects between the foreground and background. To alleviate these challenges, we reorganize the input data into a unified format and leverage the semantic prior provided by the pretrained diffusion model to facilitate the generation of natural results. Moreover, we propose a Position-Guided Light Adapter (PGLA) that condenses light information from different directions in the background into designed light query embeddings, and modulates the foreground with direction-biased masked attention. In addition, we present a post-processing module named Spectral Foreground Fixer (SFF) to adaptively reorganize different frequency components of subject and relighted background, which helps enhance the consistency of foreground appearance. Extensive comparisons and user study demonstrate that our DreamLight achieves remarkable relighting performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an image relighting method called DreamLight, which supports both image-based and text-based relighting. To generate more natural lighting effects between the foreground and background, the authors introduce a position-guided lighting adapter. Additionally, to avoid detail loss during VAE encoding, a spectral foreground fixer is proposed to preserve high-frequency details of foreground textures. However, some technical details require further explanation and experimental evaluation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. DreamLight supports both image-based relighting and text-based relighting.\n2. A direction-biased mask attention mechanism is proposed to extract illumination from the four background directions (up, down, left, right), thereby achieving more accurate foreground relighting.\n3. Qualitative and quantitative experimental results demonstrate that the proposed method achieves more natural and continuous lighting effects.\n\nWeaknesses:\n1. I noticed that the image-based relighting exhibits some incorrect shadow effects. Is it because the spectral foreground corrector directly replaces the foreground region with I_out, weakening the shadow interaction between the foreground and background?\n2. The training data for the spectral foreground fixer is obtained using random color transformations, which significantly differ from actual lighting variations. Although ablation experiments demonstrate the module's effectiveness, what is the actual distribution of the predicted alpha and beta? Visual ablation experiments are needed to prove its specific role.\n3. Why does the spectral foreground fixer predict beta parameters and directly add them to the high-frequency components? Its actual function and physical meaning are unclear.\n4. After removing the spectral foreground fixer module, metrics such as PSNR were lower than IC-Light and IH. This is strange because the position-guided light adapter should better extract illumination information from four directions, and the performance metrics should be higher than these baselines. The authors need to further explain the reason.\n5. The inference latency should be experimentally tested.\n\nQUESTIONS:\nMy main concerns are summarized in Weaknesses Section. Considering some issues with the spectral foreground fixer and the confusion brought by ablation experiments, my initial rating to this paper is borderline reject. While I will consider my final rating according the feedback of the authors during the rebuttal section.\n\nLIMITATIONS:\nThe limitations are kindly discussed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DreamLight, a universal image relighting framework that harmonizes foreground subjects with new backgrounds specified by images or text. It leverages pretrained diffusion models and proposes the Position-Guided Light Adapter (PGLA) to extract directional lighting from the background via masked attention, improving natural light interaction. A Spectral Foreground Fixer (SFF) further enhances appearance consistency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- Writing is clear and figures support the technical explanations. The Position-Guided Light Adapter (PGLA) introduces a novel direction-biased masked attention mechanism that explicitly models multi-directional lighting cues, improving relighting quality. The Spectral Foreground Fixer (SFF) effectively addresses foreground distortion by adaptively combining high-frequency foreground details with low-frequency lighting from the background, enhancing visual fidelity.\n\n\nWeaknesses: \n- The directional light queries are limited to four fixed directions, which may restrict the model’s ability to handle complex lighting environments with multiple arbitrary light sources.\n- The SFF training relies on synthetic color transformations that may not fully capture real-world lighting variations; additional evaluation on real lighting conditions would strengthen the claims.\n- Computational cost and inference speed are not thoroughly discussed, which matters for practical deployment.\n- Some ablations and deeper analysis on the relative importance of PGLA components and SFF would improve clarity.\n\nQUESTIONS:\n- Could you please clarify how your model performs in scenes with more than four dominant light directions? Are there any plans to extend the PGLA mechanism to support more flexible or adaptive directional queries?\n\n- How sensitive is the Spectral Foreground Fixer to the quality and realism of its synthetic training data? Have you evaluated its effectiveness on real-world relighting scenarios?\n\n- Would you be able to provide more details on the runtime performance, memory requirements, and inference speed of your method compared to existing baselines?\n\n- Have you encountered any limitations or failure cases when the background contains strong lighting contrasts or complex shadows? If so, how does your method address these challenges?\n\nLIMITATIONS:\nSee Questions", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes DreamLight, a method for relighting images based on either a text prompt or a background image. It introduces two main components: (1) the **Position-Guided Light Adapter (PGLA)**, a modified IP-Adapter designed to model lighting from four primary directions (left, right, up, down), and (2) the **Spectral Foreground Fixer (SFF)**, which enhances detail preservation by injecting high-frequency components from the original input into the relit result, improving areas like faces or text.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength: \n- **Quality:**  The method is technically sound and well-executed, with clear contributions\n- **Originality:** Both the Position-Guided Light Adapter (PGLA) and Spectral Foreground Fixer (SFF) present novel ideas tailored to the relighting task.\n- **Significance:** The model effectively preserves foreground identity (e.g., face details), a common weakness in prior work. This makes it a strong candidate for real-world relighting applications and downstream tasks.\n- **Clarity (high-level)**: The overall pipeline is well-structured and easy to follow.\n- **Qualitative result:** The visual comparisons clearly demonstrate improved foreground consistency and lighting harmony over prior methods.\n## Weakness:\n- **Clarity (technical detail):**  Some implementation aspects, such as the implementation of the Modulator in SFF, are underexplained, limiting reproducibility.\n- **Quantitative Evaluation:** The benchmark used for evaluation is internally created. While IC-Light did not release its test set, this raises concerns about potential cherry-picking or a lack of generalization.\n- **Ablation gap:**  The ablation study on PGLA evaluates the impact of the spectral filter but omits a crucial variant: applying the filter without the light queries (i.e., IP-Adapter with filtering). This makes it difficult to disentangle the contributions of each component.\n\nQUESTIONS:\n## 1. Clarify Modulator Implementation in SFF\n\nIn Line 245, the paper states that SFF is fine-tuned on the VAE of Stable Diffusion 1.5. However, it’s unclear how the Modulator is implemented, given that it is supposed to take two inputs high-frequency foreground ($HQ_{in}$) and low-frequency relighting result ($LQ_{out}$) and output two modulation parameters ($\\alpha$, $\\beta$), whereas the VAE has only one single input and output which a single image.\n\n**Question:** Could you clarify how the Modulator is structured and trained in this context?\n\n## 2. Text Prompt Used in Image-Based Relighting\n\nSince the model is based on Stable Diffusion 1.5, a text prompt is required at inference. However, the paper does not clearly state what prompt is used during image-based relighting. Figure 2 shows “Blend these two images,” but this is not explicitly mentioned in the text.\n\n**Question:** What text prompt is used during inference for image-based relighting, and is it the same as during training? If not, how is the prompt selected, and how sensitive is performance to prompt choice?\n\n## 3. Lack of Reflections or Subject-Light Interactions in Background\nSupplementary Figure 1 shows a dog above a reflective surface (wet floor or water), but no reflection is present.\n\n**Question:** Does the current pipeline, particularly SFF, suppress interactions such as reflections or shadows in the background? Would disabling SFF allow more light or reflection effects to propagate naturally?\n\nLIMITATIONS:\nThe authors addressed some limitations in the supplementary material. However, one additional limitation worth mentioning is that the model does not handle physical interactions between the foreground and background, such as reflections or shadows, adapting to background surfaces. This could limit realism in certain scenarios.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new method called DreamLight to perform both image-based and text-based relighting, given a foreground image and a background image. Two key components are introduced: the position-guided light adapter and the spectral foreground fixer. The authors also generate their own relighting data for the training of the model. Experiments are conducted on the generated data, and DreamLight is compared against methods like IC-Light.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1. The idea of having a unified approach for both image-based and text-based relighting makes sense. The proposed method, DreamLight, seems to be a reasonable solution to this task. \n2. The proposed Position-Guided Light Adapter is a novel design that is tailored to model lighting information in an effective format. See below for additional questions on this part for clarification. \n\n### Weaknesses\n1. One major weakness is the lack of experiments on the existing relighting datasets. Currently, the method is trained and tested on the author-generated data. It will be more convincing if the method is also tested on public benchmarks. \n2. While the Position-Guided Light Adapter is directly designed for the relighting task, the second proposed component, Spectral Foreground Fixer, seems to be less tailored for relighting. For example, some lighting effects are actually high-frequency such as specular reflection. It is unclear if the proposed fixer can handle such a relighting situation. \n3. Some clarification is needed to fully understand the effectiveness of the DreamLight method (see questions below). \n\n### Typo\n- Table 2, 'PP denotes PowerPoint'--it should be PowerPaint.\n\nQUESTIONS:\n1. The proposed method is evaluated on an author-generated dataset. What's the performance of the method on existing datasets that are used as benchmarks for relighting, such as TensoIR (https://github.com/Haian-Jin/TensoIR)?\n\n2. Related to Q1, most results shown in this paper are portraits. How does the method perform on relighting general objects with different materials?  \n\n3. The design of the direction-biased masked attention is pretty interesting. On the other hand, I don't have a good understanding of why it is working well. Why is it important to have four fixed directions? What happens if you need more complex lighting patterns? Or maybe I'm missing something here. \n\n4. While Table 4 shows the performance trade-off of adding the fixer to the relighting pipeline, what about the time cost? \n\n5. Line 214: The modulator M is trained with pseudo pairs of relighting data. Why not use actual relighting data to train it? \n\n\nOverall, the proposed method has interesting designs to solve a challenging task, while there are some missing pieces in fully understanding the effectiveness of the method. If the authors can address the questions mentioned above, I can raise my score.\n\nLIMITATIONS:\nNo discussion on the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}