Monocular Normal Estimation via Shading Sequence Estimation
Authors: Zongrui Li
Xinhua Ma
Minghui Hu
Yunqing Zhao
Yingchen Yu
Qian Zheng
Chang Liu
Xudong Jiang
Song Bai
Venue: ICLR 2026 Oral (2026)
Keywords: Video Diffusion Model
Shading Estimation
Single-view Normal Estimation
Primary area: generative models

Abstract: Monocular normal estimation aims to estimate the normal map from a single RGB image of an object under arbitrary lights. Existing methods rely on deep models to directly predict normal maps. However, they often suffer from 3D misalignment: while the estimated normal maps may appear to have a correct appearance, the reconstructed surfaces often fail to align with the 3D geometry. We argue that this misalignment stems from the current paradigm: the model struggles to distinguish and estimate varying geometry represented in normal maps, as the differences in underlying geometry are reflected only through relatively subtle color variations. To address this issue, we propose a new paradigm that reformulates normal estimation as shading sequence estimation, where shading sequences are more sensitive to various geometry information. By learning to infer the shading sequence of an object, the model can better capture underlying 3D geometry and thereby produce more accurate normal predictions. Building on this paradigm, we present RoSE, a method that leverages image-to-video generative models to predict shading sequences, which are then converted into normal maps by solving a simple ordinary least-squares problem. To enhance robustness and better handle complex objects, RoSE is trained on a synthetic dataset, MultiShade, with diverse shapes, materials, and light conditions. Experiments demonstrate that RoSE achieves state-of-the-art performance on both synthetic and real-world benchmark datasets for object-based monocular normal estimation.

Published as a conference paper at ICLR 2026
MONOCULAR NORMAL ESTIMATION VIA SHADING SE- QUENCE ESTIMATION
Zongrui Li Xinhua Ma Minghui Hu? Yunging Zhao² Yingchen Yu²
Qian Zheng** Chang Liu Xudong Jiang' Song Bai
"ccooo of Electrical and Electronic Engineering, Nanyang Technological University
BByteDance
CCollege of Computer Science and Technology, Zhejiang University "The State Key Lab of Brain-Machinn Intelligence, Zhejiang University
MoE Key Laboratory of Interdisciplinary Research of Computation and Economics, Shanghai University of Finance and Economics https://github com LMczart/ wttp:/arrrr/IR2222---2202 ROSE.
Ref. Image
DSINE
StableNomal
Neural LightRig
NiRNE
Ours
Comparison of Methods on Three Datasets
20-
22.53
21.58
Angular
20.16
19.20
18.24
GeoWicard
DSINE
StabieNormal
Letux-O
Lotua-G
Neural Lightfiig
NIRNE
Durs
Figure We present ROSE, a method using video generative model for monocular normal map estimation, built on a new paradigm that reformulates normal estimation as shading sequence estimation task. Results on complex and diverse scenario show that RoSE reconstructs fine-grained geometric details and generalizes robustly to unseen datasets, achieving state-of-the performance object-basse based monocular normal estimation on benchmark datasets.
ABSTRACT
Monocular normal estimation aims to estimate the normal map from a single RGB image of an object under arbitrary lights. Existing methods rely on deep models to directly predict normal maps. However, they often suffer from 3D misalignment: while the estimated normal maps may appear to have a correct appearance, the reconstructed surfaces often fail to align with the 3D geometry. We argue that this misalignment stems from the current paradigm: the model struggles to distinguish and reconstruct varying geometry represented in normal maps, as the differences in underlying geometry are reflected only through relatively subtle color variations. To address this issue, we propose new paradigm that reformulates normal estimation as shading sequence estimation, where shading sequences are more sensitive to various geometry information. Building on this
"quual contribution. Corresponding author.

---

Published as a conference paper at ICLR 2026
StabieNormal
Neural LightRig
GeoWizard
NIRNE
Ours
Ground Truth
Normal
Surface
Illustration of 3D misalignment. The estimated normal maps of previous methods may appear to have overall correct color distribution, yet the reconstructed surfaces often fail to align with the accurate geometry details, showing over-smooth results. Our estimated normal map has higher 3D alignment compared others.
Figure 2:
paradigm, we present RoSE, method that leverages image-to-vides generative models predict shading sequences. The predicted shading sequences then converted into normal maps by solving a simple ordinary least-squares problem. To enhance robustness and better handle complex objects, RoSE is trained on a synthetic dataset, MultiShade, with diverse shapes, materials, and light conditions. Experiments demonstrate that RoSE achieves state-of-the-arr performance on both synthetic and real-world benchmark datasets for object-based monocular normal estimation.
INTRODUCTION
Normal maps of surface encode 3D geometry by representing the orientation of the surface elements each pixel, making them essential for wide range of applications, including relighting Tiwari
et al. 2024: Sang & Chandraker 2020 Li et al.) 2018 Liu et al.) 2020 Li et al. 2023). 3D mesh reconstruction (Ye et (005)). and modern gaming pipelines (Wong & Bishop 2006 Moreira et al.) 2022). In early methods, capturing accurate normal maps of object requires specialized equipment and incurs high cost, motivating the development of more intelligent methods that can reliably infer them from casually captured RGB images of the object.
To tackle this task, previous works YYonn al. 2016 He et al.) 2024b; Li et al.) 2015 2024b Fu et al.) 2024 Bae
Shading
al. 2021) directly estimate normal maps from a single RGB image using deep models. Despite achieving promis- ing results, these methods often produce normal maps that appear have correct appearance but fail remain sistent with the underlying 3D geometry. We refer to this limitation as "3D misalignment" (see Fig. 2). We argue that this limitation arises from the current paradigm, where the model learns recover geometry primarily by aligning with the color representation of normal maps. The deep model struggles distinguish and reconstruct fine geometric de- tails because normal maps represent geometry in highly compact form, where geometry variations across different positions appear only subtle color differences. This issue becomes particularly severe in monocular normal estima- tion, where geometric details in the input are inherently more ambiguous, thus difficult to recover.
Low
Figure 3: Validation of sensitivity to geometry variations for different representations, including the pro- posed shading sequence (left) and the normal map (right), measured by av- erage total variation (TV). TV is com- puted as the mean magnitude of the first-order image's gradient in terms of different representations, where higher TV indicates stronger sensitivity to spatial geometric variation.
To reduce 3D misalignment, this work proposes
paradigm for normal estimation by modifying the training target. The main idea is to adopt a representation that is sensitive to geometry variation the training target, thereby enhancing
2

---

Published as a conference paper at ICLR 2026
the network's ' ability to distinguish and reconstruct geometric details. Guided by this intuition, propose using a shading sequence, defined as the clamped dot product between the normal map and a set of canonical light directions, the new training target. The idea of using shading sequence is motivated by two key observations. First, shading sequences capture geometry variation through brightness variation, while excluding material influences, making them sensitive to only the geometric variations, as illustrated in Fig. 3 Second, predicting shading sequence given canonical light directions is equivalent to predicting the normal map, as shading sequence can be losslessly converted to the normal map (Yu et al. 2010) using an Ordinary Least Square (OLS) solver.
Based on the new paradigm, we propose ROSE, a method that Reformulating normal estimation as the Shading sequence Estimation based on the monocular input image. Specifically, as the shading sequence can be regarded as a video, we leverage recent image-to-videg generative models (Voleti et al. 2024, |Blattmann et al., 2023) to predict shading sequences. A key benefit of this design is that the rich lighting priors encoded in large-scale video models facilitate the generation of accurate shading variations. Once the shading sequence is obtained, we recover the normal map using an OLS solver (Woodham) 1980). In practice, to enhance robustness when handling objects with more complex materials and lights, train our model on collected dataset named MultiShade, enriched with more diverse materials and light conditions compared to previous datasets. Experimental results demonstrate that our method achieves superior performance compared to state-of-the-ar methods.
Overall, we summarize the main contributions of this paper as:
We introduce new paradigm that reformulates the task of monocular normal estimation as shading sequence estimation.
Under the new paradigm, we propose RoSE, a monocular normal estimation method using an image-to-video generative model that predicts a shading sequence of an object under predefined canonical parallel lights and analytically derives normal maps from it.
We train RoSE on MultiShade, synthetic dataset with diverse material and light condi- tions. Experiments show that our method achieves state-of-the-art performance on multiple datasets. especially the widely- real-world benchmark datasets (i.e, DiLiGenT, LUCES).
2 RELATED WORKS
Monocular Normal Estimation. Despite persistent research efforts in monocular normal estimation, achieving high accuracy remains significantly challenging due to the complexity of this task, which requires the prediction of an accurate normal map with highly limited input information. Early
works (Eigen & Fergus 2015 Do al., 2020 Fouhey et al. 2013] Wang al.. 2015)) Zhang et al..
2019 Bansal et al.) 2016 Ladicky et al. 2014: et al. 2015 Wang al. 2020) relied on handcrafted features, empirical priors, or conventional deep neural networks. However, these methods often
suffer from limited generalization ability. Recent methods based on generative models (Voleti et al.)
2024: Fu et al. 2024 He et al., 2024a), thysics-inspire deep networks (Bac & Davison 2024). )) and
auto-regressive frameworks (Ye et al. 2025) have demonstrated improved generalization ability and the capacity to estimate relatively accurate normal maps. However, the estimated normal maps suffer from 3D misalignment, a problem that stems from the current paradigm where the model fails to capture the compact information in the normal maps. To address this, other works (Tiwari et al. 2024. He et al. 2024b) attempt first generate more input images under of predefined canonical lights and subsequently estimate normals from these multi- light images. Yet, the accuracy of such methods often degraded by artifacts in the generated input images, resulting more severe 3D misalignment. In contrast, we propose a new paradigm that uses shading sequences, a representation that is sensitive to geometry, as the training target, and leverage video generative models to estimate them based the input image, which achieves improved 3D alignment.
Recent advances in video generation (Rombach et al. 2022 Peebles & Xie 2023 Zhang et al. 2023 Ho et al. 2022) have significantly improved video synthesis quality. Methods such (Blattmann et al.) 2023] Deng et al., 2024; Kuaishou] 2024 Lin et al.) 2024| Guo et al. 2023) can generate high-fidelit videos by enforcing temporal consistency across frames through deep architectures such as temporal UNets (Blattmann et al.] 22023] Guo et al., (023) and Transformers (Deng et al.
2024 Lin al. 2024). In 313 generation, video diffusion models are used to facilitate cross-view
3

---

Published as a conference paper at ICLR 2026
consistency (Voleti et al. 2024; Tang et al. 2024 Dai et al. 2023) improve the quality of generated 3D models. In 3D estimation, recent work (Bin et al. 2025) employs a video diffusion model for normal estimation. They focus predicting per-frame normals for an input video. In contrast, our work leverages the capability of video generative models to predict a shading sequence that follows a predefined light path consisting of multiple canonical parallel lights, using only single input image. This enables accurate monocular normal estimation for objects with diverse shapes and materials.
Shading Utilization. In photometric stereo methods, shadings are often used explain the behavior of deep model in normal map estimation. Previous studies (Chen et al. 2020 Wei et al. 2025) show that learned features closely resemble shading sequences, motivating the use of shading sequence supervision to improve network performance (Wei et al.) (025). Inspired by these findings, we reformulate monocular normal estimation as shading sequence estimation task and train video diffusion model with shading sequences as targets.
3 METHODS
3.1 ON EQUIVALENCE OF NORMAL. ESTIMATION AND SHADING SEQUENCE ESTIMATION
Shading map and shading sequence. this paper, we define shading map (Wei et al., 2025) as:
S A {sp = max 1, 1,0)pp E P),
(I)
where is the normal map, and 1 is the direction of parallel light, max(.. 0) is the nonlinear maximum operation that clamp the negative values, P is the points that belong to the object. Shading maps remove the effects of surface reflectance and occlusion-induced cast shadows while preserving the geometry information and the attached shadow. A sequence of shading maps obtained under multiple canonical lights, defined as a shading sequence, S A {S: i 1, ff. provides sensitive cues to the underlying 3D geometry.
Normal map estimation. Given an observed image I of an object captured under arbitrary lights, the goal of monocular normal estimation is to recover the normal map N A {nppp E P). This requires learning mapping:
① II+N.
Previous methods rely deep models to learn a direct color mapping from single RGB image to a normal map. This often produces a visually aligned appearance but inaccurate 3D geometry (i.e, the 3D misalignment). A more recent line of work (He al. 2024b; Tiwari et al.) 2024) first generates a series of RGB images under simple light sources and then estimates normals from them. The main idea of these works is augment the input with additional generated images that provide additional cues, thereby improving the prediction of normal maps. However, as the materials, lights, and geometry in the input image become more complex, the process of generating additional RGB images itself introduces substantial bias, ultimately leading to more pronounced 3D misalignment artifacts.
Shading sequence estimation. The shading sequence under set of predefined, non-coplanar, parallel lights (canonical lights) can be converted into a normal map. This enables us to shift the training target to shading sequence prediction with lights that vary along a predefined path, L e (4 j ,,...,,ff f}.
t l S*
where I, denotes the grayscale input image. Then, the shading-to-normal estimation S + N can be solved via Ordinary Least Squares (OLS) (Woodham) 1980):
N arg min J - 'T
The solution is unique when is full rank. In practice, however, the max(-, 0) operation causes a truncation effect, which makes the OLS estimate biased if OLS is applied directly to the shading sequence. To address this issue, we solve for the normal using only shadings with values greater than 0, treating them as valid equations in OL.S
4

---

Published as a conference paper at ICLR 2026
f
Alls
Modul
Arsiltit Salver
Convension
Freeze
Nokyy Latans
Trainable
Concer
Paraliell
RGB
Gay Image
Predkoied Shading Bequence
Predicted Norrel
Given a monocular RGB image under arbitrary light, RoSE first converts it into grayscale image, which is then used to generate the shading sequence via video diffusion model. This generation is guided by two complementary feature representations extracted from CLIP encoder and a VAE encoder. Finally, ordinary least squares problem is solved analytically to estimate the normal map from the generated shading sequence. We train the video diffusion model while freezing the CLIP and the VAE encoder.
Figure Pipeline of RoSE.
3.2 SHADING SEQUENCE- BASED TRAINING TARGET
Reformulating monocular normal estimation shading sequence estimation introduces addi- tional flexibility in designing the training target, since different choices of L yield differ- shading sequences. As long each surface point is illuminated by least three non- coplanar parallel light sources (iee, the lighting matrix L is full rank in Eq. 4 normal maps can be recovered from the shading sequence without information loss. In our setup, this means that each surface point should correspond to at least three positive shading val- ues. this paper, we adopt a classic ring light setup from photometric stereo (Zhou & Tan 201O), where canonical lights are uniformly placed on latitude ring in the upper hemi- sphere of the object's surface, each light oriented toward the surface center (see Fig. 4 With an appropriate choice of the ring's latitude (45% in setup), these lights collectively illuminate all surface points. The remaining question is: How many lights with distinct positions do we need, in particular the minimum number binn guarantee that every surface point illuminated by at least three light sources with positive shading values? We address this Lemmal]
Lemma 1. Define surface point as illuminated if its shad- ing value is positive, i.c, max(0, S) > O. Under single parallel light, at least half of the upper hemisphere is illami- nated. Therefore, = lights are sufficient ensure that every point on the hemisphere is illuminated once. To further guarantee that every point is illuminated by least 3 different lights, the minimum number of lights is Imin m n with the light directions uniformly distributed along the ring.
Object
Ring Light
Figure 4: Ring light setup.
In our experiments, we observed that slightly increasing the number of light sources improved the accuracy of both normal and shading estimation. The best performance was achieved with light sources, yielding 0.74" improvement on LUCES Mecca et al. (202T) compared to 6 lights. While appropriately introducing additional light sources can further enhance accuracy, it also incurs longer training and convergence time as well as higher resource consumption. For instance, under the same settings, performance dropped by 1.31 under 12 lights.
3.3 ROSE: MONOCULAR NORMAL ESTIMATOR BASED ON VIDEO GENERATIVE MODEL.
Architecture RoSE shown in Fig. Firstly, the shading generator go (.) is designed to take grayscale images I as input, effectively eliminating redundant chromatic information that may distract the model from learning geometric cues. It produces grayscale shading sequences that follow predefined light path, introducing structured patterns and temporal coherence well-suited to video generation models. In this paper, we implement go(.) using a standard video diffusion U-Net composed of multiple spatial and temporal transformer blocks (Voleti al.) 2024 Blattmann et al.) 2023). The grayscale input image I, used as an additional condition to guide the denoising process during shading sequence generation.

---

Published as a conference paper at ICLR 2026
Specifically, following previous works (He al. 2024b Voleti al. 2024 Blattmann et al. 2023). we adopt a similar dual- branch conditioning strategy that combines global guidance from CLIP embedding and local guidance from VAE latent concatenation to reuse the pre-trained weights of the model. 1) CLIP embedding: We extract a global feature vector cg from the input image using a pretrained CLIP encoder. This semantic embedding is injected into the denoising U-Net via cross- attention, guiding the shading generation with object-level context. 2) VAE latent concatenation: to preserve spatial details, we encode the grayscale input Ig with a pretrained VAE encoder & and concatenate the resulting latent with the noisy latent 2 at each denoising step. Since Ig is single-channel, we replicate
three channels before feeding into the VAE and CLIP encoders: T = repeat B x H x W + B x H x W x 3), where B is the batch size. By combining both conditioning techniques, the model better preserves the fine geometric structures of the input image during generation, which is particularly important for accurate normal estimation. Finally, the generated latents are decoded by the VAE decoder D and averaged across channels to obtain the final grayscale shading sequence.
Training. During training, we use the standard training objectives on latent space encoded by E. The video generative model will learn to predict the noise given the noisy latent 2,, Zu ES*)). where Z = age + oiZg. The diffusion loss follows calculation of Z reprraaaamtttiiiiooonn (Ho et al. 2020):
Ze a.ga (ze)c.
Laitt = Ez, ||z0 2oll² 20
(5)
where bo the one-step denoised version of Z
Dataset Curation. To improve robustness when handling more complex materials and lights, we curate dataset named MultiShade, featuring diverse shapes, materials, and light conditions to ensure robust generalization. MultiShade is built upon a list of pre-filtered 3D models (90K) curated from Objaverse (Deitke et al.] [023] He et al. 2024b), widely adopted resource for 3D generation and reconstruction. For each object, we render observed images under three lighting setups: (1) parallel lights randomly placed around the object; (2) point lights with randomly sampled positions and intensities; and (3) environment lights using fetb-dynamic--aae range (HDR) maps sampled from a public collection of 780 real-world environments (Poly Haven 2025). Each object is rendered from six distinct viewpoints (top, left, right, bottom, front, and one random view) to ensure comprehensive geometric coverage. To avoid lighting from the object's backside in each view, we apply view- dependent transformations to keep light sources in the upper hemisphere relative to the view direction. During rendering, we implement material augmentation to the dataset by either retaining the object's 's original texture or applying material augmentation. With a probability of 0.5, an additional material is assigned from the MatSynth dataset (Vecchio & Deschaintre 2024), which contains 5,657 high-quality PBR materials. More specifically, we assign probability of 0.25 to sample materials from the metallic category, while 0.25 to extract materials from non-metallic categories such as plastic, wood, and fabric. This augmentation improves surface diversity and model robustness, especially for metallic materials. All images are rendered using Blender at resolution of 576 x 576 following (Voleti et al. 2024), generating approximately million image-normal pairs. Precomputed shading sequences under known canonical light sources are also provided. More details on rendering parameters, camera setup, and augmentation strategies are in the appendix.
EXPERIMENTS
4.1 EXPERIMENT SETUP
Datasets. We evaluate the proposed method on widely used benchmarks, including LUCES (Meccal ct 202I) for lear-light monocular normal estimation, DiLiGenT (Shi et al. (016)) for parallel-ligh settings, and a curated test set of 100 unseen objects from the Objaverse dataset (Deitke al. 2023) rendered with diverse materials and light conditions.
Baselines. We compare RoSE with 7 other monocular normal estimation methods, i.e, GeoWizard Fu et al. 2024). DSINE (Bae & Davison| (024), StableNormal (Ye et al. 2024), Lotus-G & Lotus-D HHe al. 2024a), Neural LightRig (He et | (024b), and NiRNE (Ye al.) 2025).
Implementation details and evaluation metrics. All training experiments are conducted on 8 x NVIDIA H100 GPUs with 80GB memory. The model is trained at learning rate of 1 x e5 using
6

---

Published as a conference paper at ICLR 2026
Table I: Quantitative comparison in terms MAE (4) of the normal map on DiLiGenT bench- mark dataset. Highlighted numbers indicate the best and second best results among monocular estimation methods.
Mothod
BALL
BUDDHA
CAT
Cow
GOBLET
HARVEST
PoT1
PoY2
READING
Mean
GooWizard
16.85
14.58
26.38
21.82
19.54
17.70
29.78
21.86
19.97
29.42
21.79
DSINE
23.82
14.15
28.09
18.22
19.35
22.63
35.90
20.90
19.14
30.25
23.25
StamleNormal
13.17
22.46
17.43
Lotus-D
36.83
11.29
21.68
23.93
22.62
13.93
34.99
21.45
17.14
25.49
22.94
Lotus-G
12.74
13.02
23.27
22.68
22.78
15.52
32.94
23.27
19.23
28.67
21.41
Neural LightRig
10.16
14.47
26.23
28.39
21.16
22.70
76.82
24.71
31.84
34.51
29.10
NiRNE
10.26
10.87
21.28
15.43
15.03
17.91
27.40
15.27
16.15
23. 23.08
17.27
Ours
5.51
9.22
20.72
15.78
13.28
16.55
28.62
16.05
14.24
23.65
16.36
Table 2: Quantitative comparison in terms MAE of the normal map on LUCES benchmark dataset (Mecaa et al. 2021). Highlighted numbers indicate the best and second best results among monocular estimation methods.
Method
BALL
BELL
BOWL
BUDDHA
BUNNY
CUP
DIE
HIPPO
HOUSE
JAR
OWL
QUEEN
SQUIRREL
TooL.
Mean
GooWizand
30.09
9.08
22.29
15.90
20.30
15.76
17.55
42.15
11.07
35.48
18:57
22.49
DSINE
26.88
15.00
9.53
22.34
15.82
22.65
32.02
14.42
36.95
16.26
27.46
23.76
25.26
17.19
21.82
StableNonaal
9,58
9.36
31.39
20.80
14.73
29.40
11.88
20,30
37.55
8:25
23.23
21.10
27.24
19.49
20.34
Lotus-D
35.30
Lotus G
17.82
8.66
10.89
19.71
12.90
23.26
12.59
16.94
35.32
10.69
18.94
20.65
24.05
11.74
17.44
Neural LightRig
9.52
11.95
21.71
20.66
15.25
18.08
25.13
18.54
39.67
19.78
23.40
23.35
25.32
20.97
20.95
NiRNE
10.55
12.00
17.35
20.62
16.14
15.78
12.57
15.85
34,99
10.37
22.46
22.41
21.90
17.34
17.88
Ours
9.09
17.58
12.70
13.80
36.79
5.93
19.99
21.34
10.66
14.48
A as the optimizer. The diffusion architecture follows previous work (Voleti et al. 2024 More details can be found in the appendix. To assess the accuracy of predicted normal maps, tollowing the common protocol in previous works (Ye et al.) 2024 2025 Bac & Davison 2024 He et al. 2024a). we use the mean angular error (MAE) the evaluation metrics for all experiments.
4.2 PERFORMANCE ON BENCHMARK DATASET
We conduct monocular normal estimation experiments on the DiLiGenTT (Shi et al. 2016) and LUCES benchmark dataset (Mecca et al. 2021) to evaluate our method's ability in handling objects captured under distant and near-fieh field light sources. For each object, we select 10 images with relatively centered lights so that the light can cover enough details. The index of the images used for testing can be found in the appendix.
Quantitative analysis normal estimation. The results in Table I and Table present the average MAE for each object across selected 10 images, and the average MAE across all objects. These quantitative results demonstrate significant advantage of our method over the state-of-the-an method (16.362 for ours vs. 17.27 for NiRNE (Ye et al. 2025) on DiLiGen'T dataset (Shi et al.) 14.48" for ours vs. 17.44" for Lotus-G He et al. (2024a) on LUCES dataset (Mecca al. 202
2016);
This validates the effectiveness of our method in achieving robust performance in normal estimation across various materials and light conditions. However, we observe that for certain objects, such as GOBLET in DiLiGenT and HOUSE in LUCES, our method does not rank within the top two. We attribute this to the model's inherent variance. Note that even the previous SOTA method, NiRNE, fails to deliver consistently strong performance across all cases. Another possible reason may be attributed to the training set used. We discuss this in Sec. 4.4
Qualitative analysis on normal estimation. We present a qualitative comparison between our method and state-of-the-art methods Fig. Our method consistently recovers accurate object details in the estimated normal maps, achieving lower MAE. In contrast, previous methods often produce overly smooth results, distorted normal distributions, or significant artifacts (He et al. 2024b) (e.g., the tail and back of the SQUIRREL). These results validate the effectiveness of the proposed shading sequence-basee based formulation and demonstrate RoSE's advantage in preserving fine shape details for accurate normal estimation.
7

---

Published as a conference paper at ICLR 2026
GT Normal / Ref RGB
Neural LightRig
GeoWizard
DSINE
StableNormal
Lotus-G
Lotus-D
NiRNE
Ours
19.35
23.11
22.78
21.16
15.03
Squirrel
82
25 23.41
22.91
24.59
50'
Figure 6: Qualitative comparison selected objects from two benchmark dataset (Cow from DILIGENT (Shi et al.) 2016) and SQUIRREL from LUCES (Meeca et al.) 2021). Row & 3: normal map comparison. Row 2 & 4: error map comparison.) Best viewed in color with zooming in.
Table 3: Quantitative comparison in terms of Mean and Median Angular Errors of the normal map on MultiShade test set, and the percentage objects below a specific error bound. Highlighted numbers indicate the best and second best results among monocular estimation methods.
Method
Mean 4
Median
3"
5" f
7.5" 1
11.25 1
22.5"
30" f
GeoWizard
20.46
11.61
12.84
25.41
37.34
49.09
68.53
76.29
DSINE
22.53
14.04
12.38
22.47
32.18
43.27
65.19
74,16
StahleNormal
19.71
11.23
6.83
18.67
34.65
50.08
71.66
79.48
Lotus-D
18.48
10.63
14.51
26.34
38.78
51.78
72.47
79.82
Lotus-G
18.76
10.65
14.67
27.13
39,19
51.63
71.83
79.54
Neural LightRig
20.59
11.36
17.65
27.59
37.90
49.69
70.85
78.54
NiRNE
19.57
11.92
42.10
71.42
Ours
15.37
26.99
38.38
60.32
78.30
84.28
Analysis on shading sequence estimation. We conduct quantitative analyses of the predicted shading sequences on the LUCES dataset illustrate RoSE's ability recover accurate shading sequences. The shading map of all other methods (including the ground truth) is computed as the dot product between the lights' directions and the surface normal, with negative values clamped to zero. We use PSNR (t), SSIM (t), and LPIPS (4) as the evaluation metrics, as shown in Table The results demonstrate that RoSE achieves SOTA performance predicted shading sequence, which also align with the results of normal estimation.
4.3 PERFORMANCE ON MULTISHADE
To further evaluate our method's 's performance across various lighting conditions and materials, the test the applied synthetic dataset consists of 100 unseen objects from Objaverse (Deitke et al.) 2023). Each object is rendered with random materials selected from the MatSynth test set (Vecchio Deschaintre 2024). For lighting conditions, we employ one random point light, one directional (parallel) light, and two environmental lights selected from Poly Haven (Poly Haven (2025) that different from the training dataset. Each object is rendered from seven viewpoints, including the front, back. left, right, and top views, well two randomly sampled views. This setup yields total of
Please refer the appendix for qualitative analyses.
8

---

Published as a conference paper at ICLR 2026
Table 4: Quantitative comparison on estimated shading sequence in terms of PSNR (1), SSIM (1), and LPIPS (L) on LUCES benchmark dataset (Mecca et al. 2021). Highlighted numbers indicate the best and second best results.
Metries
GeoWizard
DSINE
StahleNormal
Lotus-D
Lotus-G
Neural LightRig
NiRNE
Ours
PSNR (t)
16.86
17.05
18.40
18.80
19.19
17.88
18.99
20.74
SSIM (T)
0.6920
0.7199
0.7411
0.7492
0.7589
0.7139
0.7503
0.7744
LPIPS ())
0.2806
0,3100
0.2972
0,2868
0.2724
0.2831
0,2688
0.2583
Table
Quantitative analysis terms of MAE and SNE of the normal map on LUCES benchmark
dataset (Mecca et al. 2021
Highlighted numbers indicate the best and second best results.
GcoWizard
DSINE
StableNormal
Lotus-D
Lotus-G
Neural LightRig
NiRNE
Ours
MAE ( SNE (J)
22.49 37.76
21,82
20.34
18.56
17.44
20.95 32.77
17.88
14.48
33.08
29.20
33.20
29.85
26.78
26.74
2800 test samples. Following the evaluation protocol in prior work (He et al. 2024b), report the mean and median angular error (MAE) across all objects, as well as the percentage of objects with MAE below specified angular thresholds. As shown in Table our method consistently outperforms baseline approaches across all metrics, with particularly strong performance under tighter thresholds (i.e, 77.7%)) highlighting the robustness and accuracy of the proposed RoSE.
4.4 ABLATION STUDY
We conduct ablation experiments using the LUCES benchmark dataset (Mecca al. 202T] as the test set to analyze the effectiveness of the proposed RoSE and MultiShaee. Additional experiments, analysis, and discussion are in the appendix.
Validation on details alignment. Following (Ye ct al. 2025). we evaluate detailed alignment by computing the sharp normal error (SNE), i.e, the normal estimation error measured in boundary regions, on the estimated normal maps from the LUCES dataset, see Table 5 Our method achieves performance comparable to the state of the art method (NiRNE Ye et al. (2025) and shows clear advantage over other baselines. It is worth noting that NiRNE was trained on dataset nearly 10x larger and containing significantly more diverse and complex 3D models than ours. These results highlight that the proposed RoSE is capable of generating fine-grained details even with substantially lower resource consumption during training.
Validation negative-clamping on shading sequence. After clamping negative values, the shading sequence is rescaled to the range [-1, 1] (by applying a linear transformation S - x 2 1) to match the input requirements of the VAE encoder Blattaan et al. (2023). This rescaling makes the shading sequence more sensitive to geometric variations. The effectiveness of this strategy is validated in Table with comparison between 'ours' and 'ours w/o clamp'.
Validation on material augmentation. To evaluate the effectiveness of the proposed dataset, we train RoSE on the publicly available LightProp (He et al.) (024b) dataset as a reference. The key difference is that our dataset introduces additional material augmentation to increase material diversity. As shown in Table 6 training without material augmentation (w/o MA, using only the original object materials) yields slightly worse performance on the LUCES (Mecca et al., 2021) benchmark, while enabling material augmentation leads to notable improvements. These results demonstrate the effectiveness of the material augmentation.
Validation on dataset impact. We retrain the previous SOTA method on LUCES (i.e, Lotus-G) using our dataset ("Lotus-G+M"), resulting in consistent improvements and further validating the effectiveness of the proposed dataset. More importantly, when trained on the same dataset, our method still outperforms the baselines: Neural LightRig vs. "Ours+L" and "Lotus-G+M" vs. Ours, clearly demonstrating SOTA performance and highlighting its efficiency and competitiveness. Finally, also observed that for some specific objects, such as HOUSE, retraining Lotus-C G with dataset resulted in decreased performance (35.32% for Lotus-G vs. 38.90 for Lotus-G+Me This suggests that dataset variations may affect estimation accuracy certain objects.

---

Published as a conference paper at ICLR 2026
Table Ablation study in terms of MAE of the normal map on LUCES benchmark dataset (Mecca et al. 2021]. In particular, "+M"("+L") means training on Multishade (LightProp) dataset. "w/o clamp means removing clamping on shading sequence. 'w/o MA' means training on dataset without material augmentation. Highlighted numbers indicate the best and second best results.
Methool
Bala.
CUP
HouSE
QUIENN
SQUIRREL
Toor.
Mean
Latue
17:50
23.26
35.32
20.65
24.05
Neural LightRig
9.52
21.71
20.66
15.25
18.08
39.67
19.78
23.35
25.32
20.97
Our clamp
7.75
17:29
15.72
40.77
21.18
21:01
12.84
15.88
Ours MA
12:97
6.92
19.00
14.90
15.96
10:65
30:35
6.64
20.49
30.65
21.50
13:23
16.19
Lotus G+M
16.21
8.95
7.11
15:37
11.50
22:41
16.40
13.01
38.90
13.14
24.69
19:28
18.96
14.35
17.35
Oun+L
9.37
7.03
3,45
19:12
12.24
14.05
8.75
13.06
38.14
5.81
20.94
20:22
22.19
11.23
15,21
Ous w/ spiral
16.32
9:25
10.97
20.23
16.78
16:70
12.17
16.06
39.68
7.38
22.25
21. 86
22:50
14.29
17.60
Ours w/ RGB
10:36
8.56
8.99
18:28
13.89
11: 166
9.22
13.70
58. 49
5.74
19.80
20.45
20.78
13.99
15.27
Ours w/ SVD XL
8.69
7.68
9:16
18.34
12.43
12:15
5.47
14.11
57.60
0.94
18.75
19:10
19.28
11.38
14.58
Ours
9.89
5.94
6.84
17.58
12.70
13,80
8.26
14.14
36.79
5.93
19.60
19.99
21.34
10.86
14.48
Validation on model variants impact. We train RoSE using different video diffusion backbone, namely Stable Video Diffusion XL. (SVD XL.) (Blattmann et al. (023), on the MultiShade dataset. We denote this variant as "Ours w/ SVD XL". As shown in Table 6, this model achieves performance comparable to RoSE built SV3D (14.58" for "Ours w/ SVD XL" and 14.48 for Ours). This demonstrates that our framework generalises well even when the backbone is pretrained on large-scale, general-purpose video data rather than domain-specific object-centric dataset.
Validation on gray-scale input. We train a variant of RoSE that replaces the grayscale input with an RGB input (iee Ours w/ RGB). The performance drops by 0.79% the LUCES benchmark. The result indicates the effectiveness of the grayscale input, which eliminates redundant chromatic information enable accurate shading sequence estimation.
Validation on ring-light setup. We train RoSE using a different light path where the elevation decreases from 60' to 30 while rotating 360" around the z-axis. We denote this variant as "Ours w/ spiral". As shown in Table 6 this more complex light path leads to performance drop (MAE of 17.60%). This result highlights that the proposed ring-light setup is an effective and efficient design for predicting the shading sequence.
5 DISCUSSION
Conclusion. We propose ROSE, a novel method for monocular normal estimation that addresses the limitations of previous methods in their training paradigms to reduce 3D misalignment. By reformulating normal estimation to shading sequence estimation, RoSE facilitates normal estimation through an image-to-video generative model and a simple analytical solver. To further improve performance across more general scenarios, we train RoSE on MultiShade, a large-scale dataset with diverse materials and lighting conditions. Experiments show that RoSE outperforms state-of-the-art art methods.
Limitations & Future Work. While RoSE demonstrates strong performance in normal estimation across various settings, it has several limitations. First, employing video diffusion models for shading sequence generation introduces additional computational overhead, which may limit the applicability of the method in real-time scenarios. Second, RoSE may struggle under extreme lighting conditions, particularly when large regions of the object receive insufficient illumination, resulting in degraded shading quality and less reliable normal predictions in those areas. Third, RoSE fails produce high-qualit normal maps on transparent or semi- transparent objects, and extending support for such cases will be an important direction for future work. Finally, the current evaluation is primarily object- centric, with focus on robustness to varying light sources and reflectance properties. Extending RoSE to scene-centric settings remains an important direction for future work.
Acknowledgment. This work is supported by the Ministry of Education, Singapore, AcRF Tier 1 grant No. RG98/24, and ByteDance Inc.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Gwangbin Bae and Andrew J. Davison. Rethinking inductive biases for surface normal estimation. In CVPR, 2024.
Gwangbin Bac, Ignas Budvytis, and Roberto Cipolla. Estimating and exploiting the alcatoric uncertainty in surface normal estimation. In CVPR, 2021.
Aayush Bansal, Bryan Russell, and Abhinav Gupta. Marr revisited: 2d-3d alignment via surface normal prediction. In CVPR, 2016.
Yanrui Bin, Wenbo Hu, Haoyuan Wang, Xinya Chen, and Bing Wang. Normalerafter: Learning
temporally consistent normals from video diffusion priors. In ICCV, 2025.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023.
Blender Online Community. Blender 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018.
wwt.:w.wll..... www.w.blender.oo org
Xu Cao, Boxin Shi, Fumio Okura, and Yasuyuki Matsushita. Normal integration via inverse plane fitting with minimum point-to-plane distance. In CVPR, 2021.
Xu Cao, Hiroaki Santo, Boxin Shi, Fumio Okura, and Yasuyuki Matsushita. Bilateral normal integration. In ECCV, 2022.
Guanying Chen, Michael Waechter, Boxin Shi, Kwan- Yee Wong, and Yasuyuki Matsushita. What is learned in deep uncalibrated photometric stereo? In ECCV, 2020.
Zuozhuo Dai, Zhenghao Zhang, Yao Yao, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Animatcanything: Fine- grae-erained open domain image animation with motion guidance. arXiv preprint arXiv:2311.12886 2023.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: universe of annotated 3D objects. In CVPR, 2023.
Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui. Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation without vector quantization. arXiv preprint 24Xiv:2412.14166 2024.
Tien Do, Khiem Vuong, Stergios Roumeliotis, and Hyun Soo Park. Surface normal estimation of tilted images via spatial rectifier. In ECCV, 2020.
David Eigen and Rob Fergus. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In ICCV, 2015.
David F Fouhey, Abhinay Gupta, and Martial Hebert. Data-driven 3d primitives for single image understanding. In ICCV, 2013.
Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. GioWizard: Unleashing the diffusion priors for 3D geometry estimation from single image. In ECCV, 2024.
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307. 04725, 2023.
Jing He, Haodong Li, Wei Yin, Yixun Liang, Leheng Li, Kaiqiang Zhou, Hongbo Zhang, Bingbing Liu, and Ying-Cont Cong Chen. Lotus: Diffusson- based visual foundation model for high-quality dense prediction. arXiv preprint 40Xi 200.2/249../ 2024a.

---

Published as a conference paper at ICLR 2026
Zexin He, Tengfei Wang, Xin Huang, Xingang Pan, and Ziwei Liu. Neural lightrig: Unlock- ing accurate object normal and material estimation with multi luglt diffusion. arXiv preprint arXiv:2412.09559, 2024b.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistie models. NeurIPS, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. NeurIPS, 2022.
Satoshi Ikehata. Universal photometric stereo network using global lighting contexts. In CVPR, 2022.
Satoshi Ikchata. Scalable, detailed and mask-free universal photometric stereo. CVPR, 2023.
Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely, Neural gaffer: Relighting any object via diffusion. In NeurIPS, 2025. 2025.
Kuaishou. Kling AI, 2024. pttps://klingall com/
L'ubor Ladický, Bernhard Zeisl, and Marc Pollefeys. Discriminatively trained dense surface normal estimation. In ECCV, 2014.
Bo Li, Chunhua Shen, Yuchao Dai, Anton Van Den Hengel, and Mingyi He. Depth and surface normal estimation from monocular images using regression on deep features and hierarchical crfs. In CVPR, 2015.
Peng Li, Yuan Liu, Xiaoxiao Long, Feihu Zhang, Cheng Lin. Mengfei Li, Xingqun Qi, Shanghang Zhang, Wenhan Luo, Ping Tan, et al. Era3D: High-resolution multiview diffusion using efficient row-wise attention. arXiv preprint arXiv:2405. 1616, 2024a.
Zhengqin Li, Zexiang Xu, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Learning to reconstruct shape and spatially-varying reflectance from a single image. ACM TOG, 2018.
Zongrui Li, Qian Zheng, Boxin Shi, Gang Pan, and Xudong Jiang. Dani-net: Uncalibrated photometric stereo by differentiable shadow handling, anisotropic reflectance modeling, and neural inverse rendering. In CVPR, 2023.
Zongrui Li, Zhan Lu, Haojie Yan, Boxin Shi, Gang Pan, Qian Zheng, and Xudong Jiang. Spin-up: Spin light for natural light uncalibrated photometric stereo. In CVPR, 2024b.
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang. Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.0011// 2024.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3: Zero-shot one image to 3d object. In CVPR, 2023.
Yunfei Liu, Yu Li, Shaodi You, and Feng Lu. Unsupervised learning for intrinsic image decomposition from single image. In CVPR, 2020.
Roberto Mecca, Fotios Logothetis, Ignas Budvytis, and Roberto Cipolla. LUCES: dataset for near-field point light source photometric stereo. arXiv preprint 10Xiv:2104.13/33 2021.
Rodrigo D Moreira, Flávio Coutinho, and Luiz Chaimowicz. Analysis and compilation of normal map generation techniques for pixel art. In Brazilian Symposium on Computer Games and Digital Entertainment, 2022.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
Poly Haven. Poly haven, 2025. https://pol http:///polyaave...mm yhaven...... com/
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
12

---

Published as a conference paper at ICLR 2026
Shen Sang and Manmohan Chandraker. Single-shot neural relighting and svbrdf estimation. In ECCV, 2020.
Boxin Shi, Zhe Wu, Zhipeng Mo, Dinglong Duan, Sai-Kit Yeung, and Ping Tan. A benchmark dataset and evaluation for non-lambertian and uncalibrated photometric stereo. In CVPR, 2016.
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. LGM: Large multi-view gaussian model for high-resolution 3D content creation. In ECCV, 2024.
Ashish Tiwari, Satoshi Ikchata, and Shanmuganathan Raman. Merlin: Single-shot material estimation and relighting for photometric stereo. In ECCV, 2024.
Giuseppe Vecchio and Valentin Deschaintre. MatSynth: A modern PBR materials dataset. In CVPR, 2024.
Vikram Voleti, Chun-Har Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from single image using latent video diffusion. ECCV, 2024.
Rui Wang, David Geraghty, Kevin Matzen, Richard Szeliski, and Jan-Michael Frahm. Vpinet: Deep single view normal estimation with vanishing points and lines. In CVPR, 2020.
Xiaolong Wang, David Fouhey, and Abhinav Gupta. Designing deep networks for surface normal estimation. In CVPR, 2015.
Xiaoyao Wei, Zongrui Li, Binjie Ding, Boxin Shi, Xudong Jiang, Gang Pan, Yanlong Cao, and Qian Zheng. Revisiting supervised learning-based photometric stereo networks. IEEE TPAMI, 2025.
Alexander Wong and William Bishop. Adaptive normal map compression for 3d video games. Proc Future Play, 2006.
Robert Woodham. Photometric method for determining surface orientation from multiple images. Optical Engineering, 1980.
Kailu Wu, Fangfu Liu, Zhihan Cai, Runjie Yan, Hanyang Wang, Yating Hu, Yueqi Duan, and Kaisheng Ma. Unique3d: High-quality and efficient 3d mesh generation from single image. In NeurlPS, 2024.
Chongjie Ye, Lingteng Qiu, Xiaodong Gu, Qi Zuo. Yushuang Wu, Zilong Dong, Liefeng Bo, Yuliang Xiu, and Xiaoguang Han. StableNormal: Reducing diffusion variance for stable and sharp normal. ACM Transactions on Graphics (TOG), 2024.
Chongjie Ye, Yushuang Wu, Ziteng Lu, Jiahao Chang, Xiaoyang Guo, Jiaqing Zhou, Hao Zhao, and Xiaoguang Han. Hi3DGen: High-fidelity 3D geometry generation from images via normal bridging. arXiv preprint 2iXi::250332223666 3, 2025.
Youngjin Yoon, Gyeongmin Choe, Namil Kim, Joon- Young Lee, and In So Kweon. Fine-scale surface normal estimation using single nir image. In ECCV, 2016.
Chanki Yu, Yongduek Seo, and Sang Wook Lee. Photometric stereo from maximum feasible lambertian reflections. In ECCV, 2010.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023.
Zhenyu Zhang, Zhen Cui, Chunyan Xu, Yan Yan, Nicu Sebe, and Jian Yang. Pattern-fffnitive propagation across depth, surface normal and semantic segmentation. In CVPR, 2019.
Zhenglong Zhou and Ping Tan. Ring-ligt king-light photometric stereo. In ECCV, 2010.
13


Reviews:
[{"review_id": "Jl0DJuB9P6", "reviewer": "Reviewer_hBMD", "summary": "This paper addresses the task of estimating surface normal maps from a single RGB image. As I understand it, the motivation stems from the observation that prior single-image normal estimators often produce results that appear color-correct but fail to align with true surface geometry. The so-called “3D misalignment” problem in the paper. The idea of this paper is that, instead of directly predicting normal maps, the authors propose leveraging shading information, which, as shown in Figure 3, is more sensitive to surface detail (assuming Figure 3 is accurate). This provides both intuition and validation for their approach.\n\nBuilding on this idea, the authors design a new normal estimation framework that takes advantage of a pretrained video diffusion model. Rather than training the model to predict normals directly, they fine-tune the video diffusion network to generate a sequence of shading images. These predicted shading images are then used to recover the final normal map via a solver.\n\nExperimental results demonstrate that this method achieves state-of-the-art performance in single-view normal map estimation.\n\nIn summary, I find the motivation and design of this paper reasonable and well-aligned. I personally like the core idea, especially if there are no prior works that attempt to estimate normal maps from single RGB images through shading prediction (In the scope of neural network normal prediction from monocular images)\n\nAs for weaknesses, the paper could be improved by including more visualization materials to help readers better understand the advantages and effectiveness of the proposed approach. Details are provided below.", "questions": "Although I like the main idea, I am curious whether Figure 3 is truly accurate. According to Equation 1, the shading map is a linear projection (the dot product between the normal vector and the light direction). As far as I understand, if the normal map itself has “3D misalignment”, as the paper describes, where if neighboring values are similar, then its linearly projected shading map should also show similar misalignment, since the neighboring values would remain correlated. Could the authors clarify this point further?\n\nIn fact, I suspect that the main reason why learning shading images works better is because shading images are more natural than normal maps. That is, they are more consistent with the distribution or prior learned by the pretrained diffusion model. I would appreciate if the authors could help address this hypothesis.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- I appreciate the motivation and core observation of this paper, as well as how the method design naturally follows from the motivation. Specifically, I like the logical flow, identifying an underlying mechanism that may cause problems, analyzing it, and addressing it through targeted design choices. The approach feels somewhat “old school,” but I personally find it appealing.\n\n- The experimental results are good and convincing.", "weaknesses": "- The figures in the paper are of low resolution. Many images appear blurry and show visible JPEG compression artifacts when zoomed in, making it difficult to discern differences compared to the baseline methods.\n\n- As I mentioned in the paper summary, I think the authors could include more visualization results to help readers better appreciate the framework. For example, in the prediction pipeline, the authors use a video diffusion model to predict sequences of shading images. Could the authors provide video visualizations of these sequences to show how they look?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "En26OfIUda", "reviewer": "Reviewer_otsa", "summary": "The paper introduces the RoSE method to solve the 3D misalignment challenge in Monocular Normal Estimation (MNE). The authors reframe the MNE task from direct normal map prediction to estimating a shading sequence. This sequence, which captures pixel brightness under controlled lighting, is considered more sensitive to fine geometric details, providing a superior training signal. ROSE is trained using the new MultiShade dataset to enhance robustness. Experiments show the method achieves state-of-the-art performance on real-world benchmarks, successfully reconstructing finer geometric details with better 3D alignment.", "questions": "See Weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. It reformulates the task as shading sequence estimation effectively addresses the core issue of 3D misalignment and oversmoothness. It is a novel new formulation.\n2. The new synthetic dataset includes diverse materials, light conditions, and material augmentation, successfully improving the model's generalization ability. The new dataset is also a contribution.\n3. It achieves superior performance on key real-world benchmark datasets like DiLiGenT and LUCES.", "weaknesses": "1. The use of an image-to-video diffusion model for sequence generation introduces significant computational overhead, which may limit its use in real-time or resource-constrained applications.\n2. The current evaluation is restricted to object-centric normal estimation, and generalizing the approach to complex scene-centric (indoor/outdoor) settings remains a key direction for future work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1XTHZu2KHc", "reviewer": "Reviewer_uAAj", "summary": "This paper addresses the long-standing 3D misalignment issue in monocular normal estimation by introducing a novel framework, RoSE. The proposed paradigm reconceptualizes normal estimation as a shading sequence estimation problem and leverages an image-to-video generative model to predict shading sequences under predefined parallel lighting conditions. The normal map is subsequently derived through a simple ordinary least squares formulation. Extensive experiments demonstrate that RoSE achieves state-of-the-art performance on multiple real-world benchmark datasets for object-level monocular normal estimation.", "questions": "1.What is the overall inference speed of the RoSE pipeline, including shading sequence generation and normal reconstruction?\n\n2.How does the proposed method perform when estimating normals for transparent or translucent objects, where shading cues may be ambiguous or physically inconsistent?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality. The paper introduces a novel approach to monocular normal estimation by rethinking the problem in the context of shading sequence estimation. This conceptual shift is quite original, as traditional methods in normal estimation usually focus on individual lighting models or directly predicting normals from images. \n\nQuality. The paper delivers high-quality research with state-of-the-art results on multiple real-world benchmark datasets. Achieving superior performance compared to existing methods demonstrates the technical rigor of the proposed framework. The extensive experimental evaluation suggests that the approach has been thoroughly tested and validated, which is essential in establishing the robustness of the proposed method.\n\nClarity. The paper appears to be clear in presenting its approach. The explanation of the framework—reconceptualizing normal estimation as a shading sequence estimation problem—is concise and understandable. The use of an ordinary least squares formulation to derive the normal map is presented as a simple final step, which helps make the approach accessible without sacrificing technical detail.\n\nSignificance. The significance of this paper lies in addressing a long-standing challenge in computer vision, monocular normal estimation under varying lighting conditions. The ability to generate accurate normal maps from monocular images has numerous applications in 3D reconstruction, robotics, AR/VR, and object recognition. By introducing a novel framework that is not heavily reliant on complex sensor setups or assumptions about lighting, the proposed work could have a substantial impact on various real-world applications where obtaining ground truth normal maps is challenging.", "weaknesses": "1.Insufficient training details. While the paper states that RoSE is built upon the SV3D model (Voleti et al., 2024), the specific parameter configurations, training protocols, and dataset utilization are not described. Providing these details would significantly improve reproducibility.\n\n2.Limited ablation studies. The current ablation experiments primarily focus on dataset influence. It would be valuable to include analyses of the model components — such as varying SV3D settings or substituting alternative video diffusion models — to better assess the robustness and generality of the approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fK4zaDneBT", "reviewer": "Reviewer_81qY", "summary": "The paper proposes RoSE, a monocular surface normal estimator that reformulates normal estimation as “shading sequence estimation.” Instead of directly predicting a normal map from a single RGB image, RoSE leverages an image-to-video diffusion model to generate a short sequence of grayscale, lambertian shadings of the input object under a predefined set of parallel lights; the final normals are then recovered analytically via Ordinary Least Squares (OLS). This change of target is motivated by the claim that shading sequences are more sensitive to geometric variations than the compact color encoding of normal maps, reducing “3D misalignment” (over-smooth or geometrically inconsistent normals). To train and evaluate the model, the paper also curate a synthetic dataset MultiShade with diverse shapes, materials, and lighting. Experiments on DiLiGenT, LUCES, and the proposed MultiShade dataset show state-of-the-art performance among monocular normal estimation methods.", "questions": "- The closed-form OLS solution for normal map estimation requires reasonably accurate shading sequences. How robust is the generated Shading Sequence in practice? If there is bias in the estimated shading sequence from the network, to what extent does it degrade the final normal quality?\n\n- A classic way to incorporate shape-from-shading into deep learning is via a render-loss (dating back at least to [2]), where predicted normals/SVBRDF are rendered under multiple point lights and gradients are backpropagated. When training on synthetic datasets, it is possible to guarantee that the lighting condition used for render loss also exactly align with the proposed lighting setup in this paper. Hence an interesting question is: how do the trade-offs compare between (a) training with a render-loss, i.e., the network predicts normals directly but the loss is render (shading)-aware during training), and (b) the proposed approach in this paper (first generate the shading, then solve for normals offline)? \n\n[1] Wiedemer, Thaddäus, et al. “Video models are zero-shot learners and reasoners.” arXiv:2509.20328 (2025).\n\n[2] Deschaintre, Valentin, et al. “Single-image SVBRDF capture with a rendering-aware deep network.” ACM TOG 37.4 (2018): 1–15.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper proposes an interesting approach that combines the generative capabilities of video models with classic shape-from-shading priors to tackle geometric reconstruction.\n\n- Formulating the synthesis of shading images directly as a video generation problem—rather than relying primarily on image generation as in prior work is an interesting point. This also aligns with recent findings suggesting that video generators already possess strong general-purpose visual understanding and synthesis abilities [1].\n\n- The experimental results appear strong and promising.", "weaknesses": "- The generated Shading Sequence is central to the method, yet the experiments lack both qualitative and quantitative evaluations of this component (see Questions below).\n\n- The discussion of output variation (L357) and the failure cases (last section of the appendix) is too generic. At L357, the inconsistencies across cases are attributed to “the inherent variance of the model,” which is uninformative and unsatisfying. More concrete analysis is encouraged: for example, examining how input illumination, texture complexity, and BRDF properties influence the recovered normal accuracy.\n\n- The paper would benefit from additional visualizations, especially examples of the generated Shading Sequence and relighting videos.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qkR7HeKm4I", "reviewer": "Reviewer_fSqW", "summary": "This paper targets monocular normal map estimation, which aims to recover surface normal information from a single RGB object image under arbitrary lighting conditions. The core technical approach is to formulate the normal estimation problem as a shading sequence estimation task, using an image-to-video generative model to predict a sequence of shading images under predefined parallel light directions from a grayscale input image. This sequence is then converted into a normal map using an ordinary least square solver. The main contributions are to introduce the shading sequence as a more sensitive geometric representation to address 3D misalignment issues. Employing a video diffusion model to generate consistent shading sequences is also interesting. A new synthetic dataset MultiShade is also proposed. Experimental results demonstrate that RoSE achieves state-of-the-art performance on real-world benchmark datasets such as DiLiGenT and LUCES, while also excels on the synthetic MultiShade dataset.", "questions": "Will Negative-clamping introduce nonlinearity, which may violate the OLS linearity assumption?  It is not a perfect linear equivalent.\n\nThe choice of latitude for the ring light is described as an “appropriate choice” without specifying exact values or a computational formula? Is ring light setup the best choice? How do we \"ensuring that each point is illuminated by at least three sources with positive shading values\"?\n\nJust curious: How can we verify that the improvement in normal estimation accuracy comes from estimating shading, rather than the extremely dense information provided by the video model? Is it possible to achieve similar results by estimating depth or directly estimating normals instead?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The paper has some clear strengths:\nTransforming normal estimation into a shading sequence generation task and leveraging video generative models to handle temporal consistency are very interesting choice. This introduces a new perspective for normal reconstruction.\n\nThe paper uses shading sequences as the training target, which is more sensitive to geometric variations than directly predicting normal maps.\n\nThe method achieves state-of-the-art performance on real-world benchmark datasets.", "weaknesses": "The key idea and technical module, which uses a video generation model to predict shading sequences, is similar to NormalCrafter [Bin et al 2025]. However, it seems that it is not discussed in this paper.\n\nI'm not sure if using grayscale input images is a good idea. Despite being claimed in the paper that it can \"eliminating redundant chromatic information\", but did you conduct any ablation studies on this? For complex materials (e.g. metallic reflections), color may provide additional geometric hints, potentially leading to information loss.\n\nThe key claim of “reducing 3D misalignment” is only supported by qualitative comparisons and MAE metrics, without quantifying 3D reconstruction errors. If we use RoSE to estimate multi-view normal maps of a single object, can we achieve better consistency and reconstruction accuracy?\n\nEstimating monocular normals using dense video models consumes excessive additional resources. At this point, I'm still uncertain whether such overhead would be widely accepted in practical applications.\n\nThe paper claims directly estimating single normal maps will lead to 3D inconsistency. I think multi-view normal estimation may alleviate this problem to some extent. The representative works such as Unique3D and Era3D also show detailed normal estimation results via a multi-view setting, but they are not compared and discussed in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)