DualFocus: Depth from Focus with Spatio-Focal Dual Variational Constraints
Authors: Sungmin Woo
Sangyoun Lee
Venue: NeurIPS 2025 poster ()
Keywords: depth from focus
variational constraints
depth estimation
Primary area: deep_learning
TL;DR: This paper introduces a novel Depth-from-Focus (DFF) framework that leverages the focal stack’s unique gradient patterns induced by focus variation, jointly modeling focus changes over spatial and focal dimensions.

Abstract: Depth-from-Focus (DFF) enables precise depth estimation by analyzing focus cues across a stack of images captured at varying focal lengths. While recent learning-based approaches have advanced this field, they often struggle in complex scenes with fine textures or abrupt depth changes, where focus cues may become ambiguous or misleading. We present DualFocus, a novel DFF framework that leverages the focal stack’s unique gradient patterns induced by focus variation, jointly modeling focus changes over spatial and focal dimensions. Our approach introduces a variational formulation with dual constraints tailored to DFF: spatial constraints exploit gradient pattern changes across focus levels to distinguish true depth edges from texture artifacts, while focal constraints enforce unimodal, monotonic focus probabilities aligned with physical focus behavior. These inductive biases improve robustness and accuracy in challenging regions. Comprehensive experiments on four public datasets demonstrate that DualFocus consistently outperforms state-of-the-art methods in both depth accuracy and perceptual quality.

DualFocus: Depth from Focus with Spatio-Focal Dual Variational Constraints
Sungmin Woo
Sangyoun Lee Yonsei University syleee8yonsei. ac
Yonsei University smw32508yonsei. ac.
Abstract
froph-froom-ocus (DFF) enables precise depth estimation by analyzing focus cues across a stack of images captured at varying focal lengths. While recent learning-base based approaches have advanced this field, they often struggle in complex scenes with fine textures or abrupt depth changes, where focus cues may become ambiguous misleading. We present DualFocus, a novel DFF framework that leverages the focal stack's 's unique gradient patterns induced by focus variation, jointly modeling focus changes over spatial and focal dimensions. Our approach introduces a variational formulation with dual constraints tailored to DFF: spatial constraints exploit gradient pattern changes across focus levels to distinguish true depth edges from texture artifacts, while focal constraints enforce unimodal, mono- tonic focus probabilities aligned with physical focus behavior. These inductive biases improve robustness and accuracy in challenging regions. Comprehensive experiments on four public datasets demonstrate that DualFocus consistently out- performs state-of-the-art methods in both depth accuracy and perceptual quality.
Introduction
Depth estimation plays pivotal role in 3D vision, enabling wide range of applications from 3D reconstruction to augmented reality and robotics. Among various techniques available, Depth-frong from- Focus (DFF) offers passive, hardware-free approach that utilizes focal stack-a series of images captured at different focal distances. This approach is particularly attractive for consumer grade imaging systems, such smartphone cameras, which often lack specialized stereo depth sensors and must infer depth from limited inputs like a single image or a focal sweep.
Unlike monocular or stereo- based depth estimation, DFF is grounded in simple yet robust physical principle: a scene point appears sharpest when the camera's focal plane aligns with its depth. This focus-sharpnes relationship provides an interpretable signal that, with proper calibration, can yield accurate metric depth. Moreover, DFF is inherently free from scale ambiguity- an issue prevalent in monocular depth estimation 31 19 31, and does not require scene-specific priors extensive
supervision 3 30). Despite these strengths, classical DFF techniques [66] [8] 10, I 28) suffer from critical limitations. They rely heuristic focus measures (e.g.. contrast sharpness) and apply post-hoc smoothing to infer depth. As a result, these methods often falter the presence of ambiguous focus responses, fine textures, or abrupt depth changes, producing depth maps that are either too blurry or riddled with noise.
Recent advances in barning-based DFF 26 deep networks to implicitly learn focus patterns from data. While these models achieve notable improvements in benchmark performance, they typically overlook the underlying physical and geometric principles, such as the gradual changes in sharpness driven by light's focus behavior in camera. In particular, they do not fully exploit the complementary roles of spatial and focal information, which are essential for accurate and robust DFF in complex real-world scenes.
29 have tackled some of these issues by training
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this paper, we propose DualFocus, a novel framework tailored for DFF, which jointly models focus variation across spatial and focal dimensions. Unlike conventional learning-base based DFF methods that treat focal stack images similarly to all-in-focus RGB frames or depth maps, DualFocus introduces a physics-aware design that leverages the unique structure of focus cues inherent in focal stacks.
At the core of our approach are two variational constraints grounded in the optical principles of focus. First, the spatial variational constraint focuses not on estimating sharpness directly, but on analyzing the spatial gradient patterns that emerge across focal planes, Since each focal slice captures a different focus distance, the same scene point exhibits varying gradient patterns depending on whether is in or out of focus. In-focus regions tend to show coherent, strong gradients, while out-of-focte regions exhibit diffused or noisy patterns. By comparing these spatial gradient patterns across the stack, the model learns to infer sharpness indirectly and identify reliable depth edges discriminating them from texture-induced gradients, whose patterns differ from the docus-dependent changes observed at true depth edges. Second, the focal variational constraint leverages the continuous and ordered nature of focal transitions in a focal stack. At each spatial location, it encourages a unimodal and bidirectionally monotonic distribution of focus probabilities along the focal axis, ensuring that the predicted focus confidence peaks at the in-focus plane and decreases smoothly as the focal distance diverges in either direction. This physically grounded constraint reflects how real-world points appear progressively blurrier when the focal distance moves away from their true depth, and allows the model to robustly disambiguate noisy focus cues, especially in textureless reflective regions. Together, these constraints introduce domain-specific inductive priors that reflect the unique imaging characteristies of focal stacks, significantly improving robustness and generalization in diverse and challenging DFF scenarios.
Extensive experiments on four public datasets, including NYU Depth v2 25 FoD500 15 DDFF
12-Scene [ and ARKitScenes (2), demonstrate that DualFocus surpasses state-of-the-an methods in both depth accuracy and perceptual quality.
Our main contributions are:
A novel variational approach to DFF that leverages spatial and focal focus variations through constraints grounded in optical principles.
Two tailored variational constraints: (i) spatial constraints that analyze gradient pattern changes across focal planes to identify reliable depth edges, and (ii) focal constraints that enforce unimodal, monotonic focus probabilities aligned with physical focus behavior.
A unified differentiable architecture that integrates these constraints into an and optimization framework, enabling seamless training with standard depth supervision.
offte-of-the-rr performance multiple benchmarks (NYU Depth v2, FoD500, DDFF 12-Scene) and impressive generalization to unseen datasct (ARKitSccnes).
2 Related Work
Depth-from-Focte Focuus DFF, also known as
exploits the principle that objects within camera's depth of field (DoF) appear sharp, while those outside form blurred circle of confusion (CoC) due to lens effects (23 20 Early DFF techniques analyze a focal stack to find the frame where each pixel is maximally sharp, using that focal distance proxy for depth 20 [6]. Sharpness is typically measured using Laplacian-based or frequency-domain operators [16] 15). but these methods often require densely sampled stacks and degrade in low-texture regions due noise sensitivity 20)
The advent of deep learning has transformed DFF by introducing data-driven solutions. DDFFNet 9 introduced end-to-end CNN trained on the DDFF 12-Scene dataset, while AiFNet (2 combined supervised and unsupervised learning to operate with or without ground-truth depth. DefocusNet [5] exploits the CoC-based defocus cue to produce intermediate defocus maps. DEReD 24 adopts a supe-vuueee approach that reconstructs both depth and all-in-focus images from focal stacks via learned optical defocus simulation. DFVNet [29] captures first-order derivatives of volumetric features across focal planes to guide depth estimation. HebridDepth (6] leverages pretrained relative depth models and refines them into metric depth using a DFF backbone.
While these approaches effectively process focal differences and sharpness cues, they typically infer depth directly from appearance features, vulnerable to texture-induce artifacts. In contrast, our

---

DualFocus introduces spatial variational constraints that leverage focus- focus-dependent gradient variations to capture additional focus cues, and focal variational constraints that enforce unimodal, monotonic focus probability distributions aligned with physical focus transitions. This dual approach, unique to DFF, enhances accuracy and robustness complex scenes with ambiguous focus patterns.
Spatial variation modeling in depth estimation. Modeling spatial variation is a cornerstone of depth estimation, enabling the capture of geometric structure through relationships between neighboring pixels. Several methods have leveraged gradient-basee techniques or spatial priors to enhance depth prediction accuracy. Some studies [2T 4 focus on iteratively refining depth maps from off-the-ssed networks. An affinity matrix is introduced to learn pixel-neighbor depth relationships, though its unsupervised nature limits precision [5]. Li et al. [12] explore fusing gradients and depth either through unconstrained and-to-end networks or non con-differentiable optimization. More recently, VA- VA-DepthNet [14] uses first-order variational constraints to depth gradients in single-image depth estimation, reconstructing depth from gradient-aware surface field using least-squares optimization.
These techniques demonstrate the benefits of spatial regularization, yet they operate in single-image or general depth settings without access to DFF's rich multi-focus information. Our method uniquely adapts spatial variational constraints to DFF by leveraging sharpness variation across focal planes, which enables the model to distinguish true depth edges from spurious textures by contrasting gradient responses across focus levels. In this paper, we demonstrate how spatial priors from variational formulations can be reinterpreted to benefit focus- based depth estimation.
3 Method
3.1
Overview
Our method estimates depth from a focal stack by modeling the feature variations induced by focus changes and imposing variational constraints in both spatial and focal domains. As described in Section 3.2] we first construct 4D focus volume from the focal stack to capture discriminative repre- sentations across spatial and focus dimensions. We then introduce two complementary variational constraints: (1) spatial variational constraints, which exploit focus- induced gradient variation to reconstruct reliable, integrable depth gradients while suppressing texture-driven noise (Section (3:3) and (2) focal variational constraints, which encourage unimodal focus probability distributions that align with physical focus behavior (Section 3.4p. The overview of method is depicted in Figure
3.2 Focus Volume Modeling
Given a focal stack of N images captured at distinct focal distances, begin by independently extracting feature maps from each image. These feature maps are then stacked along the focal dimension to form a 4D focus volume V E: #xG XM, where H, W, Cj denote the height, width, and the number of channels, respectively. To maintain consistency in depth interpretation, the focal stack is ordered by increasing focal distance. Similar DFV 29] we compute differences along focal dimension, which captures the variation of image features across different focal settings. We concatenate these features with the original feature map in the channel dimensions to obtain the augmented focus volume V E WX2xx2 serving as the basis for subsequent focus analysis:
Vnn Va+1 Vn).
......---
Vn, V/ - Vn-1,, n N
where [.::]
represents concatenation operation.
3.3 Spatial Variational Constraints
Rather than directly regressing absolute depths, our network predicts the first-order differences between neighboring pixels, representing local depth gradients that capture how depth varies across the scene. While such gradients have often been used regularization terms, we adopt them as primary prediction targets within the context of focal-stack depth estimation. This allows the network to encode focus-dependent structural variations that are imperceptible all-in-focus approaches.
In conventional single-image depth estimation (SIDE), gradients are extracted uniformly across the image regardless of whether they arise from true depth edges from irrelevant textures. This
3

---

Docoday
Doptr Fudion
Laten: Burface Fiid
Hertiing
Focus Probablity
Depti
Foous Voluno
Focal Book
Lyg
Lyv
Lecal DE S00llam LES,
Focel Distance
Figure I: Overview of the proposed DualFocus. Predicted spatial gradients r- (T', are fitted to an implicit surface via least-squares system, where sharp, integrable gradients reconstruct coherent surfaces in in-focus regions of focal stack data. The focus probability is constrained to smooth, unimodal distribution by Liv. reflecting the gradual nature of focus shifts across the stack.
indiscriminate extraction induces -eseguuitt-seeillllll in repeating peeeating-pptterm regions-where strong gradients do not necessarily indicate depth discontinuities. Crucially, SIDE methods lack mechanism to distinguish reliable gradients from noisy ones. Focal stacks, by contrast, inherently encode focus- based variation correlated with depth: a pixel may appear sharp (strong, reliable gradients) one plane and blurred (weak, noisy gradients) in another. By comparing gradient predictions across N focal planes, our model learns to discern reliable depth cues from spurious texture signals.
For each focal plane we predict multi-channel gradient feature F,
F, [r()]
I(.)], (r.,,..... E 282 XC2
(2)
where each channel (
22/W encodes learned x-axis and y-axis depth- variation cues at relatively low spatial resolution (14 14 in our model), capturing coarse, semantically meaningful focus- induced structural patterns. As each focal plane emphasizes different depth layers, the resulting I, fields vary significantly across n, reflecting the "focus dependence" we aim to leverage.
However, direct supervision of T as depth gradients leads to noisy, spatially inconsistent results, as shown Figure 2 This arises from the absence of global integrability constraints, producing gradient patterns that no real surface could yield. To regularize this, project I, onto the space of integrable (curl-free) gradient fields by solving a least-squares optimization. This reconstructs the closest scalar surface whose gradient approximates I, in the least-squares sense. Our formulation is inspired by VA-DepthNet [14]. which applies it to exploit surface field as an intermediate depth representation in a single all-in- fol-in-ocuus image. In contrast, extend the concept to focal stacks, enabling focus- focus-dependent surface representations that better capture variations geometric consistency across focal planes.
Specifically, we solve the overdetermined system separately for each channel
P Pcn
-<(c) arg Pz P
(P P)-
(3)
where E (-1,0, 13 HHW is a fixed finite-difference operator (e.g., Sobel-style stencil) that computes horizontal and vertical derivatives for each pixel. This operator maps scalar field (e.g.. depth) to its spatial gradient, and solving the least-squares system inverts this mapping-yieldinng N E // xC2 a reconstructed surface representation whose spatial gradient best matches the predicted T, in the least-squares
The derivation of the optimal solution in Eq. 3 is provided in Appendix B
Since I, reflects the gradient of an image focused at focal distance fnx the reconstructed z naturally varies across In-focus planes tend to yield coherent structures consistent with scene geometry, while out-of-focus planes produce noisy, non man-intrerable surfaces. This discrepancy implicitly encodes the reliability of geometric inductive bias during training that only in-focus regions can be consistently reconstructed into plausible surfaces, prompting the network to learn I, fields whose gradient energy is concentrated in those regions. As a result, without explicit supervision for focus- awareeass,sssss, the model learns to produce a, representations that are inherently sensitve. Figure 2 illlstrates the effect: eeegraabliit-regulaaieee gradients highlight focused areas more effectively than directly supervised counterparts.
per focal settings. Importantly, this property introduces beneficial
To further guide this behavior, we supervise 2 only where the focal plane is actually in focus. For given pixel x, we define per-pixel, per-plane sharpness weight on indicating how close the plane

---

RGB Image
(Omeet)
(Imegrabe))
Poces Probablity p
RGa tmage
(Dtrent)
(Ireggahle
Focus Photoblity
Food
Fooal Food
Figure 2: Effect of integrability baasabd reconstruction on gradient fields I= (T",T") with N
Each I*,1" e 14xx14XC2 is upsampled and normalized, highlighting focus sensitivity.
with focal distance fn is to the ground-truth depth (x):
exp fa 1) (x)l)
(x) = ΣN exp( fm -D))x)
(4)
This weight peaks at the plane closest to the true focus depth. Using qxx,, we define the spatial variational loss
L Σ (n(x) X,I
(x)
grdd (z)) (x)
(5)
where V denotes the gradient operator, and Ggrad is a 3 convolution that fuses the C2 channels of zn into a 2-channel gradient prediction for the horizontal and vertical directions. By weighting with (x), the loss encourages alignment with true surface geometry only in sharp, in-foeus regions thus avoiding overfitting to unreliable noisy gradients in defocused areas. Visualizations of the surface field gradients provided in Appendix
Focus-guided depth fusion. The reconstructed implicit surface feature z # is then concatenated with the focus volume V e HR x XN resulting in fused feature of shape H x W x (2C) C2) N. The fused feature processed by a cascade of convolutional layers with upsampling, which jointly reason across spatial and focal dimensions. The decoder outputs focus probability maps p R xW'xN representing sharpness, with H' x W' matching the resolution of the input image, and normalized via softmax such that Σ (x (x) 1. Finally, the depth δ(x) is estimated by computing the weighted sum of the known focal distances {fn} using the predicted probabilities Pa weights:
((x) Σ (x)
(6)
In essence, by reconstructing multi-channel focus-dependent surface features z and supervising only their in-focus consistency, extract structural cues unavailable single-image methods. The differences among the 2 across focal distances serve as the key signals that our 31)-Conv decoder learns to fuse, leading sharply localized depth discontinuities and robust absolute depth estimation.
3.4 Focal Variational Constraints
DFF, the continuity of focus transitions across focal planes reflects the physical behavior of light and optics. To exploit this inherent property, introduce variational constraints that enforce smooth and unimodal focus probability distribu- tions along the focal axis. Unlike prior meth- ods that treat focal slices independently or apply limited regularization, our formulation explicitly models inter- plane consistency, enabling coher- and physically plausible depth estimation esppeiaall in complex low-texture regions.
P
Highest Probability
11 12 Is Ia fs (a) w/o Lyv
fz fo fa fs
Valid Focus Trand
(b) w/ Ly
Figure 3: Our focal variational constraints ensure plausible focus probability distribution, reflecting natural focus transitions across the focal stack.
To this end, we define a focal variation loss that promotes unimodality and monotonicity in the per-pixel focus probabilities across the focal stack. Specifically, introduce bidirectional soft
5

---

monotonicity loss:
-
Lv Σ
Σ (max(0, Pi+1 (x)))
+
Σ (max(O,Pi41 -xxx)))) R(x)
where k(x) = arg max, (x) denotes the index of the focal plane with the highest focus probability at pixel x. The first summation penalizes decreases before the peak (encouraging rising slope), while the second penalizes increases after the peak (encouraging falling slope). Together, they softly enforce a unimodal distribution centered the most probable focal plane.
This loss aligns with physical intuition: focus probability should peak at the correct depth and diminish smoothly as the focal plane deviates. By integrating Liv into training, the model learns to produce spatially and temporally coherent focus responses, which in turn improve depth accuracy and robustness across diverse focal conditions. Further analysis of the effect of the focal variational constraint is provided in Appendix C.4
3.5 Loss Function
The overall training objective integrates three complementary loss terms designed to guide both spatial and focal reasoning:
L= Loeph + AguLLv + MyyLev
(8)
where Leepth is the smooth L1 loss between the ground-truth D' and the predicted depth D, and Awy and Agy are balancing scalars. By combining these three objectives, our approach leverages the dual variational constraints to robustly infer fine-gra aene-griied depth from focal stacks. The spatial term ensures focus-dependent latent surface cues, while the focal term preserves global focus dynamics-jointl enabling precise depth localization and smooth transitions across scenes.
4 Experiments
4.1 Experimental Setup
Datasets. To evaluate our proposed method, we leverage four distinct datasets, each offering unique characteristics that enable assessment across diverse scenarios ranging from synthetic to real-world conditions. For FoD500 and DDFF 12-Scene datasets, we follow the training protocol used in DFV 12 (1) NYU Depth v2 (25) is comprehensive indoor dataset with over 24,000 RGB-depth pairs for training and 654 for testing. Since lacks native focal stack images, we generate synthetic focal stacks using the dataset synthesis technique from HybridDepth [6]. adapting it for our DFF framework. The procedure for generating the synthetic focal stacks described
Appendix A.2 the all-in-focus RGB images rather than
For single-image depth estimation (SIDE) baselines, any blurred images from the focal stack, as SIDE models are generally trained on all-in-oocus data. (2) FoD500 [15] is synthetic dataset originally designed for DFD, featuring 400 training and 100 test samples. Each includes a 5-frame focal stack and a ground truth depth map. The image resolution is 256 x 256, which is randomly cropped into 224 x 224. (3) DDFF 12-Scene [9] is a real-world dataset tailored for DFF evaluation, captured via a light-field camera across 12 scenes. We adopt the split from DFV
29 using six scenes for training and validation (e.g., kitchen, seminaroom) and six for testing (e.g., cafeteria, library). Each sample provides a 10-frame focal stack, though we use randomly selected frames for consistency. Training evaluation is performed at the original resolution of 383 x 552, consistent with prior works (2 6 9].
224 224 random crops and flips, while
(4) ARKitScenes [2] is a large-scale mobile AR dataset. We use subset of 5,600 images for zero-shot evaluation to assess the model's ability to generalize to unseen real-world environments without fine-tuning.
Metrics. We evaluate our method's performance using a suite of standard depth estimation metrics:
Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Absolute Relative Error (AbsRel), Squared Relative Error (SqRel), Accuracy (8,, 8, 8,), and Bumpiness (Bump). Detailed formulas for each metric are provided in the Appendix A.4
Implementation details. We train our model on two Titan RTX GPUs using PyTorch. The encoder is based on a ResNet- 18 FPN [13 and the decoder employs 3D-ResNet blocks (8). For optimization,

---

Table 1: Performance comparison on the NYU Depth V2 dataset with single-image depth estimation (SIDE) and depth from focus/defocus (DFF/DED) methods. f ZoeDepth- -M1-N model is used for evaluation. $ indicates results manually reproduced using the released code.
Model
Type
RMSE
AbsRel
8
82 1
53 1
ZoeDepth
SIDE
0.270
0.075
0,955
0.995
0,999
VPD [5]
SIDE
0.254
0.069
0,964
0.995
0,999
Marigold
SIDE
0.224
0.055
0,964
0.991
0.998
ECoDepth
19
SIDE
0.218
0.059
0.978
0.997
0.999
Depth Anything 30
SIDE
0.206
0.056
0.984
0.998
1.000
DefocusNet 15
DFD 0.493
HebridDepth 6
DFF
0.128
0.026
0.995
1.000
1.000
DFV 229
DFF 0.094
0.020
0.998
1.000
1.000
Ours
DFF
0.075
0.013
0.999
1.000
1.000
RGB Image
Hepth
DFV
DoalFocus
Ground Truth
Figure 4: Qualitative comparisons on the NYU Depth v2 dataset.
we use the Adam optimizer =B1= 0.9, B2 0.999) with an initial learning rate of 10 4, which is reduced to 1 x 10 via cosine annealing scheduler. The model is trained for 40 epochs on the NYU Depth v2 dataset with a batch size of 16, and for 2000 epochs on the FoD500 and DDFF 12-Scene datasets with batch size of 20. To estimate the latent surface field z", we solve the regularized normal equation closed form using aachh.inalg.sole as in [14, which is efficiently executed on the GPU. Since this computation is performed at coarse feature resolution (14 14 pixels), introduces minimal overhead and enables fast, stable inference without requiring iterative optimization. Further implementation details provided in Appendix A.I and detailed description of the model architecture given in Appendix A.3
4.2
Comparison with the State-of-the-Art
NYU Depth v2 Dataset. We evaluate our DualFocus on the NYU Depth v2 dataset, a widely adopted benchmark for indoor depth estimation. As summarized Table DualFocus consistently outperforms both single-image depth estimation (SIDE) and DFF/DFD baselines. Compared to the previous state-of-the-an DFV [29), DualFoous achieves 20.2% reduction in RMSE (from 0.094 to 0.075) and a 35.0% reduction in AbsRel (from 0.020 to 0.013), while also improving őj. Figure
7

---

Table 2: Performance comparison on the FoD500 dataset.
Model
MSE 4
RMSE 4
AbsRel |
SqRel
6 1
82 f
Bump |
RDF 10
0.112
0.322
0.46
0.240
0.395
0.646
0.761
1.54
DDFF 9
0.033
0.167
0.17
0.036
0.728
0.900
0.963
1.74
Defocus-N
15
0.022
0.134
0.15
0.036
0.811
0.933
0,966
2.52
DFV (29
0.020
0.129
0.13
0.024
0.819
0.947
0.980
1.43
Ours
0.015
0.112
0.13
0.022
0.829
0.948
0.980
1.31
Table 3: Performance comparison on the DDFF 12-Scene dataset.
Model
MSE
RMSE
AbsRel 4
SqRel 4
8 1
8g f
Bump |
RDF 10
91.8 x 10
0.0941
1.00
0.1394
0.156
0.331
0.475
1.33
DDFF 9
8.9 10
0.0276
0.24
0.0095
0.613
0.887
0.965
0.52
Defocus-Ne 15
8.6 10
0.0255
0.17
0.0060
0.726
0.942
0.979
0.46
DFV 29
5.7 x 10
0.0213
0.17
0.0063
0.767
0.942
0.981
0.42
HyhridDepth
x
0.0200
0.17
0.0060
0.789
0.947
0.981
0.47
Ours
x.7 0.0194
0.16
0.0056
0.800
0.954
0.982
0.40
Table 4: Zero-shot evaluation comparison on the ARKitScenes validation set with SIDE and DFF
methods. + ZoeDepth-M12-N model is used for evaluation.
Model
Type
RMSE +
AbsRel 4
#Params
ZeeDepth
SIDE
0.61
0.33
335M
DistDepth
27
SIDE
0.94
0.45
69M
ZepoDepth
SIDE
0.62
0.37
233M
Depth Anything 30
SIDE
0.53
0.32
336M
DFV 29
DFF
0.43
0.51
20M
HybridDepth 6
DFF
0.29
0.42
67M
Ours
DFF
0.28
0.40
27M
further illustrates DualFocus's qualitative superiority over DFV and HybridDepth, demonstrating
robustness to repeating floor patterns, consistent depth within objects, and enhanced edge preservation. This substantial improvement across quantitative and qualitative metrics demonstrates the superiority of our focal-award design, particularly in leveraging variational focus cues for more accurate depth estimation in indoor environments.
FoD500 Dataset. The FoD500 dataset contains scenes with intricate structures and frequent depth discontinuities, making it particularly challenging for DFF approaches. As presented in Table 2 DualFocus achieves the best performance across all evaluation metrics. In particular, it achieves Bump score of 1.31, outperforming the previous best (DFV: 1.43) by 8.4%. These results emphasize our model's ' capability to maintain sharp depth boundaries and effectively leverage both spatial and focal gradients, leading perceptually faithful depth reconstructions.
DDFF 12-Scene Dataset. The DDFF 12-Scene dataset includes diverse focal stack conditions, providing a rigorous testbed for evaluating DFF methods. Table shows that DualFocus achieves state-of-the-art performance across all metries. yields an MSE 4.7x 10-4 and RMSE of 0.0194, marking a notable improvement over prior methods such HybridDepthh (MSE: 5.1 10 10-4) and DFV (MSE: 5.7 10 +). The enhanced accuracy highlights our model's robustness in resolving depth ambiguities under varying focus cues, supported by spatially-aware focal supervision.
4.3 Zero-Shot Transfer
To assess generalization, we perform pero-shot depth estimation on the ARKitScenes dataset. DFF models including ours are trained solely on the NYU Depth V2 dataset, while SIDE models leverage a broad ensemble of diverse datasets. As reported in Table DualFocus outperforms other DFF baselines, including HybridDepth (6]. which leverages pretrained Depth Anything (30) results for strong zero-shot transfer, and achieves performance competitive with SIDE methods. This highlights its robust generalization to unseen focal stacks, driven by the spatio-focal variational constraints.

---

Table 5: Ablation studies of the proposed components.
Method
RMSE +
log RMSE 4
AbsRel
SqRel 4
w/o spatio-focal variational constraints
0.094
0.027
0.020
0.0038
w/o spatial variational constraints
0.090
0.025
0.018
0.0032
w/o focal variational constraints
0.078
0.022
0.014
0.0022
w/ direct supervision on gradients r
0,083
0.023
0.015
0.0026
w/o sharpness weight w/ blurness weight (1 -))
0,077
0.022
0.014
0.0022
0.079
0.022
0.014
0.0025
Ours
0.075
0.020
0.013
0.0021
4.4 Ablation Study
We conduct comprehensive ablation studies on the NYU Depth v2 dataset to validate the effectiveness of the proposed spatio-focal variational constraints, as summarized Table
Spatio-Focal Variational Constraints. When both spatial and focal variational constraints are removed, the model's performance significantly degrades across all metries (e.g.. RMSE increases from 0.075 to 0.094), underscoring the importance of incorporating variational regularization in DFF. To further disentangle their individual contributions, we evaluate the effects of removing each constraint separately: Omitting the spatial variational constraint leads to a marked performance drop, validating the effectiveness of focus-dependent spatial gradients modeling. Removing the focal variational constraint also results in degraded performance, indicating its auxiliary role in guiding the focus probability distributions reflect natural defocus behaviors. Overall, the spatial variational constraint demonstrates greater impact than its focal counterpart, as it directly encodes surface gradients at each focal plane, capturing localized geometric cues that vary with focus.
Effect of Integrability-based Regularization on To assess the effectiveness of our integrability- based regularization scheme, we compare it against a baseline that applies naive gradient-level supervision directly on the predicted gradient field I. Without enforcing integrability, this baseline yields consistently higher errors across all evaluation metrics, as unconstrained gradients yield spa- tially inconsistent or non- phyysccll surfaces. In contrast, our integrability projection reconstructs curl-free surface approximations from r, regularizing the gradient field toward geometrically consis- tent structures. This constraint not only stabilizes training but also encourages the network to encode focus-aware, surface-consistent representations that align with real-world depth geometry.
Importance of Focus-aware Weighting on Lay. We evaluate the contribution of the sharpness- based focus weighting q in our spatial variational loss Ly., which adjusts the influence of each pixel according to its sharpness. Removing or inverting its effect to prioritize defocused regions (i.c., using 1 9) results in consistent performance drops. These ablations show that emphasizing reliable in-focus pixels is critical for guiding the model toward accurate depth learning. By attenuating supervision in out-of-for oocuf regions, the focus-aware aware weighting encourages the network to concentrate on spatial regions that are both photometrically informative and structurally meaningful.
Additional ablation results, runtime analysis, and quantitative comparisons provided in Appen- dices and
5 Conclusion
We introduce DualFocus, novel DFF framework that exploits the intrinsic characteristies of focal stack data by jointly modeling spatial and focal variations through a dual variational formulation. Our method explicitly incorporates beeuuddependent gradient behaviors and focal sharpness distributions to distinguish true geometric structures from texture-inn ieature-induced ambiguities an area where previous DFF approaches often fall short. By leveraging the complementary strengths spatial and focal constraints, our framework delivers robust and accurate depth predictions, particularly in scenes with fine textures or sharp depth transitions. Extensive experiments across four benchmark datasets validate the effectiveness of our approach, setting standard in focus-based depth estimation.

---

Limitation, While DualFocus improves robustness under challenging focus conditions, limitations remain. First, in regions with low texture contrast, focus variation may offer weak depth cues. Second, the method assumes spatially aligned focal stacks, making it sensitive to motion-induced misalignment. Finally, the scene-dependent blur characteristics (e.g., lighting artifacts or lens- specific aberrations) may limit the model's generalization. Addressing these limitations through motion-invariant modeling or alignment-free representations is promising direction for future work.
Acknowledgement
This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (No. RS-2024-003407455 and the Yonsei Signature Research Cluster Program of 2025 (2025-22-0013),
References
[1] Muhammad Bilal Ahmad and Tac Sun Choi. Application of three dimensional shape from image focus in lcd/tft displays manufacturing. IEEE Transactions Consumer Electronics, 53(1):1-4, 2007.
[2] Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gehauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, al. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data, arXiv preprint arXiv:2111.0885 2021.
[3] Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero-shot transfer by combining relative and metric depth. arXiv preprint arXiv:2302.12288 2023.
[4] Xinjing Cheng, Peng Wang, and Ruigang Yang. Depth estimation via affinity learned with convolutional spatial propagation network. In Proceedings of the European conference computer vision (ECCV), pages 103-119, 2018
[5] Xinjing Cheng. Peng Wang. and Ruigang Yang. Learning depth with convolutional spatial propagation network. IEEE transactions on pattern analysis and machine intelligence, 42(10):231-2371 2019.
[6] Ashkan Ganj. Hang Su, and Tian Guo. Hybriddepth: Robust metric depth fusion by leveraging depth from focus and single-image priors. In 2025 IEEF/CVF Winter Conference on Applications Computer Vision (WACV), pages 973-982. JEEE, 2025.
[7] Vitor Guizilini, Igor Vasiljevie, Dian Chen, Rares Ambrus, and Adrien Gaidon. Towards zero-shot scale-aware aware monocular depth estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9233-9243, 2023.
[8] Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Learning spatio-temporal features with 3d residual networks for action recognition. In Proceedings of the IEEE international conference on computer vision workshops, pages 3154-3160, 2017.
Caner Hazirbas, Schastian Georg Soyer, Maximilian Christian Staab, Laura Leal-Taixé and Daniel Cremers. Deep depth from focus. In Computer Vision-ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2-6, 2018, Revised Selected Papers, Part III 14, pages 525-541. Springer, 2019.
[10] Hae-Gon Jeon, Jaeheung Surh. Sunghoon Im, and In So Kweon. Ring difference filter for fast and noise robust depth from focus. IEEE Transactions on Image Processing, 29:1045-1000 1060, 2019.
[11] Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion-basee image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 9492-9502, 2024.
[12] Jun Li. Reinhard Klein, and Angela Yao. A two-streamed network for estimating fine-scaled depth maps from single rgb images. In Proceedings of the IEEE international conference on computer vision, pages 3372-3380, 2017.
[13] Tsung-Yr Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117-2125, 2017.
[14] Ce Liu, Suryansh Kumar. Shuhang Gu, Radu Timofte, and Lue Van Gool. Va-depthnet: A variational approach to single image depth prediction. arXiv preprint 2rXiv:2302.065566 2023.
10

---

[15] Maxim Maximov, Kevin Galim, and Laura Leal-Taixé. Focus on defocus: bridging the synthetic to real domain gap for depth estimation. In Proceedings of the IEEE/CVF conference on computer vision and puttern recoggition, pages 1071-1080, 2020.
[16] Hari N Nair and Charles V Stewart. Robust focus ranging. In Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Puttern Recognition, pages 309-310. IEEE Computer Society, 1992.
[17] Shree K Nayar and Yasuo Nakagawa. Shape from focus. IEEE Transactions Pattern analysis and machine intelligence, 16(8):824 831, 1994.
[18] Illah R Nourbakhsh, David Andre. Carlo Tomasi, and Michael R Genesereth. Obstacle avoidance via depth from focus. Proc. of Image Understanding Workshop (IUW 96), pages 1339-1344, 1996.
[19] Suraj Patni, Aradhye Agarwal, and Chetan Arora. Ecodepth: Effective conditioning of diffusion models monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2024.
[20] Said Pertuz, Domenec Puig. and Miguel Angel Garcia. Analysis focus measure operators for shape- from-focus. Puttern Recognition, 16()):1415-1432 2013.
[21] Michael Ramamonjisoa, Yuming Du, and Vincent Lepetit. Predicting sharp and accurate occlusion boundaries in monocular depth estimation using displacement fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1464-11467, 2020.
[22] Lingyan Ruan, Bin Chen, Jizhou Li, and Miu-Ling Lam. Aifnet: All-in-focus image restoration network using light field-based dataset. IEEE Transactions on Computational Imaging, 7:675-655 2021.
[23] Yoav Y Schechner and Nahum Kiryati. Depth from defocus vs. stereo: How different really are they? International Journal of Computer Vision, 39: 19:1411162, 2000.
[24] Haozhe Si, Bin Zhao, Dong Wang, Yunpeng Gao, Mulin Chen, Zhigang Wang, and Xuelong Li. Fully self-supervised depth estimation from defocus clue. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9140-9149, 2023.
[25] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Computer Vision ECCV 2012: 12th European Conference Computer Vision, Florence, Italy October 7-13, 2012, Proceedings. Part 12, pages 746-760. Springer, 2012.
[26] Ning- Wang. Ren Wang, Yu-Lun Liu, Yu-Hlao Huang, Yu-Lin Chang, Chia-Ping Chen, and Kevin Jou. Bridging unsupervised and supervised depth from focus via all-in-focus supervision. Proceedings of the IEEE/CVF international conference on computer vision, pages 12621-126311 2021.
[27] King Wu, Jialiang Wang, Michael Hall, Ulrich Neumann, and Shuochen Su. Toward practical monocular indoor depth estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3814 3814-3824, 2022.
[28] Hui Xie, Weibin Rong, and Lining Sun. Wavelet-basse focus measure and 3-d surface reconstruction method for microscopy images. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 229 204.2344222 IEEE, 2006.
[29] Fengting Yang, Xiaolei Huang, and Zihan Zhou. Deep depth from focus with differential focus volume. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22642-12655 2022.
[30] Lihe Yang. Bingyi Kang. Zilong Huang. Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371-10381, 2024.
[31] Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, Jiwen Unleashing text-to-imane -o-tao-image diffusion models for visual perception. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 5729-5739, 2023.
11

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims in the abstract and introduction accurately reflect our contri- bution.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have discussed the limitations inherent to the proposed method in the Section 5
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
12

---

Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We have provided all implementation details for reproduction of results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
13

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We plan to release the code and pretrained models in the future, but they are not available at the time of camera-ready submission.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
public/guides/ hhboli///guides//ddeeuubibiiiss asionoliicy for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (httpa:
/ httppps.ccppbbllcguuceeeeeeoeeee
CodeSubmi ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have provided the necessary information in the Section 4.1 and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We simply have reported and compared the evaluations without statistical analysis, because our experiments stable across multiple runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
14

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have reported the information of compute resources used in our experiments in [4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: htt.t:pss//euurssscccc/os/ heteliic///hiithtteeeee ethhiiduueelllnneeee ines?
Answer: [Yes]
Justification:
Our research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: As our work focus on depth estimation from multi-focus images, a relatively general research improving the deep network optimization with potential societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
15

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: This work properly credits and respects the assets used.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, wwwprswi thcode. com/ con/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
16

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification:
We do introduce any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
17

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy http:p///.eerrrssscc/////teeeees for what should or should not be described.
18

---

A Implementation Details
A.1 Training Details
Our model is trained end-to-end using the Adam optimizer with cosine annealing for learning rate scheduling. We use focal stack of N = 5 images as input, and adopt ResNet-18 18 FPN [13] as the encoder and De-ResNet blocks B as the decoder. The encoder produces the focus volume V with 64-channel features (2C; = 64), while the variational module operates on 16-channel features (C2=16) Two variational loss terms are weighted by Agy 20 and 2,, 100, balancing spatial constraints and focus probability constraints, respectively. For training, we use batch size of 16 for NYU Depth V2 and 20 for other datasets, with training durations of 40 and 2000 epochs, respectively.
A full list of model and training hyperparameters is summarized in Table 6
Table 6: Model and training hyperparmeters.
Hyperparameter
Value
Focal stack size
Encoder Decoder
Resnet- 18 FPN [1
3D-ResNet blocks
Feature channel C1
32
Feature channel C2
16
Asv
20
24 Optimizer Scheduler
100
Adam (B1 = =
Cosine annealing
Initial learning rate Batch size Training epochs
0.001
16 / 20 (NYU dataset Others)
40/ 2000 (NYU dataset / Others)
A.2 Focal Stack Data Synthesis
To address the limitations of datasets lacking real focal stacks and enable robust comparisons with state-of-the-ar (SOTA) models, we adopted data synthesis method of prior work (6) to artificially generate focal stacks from a single image with ground truth depth. The synthesis process involves configuring virtual camera with adjustable focus settings, defining specific focus distances to simulate depth-based focusing, and applying a circular kernel to introduce blur based on the ground truth depth and focus distances. The extent of blur, namely the circle of confusion (CoC), is calculated using a widely adopted equation
[5 24), which is expressed as
S1- c
1
S2
N (SI - ff¹
where f the focal length, is the f-number, S₁ the in-focus subject distance, and S2 is the out-of-focus distance. Following the process in prior work HybridDepth (6), we slightly cropped the NYU Depth v2 images by removing borders from the depth maps to mitigate issucs in focal stack creation. For detailed implementation, readers are referred to (6.
A.3 Model Architecture
The proposed model is convolutional encoder-decoder network for depth estimation from stack of N focal images. It consists of four sequential components: an encoder, a variational module, a decoder with fusion modules, and focus probability prediction head.
The encoder, based on a ResNet-18 Feature Pyramid Network (FPN), processes the input focal stack using 2D convolutions and extracts multi-scale feature maps at 1/4, 1/8, 1/16, and 1/32 of the input resolution x w. At each scale, the features are reorganized into 4D focus volume, following the approach of DFV 229]. to encode focus-dependent information across focal planes.
Encoder.
19

---

Variational Module. To impose spatial variational constraints, a variational module operates at the 1/32 scale and estimates 16-channel gradient fields [r", E p2xh//2xx//2xx1655 from the focus volume. These gradients are used to solve a least-squares problem, yielding surface fields z" E RM/12Xx/332x16xX, The surface features are regularized via group normalization, treating the 16 channels as single group to encourage implicit surface consistency. A convolutional layer then expands the channel dimension from 16 to 128 for richer representation.
Decoder with Fusion Modules. The decoder comprises three hierarchical fusion modules that operate at progressively higher spatial resolutions: 1/16, 1/8, and 1/4 of the input size. At each scale, a fusion module integrates two complementary streams of information: the focus volume features extracted by the encoder and the upsampled surface features z" produced by the variational module. To effectively merge these modalities, the fusion is performed using 3D convolutions, which can jointly process spatial and focal dimensions. This design allows the network refine depth-relevant features by leveraging both the focus- specifs sharpness variations and the geometric surface cues estimated at coarser levels. As the decoder progresses through the scales, the features become increasingly detailed and structurally coherent. Finally. the refined outputs from all fusion stages are upsampled to a common resolution and concatenated, yielding a unified multi-scale representation that encapsulates rich depth cues across both spatial and focal dimensions. This multi-scale representation serves as the input to the subsequent focus probability prediction head.
The aggregated multi-scale features are further upsampled to match the resolution of the input images and then passed through the focus prediction head, which outputs a probability map p E kxxww A softmax is applied across the N focal planes at each pixel location. The final per-pixel depth then calculated as the expectation over the known focal distances { weighted by the predicted focus probabilities pn: D(x) Σ Pa (x) fn- This procedure allows the network to produce continuous depth estimates from discrete focal planes, effectively integrating the focus information learned across the entire stack.
Focus Probability Prediction.
Dual Variational Supervision. During training, the model is supervised with two complementary variational losses. The spatial variational loss computes local gradients from the reconstructed surface features and compares them to the ground-truth depth gradients in in-focus regions, guiding the network to produce coherent surface structures. The focal variational loss constrains the predicted in-focus probabilities across the focal stack to follow smooth and physically consistent transition. These two losses are applied alongside the standard depth regression loss, ensuring the network effectively learns both spatial and
cues from the multi-focus input.
A.4 Evaluation Metrics
We evaluate our method using set of widely adopted depth estimation metrics that collectively measure the accuracy, robustness, and perceptual quality of predicted depth maps. Specifically, we report Mean Squared Error (MSE), Root Mean Squared Error (RMSE). Absolute Relative Error (AbsRel), Squared Relative Error (SqRel), accuracy at multiple thresholds (8, , 8, 8g), and Bumpiness (Bump). Let D = (d:) denote the predicted depth map and D {d;} the corresponding ground truth depth map. where i indexes the N valid pixels in the image (e.g., pixels where ground-trun depth is available and non-zero).
MSE measures average squared differences, emphasizing larger errors: $ IN
(d; d,)2.
RMSE gives average error in depth units: Ed (d))
AbsRel averages absolute differences over ground-truth depth: 1/2 EN d
SqRel averages squared differences over ground-trute depth: $ EN (d (d-d))2
d;
Accuracy (8)) is pixels with depth ratio < 1.25* (k 1,2,3): 1 E (max o. $ < 1.254).
Bumpiness (Bump) assesses smoothness via the variance of the discrete Laplacian: 1 (V2d))
20

---

B Optimal Solution Derivation for Least-Squares System
RRHW and a first-order difference operator r
Given the predicted spatial gradients
1-,,0, xHW we aim to recover the surface field An E R that best satisfies the gradient constraint:
P
(10)
Since the system is overdetermined - ie., the number of equations (2HW) exceeds the number of unknowns (HW)- we seek least-squares solution by minimizing the residual:
(c)
arg P r1. 113
(11)
This objective is convex, and its minimum can be obtained by setting the gradient with respect to to zero. The closed-form solution is then given by the Moore-Penrose pseudoinverse:
(c) (P P) !
(12)
We refer readers for more details on the derivation.
Additional Experimental Results
C.1 Spatial Resolution of the Gradient Fields I and Surface Fields
To analyze the trade-off between performance and computational efficiency, we investigate the spatial resolution of the gradient fields I and surface fields z". As shown in Table increasing the resolution from 10 x 10 to 20 x 20 consistently improves accuracy, particularly in RMSE and SqRel. However, the performance gain between 14 14 and 20 x 20 marginal, while the runtime increases from 26ms to 32ms. Our final model adopts a 14 x resolution, which achieves near-optimal accuracy with lower computational cost, making a more efficient and practical choice for real-world applications.
Table 7: Ablation study the spatial resolution of the gradient fields I and surface fields z² on the NYU Depth v2 dataset. Increasing resolution improves accuracy but increases runtime.
Resolution
RMSE +
log RMSE J
AbsRel +
SqRel +
Runtime (ms) $
10 x 10
0.079
0.022
0.014
0.0024
25
14 x 14
0.075
0.020
0.013
0.0021
26
20 x 20
0.073
0.020
0.013
0.0020
32
C.2 Feature Channel C2 of the Gradient Fields I' and Surface Fields =
To investigate the impact of feature channel dimensionality in our geometric field representations, we conduct ablation study on the number of feature channels C2 used in both the gradient fields I and surface fields z"_ As presented in Table increasing the channel size from to 16 leads to consistent improvements across all evaluation metrics the DDFF 12-Scene dataset.
Specifically, the best performance is achieved when C2 16, which suggests that a richer feature representation in the geometric fields enables the model to better capture fine-grained focus cues and surface details. Conversely, overly limited channel dimensions (e.g., C2 1) may restrict the expressiveness of the learned gradient and surface structures, leading to suboptimal depth estimates.
Table 8: Ablation study on the number of feature channels C2 used in the gradient fields I and surface fields z" on the DDFF 12-Scene dataset.
Feature Channel C2
RMSE |
log RMSE 1
AbsRel +
SqRel /
1
0.0214
0.226
0.188
0.0067
0.0203
0.216
0.174
0.0060
16
0.0195
0.200
0.162
0.0057
21

---

Focal Plane
Focal Plane
Focal Plane
Focal Plane
Focal Plane
Focal Plane
Image
Probability
Focal Plane
Focal Plane
Focal Plane
Focal Plane 1
Focal Plane
Focal Plane
RGB Image
8 (2")
Focus Probability
Figure 5: Visualization of input image, surface gradient magnitude ||0grad (=t) ||. and predicted focus probability p at focal planes 1, 3, and 5 (out of N = =5).
C.3 Visualization of Implicit Surface Field Gradients
The proposed method leverages an implicit surface field z² e 12#xWxCxxt to encode multi-channel surface representations across the focal stack. As described in Section 3.3 of the main paper, z' is derived by projecting predicted gradient features onto the space of integrable surfaces, effectively reconstructing plausible local geometry per focal plane. Since z" resides in a multi-dimmnsionnl latent space, directly interpreting its content is non-trivial. To provide insights into its internal structure and focus dependence, visualize the spatial gradients Ograd (z)), which are used in the spatial variational loss Lay-
Figure presents representative examples from three focal planes (indices 1, 3, and 5, where N =5 5) and includes: (I) the RGB image at each focal plane, (2) the magnitude of Ggrad (zt), and (3) the predicted focus probability p. The visualizations reveal a strong alignment between focus probability and gradient coherence that in-focus regions with high Pa. values consistently exhibit sharper and more structured gradient patterns, whereas defocused regions result in weaker noisy gradients. This focus-aware behavior emerges despite no explicit supervision on sharpness or texture indicating that the integrable reconstruction process implicitly guides the model to concentrate gradient energy in geometrically reliable regions.
This empirical evidence supports the underlying design motivation introduced in the main paper: namely, that predicting spatial gradients rather than absolute depths allows the network to exploit focus-based depth variations and better distinguish true geometric discontinuities from texture- induced noise. The consistency and structure of the gradients in sharp regions demonstrate that the model learns to encode geometry in a focus-aware manner. Furthermore, the observed variation in gradient responses across focal planes shows that the network effectively leverages multi-plane comparison isolate reliable depth cues.
In summary, this analysis reinforces the role zt not just as an intermediate representation, but as focus-conditioned surface descriptor. The structured, integrable gradients emerging in high-focus regions validate our spatial variational formulation and highlight the importance incorporating spatial constraints tailored to the unique characteristics of DFF tasks.
22

---

Table 9: Impact of focal variational constraints on focus probability distribution and performance.
Dataset
Liv
Invalid Focus Trend (%) +
RMSE 4
log RMSE +
AbsRel +
NYU Depth v2
x
46.2
0.078
0,022
0,014
NYU Depth v2
/
6.2
0.075
0.020
0.013
DDFF 12-Scene
x
88.2
0,021
0.211
0.167
DDFF 12-Scene
/
13.4
0.020
0.200
0.162
Focal Plane
Focal Plane 2
Focul Plane 3
Focal Plane 4
Focal Plane 5
Depch
RGB Imuges &
Ground Truth
DualFocus w/o Lyv
DualFocus Lev
Figure 6: Impact of the focal variational constraint Li on focus probability and depth prediction. Without Liv, the model produces inconsistent focus probabilities across focal planes (e.g., non- monotonic trend in red-boxed region), leading to depth errors. With Lyv the focus trend is regularized to be monotonic, resulting in more accurate and stable depth maps.
C.4 Effeet of the Focal Variational Constraints
Table 9 presents the quantitative effect of applying the focal variational loss Li on both the NYU Depth v2 and DDFF 12-Scene datasets. The metric "Invalid Focus Trend" measures the percentage of pixels whose predicted focus probability does not exhibit monotonic decrease with respect to the peak focal plane, i.e., cases where the focus probability fails to drop off smoothly as the distance from the most in-focus slice increases. Lower values indicate better focus behavior that aligns with the expected defocus pattern in natural focal stacks.
Without the proposed variational constraint, substantial portion of pixels-46.291 in NYU and 88.2% in exFF-exhibit invalid focus trends. By incorporating Lv. these rates drop significantly to 6.2% and 13.4%, respectively, demonstrating that the constraint effectively enforces physically consistent focus probability distributions. This improvement in focus consistency also translates into slight gains in depth accuracy across all evaluated metrics (RMSE, log RMSE, and AbsRel), confirming that encouraging structured focus behavior leads to more reliable depth estimation.
To further illustrate the effect of the focal variational constraint, we visualize the predicted focus probability distributions across the focal stack (N 5) along with the corresponding depth maps. As shown in Figure 6, DualFocus without Liv exhibits invalid focus trend in the red-boxed region. Specifically, the Tocus probability peaks at focal plane and decreases across planes to 4, as expected. However, focal plane l-being furthest from the uaak--nnexeetedlly shows higher probability than the intermediate planes. This violates the expected monotonic decay pattern and leads to incorrect depth prediction in the affected region. In contrast, DualFocus with Liv produces a smooth and monotonic focus probability curve centered around the most in-focus plane. The probability consistently decreases as the focal plane moves away from the peak, resulting in valid focus trend. This improved structure enables the model to infer a more accurate and geometrically consistent depth map.
These qualitative results support the quantitative findings in Table demonstrating that the focal variational constraint not only enforces physical plausibility in the focus distribution but also leads to more reliable depth estimation in practice. We can observe that this approach reduces errors caused by invalid focal trends across diverse scenarios, with additional examples provided in Figure 7
23

---

C.5 Scalability to Focal Stack Size N
In this section, we evaluate the scalability of our method with respect to the number of focal stack images the NYU Depth v2 dataset. TO summarizes the performance of our method compared to prior DFF approaches, including HybridDepth [6] and DFV [29] as well as a recent single-image
depth estimation method, Depth Anything (30
The number of focal planes in DFF tasks directly affects the richness depth cues available for learning. As expected, performance generally improves with more inputs and degrades with fewer, trend observed both DFV and our method.
Despite using fewer focal planes, our method demonstrates strong scalability. With only 3 focal images, it outperforms DFV and surpasses HebridDepth trained with 5 images, well the single- image Depth Anything model trained on a large corpus of general-purpose purpose depth data. This result highlights the effectiveness of our approach in extracting meaningful depth cues even under sparse focus supervision.
When increasing the number of focal planes to 10, method achieves substantial performance gain, establishing a new state of the art across all metrics. This demonstrates that our model can effectively leverage sparse focal stacks while fully capitalizing on rich focus information when available, resulting in high-quality and geometrically consistent depth predictions.
Table 10: Performance comparison NYU Depth v2 with varying focal stack size N. Hybrid- Depth (6] results under the 3-stack setting are not reported due to reproducibility issues the released code.
Method
Focal Stack Size N
RMSE
AbsRel 4
Depth Anything
30
0.206
0.056
HybridDepth
6
3
DFV 29
3
0.134
0.030
Ours
3
0.119
0.023
HybridDepth 6
5
0.128
0.026
DFV 29
5
0,094
0,020
Ours
5
0.075
0.013
HybridDepth 6 DFV (29
10
0.083
0.015
10
0.069
0.011
Ours
10
0.052
0.008
C.6 Model Size and Runtime
To assess the efficiency of our method, we compare the model size and inference time against both single-image depth estimation (SIDE) and depth-from-focus (DFF) baselines, as summarized in Table I1 Our model achieves balanced design with small parameter count and fast runtime. Compared other baseb methods, remains lightweight and efficient, while delivering superior accuracy. These results demonstrate the practicality of our method for real-time depth estimation using focal stacks.
Comparison of model size and runtime on the DDFF 12-Scene dataset. All measurements were conducted on a single NVIDIA RTX A6000 GPU.
Table 11:
Method
Type
Focal Stack Size N
#Params (M)
Runtime (ms)
3
SIDE
335
189
Depth Anything (ViT-L)
30
SIDE
1
336
113
Depth Anything (ViT-S)
30
SIDE
1
25
37
DFV 20
DFF
5
20
16
HybridDepth
6
DFF
5
66
41
Ours
DFF
5
27
28
24

---

D Discussion on Asymmetric Defocus Blur
In practical imaging systems, defocus blur can exhibit mild asymmetry due to lens imperfections and sensor characteristics. The proposed Sopth-from-foeue framework is designed to remain robust under such conditions. The spatial variational constraint captures gradient variations induced by focus changes without assuming any parametric blur model, allowing the network to extract depth- relevant structural cues directly from the observed data. Complementarily, the focal variational constraint encourages smooth unimodal distribution of predicted focus probabilities across the focal stack, reflecting the physical principle that sharpness peaks the in-focus plane and decreases for neighboring planes. This behavior is preserved even when blur is moderately asymmetric, ensuring that the model can reliably identify in-focus regions.
Empirical evaluation across datasets with varying blur characteristics supports this robustness. Syn- thetic datasets such as NYU Depth v2 (25] and ARKitScenes 2 employ idealized symmetric defocus kernels, while FOD500 [S] also assumes circular blur. In contrast, DDFF 12-Scene [9] provides real-world focal stacks captured with a light-field camera, naturally introducing mild asymmetry. The framework maintains strong performance across all these datasets, including DDFF 12-Scene, demonstrating effective handling of moderate asymmetry.
However, extreme asymmetry or highly non-Gaussian point spread functions, which are uncommon in the considered datasets, may challenge generalization. Future work could explore explicit modeling of such scenarios to further enhance robustness.
E More Qualitative Comparisons
To further demonstrate the effectiveness of our method, we present additional qualitative comparisons on three publicly available datasets: NYU Depth v2 [25]. DDFF 12-Scene (9. and FoD500 [15]. As illustrated in Figures 10 our approach consistently produces more coherent depth maps compared
to previous state-of-the-ar methods, HybridDepth (6) and DFV (29)
All visualizations include data from publicly available datasets. Human subjects, where visible, rendered in -on-ieentee form with facial features removed.
25

---

Focal Plane
Focal Plane
Focal
Focal Plane
Focal Plane
Depth
Imuges Ground Truth
DualFocus
w/ Ltv
DualFocus
Lev
Focal Plane
Plane
Focal
Focal Plane
Focal
Depth
RGB Images & Ground Truth
DualFocus w/o Liv
DualFocus w/ Lev
Focal
Focal
Depth
RGB Images & Ground Truth
DualFocus w/o Lgy
DualFocus w/ Lyv
Focal
Depth
RGB Images & Ground Truth
DualFocus w/b I.j,
DuslFocus w/ 1i-
Figure Effect of the focal variational constraints on the NYU Depth v2 dataset.
26

---

RGB Image
HephidDepth
DFV
DualFocus
Ground Truth
Figure Qualitative comparison on the NYU Depth dataset.
RGB Image
HybridDepth
DFV
DualFocus
Ground Truth
Figure 9: Qualitative comparison on the DDFF 12-Scene dataset.
27

---

RGB Image
DFV
DualFocus
Ground Truth
Figure 10: Qualitative comparison on the FoD500 dataset.
28


Reviews:
[{"review_id": "xY0tPycoiY", "reviewer": "Reviewer_Gh3D", "summary": "The paper proposes a method to solve the depth from focus problem. The method consists of two main components: handling spatial variational constraints and focal variational constraints, respectively. Spatial variational constraints aim to match the first-order spatial gradient of depth maps to the estimated multi-channel latent representation's spatial gradient. Focal variational constraints try to estimate focus distance to follow the real-world physical behavior---the blur amount monotonically increasing and decreasing from the in-focus depth. Based on two constraints, the depth is estimated by training the focus volume encoder and depth decoder, using the multi-channel latent representation.", "questions": "For the rebuttal, I expect the following questions to be answered.\n\n- Are there any other apparent differences between VA-DepthNet, except for the extension for the multi-channel behavior? It seems the confidence weights are removed.\n- In comparison against monocular depth estimation methods (categorized as SIDE in the paper), how is the input image selected? It seems the blurred images are used. If so, which focus depth is selected from the focal stack? Are there any criteria for such selection?\n- The method seems to work on ideal blur kernel cases. Can this method handle asymmetric blur kernels? Vignetting or optical aberrations cause that asymmetry, which is not rare.\n\nIn addition, please indicate the in-focus object in the Figures, if possible, for clarity. (Figure 2, and Figures in the supplementary material)", "limitations": "Some potential limitations are raised in questions.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The proposed two constraints seem technically sound and valuable. In terms of the spatial variational constraints, although the key idea is from the VA-DepthNet, its extension to multi-channel latent can be shown as the paper's contribution. The focal variational constraints seem to work, based on ablation studies and provided results in the supplementary material. The paper well describes those contributions and also provides sufficient details for experiments. The evaluation result shows state-of-the-art performance. As a result, there is no reason to reject the paper, but with minor considerations that may be answered in the rebuttal.", "was_revised": true, "final_justification": "The authors properly handled my questions and clarified my concerns in rebuttal. Since the authors resolved my concerns, I will keep my ratings.", "rebuttal": "We thank the reviewer for the valuable comments and positive reception of our work. We address your questions as follows:\n- - -\n## 1. Comparison with VA-DepthNet\nWhile our method shares a core similarity with VA-DepthNet in leveraging gradient-to-surface reconstruction, it differs in several important aspects both in terms of motivation and implementation.\n\n**- Use of Confidence Weight:**\n\nOur final model also incorporates a confidence weighting mechanism functionally similar to VA-DepthNet. However, we found its effect to be minimal in our setting and therefore chose not to highlight it in the main paper. The table below summarizes the effect of confidence weighting on both models, evaluated on the NYU Depth v2 dataset:\n\n| Method | Setting | RMSE&#8595; | AbsRel&#8595; |\n|-------------|:-----------:|:----:|:------:|\n| VA-DepthNet | w/o confidence weights | 0.316 | 0.089 |\n| VA-DepthNet | w/ confidence weights | 0.304 | 0.086 | \n| Ours | w/o confidence weights | 0.076 | 0.013 |\n| Ours | w/ confidence weights | 0.075 | 0.013 | \n\nIn VA-DepthNet, confidence weighting is crucial because the reconstructed surface is used directly as an intermediate depth representation for final refinement. Therefore, suppressing noisy gradients is essential for robust performance.\n\nIn contrast, our method is designed to operate on multi-focus image stacks with the primary goal of identifying in-focus regions by analyzing the structural consistency of surfaces across different focal planes. Unlike VA-DepthNet, which relies on accurate surface reconstruction from a single image, our method leverages geometric consistency across focus planes, reducing dependency on precise per-image reconstruction. Additionally, we already suppress unreliable gradients from defocus-blurred regions via a sharpness-based weighting factor $q$, reducing the marginal utility of additional confidence weighting. While confidence weights still offer some benefit in filtering out invalid gradients within sharp regions (e.g., around edges or textured noise), their overall contribution is marginal in our framework, as evidenced by the small quantitative difference.\n\nTo prevent potential misunderstanding, we will clarify in the revised version that our model does incorporate confidence weighting, though it plays a limited role in our framework.\n\n**- Lightweight Design for Real-Time Inference:**\n\nWhile VA-DepthNet aims for high-precision depth estimation from a single image, our model is designed to balance accuracy with real-time inference efficiency for multi-focus depth estimation. This design goal leads to the following key architectural differences:\n\n* We perform surface reconstruction at 1/6 the spatial resolution used in VA-DepthNet. This is feasible in our case because we do not rely on high-fidelity surface recovery from a single image. Instead, we leverage geometric variations across focus planes, which are enriched by the complementary information inherent in the focal stack.\n\n* We use a ResNet-18 encoder and a lightweight decoder designed to fuse information from multiple focus planes. In contrast, VA-DepthNet employs a heavier Swin-Large Transformer and focuses on refining depth from a single image.\n\nAs a result, even though our model processes five input images, it remains significantly more efficient in terms of both parameters and runtime:\n| Method | Input Type | #Params (M) | Runtime (s) |\n|-------------------|-----------|:----:|:------:|\n| VA-DepthNet | Single Image | 249 | 0.091 |\n| Ours | 5 Multi-Focus Images | 27 | 0.026 | \n\n- - -\n## 2. Input Image Selection for SIDE Baselines\nWe appreciate this clarification request. For single-image depth estimation (SIDE) baselines, we use the all-in-focus image as input rather than any blurred image from the focal stack. This is because SIDE models are generally trained on all-in-focus data and are not designed to handle defocused inputs. Using blurred images would therefore lead to unfair underperformance. In our evaluation, the NYU Depth v2 and ARKitScenes datasets natively provide all-in-focus RGB images, which were used for SIDE comparisons. We will add this clarification to Section 4.1 to avoid confusion in the final version.\n\n\n- - -\n## 3. Handling Asymmetric Blur Kernels\nThank you for bringing up a key aspect that must be addressed for real-world applicability. Our method does not explicitly assume an ideal symmetric blur kernel in its formulation. The spatial variational constraint is designed to capture spatial gradient variations that are consistent with focus-induced sharpness changes, without requiring any parametric blur model. This allows the network to learn focus-dependent structural cues directly from data, regardless of whether the blur is perfectly symmetric or mildly asymmetric.\n\nSimilarly, the focal variational constraint encourages the predicted focus probabilities to follow a smooth unimodal pattern across focal planes, which aligns with the physical intuition that sharpness typically peaks at the in-focus depth and decreases away from it. This principle still holds even under mild asymmetric blur common in real-world camera systems.\n\nOur method has been evaluated on a diverse datasets exhibiting different blur characteristics. NYU Depth v2 and ARKitScenes use synthetically generated focal stacks with circular (symmetric) defocus kernels. FOD500 is also synthetic  and assumes idealized blur. In contrast, DDFF12 is a real-world dataset captured with a light-field camera, which naturally introduces moderate blur asymmetry due to lens and sensor characteristics. Our method achieves strong performance across all these datasets, including the real-world DDFF12, indicating its robustness to moderate asymmetry.\n\nNonetheless, we acknowledge that while our method is generally applicable to scenes with asymmetric blur patterns, its generalization capability may still be limited when trained primarily on symmetric blur and applied to scenes exhibiting severe asymmetry or highly non-Gaussian point spread functions. Performance degradation may occur in such cases due to distributional shifts. This potential limitation is briefly noted in the manuscript (Line 326), and explicit modeling of such effects can be an interesting direction for future work.\n\n- - -\n## 4. Clarification of In-Focus Regions in Figures\nThank you for the helpful suggestion. To improve clarity, we will revise Figure 2 and the figures in the supplementary material to indicate the in-focus areas wherever possible in the final version. \n\nSpecifically, we plan to use a sharpness weight map $𝑞$ to modulate the RGB input image such that in-focus regions retain their original colors while out-of-focus areas appear visually faded. This will help readers better interpret the focal properties of each input frame and understand how our model responds to focus variation. We will also clarify in the captions that this visualization is solely for interpretability and does not alter the original RGB content."}, {"review_id": "GvIbY7RELX", "reviewer": "Reviewer_WY53", "summary": "This paper presents a novel framework for Depth from Focus (DFF) that incorporates two physics-informed variational constraints previously overlooked by existing deep learning-based methods: (1) a spatial variational constraint, and (2) a focal variational constraint.\n\nThe spatial variational constraint leverages the physical observation that in-focus regions tend to exhibit coherent and strong gradients, while out-of-focus regions often display blurred or noisy patterns. This constraint is integrated into the network through:\n\n(i) depth gradient supervision with region-dependent weighting (emphasizing in-focus areas), and\n\n(ii) a least-squares operation that implicitly separates in-focus from out-of-focus regions.\n\nThe focal variational constraint enforces that the predicted focus probability distribution be smooth and unimodal along the focal sweep. This is implemented via a bidirectional soft monotonicity loss.\n\nExperimental results demonstrate that the proposed framework consistently outperforms state-of-the-art (SOTA) DFF and single-image depth estimation methods. Ablation studies further validate that each proposed component contributes to the overall performance gain.", "questions": "1. Can you provide further results to show that the proposed method is better than previous methods when texture-induced artifacts/noises exist?\n2. Can you provide further results to show that $\\\\Gamma$ serves as gradient feature, and $z$ serves as surface feature?\n3. Please unify the font style of $z$ in Equation (3) and Figure 1.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "$\\\\textbf{Strengths}$\n+ The integration of the spatial variational constraint and the focal variational constraint into deep learning-based DFF is novel and well-motivated from a physical perspective.\n\n+ The mechanisms proposed for embedding these constraints into the network are well-designed, and the authors have explored a reasonably sized design space to validate their choices.\n\n+ The method achieves strong performance gains over existing approaches, and the ablation studies are comprehensive, covering all proposed components.\n\n$\\\\textbf{Major Weaknesses}$\n\n+ Throughout the abstract, introduction, methodology, and conclusion, texture-induced artifacts/noises are emphasized as a major cause of degraded performance in prior DFF methods and serve as a key motivation for the proposed framework. However, no targeted experimental results are provided to demonstrate that the proposed method explicitly improves performance in such cases.\n\n+ In line 143, $\\\\Gamma$ is described as a multi-channel gradient feature, but no visualizations or qualitative results are shown to support this interpretation. Similarly, $z$ is referred to as a latent surface field, yet no qualitative evidence or intuitive explanation is provided to substantiate how this representation reflects surface geometry.\n\n$\\\\textbf{Minor Weakness}$\n\nThe font style of $z$ in Equation (3) and Figure 1 is inconsistent and should be unified for clarity and presentation quality.", "was_revised": true, "final_justification": "The authors provided more detailed explanation on $\\Gamma$ and $z$, and also qualitative results to support the claim that the proposed method improves robustness in texture-sensitive regions. And the authors has planned to include these in the final version, making the presentation this paper clearer and easier to follow. Together with the good performance, motivation and novelty of the proposed method, I will suggest a rating of accept.", "rebuttal": "We thank the reviewer for the valuable feedback and recommendations for improving the manuscript. We list our response to your questions below.\n- - -\n## 1. Demonstration of Robustness in Texture-Sensitive Regions\nThank you for highlighting this important point. As correctly noted, texture-induced artifacts and noise often lead to erroneous depth estimates in prior DFF methods, particularly in texture-rich but geometrically flat regions. Our framework addresses this challenge by modeling focus-induced gradient variations, which inherently suppress misleading high-frequency textures and promote structurally consistent depth estimation.\n\nDue to the NeurIPS rebuttal policy this year, we are unfortunately unable to provide additional figures or updated qualitative results beyond the original submission. However, we would like to highlight that several examples in the main paper and supplementary material already help illustrate the robustness of our method to texture-induced artifacts:\n\n- In Figure 4 of the main paper, the first row shows a carpet with repetitive circular patterns on the floor. These texture patterns confuse existing DFF methods, leading to erroneous depth in flat geometry regions. In contrast, our method appears to produce smoother and more consistent depth in this scene.\n\n- The third row in the same figure shows vertical bright dot textures on the inner wall of a brown cabinet. Competing methods often misinterpret these patterns as depth edges, whereas our method yields comparatively more reliable estimates.\n\n- In the fourth row of Figure 4 in the supplementary material, the window composed of small square glass panes contains complex reflections and visual clutter. Prior methods produce inconsistent errors in these regions, but our method regularizes such noise more effectively through the proposed focus-aware gradient-based constraints.\n\n- In the second row of Figure 6 in the supplementary material,  a vertically textured wooden cylinder with strong lighting variations is present. The previous work DFV predicts noisy depth in this area, while ours yields more consistent and clean predictions.\n\nWhile these examples were included in the original submission, we acknowledge that the associated captions and discussion did not sufficiently emphasize this aspect, and we sincerely thank the reviewer for drawing attention to it.\n\nIn the final version, we plan to (i) include more explicit commentary in the captions to draw attention to texture-sensitive regions, and (ii) add a dedicated figure showcasing representative cases of texture-induced errors. We hope these additions will better communicate the practical value of our approach in texture-challenging scenarios.\n\n- - -\n## 2. On the Geometric Interpretation of $\\Gamma$ and $z$\nWe appreciate the reviewer’s insightful question regarding the interpretation of $\\Gamma$ as a gradient-like representation and $z$ as a surface proxy. While we are limited in providing additional visualizations due to the current rebuttal policy, we would like to clarify that their roles are both mathematically grounded and qualitatively supported by the visualizations already included in the original submission.\n\n**- Theoretical Rationale for Interpreting $\\Gamma$ and $z$:**  \n\nIn our method, each focal stack image is first encoded into a multi-channel directional feature $\\Gamma$, which is then used to reconstruct a latent surface field $z$ through a least-squares optimization (Eq. 3 in the main paper). This formulation enforces the constraint that $\\nabla z \\approx \\Gamma$ in the least-squares sense, thereby encouraging $\\Gamma$ to represent a spatial gradient field and $z$ to function as the corresponding surface.\n\nTo reinforce this structural coupling during training, we supervise the local gradient of $z$ to match the ground-truth depth gradient, specifically in in-focus regions (Eq. 5 in the main paper). The gradient of $z$ here refers to the estimated 2D spatial gradient of the reconstructed surface field, extracted via local convolutional filtering (e.g., a 3×3 kernel), not the learned feature $\\Gamma$ itself. As the surface reconstruction from gradients via the least-squares system is differentiable, this supervision ensures that $z$ encodes coherent surface geometry where reliable visual cues exist, while guiding $\\Gamma$ to represent localized surface variations consistent with the underlying depth.\n\n**- Focus-Aware Inductive Bias within $\\Gamma$ and $z$:**\n\nBefore analyzing the visual evidence provided in the paper, we introduce the focus-aware inductive bias embedded in our architecture, which shapes how geometric patterns selectively emerge in $\\Gamma$ and $z$.\n\nOur model is designed to encourage the representation to respond selectively to in-focus regions, where reliable visual cues for depth exist. This is achieved through two key mechanisms: (i) focus-dependent integrability, which allows the surface reconstruction to be guided by directional consistency only where valid gradients are present (please refer to lines 162-170 in the main paper), and (ii) focus-weighted supervision, which amplifies training signals in focused areas (Eq. 5 in the main paper). As a result, the network is encouraged to learn gradient-related features that are concentrated and coherent in in-focus regions, while avoiding spurious signals elsewhere. This inductive structure shapes the formation of both the learned gradient field $\\Gamma$ and the reconstructed surface $z$ by emphasizing reliable geometric cues in focus regions.\n\n**- Visual Appearance of $\\Gamma$:**\n\nWe would like to clarify that Figure 2 in the main paper does visualize $\\Gamma$, but in a form that aggregates its multi-channel and directional nature into a single composite image. The visualization is constructed by normalizing and combining feature magnitudes to emphasize regions where strong directional responses emerge, thereby illustrating how the network focuses on in-focus areas with reliable gradient cues. While the integrated energy resembles a soft attention map, each directional channel of $\\Gamma$ captures localized gradients that collectively contributes to surface reconstruction, guided by the integrability constraint and gradient supervision.\n\n**- Support from Visualized Gradients of $z$:**\n\nIn Figure 1 of the supplementary material, we visualize the 2D gradients of the reconstructed surface $z_n$ for each focal plane $n$. These images provide intuitive support for our interpretation of $z$ as a surface proxy: in in-focus regions, the gradient maps tend to exhibit smooth and coherent structures that align with true surface geometry. Conversely, out-of-focus regions produce irregular and noisy gradients, reflecting the model's uncertainty due to unreliable depth cues. This contrast across focus levels serves as an implicit cue that allows the model to infer which regions are in focus and embed this information into its representations, consistent with the intended inductive bias.\n\nWe hope these clarifications address the reviewer’s concerns and provide a clearer understanding of our design rationale and its empirical implications. We are also preparing targeted per-channel visualizations and quantitative analyses in the supplementary material, specifically designed to directly reveal how $\\Gamma$ captures gradient-like transitions and how $z$ encodes surface-level geometry. These visualizations will serve as concrete evidence supporting our claims and enhance interpretability of the learned representations.\n\n\n- - -\n## 3. Font Style Consistency\nThank you for pointing this out. We will ensure the font style of $z$ in Figure 1 is consistent with that in Equation (3) in the final version."}, {"review_id": "iy75Hc3fdn", "reviewer": "Reviewer_hJeu", "summary": "This paper introduces DualFocus to estimate scene depth from focal stacks. Different from existing methods, DualFocus  leverages the unique gradient patterns across a focal stack by modeling both spatial and focal variations through dual variational constraints. The spatial constraint captures consistent gradient changes across focus levels to identify true depth edges, while the focal constraint ensures the predicted focus probabilities follow a physically plausible unimodal and monotonic distribution. Experiments show that the proposed constraints can help separate real geometry from texture artifacts, improving depth accuracy and robustness.", "questions": "- Can the authors demonstrate the model’s generalization ability by training on one dataset and testing on another? In the comparison, please add the results of single-image-based depth estimation methods like DepthAnything.\n- The improvement over prior DFF methods (e.g., DFV on DDFF) is minimal. Can the authors provide ablation studies or visualizations that clarify where and why the gains occur?\n- How does the method behave when using fewer or more than 5 focal images? Is the model robust to missing frames or variable stack sizes?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength\n- The DualFocus framework uses a physics-aware modeling approach that analyzes gradient structures and probability distributions across spatial and focal dimensions, and leverages the complementary strengths of spatial and focal constraints. The framework produces robust and accurate depth predictions, particularly in scenes with fine textures or sharp depth transitions. The paper is technically sound.\n- The paper is easy to follow and well organized.\n\n\nWeakness\n- Each dataset is trained and tested independently, and there is no evidence of cross-dataset generalization. This raises concerns about the scalability of the method to real-world settings, especially compared to zero-shot monocular models, such as DepthAnything, VGGT.\n- Unlike single-image methods such as DepthAnything, the proposed approach requires multiple focal stack images. Yet, its depth predictions—particularly in textured regions (see Fig. 6 in supp.)—remain suboptimal. Furthermore, it lacks qualitative comparisons with single-image baselines, and only shows marginal gains over DFV on the real-world DDFF dataset (Fig. 5 in supp.).\n- The method operates with a fixed input of 5 focal images, but it remains unclear how robust or flexible it is when using more or fewer images. A study on scalability across focal stack sizes would strengthen the evaluation.", "was_revised": true, "final_justification": "This paper introduce DualFocus, which leverages the unique gradient patterns across a focal stack by modeling both spatial and focal variations. My original rating is negative considering the tradeoff between depth estimation accuracy and the number of focal stacks required, and the marginal gain over existing methods on DDFF-12 dataset. The authors provide a comprehensive rebuttal and address my concerns. I suggest the authors put the corresponding discussion in the revised version.", "rebuttal": "We thank the reviewer for the detailed comments and valuable feedback. We list our response to your questions below.\n- - -\n## 1. Cross-Dataset Generalization and Comparison with Zero-Shot SIDE Models\nWe would like to clarify that the generalization ability of our model has already been evaluated in Section 4.3 of the main paper through zero-shot transfer experiments (please see Table 4). In this evaluation, all Depth-from-Focus (DFF) models, including ours, are trained exclusively on the NYU Depth v2 dataset and tested zero-shot on the ARKitScenes dataset without any additional training.\n\nThis experiment also includes comparisons with four different zero-shot single-image depth estimation (SIDE) models—Depth Anything, ZoeDepth, DistDepth, and ZeroDepth. These models have been trained on large-scale and diverse datasets. Depth Anything is trained on 63.5 million images from 14 datasets, and ZoeDepth-M12-N is pretrained on 12 datasets and finetuned on NYU Depth v2. DistDepth is trained on 500K images collected from approximately 1,000 diverse environments, while ZeroDepth is trained on 30.1 million images from 5 datasets.\n\nDespite their extensive training, our method demonstrates competitive zero-shot performance on ARKitScenes. Notably, it outperforms all other DFF baselines—including HybridDepth, which leverages Depth Anything predictions as input—and achieves better RMSE than all SIDE baselines.\n\nThese results underscore the robust generalization capability of our approach without relying on large-scale or diverse training data. Although we cannot provide qualitative comparisons here due to this year’s NeurIPS rebuttal policy, internal observations indicate our model tends to produce sharper object boundaries and more consistent depth across focus transitions on both NYU Depth v2 and ARKitScenes samples. While large-scale pretrained SIDE models such as Depth Anything may yield smoother and more visually coherent depth maps due to massive and diverse datasets, our method instead preserves physically grounded focus cues and local depth variations, which often lead to more faithful representations of true scene geometry, particularly near defocus transitions and fine structures.\n\nIn response to the reviewer’s feedback, we will include qualitative comparisons with Depth Anything  model in the final version. We hope this sufficiently addresses the reviewer’s concern and demonstrates our model’s scalability and applicability to unseen domains.\n- - -\n## 2. Marginal Gains over Prior DFF Methods\nWe appreciate the reviewer’s concern and would like to offer additional context to better clarify the perceived gains over prior DFF methods, particularly on the DDFF-12 dataset (Fig. 5 in the supplementary material).\n\nWhile the overall visual quality on the DDFF-12 dataset may not appear strong, we believe this is largely attributable to the limited size and diversity of the dataset—only 600 focal stacks are available for training. This scarcity of data makes it challenging for all DFF models, including ours, to produce consistently detailed or visually compelling depth maps. In contrast, our primary benchmark, NYU Depth v2, contains 24,000 focal stacks, allowing for more stable training, better generalization, and thus clearer distinctions in model performance.\n\nDespite this limitation, our method still demonstrates consistent and clear quantitative improvements, as reported in Table 3 of the main paper. Compared to DFV, our approach yields an approximately 17.5% reduction in MSE (from $5.7 \\times 10^{-4}$ to $4.7 \\times 10^{-4}$), indicating a notable performance gain. Furthermore, we would like to draw attention to specific regions where our method exhibits meaningful qualitative gains that may not be immediately apparent without closer inspection.\n\nIn Figure 5 of the supplementary material, we highlight several such cases:\n\n- **1st row:** The small loop on top of the bag has a hollow center. Our method captures its edge more accurately than prior methods.\n\n- **2nd row:** The textureless white wall on the right often leads to noise in prior methods, whereas our model produces a clean and stable depth map.\n\n- **3rd row:** The chair in the background exhibits a sharper depth edge in our result.\n\n- **4th row:** The transparent liquid container in the bottom left is clearly outlined by our method, while largely omitted by others.\n\nTo better showcase these qualitative gains, we will revise the figure in the final version by adding zoomed-in views and annotations that guide attention to regions where differences are most evident. Additionally, we plan to include new examples where the improvements are more apparent, particularly in texture-sensitive areas and around depth discontinuities, thereby enhancing visual support for the reported gains.\n\nWhile the DDFF-12 dataset provides a challenging testbed due to limited diversity, we wish to emphasize that the broader evaluation results provide further evidence of our model's effectiveness. As shown in Tables 1–4 of the main paper, our method achieves consistent and noticeable improvements across all four datasets, surpassing state-of-the-art DFF approaches in diverse scenarios.\n\nQualitative results further support these findings. For instance, Figure 4 of the main paper and Figures 4 and 6 of the supplementary materials demonstrate clear visual advantages on NYU Depth v2 and FoD500. Specifically:\n\n- In Figure 4 of the main paper, the first two rows show a carpet and a brown cabinet with repetitive circular patterns. These texture patterns confuse existing DFF methods, leading to erroneous depth in flat geometry regions. In contrast, our method produces consistent and smooth depth in this scene.\n\n- The last two rows show enhanced edge preservation in our results, highlighted by blue dotted boxes, which prior methods fail to retain.\n\n- In the fourth row of Figure 4 in the supplementary material, the window composed of small square glass panes contains complex reflections and visual clutter. Prior methods produce inconsistent errors in these regions, but our method regularizes such noise more effectively through the proposed focus-aware gradient-based constraints.\n\n- In the second row of Figure 6 in the supplementary material,  a vertically textured wooden cylinder with strong lighting variations is present. The previous work DFV predicts noisy depth in this area, while ours yields more consistent and clean predictions.\n\nTo clarify where and why the gains occur, we kindly refer the reviewer to the comprehensive ablation studies in Table 5 of the main paper, which demonstrate the effectiveness of each proposed component. For instance, removing the spatial variational constraint alone results in a clear degradation in all metrics, underscoring its role in capturing focus-dependent spatial gradients and their variation along focal planes. Similarly, we analyze the effect of the focal variational constraint in Section D.4 of the supplementary material (Table 4 and Figures 2–3), showing that its absence leads to inconsistent depth transitions across focus levels and increased noise in textureless regions. These results confirm that enforcing a physically plausible focus distribution leads to sharply localized depth discontinuities and the suppression of artifacts in defocus-dominated areas. Together, these studies demonstrate that our model’s architectural inductive biases promote localized depth transitions and effectively mitigate texture-induced noise, leading to the consistent gains observed across benchmarks.\n\nWe hope that the provided analysis, along with the planned clarifications in the final version, sufficiently address the reviewer’s inquiry regarding the gains.\n- - -\n## 3. Scalability to Focal Stack Size\nWe thank the reviewer for this insightful question regarding the scalability of our model with respect to the number of focal stack images. To address this, we conducted additional experiments on the NYU Depth v2 dataset using 3 and 10 focal planes. The table below summarizes the performance of our method compared to prior DFF approaches, including DFV and HybridDepth, as well as a recent single-image depth estimation method, Depth Anything. HybridDepth results under the 3-stack setting are not reported due to reproducibility issues in the official training code.\n\n| Method | Focal Stack Size | RMSE&#8595; | AbsRel&#8595; |\n|-|:-:|:-:|:-:|\n| Depth Anything | 1 | 0.206 | 0.056 | \n| HybridDepth | 3 | - | - | \n| HybridDepth | 5 | 0.128 | 0.026 | \n| HybridDepth | 10 | 0.083 | 0.015 |\n| DFV | 3 | 0.134 | 0.030 | \n| DFV | 5 | 0.094 | 0.020 | \n| DFV | 10 | 0.069 | 0.011 |\n| Ours | 3 | 0.119 | 0.023 | \n| Ours | 5 | 0.075 | 0.013 | \n| Ours | 10 | 0.052 | 0.008 |\n\nIn DFF tasks, the number of focal planes directly affects the richness of depth cues available for learning. As expected, performance increases with more inputs and degrades with fewer, a common trend that is clearly reflected in both DFV and our method.\n\nNonetheless, our method demonstrates strong scalability. Even with only 3 focal images, it outperforms DFV and surpasses HybridDepth trained with 5 images, as well as the single-image method Depth Anything trained on a large corpus of general-purpose depth data. This highlights the effectiveness of our design in extracting meaningful depth cues even under sparse focus supervision.\n\nNotably, when increasing the number of focal planes to 10, our method achieves a substantial performance gain, establishing a new state of the art across all metrics. This demonstrates not only that our model can effectively utilize sparse focal stacks, but also that it fully capitalizes on rich focus information when available, leading to high-quality and geometrically consistent depth predictions.\n\nWe appreciate the reviewer’s suggestion once again and are happy to include these additional experiments and clarifications in the final version of the paper."}, {"review_id": "HaeM19WzpS", "reviewer": "Reviewer_aqEp", "summary": "This paper presents DualFocus, a novel Depth-from-Focus (DFF) framework that addresses the limitations of prior methods in complex scenes with fine textures or abrupt depth changes. The approach introduces spatial and focal variational constraints to model gradient patterns across focal stacks, leveraging physical focus behavior to distinguish true depth edges from texture artifacts. Experiments on four datasets (NYU Depth v2, FoD500, DDFF 12-Scene, ARKitScenes) demonstrate state-of-the-art depth accuracy and generalization, outperforming prior DFF methods. The key contributions include the dual-constraint formulation, a unified differentiable architecture, and robust performance across diverse scenarios.", "questions": "1.\tHave you tested the impact of the number of images in the focal stack on the performance?", "limitations": "Same with weakness", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths*\n1.Innovative Dual Constraints: The spatial and focal variational constraints effectively integrate optical principles, improving robustness in challenging regions by enforcing gradient integrability and unimodal focus distributions .\n2. Intuitive Motivation：Using the gap between focal distance and true depth to model clarity, employing this as a weight to supervise gradients, and generating probabilities based on clarity to weight and compute the final depth aligns with human intuition.\n3. Comprehensive Experimental Validation: Thorough evaluations on synthetic and real-world datasets, including zero-shot transfer to ARKitScenes, validate generalizability.\n\nWeaknesses*\n1. The accuracy is highly dependent on the number of focal stacks. If the number of focal stacks is small, it may lead to inaccurate depth estimation.\n2. The estimation of probability heavily relies on gradient-related surface features, which may limit its effectiveness in low-texture regions.\n3. It seems that experiments were only conducted in indoor scenes without considering outdoor scenarios. And the proposed method in the paper has been fully validated through experiments.", "was_revised": true, "final_justification": "I have upgraded the rating.", "rebuttal": "We thank the reviewer for the constructive suggestions and positive reception of our work. We list our response to your questions below.\n- - -\n## 1. Scalability to Focal Stack Size\nWe thank the reviewer for this insightful question regarding the scalability of our model with respect to the number of focal stack images. To address this, we conducted additional experiments on the NYU Depth v2 dataset using 3 and 10 focal planes. The table below summarizes the performance of our method compared to prior DFF approaches, including DFV and HybridDepth, as well as a recent single-image depth estimation method, Depth Anything. HybridDepth results under the 3-stack setting are not reported due to reproducibility issues in the official training code.\n\n| Method | Focal Stack Size | RMSE&#8595; | AbsRel&#8595; |\n|-------------|:----------------:|:----:|:------:|\n| Depth Anything | 1  | 0.206 | 0.056 | \n| HybridDepth | 3 | - | - | \n| HybridDepth | 5 | 0.128 | 0.026 | \n| HybridDepth | 10 | 0.083 | 0.015 |\n| DFV | 3 | 0.134 | 0.030 | \n| DFV | 5 | 0.094 | 0.020 | \n| DFV | 10 | 0.069 | 0.011 |\n| Ours | 3 | 0.119 | 0.023 | \n| Ours | 5 | 0.075 | 0.013 | \n| Ours | 10 | 0.052 | 0.008 |\n\nIn DFF tasks, the number of focal planes directly affects the richness of depth cues available for learning. As expected, performance increases with more inputs and degrades with fewer, a common trend that is clearly reflected in both DFV and our method.\n\nNonetheless, our method demonstrates strong scalability. Even with only 3 focal images, it outperforms DFV and surpasses HybridDepth trained with 5 images, as well as the single-image method Depth Anything trained on a large corpus of general-purpose depth data. This highlights the effectiveness of our design in extracting meaningful depth cues even under sparse focus supervision.\n\nNotably, when increasing the number of focal planes to 10, our method achieves a substantial performance gain, establishing a new state of the art across all metrics. This demonstrates not only that our model can effectively utilize sparse focal stacks, but also that it fully capitalizes on rich focus information when available, leading to high-quality and geometrically consistent depth predictions.\n\nWe appreciate the reviewer’s suggestion once again and are happy to include these additional experiments and clarifications in the final version of the paper.\n- - -\n## 2. Concern regarding Performance in Low-Texture Regions\nWe understand the reviewer’s concern that our method, which estimates focus probability based on gradient-related surface features, may be vulnerable in low-texture regions. As also described in Line 324 of the main paper, performance can indeed degrade when large spatial areas contain little to no texture, resulting in weak or absent focus-induced gradient variations. Importantly, such regions are known to be inherently challenging for depth estimation in general, including both single-image and prior DFF-based methods, due to the scarcity of discriminative visual cues.\n\nThat said, we observe that our model still performs reasonably well in many low-texture scenarios. We attribute this to the design of our gradient-based constraints, which can respond to subtle focus transitions when present, and to the use of gradient-related features computed at a relatively lower spatial resolution. This lower-resolution representation helps mitigate the reliance on fine-grained local gradients, making the model more robust in textureless regions by allowing it to extract meaningful focus cues from broader structural differences across focal planes.\n\nWhile we are unable to include new figures under the NeurIPS rebuttal policy this year, several examples in our original submission demonstrate promising results in low-texture settings:\n\n- In the fourth row of Figure 4 in the main paper, the white recessed sink has smooth surfaces with very limited texture, particularly around its edge. Competing methods struggle to capture the depth discontinuity at the sink boundary, whereas our model produces a clearer and more structurally accurate prediction.\n\n- In Figure 3 of the supplementary material, two of the four scenes (bottom two rows) exhibit large textureless wall regions. Despite minimal texture, our model successfully identifies the in-focus areas across different focal planes and predicts depth values that closely match the ground truth.\n\n- In the second row of Figure 5 in the supplementary material, the white wall on the right side is textureless. While other prior methods exhibit significant noise in this region, our model remains robust and produces clean, stable depth predictions.\n\nWe hope that these examples and clarifications help demonstrate the reasonable robustness of our method in low-texture regions, and that the overall modeling strategy helps mitigate some of the limitations typically associated with gradient-based approaches.\n\n- - -\n## 3. Indoor-Only Evaluation\n\n\nWe agree that exploring depth-from-focus in outdoor scenarios would be a valuable extension of current research. In practice, however, most existing DFF datasets and studies have been limited to indoor environments. This is largely due to the practical challenges of capturing focal stacks with consistent focus sweeps outdoors—such as variable lighting, motion artifacts, and the difficulty of obtaining accurate ground truth depth in large-scale, unstructured scenes. Consequently, our work, like most prior DFF approaches, focuses on indoor datasets where data collection and evaluation are more tractable.\n\nNonetheless, we appreciate the reviewer’s comment, which highlights an important consideration for real-world applicability. We believe that expanding DFF research to more diverse and unconstrained environments, including outdoor scenes, represents a meaningful future direction."}]

Author rebuttal:

Meta-review: The paper describes a new technique for depth-from-focus using spatial and focal variational constraints. The spatial constraints are incorporated via gradient-based supervision to prioritize in-focus areas, while the focal constraints are based on a physically plausible unimodal and monotonic distribution. These constraints help to distinguish true depth edges from texture artifacts, leading to SOTA results.

Pros: The combination of spatial and focal constraints is novel. Its effectiveness was demonstrated through extensive experimental validation. 

Cons: Validation is based only on indoor scenes, which raises concerns about scalability of method to real-world settings. Multiple focal stack images are required as input as opposed to single-image methods such as DepthAnything. 

The main reasons for acceptance are the consensus on novelty and the SOTA results.

The authors provided effective rebuttals to the reviewers’ concerns, which led to three reviewers upgrading their scores. In particular, the authors responded that ground-truth datasets tend to be of indoor scenes only. They also showed that SOTA single-image depth estimation techniques perform more poorly than theirs.
Decision: Accept (poster)