Learnable Burst-Encodable Time-of-Flight Imaging for High-Fidelity Long-Distance Depth Sensing
Authors: Manchao Bao
Shengjiang Fang
Tao Yue
Xuemei Hu
Venue: NeurIPS 2025 spotlight ()
Keywords: Time-of-Flight; End-to-End Learning
Primary area: deep_learning

Abstract: Long-distance depth imaging holds great promise for applications such as autonomous driving and robotics. Direct time-of-flight (dToF) imaging offers high-precision, long-distance depth sensing, yet demands ultra-short pulse light sources and high-resolution time-to-digital converters. In contrast, indirect time-of-flight (iToF) imaging often suffers from phase wrapping and low signal-to-noise ratio (SNR) as the sensing distance increases. In this paper, we introduce a novel ToF imaging paradigm, termed Burst-Encodable Time-of-Flight (BE-ToF), which facilitates high-fidelity, long-distance depth imaging. Specifically, the BE-ToF system emits light pulses in burst mode and estimates the phase delay of the reflected signal over the entire burst period, thereby effectively avoiding the phase wrapping inherent to conventional iToF systems. Moreover, to address the low SNR caused by light attenuation over increasing distances, we propose an end-to-end learnable framework that jointly optimizes the coding functions and the depth reconstruction network. A specialized double well function and first-order difference term are incorporated into the framework to ensure the hardware implementability of the coding functions. The proposed approach is rigorously validated through comprehensive simulations and real-world prototype experiments, demonstrating its effectiveness and practical applicability. The code is available at: https://github.com/ComputationalPerceptionLab/BE-ToF.

Learnable Burst-Encodable Time-of-Flight Imaging for High-Fidelity Long-Distance Depth Sensing
Manchao Bao'
Shengjiang Fang shengj langfangdsmalnn nju. edu. cn
manchaobao@smail nju. edu cn
Tao Yue! yuetao@nju. edu. cn
Xuemei Hu! xuemeihuênju, edu, on
Nanjing University
NNajjing Electronic Devices Institute
Abstract
Long-distance depth imaging holds great promise for applications such au- tonomous driving and robotics. Direct time-of-fligtt (dToFF imaging offers high- precision, long-distance depth sensing, yet demands ultra-short pulse light sources and high-resolution time-to-digital converters. In contrast, indirect time-of-flight (iToF) imaging often suffers from phase wrapping and low signal-to-noise ratio (SNR) the sensing distance increases. In this paper, we introduce novel ToF imaging paradigm, termed Encooablec Time-of-Flight (BE-ToF), which cilitates high-fidelity, long-distance depth imaging. Specifically, the BE-ToF sys- tem emits light pulses in burst mode and estimates the phase delay of the reflected signal over the entire burst period, thereby effectively avoiding the phase wrap- ping inherent to conventional iTToF systems. Moreover, to address the low SNR caused by light attenuation over increasing distances, propose end-to-end learnable framework that jointly optimizes the coding functions and the depth reconstruction network. A specialized double well function and first-order dif- ference term are incorporated into the framework to ensure the hardware imple- mentability of the coding functions. The proposed approach is rigorously vali- dated through comprehensive simulations and real-world prototype experiments, demonstrating its effectiveness and practical applicability. The code is available at: https::/// //ggt/gtt...//////// com/ ieapetaaiiiiaeeeppptpiiiiiii
Introduction
Achieving high-precision depth imaging over long distances has remained a fundamental objective in fields such as computer vision, robotics, and autonomous systems. Time-of-fligee (ToF) imag- ing [1, 2, 3], as key approach depth imaging, can be further categorized into direct ToF (dToF) and indirect ToF (iToF) based on differences in working principles. Direct ToF imaging [4] estimates depth by directly measuring the round-trip time of light, enabling high-precision and long- rangerange sensing. Despite its advantages, this approach requires ultra-short pulsed light sources and high- resolution time-to-digital converters (TDCs), imposing stringent hardware demands that increase system complexity and cost, thereby limiting its practicality for widespread deployment. Indirect ToF systems [5, 6, 7, 8, 9], in contrast, emit amplitude-mont moduulldd continuous wave (AMCW) sig- nals and infer depth by analyzing the phase shift between the transmitted and received signals. Due to their relatively lower hardware complexity and cost, iToF systems offer a more practical and hardware-friende solution. Nevertheless, existing iToo technologies face significant challenges in long-range imaging, primarily due to phase wrapping [10] and low signal-to-noiss ratio (SNR) re- sulting from optical attenuation [1]]. To address the phase wrapping, fual-frequency modulation
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

techniques [12, 13] have been proposed, albeit at the cost of increased computational complexity and stricter hardware synchronization requirements. Alternative approaches have sought to mitigate phase wrapping under single- frequeeyy modulation by incorporating scene priors [5, 14], however, these methods do not fundamentally resolve the intrinsic ambiguity introduced by periodic modula- tion.
In this paper, we propose novel ToF imaging paradigm termed Burst-Encodable Time-of-Flight (BE-ToF). Our BE-ToF system operates in low-frequency burst mode for light pulse modulation and demodulation, such that the phase of the reflected signal sweeps the entire range [0, 2x] within a single, long burst period. This facilitates high-fidelityy long-distance depth imaging using only single frequency modulation. Moreover, considering the significant variation in SNRs caused by the light-falloff, we propose an end-to-end learnable framework that jointly optimizes the coding functions and the depth reconstruction network, thereby ensuring high-precision depth estimation. In particular, we incorporate constraints based on double well function and first-order difference to ensure the hardware implementaiility of the learned coding functions. We evaluate our method on a synthetic dataset and compare it with conventional iToF approaches, including single- frequency and multi-frrquency modulation techniques. Finally, we built a prototype system to prove the effective- ness of our method in real-world experiments.
In general, we make the following contributions:
We present a novel Burst-Encodable Time-of-Flight imaging system that enables high- fidelity long-distance depth sensing using only single modulation frequency, thereby fun- damentally mitigating the issue phase wrapping inherent in traditional iToF systems.
We propose an and-to-end learnable framework that jointly optimizes the coding functions and the depth reconstruction network to ensure high- nigh-precision depth estimation across vary- ing distances.
We uniquely incorporate double well function and first-order difference loss function to ensure the hardware implementability of the learned coding functions.
We develop a prototype of our BE-ToF system and demonstrate its superior performance on both synthetic datasets and real-world scenarios.
2 Related Work
ToF imaging. Time-of-fight (ToF) imaging has become widely adopted and effective modal- ity for depth acquisition. Direct ToF (d'oF) enables long-range depth estimation by measuring the round-trip time of short optical pulses [15]. However, attaining high precision with dToF places stringent demands on the illumination and timing hardware, typically requiring nanosecond- or even picosecond-scale pulse widths [4, 16, 17] as well as TDCs with
resolution and low timing jitter [18, 19]. These requirements substantially hinder practical implementation and large-scale deployment. In contrast, indirect ToF (iToF) imaging leverages cost-effective CMOS sensors to achieve high-resolution depth estimation, yet it is inherently susceptible to phase ambi- guity due to phase wrapping in long-range scenarios. common strategy to alleviate this issue is multi-frequency modulation [12, 20, 21], where low modulation frequencies extend the maximum unambiguous range and high frequencies preserve depth precision. For example, Hanto er al. [22] developed a ToF LiDAR range finder based on dual-modulatie frequency switching to extend the imaging range, and Su et al. [23] proposed an end-to-end ToF framework for high-quality depth reconstruction under multi-frequency modulation. Nevertheless, multi-frequency operation often increases hardware complexity and computational cost. Alternatively, single-frequency phase un- wrapping has been explored using amplitude correction [5], surface normal constraints [14], and RGB fusion [24]. but such approaches typically rely heavily on scene priors and may degrade under challenging conditions. In this paper, propose
Time-of-Flighh Imaging (BE- ToF) to fundamentally address phase wrapping in iToF, enabling high-fidelity, long-distance depth estimation.
End-to-end learning. End-to-end learning is a method aimed at jointly optimizing optical sys- tems and reconstruction algorithms. Metzler et al. [25, 26] obtained high dynamic range (HDR) images from a single-shot by jointly optimizing the optical encoder and the electronic decoder. Nie et al. [27] leveraged end-to-end network for hyperspectral reconstruction, enabling simultancous

---

learning of optimized camera spectral response functions and a mapping for spectral reconstruction. For dense 3D localization microscopy, Nehme et al. [28] proposed a deep STORM-based method to achieve end-to-end optimization of point spread function engineering and accurate 3D) localization. To achieve extended depth of field (EDOF), Sitzmann al. [29] proposed jointly optimize the optical system and the reconstruction algorithm's s parameters to achieve achromatic EDOF imaging. Guo et al. [30] put forward an end-to-end framework capable of jointly optimizing the coding func- tions and the exposure time to improve the accuracy of fluorescence lifetime imaging. Moreover, in iToF imaging, Chugunov et al. [31] proposed to jointly learn a microlens amplitude mask and an mncoer-decoder network to reduce flying pixels in depth captures. Li et al. [11] put forward a Fisher-inforee wwt.riiiooomaaogguueeeeee framework for the joint optimization of the coding functions and the reconstruction network. Given the remarkable potential of end-to-end learning in elevating imaging performance, we propose an end-to-end learnable framework that jointly optimizes the coding func- tions and the depth reconstruction network of our BE-ToF, ensuring high-quality depth performance across varying distances.
Learnable Burst-Encodable Time-of-Flight Imaging
(x) IToF
Modziated
Notalmed Sigral
Long Distamee
Signal
High Fidelity
Pluse Shiit 1
Ranger -
Ten
Relaciad
Signel
Rulleded
Sipsa
(b) BE-ToF
Long Distance
Coding
Codine
Tima Dally
High Fiielity
Range:
T+Te
IToF Imaging
(b) BE-ToF Imaging
Figure/ 1: Comparison between iToo and BE-ToF. (a) Principle of iTo imaging, which suffers from a trade-off between sensing distance and precision; (b) Principle of BE-ToF imaging, enabling long- distance and high-fidelity depth sensing through modulation and demodulation in burst mode.
In this section, first introduce the working principle of BE-ToF. As shown in Fig. 1(a), con- ventional iToF is fundamentally constrained by trade-off between maximum unambiguous range and depth precision, governed by the modulation period Tm To handle this, BE-ToF performs short-period light pulse modulationndemmodulttooo in a low-frequency burst mode. As illustrated in Fig. 1(b), within each long burst period Tiuest, single modulated signal is emitted. When the reflected signal returns with a phase shift é, it can be demodulated by coding functions with con- trollable time delay Specifically, the total phase shift can be decomposed into two components: Φ1, which is primarily determined by the controllable time delay and Φ2. which can be recovered using demodulation techniques like 4-step phase shift [5] or deep learning [23, 11]. In summary, the depth d can be defined as Eq. 1
d C (0) + )22 T Tousst
CT
+ D(12),
4T
2
where e is the light speed and ((22) represents the demodulation process of Φ2.
Thus, in BE-ToF system, the maximum unambiguous range dmar is primarily determined by burst period Toursts as defined in Eq.2 2
Thuret
dmar
2 2fburst
Regarding depth precision, since we divide the phase delay é into two components Φ1 and 62, where Φ1 is entirely determined by the time delay т. Consequently, the depth error in our BE-ToF system mainly arises during the demodulation of Φ2. Thus, the depth error of our BE-ToF system can be represented as Eq.
C. Ed2
C. EØg Tm
Ed
(3)
4m fm
4T
3

---

where 6d2 is the phase error due to several factors like photon noise, readout noise, and multi multi-path path in- terference. For fixed phase error, the depth error is chiefly governed by the modulation/demodulatiin period Tm-
Based the above analysis, BE-ToF substantially extends the maximum unambiguous range while maintaining the same depth precision as conventional iToF, thereby enabling long distance and high- fidelity depth imaging. Moreover, the depth sensing range of our BE-ToF spans from to (r+TTm. which can be flexibly adjusted by tuning the time delay т.
3.1 Differential BE-ToF Imaging Model
(a) End-to-EEn BE-ToF Imaging Framework
-bantll Recomstmattoo Networ
Differentalle
Freward
MSE
Mede
Predution
Ground Trush
Deribe Well Bustios
Forword Bookgropppriio
thiference
(b) Differentiable Forward Model
Modulation
Signal
Learralee Binarized Functions
Reflectel Signal
Ginad Truth
Excedalle Caocr
Expoore
0,10
Figure 2: (a) End-to-end BE-ToF imaging framework for jointly optimizing the coding functions and reconstruction network, (b) The differentiable physical model of the BE-ToF system.
Based on the operating principle of BE-ToF, which can be realized with a pulsed laser and an exposure-encodable camera [32], we propose end-to-end imaging framework to ensure high- quality reconstruction across varying distances and SNRs. As shown in Fig. 2(a), the framework comprises two components: differentiable forward model that synthesizes BE-ToF measurements in our simulation pipeline, and a reconstruction network that estimates depth from multiple mea- surements. By jointly optimizing the coding functions and the reconstruction network, the system delivers accurate depth reconstructions under challenging conditions.
In this section, we first establish the differentiable forward model of our BE-ToF for end-to-end optimization, as illustrated in Fig. 2(b). Assuming ML) the modulated signal emitted by pulse laser, the reflected signal of scene point R3 can be defined as Eq. 4
R(s, t) = P. d(s))
+
Tamb
where Ps is the inherent reflectance of the scene point Lamb is the ambient light, d(s) denotes the depth value point s. Furthermore, considering the attenuation light intensity with distance during propagation, we incorporate the attenuation function into our model as Eq. 5
d(s)
R(,, 1) Fale)
Tamb:
where Fa(s) is the attenuation coefficient of the emitted light Mt) depth d(s). which is typically inversely proportional to the square of the distance [33]. Finally, the whole BE-ToF imaging process can be formulated as Eq. 6
T..... R(s, (t) ie L,,,,,,
L(s)
(6)
where (s)s) is the measurement value of the camera, Dt(t) denotes the coding functions and K denotes the number of measurements. Taking into account the inherent noise of the sensor, the final measurement can be expressed as Eq.
X((s) ((s) + nd nr, ns ((B((n)), n, ~0,0

---

where nd is the dark noise following the Poisson distribution with expectation E(na) and n, is the readout noise following Gaussian distribution with standard deviation o,-
Considering that (s) contains three unknowns: Ps. Jamb, d(s). Therefore, at least 2 3 mea- surements required to solve for the depth d((s).
3.2 Reconstruction Network
CFEB
CFER
CFEB
CFEB
CERR
Hx4x4 Messurements
HoWW33
Restormer
Restomeer
Mrr
Block
Block
Depth Map
Restormer
Restormer Block
MPPH
Add
Block
Concate
DawnSample
(a) Channel Feature Extraction Block
UpSample
Restormer
Restormer Block
MPFF
Efficiens
Block
ECA
Channel
Attention
Convolution
Restomeer
3x3 Convolution
(b) Multi-scale Feature Fusion Block
Block
MFFB
Figure 3: Architecture of the Restormer-basee Spatial-Channe Fusion Network (a)
Channel Feature Extraction Block(CFEB), (b) Multi-scale Feature Fusion Block(MFFB).
With the proposed differentiable forward model, we can simulate the K measurements the BE- ToF imaging process. To recover high-fidelity depth map from this set of measurements, propose a Restormer-based
Fusion Network adopts Restormer [34] as the backbone, featuring four-level encoder-decoder structure in which each level comprises multiple Restormer blocks. In contrast to the conventional skip connections integrate Efficient Channel Attention (ECA) module [35] to
As shown in Fig. 3, our network
used in the original Restormer, enhance the fusion of features between encoder and decoder branches. Furthermore, recognizing the inherent differences between depth reconstruction and the image restoration tasks for which Restormer was originally designed, we augment our network with two additional components: the Channel Feature Extraction Block (CFEB) and the Multi-scale Feature Fusion Block (MFFB). The CFEB is composed of multiple connece--onnected 1 1x1 convolutional layers, designed to extract inter-channel relationships across multiple per- per-pixel measurements.
the other hand, the MFFB emphasizes spatial structure by performing preliminary depth estimation at each decoder level and progressively integrating features from multiple scales in a coarse-to- to-fine manner. The outputs of CFEB and MFFB are subsequently fused produce the final high-fidelity depth map.
3.3 Loss Function
During the training process, we jointly optimize the coding functions and the reconstruction net- work. Given that our exposure encodable camera supports only binary coding functions, we enforce hardware implementability by applying constraints based difference. Additionally, Fisher information incorporated into the loss to improve reconstruction quality, while Mean Squared Error (MSE) is used as the objective guide the final output. Here we give more details about these losses.
double well function and first-order
Mean Squared Error Loss. map, as defined Eq.
We employ MSE the fidelity loss supervise the predicted depth
LMSE
Σ Ildpre dsc(3)
(8)
5

---

Fisher Guidance Loss. The SNR is one of the key factors influencing the quality of ToF imag- ing. Inspired by [11], we introduce the fisher guidance loss enhance the quality of our depth reconstruction, which can be summarized as Eq. 9
K
[B(((()))
fisher -E
(9)
(0) of(s)
ad
where (s)( is the expectation of Ti(s) and (s) /B((())) + B(nd) + of
Double Well Function Loss. To enable the optimization of binary coding functions within the differentiable physical model. We introduce the double well function from quantum mechanics [36], which is formulated in Eq. 10
(z) = 4(x - 0.5)* 2(x 0.5)²
(10)
As shown in Fig. 4, this function has two valleys x 0 and x = 1, thereby encouraging the coding functions to converge toward binary states during the optimization process. There- fore, our double well function loss can be defined as Eq. 11
0
-0.5
K
M
Figure 4: Demonstration of the
Low
Sdn
(11) double well function with two iden- tical minima located at x. 0 and 1.
==1
where M is the sampling points on each coding function.
First-order Difference Loss. Although the double well function effectively constrains the coding functions to a binary state, we observe that the learned functions often exhibit extremely narrow peaks, which pose challenges for practical hardware implementation. To mitigate this issue, in- troduce a first-order difference loss, as defined in Eq. 12. By minimizing the first-order difference loss, narrow peaks can be effectively suppressed, thus ensuring feasibility for hardware implementa- tion.
K M
List
Σ
Σ -
(12)
Finally, our complete loss can be summarized Eq. 13
[ = LMSE + +n fisher + 72 Ldw + Va List
(13)
where 71. 72 and 73 are loss balance coefficients.
4 Synthetic Assessment
4.1
Implementation Details
Dataset. We use the NYU-V2 dataset [37] to train and test our end-to-end framework. The NYU- V2 dataset a high-quality RGB-D dataset captured by Kinect. contains a total of 1449 pairs of precisely aligned RGB and depth images collected from 464 indoor scenes, which enables its extensive application in academic research. For each RGB-D pair, we first apply intrinsic image decomposition [38] to the RGB image to obtain reflectance and ambient light maps. Subsequently, as detailed in Sec. 3.1, given the reflectance Pa. ambient light Lambs and depth d(s). we can synthesize multiple BE-ToF mcasurements. We divide the dataset in detail, using 1000 pairs of data as the training set and the remaining 449 pairs as the test set [39, 40].
Incremental Training Method. In our BE-ToF system, the SNR varies not only with distance but also significantly under the same distance due to ambient light Iamb. Therefore, we introduce an incremental training strategy [41] to ensure robust depth estimation of our network under varying SNR levels. Specifically, for each distance, we define three distinct SNR scenarios arranged from high to low. The network is trained with input data of varying SNRs, progressively transitioning from high to low every 10 epochs. When data all SNRs are traversed, samples with random SNR are generated and fed to the network for the convergence of the network.

---

Hyur
3m
Lenr
Umu
Hsnr
33m
Lynr
30m
Haur
63m
Lanr
60m
Henr
93m
laar
90m
Scene
Single Proquency FisherToP
Dual Proquency Simusnid
Dual Prequency
Dual Prequency DesTToP
Single Frequency Ours
GT
Figure 5: Overall comparisons with traditional iToF methods under various distances and SNRs, including FisherToF [11] under single frequency modulation; Sine/Square + PS algorithm [5] and Sine + DeepToF [23] under dual frequency modulation.
Training Parameters. We choose K in Eq. 6 as 4 and M in Eq. 11 as 1000. The number of restormer blocks in the network
[4, 6, 6, 8). We train the network for 200 epochs using the ADAM optimizer [42] with batch size of 20. The learning rate is initialized at 0.01 and decays by factor of 0.7 every epochs. The loss balance coefficients ? and 72 are empirically set to Se-4 and 5e-2 initially, and are updated to 5e-5 and I after 40 epochs. 73 is always set to 5. Xavier initialization used for the learnable coding functions. All experiments are conducted on the Py Porch platform [43], using an NVIDIA GeForce RTX 4090 GPU.

---

Table 1: Quantitative comparison of overall performance, coding schemes, and reconstruction net-
works. All metrics are reported as MAE (mm).
0.3m
30-33m
60 63m
90.99 93m
Harr
Har
Low
Marr
Luw
H
(a) Overall Performance
FisherToF []]]
7.19
10.46
16.91
20.80
24.54
31.61
34.58
42,43
54.73
75.19
77.68
138. I1
Sinc+PS [5]
43.26
58.08
78.00
56.96
77.89
107.25
79.20
111.49
158.40
117.21
170.76
244.29
Square-PS [5]
33.21
40.64
51.29
40.06
51.19
66.90
51.90
69.12
93.62
72.16
100.12
140.98
Deep Tol [23]
17.76
19.19
29.51
26.54
29.25
37.49
31.50
33.02
45.54
42.64
45.12
56.97
(b) Coding Scheme
Square
12.66
15.18
21.35
16.82
22.10
29.05
20.85
24.54
32.77
25.51
30,08
44.47
Reconstruction Network
Deep [23]
14.76
16.20
20.32
18.25
23.30
34.12
24.95
31.50
37.90
28.12
38.17
48.55
TaskToP [31]
11.73
12.89
17.41
14.72
19.44
26.73
15,94
21.38
33.55
25.71
27.86
38.60
FisherToof [1]]
6.94
8.10
16.26
10.22
15.71
21.68
14.62
18.31
29.73
22.60
24.89
31.83
Ours
5.90
6.95
12.71
8,03
12.25
18.29
11.93
16.60
26.08
18.96
21,99
29.58
4.2
Comparison with the State-of-the-art Methods
To demonstrate the superiority of our method, we conduct a detailed comparison with traditional iToF approaches, including single frequency modulation and dual frequency modulation. The sce- narios encompass multiple distance ranges (0-3m, 30-33m, 60-63m, and 90-93m) combined with varying SNRs, specifically high (H 22 dB), medium(M 19 dB) and low (Lgnr 16 dB). As shown in Fig. 5, we first compare our method with FisherTof [11] under single frequency mod- ulation. While FisherToF achieves precise depth reconstruction close range, it still suffers from the rapid decline in imaging quality over distance. We then compare our method with a variety of dual frequency modulation approaches, including sinusoid and square coding functions with Phase Shift (PS) algorithm [5] and the learning based DeepToF [23] method. Our method achieves the best performance across various distances and SNRs, using only single frequency modulation. We present detailed quantitative comparison in Tab. (a), with Mean Absolute Error (MAE) as the evaluation criterion.
We further substantiate the superiority of our method through an analysis of the learnable coding function and the proposed RSCF-Net. As for the coding function, considering the practical hardware implementability, we compare our learnable coding functions with the square coding function with the RSCF-Net. The quantitative results presented in Tab. (b) prove that learnable coding functions provides superior depth reconstruction and enhanced robustness to noise. Additionally,
33.
372
6ba
02s
Scene
Square RSC
Leamed Codes
+ DeepTooF
Learned Codes MaskToF
Leamed Codes
FisherTToF
Ours
Figure 6: Comparisons with different coding scheme and reconstruction networks.

---

we perform a thorough comparison of our RSCF-Net with existing depth reconstruction networks with the same learned coding functions. including DeepToF [23], MaskToF [31] and FisherToF [11]. The quantitative results in Tab. I (c) confirm the effectiveness of our network. Fig. 6 presents the visual results of different methods across four distances under low SNR conditions, intuitively demonstrating the advantages of our approach.
4.3 Ablation Study
Law
List
with Lew
List
Table 2: Quantitative ablations with MAE(mm) the evaluation metric.
Distance(m)
0-3
30-33
60-63
90-93
w/o C risher
15.36
19.41
27.02
37.45
w/o CFEB
58.69
73.64
78.56
86.94
Horthorm
Horchorm
Handwaee Unimplenerrable
w/o MFFB
15.77
28.40
Implementable
Unimplementaiee
w/o EC'A
10.22
13.01
21.41
26.06
Figure 7: Visual ablations on Law and Liet-
We first perform ablations on the proposed double well function loss Ldw and first-order difference loss List. Since the learned coding functions are primarily used to control the camera's 's exposure, they must strictly binary. As illustrated in Fig. the absence of Law and L results in coding functions that are entirely impractical to implement in hardware. With only the Cdtus the coding functions do converge to binary states; however, the proliferation of narrow peaks still makes them impossible to implement real hardware.
In the next, we present a quantitative analysis to evaluate the impact of the fisher guidance loss and different network blocks. The values in Tab. 2 represent the average MAE measured under different SNRs at the same distance. The experimental results demonstrate the effectiveness of the introduced Fisher loss in guiding the network to learn an optimal coding functions. The ablation studies on different network blocks further validate the significant improvement reconstruction quality brought by the proposed CFEB and MFFB.
5 Physical Experiment Results
Hardware Prototype. As shown in Fig. 8, to validate the effectiveness of our BE-ToF approach in real world scenarios, we built a prototype system comprising solid-state pulsed laser and an exposure-ens eepodbee-encddabee ICMOS sensor. The laser operates at 532 nm with a ns pulse width, a fixed kHz repetition rate, and up I mJ single-pulss energy. To realize the beam with a diffuser and expand it using a beam expander. The ICMOS is fitted with a zoom lens (300-800 mm) and supports minimum exposure gate of 3 Timing synchronization is provided by a fast photodiode that detects each laser pulse and issues hardware trigger to a signal generator, which then drives the ICMOS with the learned coding functions. This hardware chain achieves picosecond-scabe synchronization, ensuring high-quality imaging.
illumination, homogenize
Experimental Results. As shown in Fig. 9, evaluate approach across diverse indoor and outdoor scenarios, including a hand model and kettle indoors and stone model and stair out-
Table 3: Quantitative results on real world experiments with MAE(cm) as the evaluation metric.
Square + PP
Square RSCF-Net
Learned Codes
Learned Codes FisherToF
Signal
Ours
+ DeepToF
Hand
10.57
3.96
4.73
2.47
1.78
Kettle
9.13
3.51
4.14
1.99
1.41
Zoot Lets
Stone
12.16
4.76
5.94
3.78
2.50
Stair
15.79
5.92
6.70
4.26
3.83
Figure 8: System Prototype.

---

35.7m
35.3m
7m
35.3m
53.8m
53.4m
78.7m
57.7m
Square PS
Square RRCCFNNet
Learned Learned
Scone
Ours
GT
DeepToF
FisherToF
Figure 9: Real world experiments across indoor and outdoor scenarios.
doors. All experiments use the same settings as simulation: we apply the coding functions learned in simulation and reconstruct with RSCF-Net. The modulation period Tmm is fixed at 20 ns, and the burst period Tourst is set to Ims, corresponding to the laser repetition rate of 1 kHz. We perform detailed comparison against other methods, including square coding function and several recon- struction networks. Quantitative results are summarized in Tab. 3. Ground truth is acquired via time-delay scan the minimum exposure time ns) with step. Both qualitative and quantita- tive results demonstrate that our system consistently achieves centimeter-level depth accuracy across these scenarios and outperforms existing methods.
Conclusion, Limitations, and Broader Impact
In conclusion, we propose novel ToF imaging paradigm, termed BE-ToF. The BE-ToF system enables long-distance high-fidelity depth imaging by modulating and demodulating pulsed signals in burst mode using only single-freqeency modulation. Additionally, we introduce a learnable end- to-end framework that jointly optimizes binary coding functions and the reconstruction network to effectively handle varying SNRs across different distances, achieving state-of-the-art performance.
Limitations. Despite achieving both long-disten dont-distance and high-fidelity depth imaging, our BE-ToF system is subject to limitations in its imaging range. As shown in Fig. 1, the operational range is confined between and e-(rTTm) with higher precision resulting in narrower imaging range. We are currently exploring several promising directions to mitigate these limitations. First, we can exploit BE-ToFs flexible time-delay control to perform temporal scanning and synthesize wide- range depth map. Second, because temporal scanning can incur significant latency, we favor coarse-to-fine strategy: first capture a wide-range, low-resolution depth map, then BE-ToF to selectively acquire high-precision depth in regions of interest (ROIs).
Broader Impact. The proposed BE-ToF system demonstrates strong potential for applications such as autonomous driving and topographic surveying, offering enhanced reconstruction quality and improved processing efficiency. However, its ability perform long-distance depth imaging raises potential privacy concerns, particularly in scenarios where individuals may be unknowingly captured. Addressing these concerns responsibly is essential for real-world deployment.
Acknowledgments and Disclosure of Funding
This work was supported by the National Key Research and Development Program of China No. 2022YFA 1 National Natural Science Foundation of China No. 62522113, 62505132, the Fundamental Research Funds for Central Universities No. 021014380227, 021014380260, and Open Research Project of Suzhou Laboratory No. 20002-11100-2222
10

---

References
[1] Ruixuan Chen, Haowen Shu, Bitao Shen, Lin Chang, Weiqiang Xie, Wenchao Liao, Zihan Tao, John Bowers, and Xingjun Wang. Breaking the temporal and frequency congestion of lidar by parallel chaos. Nature Photonics, 17(4):306-311 314,314, 2023.
[2] Ricardo Roriz, Jorge Cabral, and Tiago Gomes. Automotive lidar technology: A survey, IEEE Transac- tions on Intelligent Transportation Systems, 23(7):6282-629, 2021.
[3] Simone Zennaro, Matteo Munaro, Simone Milani, Pietro Zanuttigh, Andrea Bernardi, Stefano Ghidoni. and Emanuele Menegatti. Performance evaluation of the 1st and 2nd generation kinect for multimedia applications. In IEEE International Conference on Multimedia and Expo, pages 1-6. IEEE, 2015.
[4] David Delpy, Mark Cope, Pieter van der Zee, Simon Arridge, Susan Wray, and IS Wyatt. Estimation of optical pathlength through tissue from direct time of flight measurement, Physics in Medicine & Biology. 33(12):1433, 1988.
[5] Miles Hansard, Seungkyu Lee, Ouk Choi, and Radu Patrice Horaud. Time-of-fight cameras: principles, methods and applications. Springer Science & Business Media, 2012.
[6] Larry Li et al. Time-of flight cameraan introduction. Technical White Paper, (SLOA190B), 2014.
[7] Mario Frank, Matthias Plaue, Holger Rapp, UlIrich Köthe, Bernd Jähne, and Fred A Hamprecht. Theoreti- cal and experimental error analysis of continuous wave time-of- range cameras. Optical Engineering, 48(1) (81103136022 01 3602, 2009.
[8] Yu Meng, Zhou Xue, Xu Chang, Xuemei Hu, and Tao Yue. itof-flow-based high frame rate depth imaging. In Proceedings of the IFEE Conference on Computer Vision and Pattern Recognition, pages 4929-4938,
2024.
[9] Yu Meng, Tao Yue, and Xuemei Hu. Alignment-free 3d motion compensation for itof imaging via local linear cansfer-enhanced joint optimization. Information Fusion, page 103636, 2025.
[10] Radu Horaud, Miles Hansard, Georgios Evangelidis, and Clément Ménier. An overview of depth cameras and range scanners based on time-of-flight technologies. Machine vision and applications, 27(7): 1020, 2016.
[11] Jiaqu Li, Tao Yue, Sijie Zhao, and Xuemei Hu. Fisher information guidance for learned time-of-flight imaging. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 16334 16343, 2022.
[12] Stephane Poujouly and Bernad Journet. twofold modulation frequency laser range finder. Journal of Optics A: Pure and Applied Optics, 4(6):S356, 2002.
[13] Adrian PP Jongenelen, Donald G Bailey, Andrew D Payne, Adrian A Dorrington, and Dale A Carnegie. Analysis of errors in tof range imaging with fuel-frey modulation. IEEE Transactions on Instru- mentation and Measurement, 60(5):1811-1868 2011.
[14] Ryan Crabb and Roberto Manduchi. Fast single-frequency time-of-fligh range imaging. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition Workshops, pages 58 65, 2015,
[15] Aongus McCarthy, Gregor G Taylor, Jorge Gareia-Armenta, Boris Korzh, Dmitry V Morozov, Andrew D Beyer, Ryan M Briggs, Jason P. Allmaras, Bruce Bumble, Marco Colangelo, et al. High-resolution long- distance depth imaging lidar with ultra-low timing jitter superconducting nanowire single-photon detec- tors. Optica, 12(2):168-177 17788-17777000 2025.
[16] Mamadou Diop and Keith St. Lawrence. Improving the depth sensitivity of by extracting the distribution of times-of offfihhf- Biomedical Optics Express, 4(3):44--455 2013.
measurements
[17] Shinzo Koyama, Motonori Ishii, Shigeru Saito, Masato Takemoto, Yuga Nose, Akito Inoue, Yusuke Sakata, Yuki Sugiura, Manabu Usuda, Tatsuya Kabe, et al. A 220 m-range direct time-of-fligh 688x 384 emos image sensor with sub-photon signal extraction (spse) pixels using vertical avalanche photo- diodes and 6 khz light pulse counters. JEEE Symposium on VLSI Circuits, pages 71-72, IEEE, 2018.
[18] Matteo Perenzoni, Daniele Perenzoni, and David Stoppa. 64 64x64- 64--iells digital silicon photomulti- plier direct tof sensor with 100- 200-mphotonsss/pixx. background rejection and imaging/altimeter mode with 0.14% precision up to 6 km for spacecraft navigation and landing. IEEE Journal of Solid-Suor Circuits, 52(1): 2016.
11

---

[19] Tang Xu, Qianyu Chen, Dajing Bian, and Yue Xu. A near-infrared single-photon detector for direct time- of-flight measurement using tin-----tipittddiiiiiiii hybrid conversion method. IEEE Transactions on Instrumentation and Measurement, 73:1-9, 2023.
[20] David Droeschel, Dirk Holz, and Sven Behnke. Multi- frequenyy phase unwrapping for time-of time-of-flight cameras. In IEEE International Conference on Intelligent Robots and Systems, pages 1463-1469, IEEE, 2010.
[21] Qi Guo, Iuri Frosio, Orazio Gallo, Todd Zickler, and Jan Kautz. Tackling 3d tof artifacts through learning and the flat datasct. In Proceedings of the European Conference Computer Vision, pages 368 383,, 2018.
[22] Dwi Hanto, Hari Pratomo, Agitta Rianaris, Andi Setiono, Sartika Sartika, Mohamad Syahadi, Eko Joni Pristianto, Dayat Kurniawan, Dwi Bayuwati, Hendra Adinanta, et al. Time of flight lidar employing dual-modulation frequencies switching for optimizing unambiguous range extension and high resolution. IEFE Transactions on Instrumentation and Measurement, 72:1-8, 2023.
[23] Shuochen Su, Felix Heide, Gordon Wetzstein, and Wolfgang Heidrich. Deep end-to-end time-of-figg flightt imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6383-6392, 2018.
[24] HyunJun Jung. Nikolas Brasch, Ales Leonardis, Nassir Navah, and Benjamin Busam. Wild tofu; Improv- ing range and quality of indirect time-of-fligt depth with rgb fusion in challenging environments. In International Conference on 3D Vision, pages 239-248. IEEE, 2021.
[25] Christopher A Metzler, Hayato Ikoma, Yifan Peng, and Gordon Wetzstein. Deep optics for single-shot high-dynamic-range imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1375-1385, 2020.
[26] Qilin Sun, Ethan Tseng, Qiang Fu, Wolfgang Heidrich, and Felix Heide. Learning rank-1 diffractive optics for single-shot high dynamic range imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1386-1399 2020.
[27] Shijie Nie, Lin Gu, Yinqiang Zheng, Antony Lam, Nobutaka Ono, and Imari Sato. Deeply learned filter response functions for hyperspectral reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4767-4776, 2018
[28] Elias Nehme, Daniel Freedman, Racheli Gordon, Boris Ferdman, Lucien Weiss, Onit Alalouf, Tal Naor, Reut Orange. Tomer Michaeli, and Yoav Shechtman. Deepstorm3d: dense 3d localization microscopy and psf design by deep learning. Nature Methods. 17(7):734-440 2020.
[29] Vincent Sitzmann, Steven Diamond, Yifan Peng, Xiong Dun, Stephen Boyd, Wolfgang Heidrich, Felix Heide, and Gordon Wetzstein. End-to-end optimization of optics and image processing for achromatic extended depth field and super- resolution imaging. ACM Transactions on Graphics. 37(4):1- 13, 2018.
[30] Ziyi Guo, Jiaqu Li, Kanghui Wang. Tao Yue, and Xuemei Hu. End-to-end fluorescence lifetime imaging with optimized encoding and exposure allocation. In IEEE International Conference on Computational Photography, pages 1-12. IEEE, 2024.
[31] Ilya Chugunow, Seung-Hwan Back, Qiang Fu, Wolfgang Heidrich, and Felix Heide. Mask-tof: Learn- ing microlens masks for flying pixel correction in time-of- imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9116 9126, 2021.
[32] Amit Agrawal and Yi Xu. Coded exposure deblurring: Optimized codes for psf estimation and invert- ibility. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2066-2073. IEEE, 2009.
[33] Miao Liao, Liang Wang, Ruigang Yang, and Minglun Gong. Light fall-off stereo. In Proceedings of the IEEE Conference on Computer Vision and Patiern Recognition. pages 1-8. IEEE, 2007.
[34] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- -ssua Yang. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5728-5739, 2022.
[35] Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu. Eca-net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 11534-1151 11542, 2020.
12

---

[36] V Jelic and F Marsiglio. The double-well potential in quantum mechanics: simple, numerically exact formulation. European Journal of Physics, 33(6))1651,,011 2012.
[37] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Proceedings of the European Conference Computer Vision, pages 746-760. Springer, 2012.
[38] Junho Jeon, Sunghyun Cho, Xin Tong, and Seungyong Lee. Intrinsic image decomposition using tiruttrretexturee separation and surface normals. In Proceedings of European Conference on Com- pider Vision, pages 218 233. Springer, 2014.
[39] Beomjun Kim, Jean Ponce, and Bumsub Ham. Deformable kernel networks for joint image filtering. International Journal of Computer Vision, 17902):57--600, 2021.
[40] Lingzhi He, Hongguang Zhu, Feng Huihui Bai, Runmin Cong, Chunjie Zhang, Chunyu Lin, Meiqin Liu, and Yao Zhao. Towards fast and accurate real-world depth super-resolution: Benchmark dataset and baseline. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9229-9238, 2021.
[41] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Pro- ceedings of the International Conference Machine Learning, pages 41 48, 2009.
[42] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
[43] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary Viti,o, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
[44] Felipe Gutierrez-arrragan, Syed Azer Reza, Andreas Velten, and Mohit Gupta. Practical coding function design for time-of-flighe imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1566 1574, 2019.
[45] Achuta Kadambi, Refael Whyte. Ayush Bhandari, Lee Streeter, Christopher Barsi, Adrian Dorrington, and Ramesh Raskar. Coded time flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Transactions on Graphics, 1-10);1-10, 2013.
[46] Katrin Honauer, Ole Johannsen, Daniel Kondermann, and Bastian Goldluecke. dataset evaluation methodology for depth estimation on 4d light fields. In Asian Conference on Computer Vision, pages 19-34. Springer, 2017.
[47] Shuran Song. Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d seene understanding benchmark suite. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 576- 2015.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims in the abstract and introduction are consistent with the con- tributions this paper and align with both the simulation and real-world experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of proposed method in detail in Sec.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't it acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
All the theories in the paper are provided with complete and correct proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: provide a detailed description of our experimental setup in Sec. 4.1, which will facilitate the reproducibility of our main experimental results by others.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We will provide our dataset and code with sufficient instructions.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/ httpa//gguiiee/ooeeuaaiaiissoooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so No is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
"tspp.cc/public/guiie hwtp//uddee/////eee /ooeeee/o wetoommmssiooooooooo ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide a detailed description of training and test details in Sec. 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
The error bars reported in this paper are suitably and correctly defined.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
16

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Experiments compute resources are detailed descried in Sec, 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't it make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httppp//////orppp..//aaaaaaaaaaa
Answer: [Yes]
Justification: Research conducted in the paper conforms to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: discuss broader impacts of the proposed method in detail Sec. 6. Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided relcasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The original authors of the code are properly credited, and the dataset used in this paper is properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, pwperswithheeee thcode com/ dataatss has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [Yes]
Justification: New assets introduced in the paper are well documented and accompanied by appropriate documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
19

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our L.M policy (https:/ /ttps/////eeup./eeeeee /tteurppp..coooofeeeeeeeeeeeeeee for what should or should be described.
20

---

A Technical Appendices and Supplementary Material
A.1 Comparison with Other Coding Functions
To fully highlight the advantages of our learnable binary coding functions, conduct detailed comparison against alternative coding functions, including sinusoid, square, Hamiltonian [44], and M-sequence [45]. To ensure fair comparison of coding functions, employ the same imaging setup (with
measurements) and reconstruction network, varying only the coding functions. The quantitative results, evaluated using Mean Absolute Error (MAE) in millimeters, rized in Tab. 4. can be observed that, under the same reconstruction network, our coding functions delivers the best performance. Notably, although compare multiple coding functions in simula- tion, only the square coding function and our learnable binary coding functions are implementable on actual hardware. The sinusoidal coding function is excluded due its non-binary nature, while Hamiltonian codes and M-sequences contain narrow peaks that are impractical to implement given hardware constraints on minimum exposure time.
summa-
To further demonstrate the advantages of optimizing coding functions using neural networks, we Fisher information evaluate the quality of each coding function. As discussed in [11], Fisher information can be used as a metric to assess the optimality of different coding schemes higher Fisher Information value indicates more optimal coding scheme. Therefore, list the Fisher Information different coding functions Tab. for a straightforward comparison. can be observed that our learnable binary coding functions achieves the highest Fisher information, which demonstrates its optimality.
Table 4: Quantitative comparison with other coding functions.
0-3m
30-33m
60-63m
90-93m
Sinusoid
24.67
31.12
39.80
45.39
Square
16.40
22.66
26.05
33.35
Hamiltonian [44]
11.28
14.53
21.74
27.11
M-sequence [45]
15.19
21.24
28.33
38,81
Ours
8.52
12.86
18.20
23.51
Table 5: Quantitative comparison of Fisher Information with other coding functions.
Sinusoid
Square
Hamiltonian
M-sequence
Ours
Fisher Information 1.27 10c 2.29 x 106
2.92 x 10
2.18 106
3.93
10
A.2 Experiments with Fewer Measurements
As discussed in Sec. 3.1, at least K 2 3 measurements are required to recover depth. In our work, choose K 4 to ensure high reconstruction quality and robustness noise. In Tab. 6, present quantitative results for cases with I 4, evaluated using Mean Absolute Error (MAE) in millimeters. When K < the reconstruction quality significantly degrades, which is reasonable given the limited information available for depth recovery. With K = 3, depth can be reasonably reconstructed, though still slightly inferior K 4, where the additional measurement improves robustness to noise and other perturbations.
Table 6: Quantitative comparison with different measurements.
0-3m
30-33m
60-63m
90-93m
K 133.22
166.90
156.18
198.19
K 37.84
42.94
51.96
49.18
K 14.58
17.59
21.64
29.88
K 8.52
12.86
18.20
23.51
21

---

A.3 Ablations on Loss Balance Coefficients
In the loss defined in Eq. 13, the MSE term is the principal objective driving accurate depth recon- struction, while the three additional regularization terms serve as auxiliary constraints that guide the learning of the binary coding functions. To validate our choice of loss weights, we conduct comprehensive ablation over the coefficients and report the results. All experiments are performed distances of 0-3 m.
For coefficient ?1 before 40 epochs, as shown in Tab. 7, setting "1 < 5 x 10 makes its effect weak for the network to learn effective coding functions, leading to performance drop. When exceeds 5 x 10-4, it disrupts the double-well and first-order difference losses, resulting in coding functions unimplementable for hardware. Thus, we set Y to x 10-4 during the first 40 epochs.
Table 7: Ablations on 71 before 40 epochs.
71 Before 40 Epochs 5 x 10-7 5 x 10-6
5 x 10-5
x 10-4
x 10-3
5 x 10-2
Hardware Unimplementable Unimplementable
Hardware
MAE(mm)
17.23
12.47
9.76
8.52
For coefficient "1 in epochs after 40, shown in Tab. 8, we find that the value of n has little impact on the final performance. However, setting it too high can slow down the convergence of the network. Therefore, we set to 5e-5 after 40 epochs to balance performance and convergence speed.
Table 8: Ablations 71 after 40 epochs.
After 40 Epochs
5 x 10-7
5 10-6
5 x 10-
5 x 10-4
x 10
x 10-2
MAE(mm)
9.94
8.73
8.52
14.58 123
11.62
10.17
Convergence Epochs
107
103
112
133
137
For coefficient "/2 in epochs before 40, we set "/2 to small value so that the first-order difference loss dominates and helps suppress narrow peaks. As shown in Tab. 9, when 7/2 exceeds the learned coding functions exhibit narrow peaks and become unsuitable for hardware implementation. Thus, we set ?2 to x 10-2 during the first 40 epochs training.
Table 9: Ablations on 72 before 40 epochs.
?/2 Before 40 Epochs
5 x 10
x
10
x 10
5 x 10-1
5
Hardware Unimplementable
MAE(mm)
11.98
12.35
8.52
12.67
10.27
For coefficient 722 in epochs after 40, we increase the value of 72 encourage the coding functions to converge more rapidly to a binary state. As shwon in Tab. 10, setting the coefficient below x 10-2 prevents the coding functions from reaching binary state, while values above 30 cause noticeable performance degradation. Therefore, we set 7/2 to after 40 epochs.
Table 10: Ablations on 72 after 40 epochs.
7/2 After 40 Epochs
5 x 10
10
x 10
12
20
30
40
Hardware Unimplementable
MAE(mm)
10.26
9.02
8.52
9.29
11.41
10.68
17.92
For coefficient "3. as shown in Tab. 11, when the coefficient is too small, narrow peaks appear, making hardware unimplementable. Conversely, large coefficient results in degraded depth recon-
22

---

struction quality. A balanced performance is achieved with values between 0.05 and 10; we set it to 5 in our experiments.
Table 11: Ablations on 73-
5 X 10"4
5 x 10-3
5 x 10-
5
10
20
30
Hardware Unimplementable
Hardware
MAE(mm)
9.02
8.98
8.52
8.53
16.88
23.76
Unimplementable
A.4 Simulation Method for the Long-Range Indoor Dataset
We use the NYU Depth V2 RGB-D dataset to train and evaluate our network. First, we scale the depth values to fixed maximum range of 3 m (corresponding to =20nss,, and apply this setting consistently across all simulation experiments for fairness. As discussed in Sec. 3.1, ToF imaging is principally affected by ambient light, scene reflectance, and depth. To model these factors, we perform intrinsic image decomposition [38] on the RGB images to separate ambient light and re- flectance components, and calibrate distance-dependent attenuation curve under long-range conditions within the simulation environment. Because ambient illumination and scene reflectance are depth-invariant in this model, variations across distance are primarily attributed to attenuation. Accordingly, given a fixed emitted signal, we apply eestance-deendde attenuation coefficients to the reflected signal and then synthesize the corresponding ToF measurements. In addition, we ex- plicitly model sensor noise as in Eq. 7, incorporating both dark current and readout noise to obtain more realistic measurements.
is worth noting that our setting still differs from real long-range outdoor scenarios in several respects: (i) the current simulation does not model atmospheric effects, which can significantly in- fluence ToF imaging outdoors; and (ii) beam divergence remains a key factor affecting image quality. To mitigate the latter, employ laser beam expander generate area illumination, though some deviation from ideal uniform lighting persists. In future work, we plan to incorporate atmospheric ef- fects into the simulation framework better align with real-world experiments; meanwhile, spatial filtering is used to further improve illumination uniformity.
A.5 Precision of Binary Coding Function Optimization
In this work, we adopt differentiable double well function to drive the coding functions toward 0 or 1, thus ensuring that our end-to-end framework is fully differentiable. While the optimized coding functions effectively binary, they do not attain exact binary values in floating-poin arithmetic. Empirically. the learned coding functions lie extremely close to the binary extremes (e.g., around 0.0001 or 0.999), and regard such deviations conclusion, we apply a round function during testing to convert coding functions into strict binary states(O 1) and compare results without it. The quantitative results in Tab. 12 show that striet bina- rization does not cause performance degradation; on the contrary, slightly improves performance.
negligible for network. To validate
Table 12: Quantitative comparison of coding functions with/without round function with MAE(mm) as the evaluation metric.
0-3m
30-33m
60-63m
90-93m
Without Round Function
8.52
12.86
18.20
23.51
With Round Function
8.51
12.78
17.97
21.97
A.6 Additional Results on Other Datasets
To rigorously assess the generalization capability of our method, we conduct experiments on the 4D Light Field [46] and SUN RGB-D [47] datasets, selecting 16 scenes from the former and 298 scenes from the latter as the test sets. Both datasets are processed according to the approach detailed in the main manuscript: each RGB-D pair undergoes intrinsic image decomposition to separate into an albedo map and shading map. Specifically, the R-channel of the albedo map is utilized as the
23

---

albedo component, while the average of its three RGB channels serves as the ambient illumination. The network, trained exclusively on the NYU-V2 [37] dataset, then evaluated on these datasets without any fine-tuning. For comparison, we include several baseline methods: FosherTof [11] under single frequency modulation, and sinusoidal and square coding functions combined with the Phase Shift (PS) algorithm [5], as well as the learning-basse DeepToF [23], all under dual-frequency mod- ulation. The quantitative results are summarized in Tab. 13. As shown, our method exhibits strong generalization performance across both datasets and achieves the highest reconstruction accuracy among all compared approaches. Additionally, we present qualitative results on the 4D Light Field and SUN RGB-D datasets in Fig. 10 and Fig. 11, respectively. The visualizations further confirm that our method delivers robust reconstruction performance, even under challenging conditions such long-range scenes or low signal-to -o-niss ratios.
Table 13: Quantitative comparison of overall performance on 4D Light Field [46] and SUN RGB- D [47] dataset.
0-3m
30-33m
60-63m
90-93m
Mar
Lyx
Mar
Lux
Hire
Mar
Lux
(a) Overall Performance on Light Field Dataset
Sine+PS [5]
43.39
58.20
78.10
57.07
78.01
106.93
79.37
111.08
157.91
116.95
170.50
247.06
Square+PS [5]
33.15
40.58
51.20
39.99
51.11
66.80
51.83
69.04
93.32
72.05
99.72
140.14
DeepTol [25]
19.16
23.27
40.73
34.10
36.91
45.53
38.48
42.08
52.98
54.44
55.25
66.11
Tolf [1]]
8.83
12.81
19.19
27.19
31.99
39 39.04
44.72
52.95
65.41
83.79
98.05
140.15
Ours
6.79
11.06
19.64
11.35
16.29
26.75
15.11
21.13
32.25
26.78
28.78
34.17
Overall Performance
RGB-D Dataset
Sine+PS [5]
40.46
54.10
72.39
53.06
72.20
98.66
73.39
102.39
144.44
107.51
155.64
224.36
Squarc4PS [5]
31.90
38059
48.25
48.15
62.44
64.46
67.20
92.42
129.06
Doep'oof [25]
16.47
23.45
30.71
25.12
32.67
36.78
27.99
33.80
40.99
36.07
45.11
$5.05
FisherToF [I[]
8.08
11.80
18.96
21.35
25.39
31.83
33.34
40.62
51.86
74.61
76.92
132.77
Ours
6.63
7.53
12.11
9.00
12.55
18.03
11.59
16.85
29.20
17.93
21.62
31.70
24

---

Hgnr
3m
Lynr
0m
Hgnr
33m
Lynr
Hsnr
63m
Linr
60m
Hynr
93m
Lgnr
90m
Scme
Single Prequency FiaberToF
Prequency Sinusoid PS
Proguency Square PS
Single Frequancy Sime DisyTuP
Frequency
Ours
Figure 10: Overall comparisons with traditional iToF methods under various distances and SNRs on 4D Light Field dataset, including FisherToF [11] under single frequency modulation; Sine/Square + PS algorithm [5] and Sine DeepTToF [23] under dual frequency modulation.
25

---

3m
byar
Byyr
hyur
30m
Hanr
63m
60m
Haor
93m
lenr
Scune
Single Prequency PishedToF
Frequanccy Sinusoid PS
Dual Proquency Square +S
Prequency Size DeepToF
Single Frequency Ous
GT
Figure 11: Overall comparisons with traditional iToo methods under various distances and SNRs on SUN RGB-D dataset, including FisherToF [11] under single frequency modulation; Sine/Square + PS algorithm [5] and Sine DeepTooF [23] under dual frequency modulation.
26


Reviews:
[{"review_id": "mkQuaizmMb", "reviewer": "Reviewer_uajT", "summary": "The manuscript proposes a long-distance, high-depth-resolution indirect time-of-flight (iToF) imaging system. The core contribution is the design of learnable modulation and demodulation codes that improve depth resolution, particularly under low and spatially varying signal-to-noise ratios. These codes are burst-type, targeting specific long-distance depth ranges rather than continuous coverage. The authors validate their approach through simulations and demonstrate hardware implementation, presenting a single real-world depth image.\n\nWhile the proposed method shows promise, I am currently inclined against accepting the manuscript due to (1) lack of simpler and fairer baseline comparisons and (2) lack of more real world results despite the availability of hardware.", "questions": "Please answer the questions asked in Strengths And Weaknesses section.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 4, "originality": 2, "strengths_and_weaknesses": "**strengths**\n- The manuscript is well written and easy to follow. It builds a hardware prototype for validation. \n- Usage of the double well loss function to encourage the decoding function to be binary valued at any time is quite interesting. I haven't seen this loss function before.  \n- first-order difference loss to avoid narrow peaks (high BW) is also interesting. However, why not sample the D(t) functions at the fastest switching rates possible by hardware and thereby avoid this loss function altogether. Also, loss functions encourage a behaviour but do not guarantee that it will happen. How is that handled? \n**weaknesses**\n- While the proposed technique enhances long-distance sensing, it does not extend the depth range---unlike dual-modulation, frequency switching, or M-sequence-based methods, which are specifically designed for that purpose. As a result, the current comparisons may be somewhat unfair to these techniques, as they are evaluated on metrics outside their intended strengths. \n-  A more fair and complete comparison---currently missing---is an ablation study where a single-frequency waveform (e.g., a sinusoid with a frequency chosen to match the same depth range as used in this manuscript) is employed, but temporally delayed by appending zeros. This raises the question: is the use of neural-optimized codes truly necessary to achieve good performance?\n- Comparison with space filling codes (optimal codes for ToF imaging [1]) or M-sequences [2] which can also lead to long distance (and range) is missing.\n     - [1] Gupta, M., Velten, A., Nayar, S.K. and Breitbach, E., 2018. What are optimal coding functions for time-of-flight imaging?. ACM Transactions on Graphics (TOG), 37(2), pp.1-18.\n     - [2] Kadambi, A., Whyte, R., Bhandari, A., Streeter, L., Barsi, C., Dorrington, A. and Raskar, R., 2013. Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Transactions on Graphics (ToG), 32(6), pp.1-10.\n- While the manuscript builds the hardware, it includes only a single experimental result, even in the supplementary material. Moreover, the result lacks quantitative or qualitative comparisons with alternative coding strategies. Capturing and evaluating multiple scenes under varying conditions---such as different times of day---and comparing the performance across different codes would significantly strengthen the manuscript and provide a more comprehensive validation of the proposed system.\n\n- Eq. 5 is inaccurate. The light itself is Poisson distributed. Given that the system is operating in low-light conditions, this is quite important. \n   - I am at Eq. 9 and noticed that light is considered Poisson accurately here. So, it is just a typo in Eq. 5. Please fix it. \n\n**Composition: **\n-  Equations are part of the text and should therefore include appropriate punctuation. Specifically, Eqs. 1–5, 7, 9, and 11 should end with commas, while Eqs. 6, 9, and 10 should end with periods.\n- Lines 133–134: Names derived from proper nouns must be capitalized. Use 'Poisson' and 'Gaussian' instead of 'poisson' and 'gaussian'.\n- Move Fig 5 to top of the page as well, like the rest of the figures. If not, it comes after the text and appears like it is part of the text.", "was_revised": true, "final_justification": "The manuscript provides reasonable answers to all my concerns. Therefore, I increased my rating to accept. \n\nIn the final version, the authors should include the new results they provided in rebuttal—especially new qualitative results on multiple scenes showing the improvement over existing codes.", "rebuttal": "We sincerely appreciate the reviewer’s recognition of our paper as well written and interesting. In this rebuttal, we would like to provide our responses to the following questions.\n\n>**Q1: The necessity of first-order difference loss and how to guarantee the avoidance of narrow peaks.**\n\n**A1:** Thank you for your interest in this question. Since our coding functions are primarily used to modulate the sensor’s exposure, they are constrained by the minimum exposure time achievable by the hardware, which is currently limited to **3 ns**. This restriction makes it infeasible to finely sample the coding functions to avoid first-order difference loss altogether. The avoidance of narrow peaks is guaranteed by adjusting the loss balance coefficient $\\gamma_3$ in Eq. 11. To validate the appropriateness of our coefficient selection, we present an ablation study on $\\gamma_3$ below. It can be observed that when the coefficient is too small, it fails to avoid the appearance of narrow peaks, while a large coefficient leads to degraded depth reconstruction quality. A relatively balanced performance can be achieved when the coefficient is chosen within the range of 0.05 to 10. In our experiments, we set it to 5.\n\n| $\\gamma_3$ |          0.0005          |          0.005           |    0.05     |      1      |      5      |     10      |     20      |     30      |\n| :--------: | :----------------------: | :----------------------: | :---------: | :---------: | :---------: | :---------: | :---------: | :---------: |\n|   Result   | Hardware Unimplementable | Hardware Unimplementable | Appropriate | Appropriate | Appropriate | Appropriate | Low Quality | Low Quality |\n\n> **Q2: Ablation study on coding strategies and the necessity of optimizing the coding function via neural networks.**\n\n**A2:** Thank you for your valuable suggestion. Our comparison with the dual-frequency modulated iToF system in the manuscript is primarily intended to highlight the advantages of our BE-ToF system—achieving high-precision, long-distance depth imaging using only a single modulation frequency. To ensure a fair comparison of coding functions, we employ the same imaging setup (with **K = 4** measurements) and reconstruction network, varying only the coding function. The quantitative results, evaluated using Mean Absolute Error (MAE) in millimeters, are presented below. It can be observed that, under the same reconstruction network, our coding function delivers the best performance. Notably, although we compare multiple coding functions in simulation, **only the square wave and our learned binarized coding function are implementable on actual hardware.** The sinusoidal wave is excluded due to its non-binary nature, while Hamiltonian codes and M-sequences contain narrow peaks that are impractical to implement given hardware constraints on minimum exposure time.\n\n|             |  0-3 m   |  30-33 m  |  60-63 m  |  90-93 m  |\n| :---------: | :------: | :-------: | :-------: | :-------: |\n|  Sinusoid   |  24.67   |   31.12   |   39.80   |   45.39   |\n|   Square    |  16.40   |   22.66   |   26.05   |   33.35   |\n| Hamiltonian |  11.28   |   14.53   |   21.74   |   27.11   |\n| M-sequence  |  15.19   |   21.24   |   28.33   |   35.81   |\n|  **Ours**   | **8.52** | **12.86** | **18.20** | **23.51** |\n\nTo further demonstrate the advantages of optimizing coding functions using neural networks, we use **Fisher information** to evaluate the quality of each coding function. As discussed in Reference [10] of the manuscript, Fisher information can be used as a metric to assess the optimality of different coding schemes — a higher Fisher information value indicates a more optimal coding scheme. Therefore, we list the Fisher information of different coding functions in the table below for a straightforward comparison. It can be observed that our learnable binarized coding function achieves the highest Fisher information, which demonstrates its optimality. \n\n|  Coding Function   |      Sinusoid      |       Square       |     M-sequence     |    Hamiltonian     |        Ours        |\n| :----------------: | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: |\n| Fisher Information | $1.27 \\times 10^6$ | $2.29 \\times 10^6$ | $2.18 \\times 10^6$ | $2.92 \\times 10^6$ | $3.93 \\times 10^6$ |\n\n> **Q3: Comparison with space filling codes and M-sequences.**\n\n**A3:** Thank you for your valuable recommendation and these are indeed two very important works. We provide a detailed comparison of the two coding functions in **A2**, using the same reconstruction network and replacing only the coding function to ensure a fair comparison. The results show that our learnable binarized coding function is not only hardware-implementable but also delivers the best performance.\n\n> **Q4: Additional real-world experimental results.**\n\n**A4:** Indeed, presenting more real-world experiments and comparisons with different methods is crucial to demonstrating the effectiveness of our BE-ToF system. Therefore, in this rebuttal, we upgrade our hardware setup and conduct experiments across a wider range of scenarios. **The specifications of the upgraded hardware are detailed below:** the system primarily consists of a solid-state laser and an exposure-encodable ICMOS sensor. The laser operates at a wavelength of 532 nm, with a pulse width of 5 ns, a fixed repetition rate of 1 kHz, and a single-pulse energy of 1 mJ. To enable area illumination, we use a laser beam expander. The ICMOS sensor is equipped with a 300–800 mm varifocal lens and a 532 nm band-pass filter (FWHM ±10 nm) to suppress ambient light interference. \n\nWe conduct experiments across a variety of scenarios, including indoor scenes (\\~40 m) such as a hand model and a kettle, as well as outdoor scenes (\\~60 m) such as a stone and a staircase. All experiments are conducted under conditions consistent with the simulation setup: we apply the same learned coding function obtained from simulation and use our RSCF-Net for reconstruction. The modulation period $T_m$ is also kept the same as in simulation, set to 20 ns. We conduct a detailed comparison with other methods, and the quantitative results—evaluated using **Mean Absolute Error (MAE) in centimeters**—are presented below. The ground truth is obtained by performing a time-delay scan using our minimum exposure time (3ns), with a step size of 1 ns. The results show that our system consistently achieves centimeter-level depth estimation across various scenarios and outperforms existing methods in terms of accuracy. **We promise to provide reliable visualizations in the camera-ready version.**\n\n|            | Square + PS | Square + RSCF-Net | Learned Code + DeepToF | Learned Code + FisherToF | Ours |\n| :--------: | :---------: | :---------------: | :--------------------: | :----------------------: | :--: |\n| hand model |    10.57    |       3.96        |          4.73          |           2.47           | 1.78 |\n|   kettle   |    9.13     |       3.51        |          4.14          |           1.99           | 1.41 |\n|   stone    |    12.16    |       4.76        |          5.94          |           3.78           | 2.50 |\n| staircase  |    15.79    |       5.92        |          6.70          |           4.26           | 3.83 |\n\n> **Q5: Clarification on Eq. 5.**\n\n**A5:** We appreciate your careful and thorough feedback. In our BE-ToF imaging model, the light signal $I_i(s)$ in Eq. 5 **indeed follows the Poisson distribution**, expressed as $I_i(s) \\sim \\mathcal{P}(\\mathbb{E}[I_i(s)])$. Regarding $n_d$ and $n_r$, they correspond to the dark current noise and readout noise of the camera sensor, which follow Poisson and Gaussian distributions, respectively. Both noise components are incorporated into the imaging model to more accurately simulate real-world measurements. We apologize for any misunderstanding caused. We will carefully revise the description of Eq. 5 to clarify the meaning of each variable.\n\n> **Q6: Writing errors and formatting adjustments.**\n\n**A6:** We sincerely thank you for your expert and insightful suggestions, which have greatly enhanced the clarity and quality of our manuscript. In response, we have carefully revised the equations, figures, and proper nouns as you recommended to further improve the overall readability."}, {"review_id": "CqR9KATWOS", "reviewer": "Reviewer_XQfn", "summary": "The paper introduces Burst-Encodable Time-of-Flight (BE-ToF) imaging, a novel approach for long-distance depth sensing. By transmitting light in burst patterns and capturing the phase shift over an entire burst period, BE-ToF overcomes limitations of traditional iToF systems such as phase ambiguity. The method employs a learnable, end-to-end framework that optimizes optical coding and the neural network for depth reconstruction. The effectiveness of BE-ToF is demonstrated through both simulated experiments and a prototype setup showing superior results in long-range scenarios.", "questions": "Could the authors explicitly explain the methodology behind the synthetic long-range indoor dataset? Are conditions such as lighting, materials, and atmospheric factors reflective of outdoor long-range environments? Addressing this clearly would strengthen the confidence in reported results.\n\nCould the authors provide quantitative evaluation results of the hardware prototype using common ground-truth methods (e.g. LiDAR)? Demonstrating quantitative comparisons with previous methods on real-world data would significantly enhance credibility.\n\nHave the authors considered dynamically shifting the time delay to extend the practical depth range for applications like autonomous driving? \n\nCould the authors include experiments demonstrating performance in dynamic or moving scenarios?\n\nCould the authors clarify whether system parameters such as laser power, measurement count (K), and network complexity match those of baseline methods? Including a runtime analysis would help readers assess the practical implications and feasibility of the proposed approach.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper proposes a technically novel method for long-range depth imaging by introducing the burst-encodable time-of-flight (BE-ToF) paradigm, enabling phase unwrapping with just a single modulation frequency. This significantly simplifies system complexity. The approach integrates a hardware-aware, end-to-end learning framework optimizing binary code design and depth reconstruction jointly. The authors demonstrate clear performance improvements over baselines on synthetic data. Additionally, comprehensive ablation studies and validation using a real-world prototype indicate promising practical applicability for high-precision long-range depth sensing.\n\n\nWeaknesses:\n\nThe evaluation methodology for indoor long-range scenarios lacks clarity. It appears that the dataset involves artificially extending indoor scenes to represent longer distances, but this is not explicitly stated or adequately explained. It is uncertain whether simulated indoor long-range scenes sufficiently capture conditions typically found in long-range (usually outdoor) scenarios, such as differing ambient illumination, diverse materials, and atmospheric effects. Furthermore, important practical factors like real sensor noise or beam divergence seem unaccounted for. Thus, the strong quantitative results reported might not accurately reflect real-world performance, necessitating additional validation with genuine long-range data.\n\nWhile the hardware prototype provides a convincing qualitative depth map demonstration for distances between 22–25 meters, it lacks quantitative evaluation against ground-truth data (e.g., LiDAR). Quantitative assessment on real-world scenarios and comparative analysis with existing methods would significantly strengthen the reliability and practical relevance of the results.\n\nMinor:\n\nOptimizing coding and reconstruction through learning follows previous work, limiting novelty primarily to the burst-encodable concept itself.\n\nThe stated motivation targets autonomous driving and robotics; however, the depth window presented is narrow. Practical automotive applications would require extending this range dynamically, which is not demonstrated. Additionally, experiments involving dynamic or moving scenes are absent but crucial to understanding potential method limitations.\n\nWhile hardware specifications are provided, comparisons regarding laser power, measurement number (K), and network complexity against baselines remain unclear. Clarifying these details and including runtime analysis would improve understanding of the method's practical feasibility.", "was_revised": true, "final_justification": "The manuscript provides clear and detailed answers to all my concerns, with new experiments, clarifications, and fairer comparisons that strengthen the work. Therefore, I increased my rating to accept. \n\nIn the final version, the authors should incorporate the additional results from the rebuttal, particularly the new real-world quantitative and qualitative results across multiple scenes, to highlight the improvements over existing methods.", "rebuttal": "We are grateful to the reviewer for recognizing the novelty and promising applicability of our work. In this rebuttal, we would like to offer our responses to the following questions.\n\n> **Q1: The methodology behind the synthetic long-range indoor dataset and the differences compared to long-range outdoor scenarios.**\n\n**A1:** Thank you for point this out. We use the NYU-V2 RGB-D dataset to train and test our network. First, we scale the depth data to a fixed range of 3 meters (corresponding to $T_m$=20ns), which is consistently applied across all simulation experiments to ensure fairness. As analyzed in Sec. 3.1 of the manuscript, the ToF imaging process is mainly influenced by ambient light, scene reflectance, and signal attenuation. To model these factors, we apply intrinsic image decomposition [35] to the RGB images to extract ambient illumination and reflectance components. We also calibrate a signal attenuation curve under long-range conditions within the simulation environment. Since ambient light and scene reflectance are invariant with respect to depth, variations in ToF imaging across distances are primarily attributed to signal attenuation. Therefore, given a constant emitted signal, we apply distance-dependent attenuation coefficients to simulate the reflected signal, which is subsequently used to generate the corresponding ToF measurements. Moreover, sensor noise is explicitly modeled in our simulation as described in Eq. 5, by incorporating both dark current noise and readout noise to produce more realistic ToF measurements. **Our key distinction from outdoor long-range scenarios lies in several aspects:** First, our current simulation does not yet incorporate atmospheric effects, which can significantly influence ToF imaging in outdoor environments. Additionally, beam divergence remains an important factor affecting image quality. To address this, we employ a laser beam expander to generate sheet laser illumination; however, this still deviates somewhat from ideal uniform lighting. In future work, we plan to integrate atmospheric effects into our simulation framework to better align simulated results with real-world experiments. Meanwhile, techniques such as spatial filtering are used to enhance the uniformity of the sheet laser illumination.\n\n\n\n> **Q2: Additional real-world experimental results and quantitative evaluation against ground-truth data.**\n\n**A2:** Indeed, presenting more real-world experiments and comparisons with different methods is crucial to demonstrating the effectiveness of our BE-ToF system. Therefore, in this rebuttal, we upgrade our hardware setup and conduct experiments across a wider range of scenarios. **The specifications of the upgraded hardware are detailed below:** the system primarily consists of a solid-state laser and an exposure-encodable ICMOS sensor. The laser operates at a wavelength of 532 nm, with a pulse width of 5 ns, a fixed repetition rate of 1 kHz, and a single-pulse energy of 1 mJ. To enable area illumination, we use a laser beam expander. The ICMOS sensor is equipped with a 300–800 mm varifocal lens and a 532 nm band-pass filter (FWHM ±10 nm) to suppress ambient light interference. \n\nWe conduct experiments across a variety of scenarios, including indoor scenes (\\~40 m) such as a hand model and a kettle, as well as outdoor scenes (\\~60 m) such as a stone and a staircase. All experiments are conducted under conditions consistent with the simulation setup: we apply the same learned coding function obtained from simulation and use our RSCF-Net for reconstruction. The modulation period $T_m$ is also kept the same as in simulation, set to 20 ns. We conduct a detailed comparison with other methods, and the quantitative results—evaluated using **Mean Absolute Error (MAE) in centimeters**—are presented below. **The ground truth is obtained by performing a time-delay scan using our minimum exposure time (3ns), with a step size of 1 ns.** We do not use LiDAR as the ground truth because LiDAR typically provides sparse point clouds, whereas we obtain dense depth maps. The significant difference between the two makes direct comparison difficult. The results show that our system consistently achieves centimeter-level depth estimation across various scenarios and outperforms existing methods in terms of accuracy. **We promise to provide reliable visualizations in the camera-ready version.**\n\n|            | Square + PS | Square + RSCF-Net | Learned Code + DeepToF | Learned Code + FisherToF | Ours |\n| :--------: | :---------: | :---------------: | :--------------------: | :----------------------: | :--: |\n| hand model |    10.57    |       3.96        |          4.73          |           2.47           | 1.78 |\n|   kettle   |    9.13     |       3.51        |          4.14          |           1.99           | 1.41 |\n|   stone    |    12.16    |       4.76        |          5.94          |           3.78           | 2.50 |\n| staircase  |    15.79    |       5.92        |          6.70          |           4.26           | 3.83 |\n\n\n\n\n\n> **Q3: Potential solutions to extend the practical depth range such as dynamically shifting the time delay.**\n\n**A3:** We sincerely appreciate your interest in our work. While we acknowledge the current limitations of BE-ToF, we are actively investigating several promising directions to address them in future research. One practical approach leverages BE-ToF’s flexible control over the depth range via time-delay adjustment to perform temporal scanning, thereby synthesizing a comprehensive, wide-range depth map. Considering the high latency potentially introduced by temporal scanning, we propose leveraging compressive sensing to reduce the amount of data acquisition. In addition, a more ideal approach is to adopt a coarse-to-fine strategy: initially capturing a wide-range but low-resolution depth map, followed by selective high-precision depth acquisition in regions of interest (ROIs) using BE-ToF. This approach can be further enhanced with AI-driven techniques to enable intelligent and adaptive depth sensing. Overall, I believe our method holds great promise for a wide range of applications, such as terrain mapping and autonomous driving.\n\n\n\n\n\n> **Q4: Experimental performance in dynamic or moving scenes.**\n\n**A4:** We appreciate the reviewer’s visionary suggestions. The ability to handle dynamic scenes is indeed vital for real-world deployment of our BE-ToF system. To evaluate this, we conduct experiments on the 4D Light Field RGB-D dataset. As the dataset contains static scenes, we simulate dynamics by introducing synthetic motion through data augmentation. Quantitative results, measured by Mean Absolute Error (MAE) in millimeters, are shown below. It can be observed that our network exhibits a performance drop in dynamic scenes. This is reasonable cause we have not yet optimized the model for dynamic scenarios. Nevertheless, we recognize the critical importance of dynamic scene depth reconstruction and will prioritize it as a key focus in our future work.\n\n|               | 0-3 m | 30-33 m | 60-63 m | 90-93 m |\n| :-----------: | :---: | :-----: | :-----: | :-----: |\n| Static Scene  | 12.50 |  18.13  |  22.83  |  29.91  |\n| Dynamic Scene | 27.46 |  34.34  |  39.81  |  49.07  |\n\n\n\n> **Q5: Clarification on system parameters such as laser power, measurement count (K), network complexity and runtime.**\n\n**A5:** Thank you for your concern regarding our system. In the manuscript, the laser used is a fiber laser with a wavelength of 905 nm, a repetition rate of 200 kHz, and an average power of 100 mW. In the upgraded hardware described in **A2**, a solid-state laser is employed, featuring a wavelength of 532 nm, a repetition rate of 1 kHz, and an average power of 1 W. Both hardware setups utilize coding functions learned from simulation, with the number of measurements **K = 4**. The comparison of network complexities is summarized in the table below, demonstrating that our RSCF-Net achieves the best reconstruction performance while maintaining relatively low computational complexity and parameter count. Regarding runtime, our system currently runs in offline mode. In the future, we will deploy the algorithm on an FPGA to achieve real-time depth sensing.\n\n|                    | DeepToF | MaskToF | FisherToF | RSCF-Net (Ours) |\n| :----------------: | :-----: | :-----: | :-------: | :-------------: |\n|   **FLOPs (G)**    |  12.65  |  5.62   |   26.72   |      14.69      |\n| **Parameters (M)** |  42.57  |  18.12  |   27.87   |      13.04      |"}, {"review_id": "H6mm1a0Tr1", "reviewer": "Reviewer_ouhA", "summary": "This paper addresses the problem of long-distance high-fidelity depth imaging with time-of-flight (ToF) systems. Mainly, this work proposes a joint optimization of the coding function with a depth estimation neural network in an end-to-end manner. The work uses a differentiable modeling of the acquisition model, including two regularization functions to achieve implementable coding functions, i) a polynomial function to achieve a square wave, i.e., binary valued functions ii) total variation, so that the coding function does not transition from 1 to 0 so fast. The paper also introduces a restormer-based depth estimation network which includes, beyond the traditional MSE recovery loss function, an information-theory loss function based on the Fisher information matrix to overcome the SNR in the measurements. A comprehensive evaluation of the proposed method, along with SOTA comparison, is provided, showing improvements of the paper in determined distance ranges. This is also validated with an experimental ToF prototype, showcasing the method in real acquisition scenarios.  Supplementary material includes a detailed mathematical derivation of the sensing model and additional results with other datasets.", "questions": "1. Why in Figs 5, 6 of the main manuscript and Figs 2 and 3 of the supplementary material not include a quantitative metric to fairly evaluate the quality of the proposed approach? \n\n2. Since for promoting values a degree-two polynomial is used, is it guaranteed that this polynomial achieves exact 0 and 1 values? Why not simply use a binary activation function with a Straight-Through Estimator for the gradient as widely used in binary neural networks? [1]\n\n3. Did you try with fewer measurements? $k<4$? \n\n4. Usually, loss functions are chosen to have range $[0,\\infty]$, but the used regularizer has negative values. Why not simply choose a polynomial function $f(x) = (x-1)^2 (x)^2$? Is there any advantage to the proposed approach?\n\n[1]  Le, Huu, et al. \"Adaste: An adaptive straight-through estimator to train binary neural networks.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.", "limitations": "Although the paper mentions that the method is constrained to a narrow range of distance $\\sim 3 $m,  it does not mention how to address those limitations regarding the limited range in future work.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths: \n\n1. The proposed method addresses the physical constraints for the coding functions, which are required for implementation. Three different regularizations are proposed to obtain wide binary coding functions and maximize the depth quality.\n\n2. The results are superior to the state of the art in all cases, both quantitatively and qualitatively. The results in terms of MAE are better for different levels of signal-to-noise ratio, which guarantees the robustness to noise and general improvement.\n\n3. The proposed method bridges a gap present in dToF and iToF to achieve accurate and long-distance depth imaging in a limited depth range.\n\nWeaknesses\n\n1. An in-depth analysis of the coefficients of the regularizers ($\\gamma_1, \\gamma_2, \\gamma_3$) used in Eq. (11) is required. Ablation studies and convergence analysis could be added.\n\n2. Although higher quality is achieved for long distances, the range of action is limited by $T_m$. The difference between the minimum and maximum depth is only 3 meters. A discussion of how this could be increased would improve the work.\n\n3. While the method is compared to other state-of-the-art approaches, the evaluation is limited to using their reconstruction networks with the proposed coding functions. A more comprehensive and fair comparison would involve training the coding functions from other methods with the same reconstruction networks used in this paper, to better isolate and assess the contribution of the coding stage itself.\n\n4. There should be an analysis of the “optimality” of the resulting coding functions. Is there a way to interpret them? Additionally, there should be a comparison with other coding functions  [1,2].\n\n[1] Gupta, Mohit, et al. \"What are optimal coding functions for time-of-flight imaging?.\" ACM Transactions on Graphics (TOG) 37.2 (2018): 1-18.\n[2] Gutierrez-Barragan, Felipe, et al. \"Practical coding function design for time-of-flight imaging.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.", "was_revised": true, "final_justification": "The paper presents a new method for end-to-end learning of coding functions for long-range depth estimation, jointly trained with a depth-estimation network. It proposes a differentiable framework to design binary coding sequences, with carefully designed regularizers that promote physically implementable solutions. Although end-to-end imaging-system design has been widely studied across modalities, this work provides clear design criteria and is well written. The original submission offered a comprehensive evaluation in both simulation and real data, showing improved performance over baselines across three depth ranges. During the rebuttal and discussion period, the authors added important comparisons with state-of-the-art coding-function designs (showing consistent gains), conducted ablation studies on the regularization hyperparameters, and included coding-function optimality metrics based on the Fisher information matrix, improving the interpretability of the learned patterns. I encourage the authors to incorporate these analyses into the manuscript and/or supplementary material. Based on these points, I raise my score to 5 (accept).", "rebuttal": "We are sincerely grateful to the reviewer for acknowledging the high imaging quality and strong noise robustness of our work. In this rebuttal, we would like to respond to the following questions raised by the reviewer.\n\n> **Q1: Ablation studies and convergence analysis about coefficients in Eq. 11.** \n\n**A1:** Thank you for your suggestion. In our loss function, MSE serves as the primary term to optimize the network’s depth reconstruction performance, while three additional regularization terms act as auxiliary constraints to guide the learning of an optimal binarized coding function. To validate our coefficient choices, we perform comprehensive ablation studies and present the following conclusions.\n\n**For coefficient $\\gamma_1$**, setting it below $5 \\times 10^{-6}$ prevents the learned coding function from capturing sufficient Fisher information. Conversely, if it exceeds $5 \\times 10^{-3}$, it impairs the dominance of the MSE loss, resulting in slower convergence and reduced reconstruction quality. To balance these effects, we set $\\gamma_1$ to $5 \\times 10^{-4}$ during the first 40 epochs. After observing a notable decline in the MSE loss beyond 40 epochs, we further reduce it to $5 \\times 10^{-5}$ to ensure optimal reconstruction performance.\n\n**For coefficient $\\gamma_2$**, setting it below $5 \\times 10^{-2}$ prevents the coding function from achieving effective binarization; however, when it exceeds 30, it slows down the network’s convergence. Therefore, we set $\\gamma_2$ to $5 \\times 10^{-2}$ during the first 40 epochs to allow the network to learn an initially optimal coding function. After epoch 40, we increase it to 1 to enforce strict binarization of the coding function.\n\n**For coefficient $\\gamma_3$**, setting it below $5 \\times 10^{-2}$ results in a coding function with narrow peaks, which is unsuitable for hardware implementation. On the other hand, values above 10 impose excessive constraints, degrading the reconstruction performance. Therefore, we set $\\gamma_3$ to 5 in our network to strike a balance between hardware feasibility and reconstruction quality.\n\n> **Q2: Discussion on how to improve the depth range of BE-ToF system.**\n\n**A2:** We sincerely appreciate your interest in our work. While we acknowledge the current limitations of BE-ToF, we are actively investigating several promising directions to address them in future research. One practical approach leverages BE-ToF’s flexible control over the depth range via time-delay adjustment to perform temporal scanning, thereby synthesizing a comprehensive, wide-range depth map. Considering the high latency potentially introduced by temporal scanning, we propose leveraging compressive sensing to reduce the amount of data acquisition. In addition, a more ideal approach is to adopt a coarse-to-fine strategy: initially capturing a wide-range but low-resolution depth map, followed by selective high-precision depth acquisition in regions of interest (ROIs) using BE-ToF. This approach can be further enhanced with AI-driven techniques to enable intelligent and adaptive depth sensing. Overall, I believe our method holds great promise for a wide range of applications, such as terrain mapping and autonomous driving.\n\n> **Q3: Comparison with other coding functions with the same reconstruction network.**\n\n**A3:** Thank you for your suggestion. This experiment indeed plays a key role in demonstrating the strength of our learnable binarized coding function. To ensure a fair comparison, we employ the same imaging setup (with **K = 4** measurements) and reconstruction network, varying only the coding function. The quantitative results, evaluated using Mean Absolute Error (MAE) in millimeters, are presented below. It can be observed that, under the same reconstruction network, our coding function delivers the best performance. Notably, although we compare multiple coding functions in simulation, only the square wave and our learned binarized coding functions are implementable on actual hardware. The Sinusoid wave cannot be used because it is non-binary, while Hamiltonian codes and M-sequences exhibit narrow peaks that cannot be realized due to hardware constraints on minimum exposure time.\n\n|             |  0-3 m   |  30-33 m  |  60-63 m  |  90-93 m  |\n| :---------: | :------: | :-------: | :-------: | :-------: |\n|  Sinusoid   |  24.67   |   31.12   |   39.80   |   45.39   |\n|   Square    |  16.40   |   22.66   |   26.05   |   33.35   |\n| Hamiltonian |  11.28   |   14.53   |   21.74   |   27.11   |\n| M-sequence  |  15.19   |   21.24   |   28.33   |   35.81   |\n|  **Ours**   | **8.52** | **12.86** | **18.20** | **23.51** |\n\n> **Q4: Analysis of the optimality of the learned coding functions.**\n\n**A4:** Thank you for your valuable suggestion. This is quite important to demonstrate the optimality of our learned coding functions. As discussed in Reference [10] of the manuscript, Fisher information can be used as a metric to assess the optimality of different coding schemes — a higher Fisher information value indicates a more optimal coding scheme. Therefore, we list the Fisher information of different coding functions in the table below for a straightforward comparison. It can be observed that our learnable binarized coding function achieves the highest Fisher information, which demonstrates its optimality.\n\n|  Coding Function   |      Sinusoid      |       Square       |     M-sequence     |    Hamiltonian     |        Ours        |\n| :----------------: | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: |\n| Fisher Information | $1.27 \\times 10^6$ | $2.29 \\times 10^6$ | $2.18 \\times 10^6$ | $2.92 \\times 10^6$ | $3.93 \\times 10^6$ |\n\n> **Q5: Quantitative metrics for Fig. 5 and 6 in the main manuscript, and Fig. 2 and 3 in the supplementary material.**\n\n**A5:** We sincerely apologize for the misunderstanding caused. In fact, we have conducted detailed quantitative analyses for each experiment. **In the main manuscript**, Fig. 5 primarily compares our BE-ToF method with several traditional iToF approaches, with the corresponding quantitative metrics summarized in Table 1(a). Fig. 6 presents two sets of experiments: the first compares traditional square demodulation functions with our RSCF-Net, with quantitative results shown in Table 1(b); the second compares the performance of different networks using our learned coding functions, with corresponding metrics provided in Table 1(c). **In the supplementary material**, Fig. 1 illustrates the generalization results on the 4D Light Field Dataset compared to traditional iToF methods, with quantitative metrics summarized in Table 1(a). Fig. 2 shows the generalization results on the SUN RGB-D Dataset, with corresponding quantitative metrics provided in Table 1(b). We will revise the manuscript to clearly present the results of each experiment, enhancing readability and avoiding potential misunderstandings.\n\n> **Q6: The optimization precision of the binarized coding functions and the reason for not using a binary activation function with a Straight-Through Estimator.**\n\n**A6:** We appreciate your attention to this issue. Since we employ a differentiable double well function to encourage the coding functions to converge toward 0 or 1, it cannot numerically attain exact binary values. Nevertheless, our examination shows that the learned coding functions are already very close to 0 and 1 (e.g., around 0.0001 or 0.999), and we consider such a small deviation negligible and acceptable for our network. We choose the double well function rather than the Straight-Through Estimator because we aim to implement a **fully differentiable** approach for learning a binarization function, without relying on any gradient approximation. We believe this represents a meaningful and unexplored direction, with our experiments validating the simplicity and effectiveness of the double well function.\n\n> **Q7: Experiments with fewer measurements like $K < 4$.**\n\n**A7:** As noted in lines 135–136 of the manuscript, at least K ≥ 3 measurements are required to recover depth. In our work, we choose K = 4 to ensure high reconstruction quality and robustness to noise. Below, we present quantitative results for cases with K < 4, evaluated using Mean Absolute Error (MAE) in millimeters. When K < 3, the reconstruction quality significantly degrades, which is reasonable given the limited information available for depth recovery. With K = 3, depth can be reasonably reconstructed, though still slightly inferior to K = 4, where the additional measurement improves robustness to noise and other perturbations.\n\n|       | 0-3 m  | 30-33 m | 60-63 m | 90-93 m |\n| :---: | :----: | :-----: | :-----: | :-----: |\n| K = 1 | 133.22 | 166.90  | 156.18  | 198.19  |\n| K = 2 | 37.84  |  42.94  |  51.76  |  49.18  |\n| K = 3 | 14.58  |  17.59  |  21.64  |  29.88  |\n| K = 4 |  8.52  |  12.86  |  18.20  |  23.51  |\n\n> **Q8: Clarification on the choose of the double well function loss.**\n\n**A8:** Thank you for pointing this out. Traditional loss functions like MSE are non-negative because they quantify the error between predictions and ground truth. In contrast, our double well loss serves as a regularization term applied directly to the coding function and is not based on ground truth, so it may take negative values. Moreover, many other loss functions [1,2] also contain negative values, and it has been demonstrated that this does not hinder the network’s optimization process. As for why we don’t use $x^2(x-1)^2$ instead: Although both functions have a similar shape, our double well function exhibits more pronounced gradient variations during [0, 1], which helps the network converge more quickly.\n\n\n\n[1] Adler J, Lunz S. Banach wasserstein gan[J]. Advances in neural information processing systems,2018\n\n[2] Wang Q, Ma Y, Zhao K, et al. A comprehensive survey of loss functions in machine learning[J]. Annals of Data Science,2022"}, {"review_id": "7115Wf37r4", "reviewer": "Reviewer_56Ri", "summary": "This paper proposes Burst-Encodable Time-of-Flight (BE-ToF) and an end-to-end learnable reconstruction framework that jointly optimizes the coding function and ToF signals to obtain a better quality depth map.\n\nBE-ToF uses a short-period Modulation signal to have the advantage of using a high-frequency modulation signal (e.g. better details in the long distance) while adding a time delay in the demodulation signal to get over a phase wrapping issue on the far distance region.\n\nThe reconstruction network is based on RSCF-Net and takes synthetically generated raw ToF images using the BE-ToF Imaging model. This network can optimize both learnable coding functions that can optimize the demodulation signal of the ToF, and the network itself, which denoises and improves the ToF signals to produce high-quality depth in the end.\n\nIn the end, the paper shows the prototype of BE-ToF depthmap that uses the learned coding function and BE-ToF principle.", "questions": "1. Lack of Clarity\n\n- Somehow, the explanation regarding the advantage of BE-ToF in Sec.3 is not really clear. Personally, what is written in the first and second paragraphs of the supplementary material helped me more to understand the purpose of BE-ToF. It would be great to merge the Suppl. material's BE-ToF principle to Sec.3\n\n- How the differential BE-ToF Imaging Model is used is not well written in the paper. It would be great to refer to Fig.2 in Section 3.1, then the use of the model will be clearer.\n\n- Also, It would be great to mention in the dataset paragraph  (4.1)  that the raw ToF images are generated/synthesized from Sec. 3.1, Eq.5. It was not really clear how the train is done from the conventional depth dataset but with the input of raw ToF signal in the first iteration of reading. Mentioning that briefly will make readers understand the paper much more easily. \n\n- It would be great to mention the reasoning behind the selection of RSCF-Net briefly.\n\n- The depth range in Fig.5 is created by rescaling the GT depth? Also, I see the range is always 3m. Is it because the short-period signal that goes up to 3m is used for the experiment? Then it would be great to mention briefly in Sec.4.2.\n\n- It's hard to understand lines 189-190 regarding albedo. In Eq2, roh_s is reflectance (line 123), but in lines 189-190 it is mentioned as albedo. Also, it's hard to understand why using the R channel of albedo is used as roh_s\n\n2. BE-ToF's limitation of being able to acquire only in a specific range of depth maps\n\n- This is more of a limitation of BE-ToF functionality. It's great to have a ToF that can capture the far distance without having issues with phase wrapping. But having a small distance window limits its real-life use case. It would be great to have some way to improve this issue or have a discussion on this as future work.\n\n3. Limited Originality\n\n- This paper mainly shows details about the reconstruction network and relatively small details about BE-ToF, but the network part seems to be an incremental work from [10] (in the main paper), which changed the reconstruction network backbone and added more loss in the forward model, and the network is not necessarily tied to the BE-ToF modality as well. \n- As proposing BE-ToF is the main differentiation from [10] (in the main paper), i'd love to see experiments that run more with real-life BE-ToF instead of synthetic training. I posted more comments regarding the experiment in the next section.\n\n4. Limited real ToF dataset experiment\n\n- It's understandable that doing a real-life experiment is challenging in this setup. But the experiments are mostly synthetic, and only one picture of a real-life ToF image is included. In a real-life scenario, raw ToF images are really noisy, sparse, having physically induced noise like MPI, Motion Blur, and artifacts from surface material of the objects in the scene, which can make the input of the raw ToF signal significantly different from the simple ToF model introduced in the paper. \n- Also, experiment on Sec.5 doesn't seem to use what was trained from the network training (at least its not mentioned), if the network is not used in real, the readers can understand that the real raw signal is not really close to what is used in the synthetic trained input. Synthetic-based trained ToF models still show results on the real ToF setup, for example, from DeepToF or Gu et al [1]. If the depth in Fig.8 is from the BE-ToF and Network, then please specify. And if not, it would be great to see how the result looks on the trained network.\n- As far as I see, the reconstruction network itself is independent to how the raw ToF signal is acquired. One suggestion is to test the reconstruction network on the real-life ToF dataset that contains raw ToF and depth map as GT. This way, the authors can prove the performance of the proposed network compared to the existing ToF pipelines. As far as I checked, there are two real datasets with raw ToF included (CroMo dataset [2], HAMMER dataset [3])\n- Another possible experiment is to modify the system prototype to make the signal compatible with other trained baselines (e.g. changing frequency and without delay to try FisherToF model, adding dual frequency model to test on DeepToF model) and show the qualitative experiment on the depth images. It would be great to see whether the real-life quality of depth on different setups acts similarly to what is shown in Fig.5.\n\nMinor suggestion but, there is not much related work on ToF imaging. There are two more works that are quite relevant to this paper to be added in the related work: [1] (as multi-frequency modulation, and learnable kernel), [4] (RGB-based phase unwrapping from raw ToF)\n\n[1] Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset, Guo. et a.\n\n[2] CroMo: Cross-Modal Learning for Monocular Depth Estimation, Verdié et.al.\n\n[3] On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks, Jung et.al.\n\n[4] Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments, Jung et.al", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strength :\n1. The paper proposes a fundamental solution to the existing issue with i-ToF sensing\n2. The authors built a real-life prototype of the sensor that is proposed in the paper and shows it actually works in a real-life scenario\n3. Both Qualitative and Quantiative experiments show superior quality of the proposed modality and the nework\n\nWeakness (suggestions for each weakness are written in the Qestions part)\n1. Lack of Clarity: The paper includes many details in different parts, but the connection is very weak between the details. \n2. BE-ToF's limitation: BE-ToF is able to acquire only in a specific range of depth maps.\n3. Limited Originality:  The paper's writing focuses a lot on the network and training part, while the network and training rather seem to be an incremental work from [10], which is rather chaing the backbone and adding more losses\n4. Limited real ToF dataset experiment: Experiments are mainly done on the synthetic training while only one image is added for the real life scenario.", "was_revised": true, "final_justification": "The authors clarified the questions regarding the setup used in the paper and especially provided extra real-life experiments with quantiative evaluations, which were the major drawback of the paper. Given that the authors have polished the paper according to the rebuttal, the paper will be in a reasonable state.", "rebuttal": "We appreciate the reviewer’s recognition of the effectiveness of our method. In this rebuttal, we provide detailed responses to the main concerns raised in the review.\n\n> **Q1: Lack of clarity.**\n\n**A1:** \n\n1. The key advantage of our BE-ToF system is its capability to achieve long-distance and high-precision depth imaging with only a single modulation frequency. In addition, as shown in Fig. 1(b), our depth accuracy and range depend solely on the period $T_m$. Therefore, our BE-ToF system is highly robust to the laser's pulse repetition frequency (PRF) and can operate reliably across a wide range (ranging from tens of Hz to hundreds of kHz), significantly reducing the hardware implementation complexity. We sincerely appreciate your valuable suggestion and will integrate the description of the BE-ToF principle in the camera-ready version.\n\n2. As illustrated in Fig. 2(b), the differentiable forward model is designed to simulate ToF measurements from ground-truth depth. We first compute the reflected signal $R(s,t)$ in Eq. 3 based on the GT depth. Subsequently, the ToF measurements $I_i(s)$ in Eq. 4 are synthesized using the learnable binarized coding functions. Taking into account the noise characteristics of camera sensor, we introduce readout noise and dark noise to obtain more realistic measurements $X_i(s)$ in Eq. 5. We will carefully revise Sec. 3.1 and Fig. 2 to ensure better consistency and clarity.\n\n3. It‘s true that the ToF measurements are simulated from the depth dataset following the method described in Sec. 3.1. We will clarify this point in Sec. 4.1 to ensure better readability.\n\n4. We appreciate your suggestion. In RSCF-Net, we adopt Restormer as the backbone due to its strong ability to model long-range pixel interactions and  its lightweight architecture. Moreover, unlike conventional image restoration, ToF depth imaging involves reconstructing a depth map from multiple frames of ToF measurements, which places greater emphasis on capturing inter-channel dependencies. To address this, we design the CFEB and MFFB modules to more effectively fuse information across channels. As presented in Tab. 1 of the main manuscript, the proposed RSCF-Net achieves state-of-the-art performance. Furthermore, Tab. 1 in the supplementary material demonstrates its strong generalization capability across diverse datasets. We also conduct a systematic comparison of various networks regarding their computational complexity (FLOPs), parameter count. The detailed results are summarized below.\n\n|                    | DeepToF | MaskToF | FisherToF | RSCF-Net (Ours) |\n| :----------------: | :-----: | :-----: | :-------: | :-------------: |\n|   **FLOPs (G)**    |  12.65  |  5.62   |   26.72   |      14.69      |\n| **Parameters (M)** |  42.57  |  18.12  |   27.87   |      13.04      |\n\n5. To ensure fair comparisons, all experiments in Sec. 4 are conducted with a fixed depth range of 3 meters (corresponding to $T_m$=20 ns). For the NYU-V2 dataset, we first scale the depth values to a 3-meter range. Then, to simulate the same depth range at different distances, we adjust the light attenuation coefficients accordingly. We will incorporate these clarifications into Sec. 4.2 as suggested.\n\n6. We appreciate your careful reading. As you correctly noted, $\\rho_s$ denotes reflectance as defined in line 123. We will revise the terminology in line 189 for consistency. For the definition of reflectance, we refer to the intrinsic image decomposition introduced in reference [35] of the manuscript. In [35], given $p$ is a pixel position, an image $I(p)=R(p) \\cdot S(p)$ can be decomposed into shading layer $S(p)$ and reflectance layer $R(p)$. The shading layer $S(p)$ depicts the amount of reflected light, while the reflectance layer $R(p)$ depicts the intrinsic color of the material at $p$. Furthermore, $R(p)$ is decomposed as $R(p) = R_B(p) \\cdot T(p)$, where $R_B$ is the base reflectance and $T$ represents texture details. Based on this, we perform intrinsic image decomposition on the NYU-V2 dataset and use the base reflectance layer $R_B$ as $\\rho_s$ in Eq. 2.\n\n\n\n\n\n\n\n> **Q2: BE-ToF's limitation of being able to acquire only in a specific range of depth maps.**\n\n**A2:** We sincerely appreciate your interest in our work. While we acknowledge the current limitations of BE-ToF, we are actively investigating several promising directions to address them in future research. One practical approach leverages BE-ToF’s flexible control over the depth range via time-delay adjustment to perform temporal scanning, thereby synthesizing a comprehensive, wide-range depth map. Considering the high latency potentially introduced by temporal scanning, we propose leveraging compressive sensing to reduce the amount of data acquisition. In addition, a more ideal approach is to adopt a coarse-to-fine strategy: initially capturing a wide-range but low-resolution depth map, followed by selective high-precision depth acquisition in regions of interest (ROIs) using BE-ToF. This approach can be further enhanced with AI-driven techniques to enable intelligent and adaptive depth sensing. Overall, I believe our method holds great promise for a wide range of applications, such as terrain mapping and autonomous driving.\n\n> **Q3: Limited originality.**\n\n**A3:** Thank you for your attention to this issue. Our work is fundamentally different from [10]. While [10] seeks an optimal coding scheme within the conventional iToF framework, BE-ToF is designed to enable long-distance imaging, which traditional iToF systems cannot support. The double well function loss and first-order difference loss are tailored to our hardware constraints, as non-binary codes are incompatible with our system. Our network employs a lightweight backbone and multi-channel fusion modules, improving both performance and efficiency compared to [10]. We will expand the discussion of BE-ToF in the manuscript to better highlight its contributions.\n\n> **Q4: Limited real ToF dataset experiment.**\n\n**A4:** \n\n1. Indeed, presenting more real-world experiments and comparisons with different methods is crucial to demonstrating the effectiveness of our BE-ToF system. Therefore, in this rebuttal, we upgrade our hardware setup and conduct experiments across a wider range of scenarios. **The specifications of the upgraded hardware are detailed below:** the system primarily consists of a solid-state laser and an exposure-encodable ICMOS sensor. The laser operates at a wavelength of 532 nm, with a pulse width of 5 ns, a fixed repetition rate of 1 kHz, and a single-pulse energy of 1 mJ. To enable area illumination, we use a laser beam expander. The ICMOS sensor is equipped with a 300–800 mm varifocal lens and a 532 nm band-pass filter (FWHM ±10 nm) to suppress ambient light interference. We conduct experiments across a variety of scenarios, including indoor scenes (\\~40 m) such as a hand model and a kettle, as well as outdoor scenes (\\~60 m) such as a stone and a staircase. All experiments are conducted under conditions consistent with the simulation setup: we apply the same learned coding function obtained from simulation and use our RSCF-Net for reconstruction. The modulation period $T_m$ is also kept the same as in simulation, set to 20 ns. We conduct a detailed comparison with other methods, and the quantitative results—evaluated using **Mean Absolute Error (MAE) in centimeters**—are presented below. The ground truth is obtained by performing a time-delay scan using our minimum exposure time (3ns), with a step size of 1 ns. The results show that our system consistently achieves centimeter-level depth estimation across various scenarios and outperforms existing methods in terms of accuracy. **We promise to provide reliable visualizations in the camera-ready version.**\n\n|            | Square + PS | Square + RSCF-Net | Learned Code + DeepToF | Learned Code + FisherToF | Ours |\n| :--------: | :---------: | :---------------: | :--------------------: | :----------------------: | :--: |\n| hand model |    10.57    |       3.96        |          4.73          |           2.47           | 1.78 |\n|   kettle   |    9.13     |       3.51        |          4.14          |           1.99           | 1.41 |\n|   stone    |    12.16    |       4.76        |          5.94          |           3.78           | 2.50 |\n| staircase  |    15.79    |       5.92        |          6.70          |           4.26           | 3.83 |\n\n2. The experiment shown in Fig. 8 of the manuscript follows the same setup as the simulation. The ToF measurements are obtained using the coding functions learned from simulation, and the depth is reconstructed using the pre-trained RSCF-Net.\n\n3. We sincerely thank the reviewer for recommending these works. Both datasets are of high quality and provide valuable benchmarks for evaluating the performance of our network. However, due to time constraints, we have not yet completed the retraining and evaluation of our network and other comparative methods on these datasets. We will supplement our experimental results at a later stage.\n\n4. We regret that we cannot modify our system prototype for a direct comparison with FisherToF and DeepToF, as our laser operates at a fixed repetition rate of 1 kHz. Conventional iToF systems rely on high-frequency modulated lasers (tens to hundreds of MHz), which inherently have limited output power and are thus restricted to short-range sensing. In contrast, our BE-ToF system eliminates the need for high modulation frequencies, allowing the use of a low-repetition-rate laser with high-energy pulses, making it well-suited for long-range depth imaging.\n\n> **Q5: Supplementary references in related work.**\n\n**A5:** Thank you for your valuable suggestion. The suggested papers are indeed closely related to our work, and we have now included them in the revised manuscript."}]

Author rebuttal:

Meta-review: This paper received four positive reviews, with all four reviewers converging to accept recommendations --- 4 Accepts. 

There was general appreciation for the significance of the problem considered here, the novelty of the solution, the strength of the results and the overall presentation. 

While there were some concerns raised in the original reviews, they were adequately addressed during the author-reviewer discussion phase. As a result, an accept decision was reached.
Decision: Accept (spotlight)